Moving toward Effectiveness, Equity and Efficiency (RESUME3) Contents Acknowledgements ................................................................................................................................... 4 List of Acronyms ...................................................................................................................................... 5 Overview ..........................................................................................................................................7 1. How does Ukraine’s education system perform?.............................................................................. 8 2. What does Ukraine need to do to strengthen its education reform agenda? ................................... 14 Introduction.................................................................................................................................... 32 Chapter 1: The Context for Education Reform in Ukraine ............................................................. 35 Education System Structure .................................................................................................................... 35 Trends Shaping Education Reform in Ukraine ....................................................................................... 36 Ukraine’s Ambitious Agenda for Education Reform.............................................................................. 40 Public Perceptions of Education Reform ................................................................................................ 44 Chapter 2: Assessing Education and Labor Market Outcomes in Ukraine ..................................... 48 Learning Outcomes and Skills Supply .................................................................................................... 48 Labor Market Outcomes ......................................................................................................................... 52 Returns to Education ............................................................................................................................... 55 Employment Structure and Demand for Tertiary Education .................................................................. 58 School to Work Transition ...................................................................................................................... 62 Chapter 3: Ensuring that All Students Learn in the New Ukrainian School .................................... 69 Ensuring Readiness to Learn .................................................................................................................. 69 Aligning Incentives for Teachers ............................................................................................................ 73 Spending Wisely for Capable School Networks ..................................................................................... 83 Assessing Learning and Monitoring Quality .......................................................................................... 92 Providing Equal Opportunities to Prepare for Higher Education ........................................................... 94 Chapter 4: Strengthening the Sustainability and Transparency of Higher Education .................. 104 Massification and Inefficiency of the Higher Education System.......................................................... 105 Improving Efficiency and Transparency of Financing ......................................................................... 115 Ensuring Transparent and Equitable Access to Higher Education ....................................................... 120 Governing the Higher Education System.............................................................................................. 123 Strengthening Quality, Relevance, and Integrity .................................................................................. 125 Chapter 5: Directions for the Future of Education in Ukraine ...................................................... 138 Articulating a Vision for Improving Education Quality and Relevance ............................................... 138 What needs to be done to achieve that vision? ..................................................................................... 140 2 References .................................................................................................................................... 151 Annex 1: Education System Structure .......................................................................................... 159 Annex 2: Literature Review on Returns to Education in Ukraine ................................................. 160 Annex 3: Methodology and Estimation Results for Analysis of Secondary School Learning Outcomes ...................................................................................................................................... 162 Annex 4: Donor Mapping ............................................................................................................. 172 Annex 5: Stakeholder Mapping for Higher Education Governance and Policymaking ................. 175 Annex 6: Universities with the Largest Share of Public Funding .................................................. 177 3 Acknowledgements This report was written by a team led by James Gresham and Diego Ambasz, and included Suhas Parandekar, Juan Manuel Moreno, Simon Thacker, Lucia Casap, Subhashini Rajasekaran, Iryna Kohut, Inna Sovsun, Olga Kupets, and Gülcan Yayla. The CEDOS think tank led by Yegor Stadny provided data and background analysis. Additional data and analysis were also provided by Yevhen Nikolaiev and Lyudmyla Sokuryanska. Support was provided by Larysa Khaletska and Patrick Biribonwa. The analysis was carried out under the overall supervision of Satu Kähkönen, Country Director, Ukraine and Harry A. Patrinos, Practice Manager, Education. Valuable comments at various stages of the project were received from Feng Zhao, Halsey Rogers, Elizabeth Ninan, Shwetlena Sabarwal, Jason Weaver, Soren Nellemann, Lars Sondergaard, Anthony Gaeta, Klavdiya Maksymenko, and Anastasia Golovach. The team benefited greatly from discussions at various formal and informal meetings with representatives of various government agencies, local authorities, and the donor community. In particular, valuable comments were received from experts of the Ministry of Education and Science, the Institute for Education Analytics, the Ukrainian Center for Education Quality Assessment, the Institute for Educational Content Modernization, and the National Agency for Quality Assurance in Higher Education. 4 List of Acronyms AH Amalgamated hromada CIS Commonwealth of Independent States CMU Cabinet of Ministers of Ukraine CPD Continuous Professional Development DISO State Information System of Education (EMIS) EACEA Education, Audiovisual and Culture Executive Agency (EU) ECA Europe & Central Asia ECTS European Credit Transfer System EDEBO Unified State Electronic Education Database EHEA European Higher Education Area EIT External Independent Test EMIS Education Management Information System EQF European Qualifications Framework ESG European Standards and Guidelines ETF European Training Foundation EU European Union EUR Euros GDP gross domestic product HCI Human Capital Index HEI Higher education institution HEMIS Higher education management information system HLCS Household Living Conditions Survey ICT Information and communication technology IDP Internally displaced persons IEA Institute for Education Analytics IERC Inclusive Education Resource Center INSET In-Service Teacher Education and Training ISCED International Standard Classification of Education ISCO International Standard Classification of Occupations LFS Labor Force Survey LLL Lifelong learning MELQO Measuring Early Learning Quality and Outcomes initiative MOES Ministry of Education and Science MOEDT Ministry of Economic Development and Trade MOF Ministry of Finance MOH Ministry of Health NAQAHE National Agency for Quality Assurance in Higher Education NASU National Academy of Sciences of Ukraine NBPTS National Board for Professional Teaching Standards NQF National Qualification Framework NUS New Ukrainian School OECD Organization for Economic Cooperation and Development PD Professional development PES Public Employment Services PIAAC Programme for the International Assessment of Adult Competencies PISA Programme for International Student Assessment PPP Purchasing Power Parity PRESET Pre-Service Teacher Education and Training 5 R&D Research and development SABER Systems Approach for Better Education Results SAC State Accounting Chamber SAS State Audit Service SLC School Leaving Certificate SRS State Regulatory Service SSEQ State Service of Education Quality SSSU State Statistics Service of Ukraine STEP Skills Toward Employment and Productivity STI Science, Technology and Innovation TIMSS Trends in International Mathematics and Science Study UAH Ukrainian hryvnias UCEQA Ukrainian Center for Education Quality Assessment UIS UNESCO Institute for Statistics UK United Kingdom ULMS Ukrainian Longitudinal Monitoring Survey USA United States of America USD United States dollars USSR Union of Soviet Socialist Republics (Soviet Union) VET Vocational education and training WDI World Development Indicators WDR World Development Report 6 Overview Today, Ukraine is at a crossroads: despite impressive transformation in some sectors of the economy, the foundations of the emerging new economy are still fragile. Openness to the outside world, adoption of new technologies, and a vibrant entrepreneurial spirit is driving this transformation. However, growth has been volatile and unsustainable with a reliance on commodity-based exports, short-term foreign savings, and foreign remittances. The Euromaidan revolution created further acute political and economic challenges which led to a broad agenda of structural reforms. Going forward, the old growth model that relied on legacy industries will not deliver Ukraine’s aspirations.1 Education and human capital development are critical for driving high, sustainable and inclusive growth in Ukraine, yet human capital remains a small share of national wealth. Historically, Ukraine has benefited from a strong education system that has propelled the country’s economic and social development. Since independence, Ukraine has been able to sustain many of its comparative advantages in educational excellence, contributing to high levels of educational attainment and human capital development. However, skills demanded by the expanding sectors are different than those supplied by the education system, and change has been slow. According to recent wealth estimates of 141 countries, human capital comprises only 34 percent of total national wealth in Ukraine, compared to 51 percent for lower- middle-income countries and 62 percent for the Europe and Central Asia (ECA) region.2 Despite high levels of education, human capital has been a relatively weak factor of production in driving economic growth. Per capita income and labor productivity also remain among the lowest in the region, and the population continues to decline. Between 1999 and 2017, the period over which Ukraine’s private sector emerged, the average rate of growth of per capita income was 3.3 percent per year, compared to the average of Commonwealth of Independent States (CIS) countries at 5.6 percent or non-CIS countries at 3.6 percent. Ukraine’s economic transformation remains incomplete, and although the economy has grown, per capita income growth has been volatile.3 At the same time, Ukraine’s population continues to decline due to declining birth rates and emigration. The population of Ukraine has shrunk by around 15.0 percent since 1999, and the youth population has declined by nearly 25.0 percent. There is strong reason to believe that the education system needs to change or risk falling behind, and Ukraine recognizes this need. While there are competing visions of what knowledge and skills will be needed in the future, education and the systems that educate the next generation must constantly evolve and adapt to a fast-changing world. The increasing role of technology in economic activities and everyday life has already led to significant changes in the demand for skills, with a greater need for advanced skills in all types of work.4 However, upgrading cognitive skills alone is not enough: ‘soft’ skills are increasingly important given that interpersonal relations between humans cannot (yet) be replaced by the intervention of technology. Adaptability requires a strong and balanced toolkit of skills, which requires a rethinking of the traditional dividing lines between academic and technical disciplines. Although Ukraine has taken bold steps toward reforming its system, the reform process has been uneven and additional reforms are needed to ensure success and continuity of the process. In Ukrainian discourse, ‘reform’ is often believed to consist of legislative changes only.5 In this sense, reform is well underway with the passage of recent laws, particularly in the general secondary and higher education sectors. However, legislative changes must be implemented materially in practice to create the change in results that is desired. This requires resources, technical know-how, public support, and political leadership. Furthermore, additional reforms are critical to address imbalances that remain. This report seeks to answer two key policy questions: 7 1. How does Ukraine’s education system perform in terms of effectiveness, equity, and efficiency? 2. What does Ukraine need to do to address constraints to progress under the ongoing reform agenda and position education as a driver of growth? 1. How does Ukraine’s education system perform? Ukraine is committed to developing a modern education for the 21st century, and it has taken a number of key steps to make this happen. The reforms introduced following the Euromaidan Revolution have generated great optimism by decentralizing and democratizing the education system, while laying the foundations for greater alignment and integration with European norms and standards in education. Despite Ukraine’s impressive foundations in and history of education, the system appears misaligned with the changing needs of the economy and the population. This section looks at system performance along three dimensions: effectiveness, including quality and relevance; equity and inclusion; and efficiency of resource use. Ukraine has a highly educated population, but the education system needs to focus more on quality over quantity and on meeting the evolving skills needs of the labor market Educational attainment in Ukraine has increased considerably over the last 30 years, particularly at the tertiary level. Between 1980 and 2010, the average years of schooling for the population over age 15 has increased nearly three-fold (figure O.1).6 Educational attainment for the average Ukrainian now exceeds that of other high-income countries, such as the United Kingdom and Germany. Ukraine has also become one of the top countries in terms of higher education coverage of the population, a trend that has accelerated rapidly after independence. This is partially explained by Ukrainian legislation, by which colleges and technical schools became part of the higher education system until recently. Still, as of the 2017/18 academic year, the higher education coverage rate was 82 percent. The share of the population age 25+ with at least some tertiary education has reached about 40 percent, exceeding that of the OECD average and many other countries. Figure O.1 Educational attainment of population in Ukraine and selected countries, 1980–2010 Population Age 25+ with Tertiary Education 45 40 Ukraine 35 Percent of population age 25+ 30 OECD Sweden 25 Germany Poland 20 Hungary Chile 15 Portugal Argentina 10 5 0 1980 1985 1990 1995 2000 2005 2010 Source: World Bank EdStats. However, learning outcomes among secondary school students and literacy proficiency levels among tertiary educated adults lag other countries. While the Human Capital Index predicts that a child born 8 today in Ukraine will receive 13 years of schooling by the time she reaches age 18, this figure drops to 10.2 years after adjusting for the quality of learning. This means that for the average student, 2.8 years of schooling time is ineffective or wasted in the sense that it does not result in learning. This learning gap is substantially higher in Ukraine than in high-performing education systems such as Canada, Finland, or the Republic of Korea. Furthermore, literacy proficiency scores collected through the World Bank’s Skills Towards Employment and Productivity (STEP) survey indicate that higher educational attainment does not guarantee even basic cognitive skills or ensure such skills relative to other countries. Although university graduates have higher reading proficiency scores on average, the top-performing individuals with only a general secondary education scored higher than over 50 percent of university graduates, suggesting wide variation in learning outcomes (figure O.2).7 Figure O.2 Learning outcomes and literacy proficiency in Ukraine and selected countries Learning gap based on HCI Literacy proficiency of adult population 15 (2011-2013) 320 Literacy scale (range 0-500) 14 310 300 13 290 Years of School 280 12 270 Japan Korea, Rep. Finland 260 Canada Germany Sweden Austria 11 United Kingdom 250 Estonia Poland Latvia United States 240 10 USA Korea, Rep. Canada Armenia Poland Japan Finland UK Austria Estonia Sweden Germany Ukraine Ukraine 9 8 Total adult population Expected Years of School Learning-Adjusted Years of School Adult population with tertiary education (ISCED-1997 5A/6) Source: World Bank HLO Database and STEP Survey. The rapid expansion of the higher education system has produced an increasing number of graduates, many of whom have not been absorbed into the labor market except in jobs that do not require a university-level education. Thus, relative to demand, there is a larger supply of tertiary graduates along with a diminished relevance of credentials. This has contributed to education-job mismatch, particularly for young university graduates. Consequently, the share of tertiary educated workers among the unemployed has increased from 32 percent in 2004 to 47 percent in 2013.8 Moreover, as of 2013, 40 percent of young university graduates were working in lower-level jobs that did not require university-level education, in comparison to 29 percent for prime-age and older workers. In terms of employee satisfaction, a serious skills mismatch is identified as 40 percent of employers reported significant skills gaps which harm business objectives. Teaching and learning in universities remain focused on knowledge acquisition, often with outdated content that is not adjusted to the needs of modern students or new trends, research, or technology. Although there is little systematic information on quality of learning outcomes in higher education, there is a common view that curriculum and pedagogical teaching methods need to be modernized to reflect the evolving needs of society and the economy. For example, applied business, financial and management skills are in short supply in Ukraine, and only a few universities actively promote entrepreneurship training and programs.9 Globally, higher education is trending toward multidisciplinary study programs that emphasize experiential learning and project-based approaches to solve complex problems. Ukraine would 9 benefit from more support for thematic teaching, entrepreneurship, and student-centered approaches to learning, as well as involving employers in study program revisions to improve employability. Corruption, including a mass disregard for academic integrity and a high tolerance for academic violations, also poses an ongoing challenge to education quality and the signaling power of credentials. Although corrupt access to the higher education system has diminished since the introduction of the External Independent Test (EIT) in 2008, other issues persist, such as academic dishonesty and systematic violations of academic integrity principles, including cheating during exams and plagiarism. These instances are commonplace and are not perceived as abuses. Different estimates indicate that at least 25–30 percent of students have directly engaged in academic misconduct or bribery, with a much larger share exposed to and familiar with such practices. Given that higher education is central to public and private sector development and is a primary driver of social mobility, this challenge remains a major concern for the higher education system.10 Despite evidence of diminishing quality and skills mismatches, economic returns remain relatively high at the tertiary education level, though they vary by field of study. Available estimates from the period 2006–2016 indicate that the private returns to an additional year of schooling in Ukraine are approximately 6 percent, compared with the average in Europe and Central Asia of 7.3 percent and the world average of 8.8 percent.11 However, workers with tertiary education have much higher returns than graduates of secondary vocational schools as well as lower-educated workers with general secondary education or less, though the returns vary by field (Figure O.3). High returns despite diminishing quality and skills mismatches are driven by a confluence of factors, including an over-emphasis on educational credentials in the labor market, high selectivity into tertiary education by program and institution, and the combined effect of older and younger cohorts of tertiary educated workers comingled together in the labor market. The variation in returns by field of study indicates the importance of strategic development of higher education and the need to provide information to students on their prospects when making choices about their educational pathways. Figure O.3 Returns to education by level and field of study Relative to upper secondary education and less on yearly earnings of employees at their main job ( %) Tertiary, Agriculture Tertiary, Military and defense Tertiary, Arts Tertiary, Education Tertiary, Humanities Tertiary, Natural sciences Tertiary, Technical sciences Tertiary, Medicine Tertiary, Law Tertiary, Economics Vocational 0 10 20 30 40 50 60 70 2006 2016 Source: Authors’ calculations based on HLCS-2006 and 2016 (individual-level data for the fourth quarter). Poor governance and limited labor market relevance have contributed to a decline in public trust and a focus on credentials rather than skills, which threaten Ukraine’s human capital edge. Prior to the Euromaidan Revolution of 2014, nearly 75 percent of Ukrainians believed that corruption was pervasive or widespread in higher education, and about 50 percent reported the same for secondary education. 12 10 According to the 2016 Life in Transition Survey, Ukraine is an outlier among other countries, with relatively low levels of satisfaction with public education, along with high levels of experience with corruption.13 At the same time, the share of tertiary educated workers among the unemployed increased from 32 percent in 2004 to 47 percent in 2013.14 Low accountability and skills mismatches contribute to a growing level of credentialism and to the public sense that the education system is no longer meeting the needs of students, families and employers. Inequities in Ukraine’s education system start early and limit the potential of learners and the system as a whole High-performing education systems prioritize equity and inclusion of all learners, maintaining the vision that all students are capable of high achievement with the right level of support. Ensuring that every child benefits from high-quality instruction is not only an important end in itself. The evidence from international assessments suggests that strong performance for the system as a whole is dependent on the need to deliver for every child.15 Unequal access to preschool means that inequalities in the opportunity to learn and benefit from schooling start early, particularly in rural areas and for poor families. Ukraine has high rates of enrollment in pre-primary education by international standards, but access remains unequal. The net enrollment rate for children ages 3-5 in urban areas is 85 percent on average, compared to 58 percent in rural areas. The rural enrollment rate drops even further in some oblasts, such as Kharkiv (54 percent), Lviv (44 percent), and Ivano-Frankivsk (39 percent).16 This is concerning, given the results of the recent UCEQA monitoring study of primary school graduates, conducted in 2018, which clearly shows that Grade 4 students who attended preschool scored significantly better in mathematics than those who did not.17 Early inequities persist over time, which prevents many students from acquiring the foundational skills needed to succeed in higher education or the labor market. Ukraine performs well on available international assessments compared to other countries at similar income levels. However, this masks a high degree of inequality. The latest available international assessment, TIMSS 2011, indicates that 28 percent of Ukrainian students reached only the low benchmark for mathematics performance, and another 20 percent of students failed to reach the low benchmark. This means that nearly 50 percent of students in total are at the lower end of the mathematics achievement distribution. School segregation and poor-quality learning environments, particularly in rural areas, contribute to the challenge. Evidence from external learning assessments in Grade 4 and Grade 11 indicate that inequality is driven by (a) clustering of poor students in poor schools; (b) inadequate learning environments in small-sized and rural schools; and (c) selectivity that creates between-school inequality such as the difference in performance between selective, “elite” schools (gymnasiums, lyceums, specialized schools) and regular non-selective schools. Rural schools have substantially less access to learning materials and information and communication technologies (ICTs) and are more likely to have shortages of subject teachers, according to DISO data from 2018. The hub school program is not yet having its intended effect on learning outcomes. Although the hub school program is intended to give students in rural areas access to better quality learning environments, the effects cannot yet be observed in terms of student learning outcomes. In fact, average performance in hub schools is worse than in other schools. This likely reflects the mixed fidelity of implementation of the program, along with the fact that the program is still new so the learning impact has not materialized yet. Higher education represents the top objective and most common path for most young Ukrainians, but unequal access to high-quality curricular options and academic/career guidance in secondary 11 schools means many are not adequately prepared to enter higher education. About 60 percent of grade 9 students choose to continue their education in general secondary schools. Those who enter elite urban schools with specialized curriculum are more likely to secure top scores on the EIT. However, these schools use competitive admissions procedures determined at the school level, meaning that high performance on the EIT is partly the result of stringent admissions criteria at entry. 55 percent of these students choose to take the mathematics EIT, considered to be one of the most difficult EIT subjects, and it is required for admission to many in-demand fields in higher education. Students in urban regular (nonselective) schools are less likely to choose the mathematics EIT (45 percent), compared to only 40 percent in rural regular schools. Rural students are less likely than urban students to exceed the EIT cutoff thresholds and are considerably less likely to apply to and ultimately enroll in higher education. Whereas nearly 70 percent of urban students passed, applied, and ultimately enrolled, this figure drops to 40 percent for rural students.18 Furthermore, rural students are less likely to achieve the high levels of EIT performance required to access state-funded places in HEIs: only 17 percent of state-funded places for bachelor’s programs in 2018 went to students from rural areas (figure O.4). Figure O.4 Higher education admissions by student origin and EIT performance (2018) 70 175 Average EIT score (Ukrainian Admitted students (thousands) 170 60 165 50 160 155 language) 40 150 30 145 20 140 135 10 130 0 125 Urban Rural Urban Rural State-funded places Privately funded contract places Number Average EIT score Source: Authors’ analysis of EDEBO database. Ukraine spends a high share of public resources on education, but with a large network of institutions and a declining student-age population, those resources are not used efficiently Ukraine spends more of its GDP on education than most EU and OECD countries. After a steep devaluation of the hryvnia in 2013, public education spending declined by 35 percent in real terms over two years. Between 2013 and 2017, budget financing shrank from 7.2 to 6.0 percent of GDP. The decline brought Ukraine closer to international benchmarks in terms of the share of national wealth devoted to education, but spending remains high: with public spending on education at 6.0 percent of GDP and with private spending adding another percentage point of GDP, Ukraine’s education spending is among the highest in the world (figure O.5).19 This is driven in part by the law which requires the state to allocate at least 7 percent of GDP for education. However, Ukraine faces serious macroeconomic vulnerabilities and fiscal pressures, including significant debt repayments.20 This means that spending more to implement the education reform agenda and make needed investments in the sector is not an option: making better use of existing resources is an imperative. 12 Figure O.5 Public and Private Spending on Education, Percent of GDP (%) 8 7 6 5 Percent of GDP 4 3 2 1 0 Canada Israel Australia Denmark Belgium Finland Argentina Mexico Estonia Germany Colombia EU22 average Ireland OECD average Poland Korea France Italy Sweden Slovak Republic Turkey Hungary Ukraine United States Luxembourg United Kingdom Norway Netherlands Japan Czech Republic Indonesia Iceland Chile Lithuania Portugal Austria Latvia New Zealand Slovenia Spain Public expenditure on education institutions Private expenditure on education institutions Source: World Bank (2018c). Note: Ukraine data from 2017; OECD member and partner country data from 2014. Spending on education covers preprimary through tertiary, including expenditures not allocated by level. Data covers the majority of OECD member and partner countries as well as EU countries. To meet a growing demand, the tertiary education system expanded significantly in the 2000s, but the subsequent decline in the student population has left the sector oversized and inefficient. Higher education coverage of the population increased from around 45 percent in 1992 to over 80 percent today, according to data from SSSU. This significant increase has been driven in part by a large expansion of the sector. However, the student population has been shrinking in past years: over the same period, the youth population aged 0-17 declined by around 40 percent. While the private higher education sector shrank by almost 4 times between 2007 and 2017, the public sector has shrunk by only 1.6 times. Now, with a total population of around 42.4 million, Ukraine’s network of 327 universities, academies and institutes—of which 231 are public—is quite large. This amounts to 7.7 HEIs per 1 million population, or 5.4 public HEIs per 1 million. Even after considering population size, Ukraine appears to have a relatively large public higher education sector compared to other countries in Europe. Furthermore, many HEIs are relatively small, especially considering specialized institutions (for example, academies and institutes) and branches of universities, which can hinder the quality and conditions of learning. Although the number of HEIs has decreased over time, along with the number of students, the number of university faculty members has decreased more slowly, leading to low student-faculty ratios relative to neighboring countries (figure O.6). Figure O.6 Inefficiencies in higher education Density of HEIs relative to population (2017) and student-faculty ratio in tertiary education (2012–2017) 12.0 Romania 24 HEIs per 1 million population 10.0 22 Student-faculty ratio 8.0 (headcount basis) 6.0 Poland 20 Finland 4.0 UK 18 2.0 Moldova Hungary 16 - Sweden 14 Belarus Ukraine 12 All HEIs Public HEIs 10 2012 2013 2014 2015 2016 2017 Source: Authors’ analysis of SSSU and UIS data. 13 The same demographic and fiscal pressures are affecting general secondary schools, though the situation is more complicated due to the role of local governments in managing their school networks. The need to adapt a general secondary network to the demographic reality has long been recognized in Ukraine. While some areas have seen modest increases in birth rates, and urban student populations have increased as a result of rural-to-urban migration, the student-age population continues to decline in most of the country. This has led to low student-teacher ratios, particularly in grades 5-11 when students switch to subject teaching. The average student-teacher ratio in lower and upper secondary education in Ukraine is 10.9, compared to the OECD average of 13.1.21 Small schools and small classes constitute a fiscal strain on the budget as well as a deficient learning environment that detracts from students’ opportunities to learn and work together with a diverse peer group. However, local governments have responsibility for their school networks, and years of confused responsibilities and poor sectoral and budget management have perpetuated inefficiencies in the school network.22 2. What does Ukraine need to do to strengthen its education reform agenda? Ukraine is a middle-income country with significant potential for growth. Ukraine has historically benefited from its human capital, particularly its robust education system and highly skilled labor force. However, Ukraine has not translated this human capital into productivity and national wealth. The rapid expansion of the education system has led to high levels of educational attainment, but quality and relevance of learning have deteriorated while lack of innovation, isolation, and corruption have led to dissatisfaction, skills mismatches, and credentialism. Despite having a highly educated labor force, human capital represents only 34 percent of total wealth in Ukraine (compared to the ECA average of 62 percent), and labor productivity is only 22 percent of that in the European Union (Figure O.7). This suggests that education is not contributing its full potential to the wider economy. Figure O.7 Despite high levels of education, Ukraine is not tapping its full human capital potential 120% 100% 80% 60% 40% 20% 0% Labor productivity as share of European Union average (2017) Human capital wealth as share of total per capita national wealth (2014) Source: World Bank World Development Indicators and Wealth Accounting databases; World Bank (2018a). Demographic decline and rapid technological change mean that education must constantly evolve and adapt. The increasing role of technology in economic activities and everyday life has already led to significant changes in the demand for skills, with a greater need for advanced skills in all types of work. 23 However, upgrading cognitive skills alone is not sufficient: ‘soft’ skills are increasingly important given that interpersonal relations between humans cannot (yet) be replaced by the intervention of technology. 14 Adaptability requires a strong and balanced toolkit of skills, which means that the dividing lines between academic and technical disciplines will need to change over time. This leads to several conclusions: Ukraine cannot afford to take its human capital endowment for granted, and the cost of underperformance is rising. The workers of tomorrow—including the teachers, nurses and doctors that develop and safeguard human capital—are the product of today’s education system. The education reforms introduced since 2014 represent a major departure from the past and are in line with many good practices in high-performing education systems in Europe and elsewhere. They have generated great optimism by decentralizing and democratizing the education system. However, the large-scale transformations remain nascent. Reforms in such an environment inevitably produce imbalances between the objectives of reform—the movement towards a new education paradigm—and the inertia of history. Therefore, how can Ukraine strengthen its reform agenda moving forward? This review has identified five priority areas for reform going forward in order to address systemic imbalances which undermine Ukraine’s education reform agenda. These imbalances, which are interconnected, pertain to vision, governance, financing, incentives, and information. To rebalance the system, this review proposes reforms in these five key areas in order to address the identified imbalances, strengthen the reform agenda, and steer the system towards longer term competitiveness, innovation, productivity growth, and sustainability. The sections below describe the five imbalances, as well as what Ukraine needs to do to address them. Key priority actions for the short-term and medium-term are also summarized in Table O.1 at the end of the Overview. Figure O.8 Priority areas to ensure that reforms promote effectiveness, equity, and efficiency Source: Authors’ elaboration. 15 A. Extend the NUS vision for competency-based and student-centered learning across the sector, particularly in higher education The vision for reform in general secondary education—articulated in the New Ukrainian School—is a positive step in the right direction. This vision is rooted in a clear argument for change: today’s Ukrainian school should better equip pupils with the skills needed to learn throughout life, think critically, set and achieve goals, work in teams, and communicate in a multicultural environment. The attempts to modernize Ukraine’s general secondary school system to address these fundamental challenges are ambitious and badly needed. The Law on Education and the New Ukrainian School will prepare students for the 21st century through a combination of several elements: new educational content, more motivated teachers, greater decentralization and autonomy, child-centered approaches to teaching, a new schooling structure with 12 years of secondary education, fair allocation of public funds, and contemporary educational environments.24 Going forward, it will be important for Ukraine to accelerate the expected reform of upper secondary education to introduce the new three-year curriculum structure and streamline educational pathways from secondary into tertiary education. The current structure of upper secondary education in Ukraine is fragmented, with upper secondary general education offered in general secondary schools as well as vocational schools, colleges, and technical colleges. The movement toward a student-centered and competency-based approach to learning in line with the NUS vision will be challenging at the upper secondary level without advancing reform to concentrate resources —schools, teachers, funding, and instructional time—on what matters most. This reform will increase the duration of upper secondary education from 2 to 3 years, creating more time for students to gain exposure to the curriculum at a critical transition point in their educational trajectories, while also affording more choice, similar to the upper secondary school reforms that took place in Finland in the 1980s and Poland in the 1990s.25 The reform would also concentrate resources in a smaller network of specialized institutions, allowing schools to offer a higher quality of education with more elective subjects for students while also using fiscal resources and educational facilities more efficiently. The Law on Education requires grade 12 to be introduced by 2027 but accelerating the reform would ensure the transition to three-year upper secondary education sooner. This reform also presents an opportunity to accelerate optimization of the school network in upper secondary education, as underutilized schools and colleges could be merged or closed by local authorities. In higher education, the challenge is greater and increasingly urgent. There is no clear vision that links higher education to the positive developments for reform in secondary education or to the skills requirements of the labor market. The Law on Higher Education was the first large systematic reform measure adopted by Ukraine’s parliament in the immediate aftermath of the Euromaidan Revolution. It represented a compromise on the part of many different interest groups, coming on top of years of absence of a clear development strategy for higher education: a policy of nonpolicy.26 However, it did not address this challenge. To this day, there is no clear or coherent vision for the development of the higher education sector or individual universities. This is a fundamental problem: higher education in Ukraine cannot serve the needs of the people and the economy without clear objectives and a strategy for how to achieve them. Although the 2014 Law on Higher Education made a major step toward dismantling the centralized structures of the past, it provided more autonomy without the attending accountability mechanisms or financial flows. Without a strategy for higher education development, individual HEIs use their autonomy to achieve individual goals rather than working to achieve a broader goal for the system and nation. Furthermore, many important decisions governing the sector need to be taken by the Council of Ministers, including any that would lead to changes in funding and because many government agencies oversee subordinate HEIs. This further complicates the lack of a strategic vision. 16 The vision for higher education in Ukraine needs to prioritize modern approaches to curricula, pedagogical teaching methods, and learning support systems in line with the NUS and labor market needs, while also transforming the system to promote diversity and sustainability. Successful modern mass higher education systems are characterized by a high level of institutional diversity in which individual institutions have different missions and profiles.27 Currently, the higher education system is expansive, with a large number of specialized HEIs that are relatively small in size and scope. This has caused the system to become oversized and inefficient, especially in relation to the shrinking student population. Despite spending a relatively high share of public resources, funds are spread thinly across many institutions and staff, contributing to the incoherence: although the system spends a lot, rectors and faculty believe that lack of funding is the key problem, limiting their ability to invest in modern curricula or equipment.28 This also contributes to institutional stratification. A strategic vision for higher education should reflect modern approaches to curriculum and pedagogical teaching methods, stronger linkages with employers and the labor market, and greater institutional diversity with larger and more comprehensive HEIs. It is also important to maintain a focus on Ukraine’s vision for equitable and inclusive education. High-performing education systems prioritize equity and inclusion of all learners, maintaining the vision that all students are capable of high achievement with the right level of support. Global evidence from international assessments suggests that strong performance for the system as a whole is dependent on the need to deliver for every child. Top-performing systems show a low correlation between learning outcomes and the home background of the individual student, meaning that these systems have produced mechanisms and approaches to ensure that schools can compensate for the disadvantages that result from the student’s home environment.29 Maintaining a focus on equity and inclusion should be a key aspect of Ukraine’s vision for education going forward. In particular, priority should be given to (a) expanding access to quality pre-primary education with a focus on vulnerable groups, and (b) strengthening the capacity of Inclusive Education Resource Centers to support the transition to inclusive education within a decentralized context. B. Strengthen institutional capacity and governance structures On one hand, reforms have greatly expanded the autonomy of HEIs, local governments, schools, and teachers. This reflects a major departure from the centralized direction of the system in the past. However, the capacity of decentralized institutions and governance structures which are designed to ensure quality and promote accountability remain weak. The newly established State Service for Education Quality (SSEQ) is charged with developing a quality assurance system for secondary education, including audits of education institutions, supervision and monitoring compliance with requirements of the Law on Education, and monitoring education quality. Unlike the previous form of inspection which focused on detecting violations through a rigid structure, the new SSEQ is expected to support education institutions and local authorities to improve outcomes. This shift moves Ukraine closer to the norm in other European countries with external school evaluations, and this is a positive step given the research showing that school evaluations and support for school improvement can have positive effects at the school and system levels.30 However, this institution and its functions are new, and capacity at the central and regional levels remain weak. The capacity of local authorities to manage their school networks, particularly in rural areas, remains weak as well. However, there are some good practice examples of effective and collaborative approaches in newly amalgamated hromadas which could be further developed to strengthen capacity. Many HEIs are similarly ill-equipped to use their greater degree of autonomy to improve the quality of teaching and learning. Managerial capacity within HEIs for internal quality assurance and institutional self-improvement is limited in many cases, and the limited degree of financial autonomy afforded to HEIs constrains capacity even further. Ukraine is moving towards a system of accreditation and quality assurance more in line with the Standards and Guidelines for Quality Assurance in the European Higher Education Area, and the newly established National Agency for Quality Assurance in Higher Education (NAQAHE) is leading the effort for institutional and program accreditation, along with university ranking mechanisms, 17 regulation of doctoral degrees, and support for internal quality assurance systems. However, accreditation procedures are only now being developed, and there is a substantial shortage of trained experts.31 Evidence on effective education service delivery shows that autonomy needs to be paired with accountability and capacity to operate in a decentralized approach. As such, Ukraine should prioritize three areas of support: (a) capacity for decentralized management and education service delivery; (b) internal governance capacity within HEIs; and (c) mechanisms for accreditation and quality assurance. (a) Strengthen capacity for decentralized management and delivery of education at regional, local, and school levels A key area for accelerating education reforms in secondary education is the development of managerial capacity for decentralized service delivery. Reforms have provided local authorities and schools with more autonomy in how they use their budgets and organize curricula, but they need to have proper levels of capacity in order to manage this autonomy. This is a long-term objective, but the Government can support this through the State Service of Education Quality (SSEQ) and through the dissemination of information and management tools needed to build managerial capacity. For example, in Brazil, the structured planning and management decision-making process known as the “management circuit” introduced through the Jovem de Futuro program prompted positive changes on various managerial practices as well as learning outcomes.32 Improving communication, outreach and stakeholder engagement is also key to building this capacity and generating support at the local level. Ukraine can also build on good practices from across the country. For example, the Swedish project supporting decentralization in Ukraine has developed and implemented an interactive tool to help communities prepare for the school network optimization process, as well as a database33 of good practices from newly amalgamated communities. Both these initiatives could be developed further and institutionalized in the system. (b) Strengthen internal governance capacity in HEIs As in secondary schools, HEIs also face challenges in managing the increased levels of autonomy provided to them. Many HEIs have demonstrated limited capacity for democratic decision making and strategic planning and management aimed at improving quality of teaching and learning.34 The decision from 2014 to allow HEIs to elect rectors without external interference did protect them from external political influence, though it also led to a consolidation of the status quo. Proper training and support systems should be developed to support internal transformation and build the managerial capacities needed in HEIs. One option is to strengthen the capacity and authority of the university governing boards, which is a common feature in many European higher education systems. 35 These supervisory boards should include representatives from society and the economy, such as internationally acclaimed academics, business, civil society, and government representatives. In 20 of 28 European higher education systems investigated by the European University Association (EUA) in 2010, institutions were required to include external stakeholders in the internal governance of their institution at the central level. These boards should have the reputation and authority to carry out strategic management of the HEI, coordinate efforts to achieve designated goals, and provide independent assessment of the progress of the HEI. (c) Strengthen systems for accreditation and quality assurance There is an urgent need to strengthen internal and external quality assurance (QA) functions in the higher education system, including through improved capacity of the newly established National Agency for Quality Assurance in Higher Education (NAQAHE), as well as in individual HEIs. In particular, there is a need to accelerate QA reforms in line with the Standards and Guidelines for Quality Assurance in European 18 Higher Education Area (ESG). In terms of accreditation, Ukraine could also explore the potential for partial or conditional accreditation. Currently, there are two potential outcomes of accreditation: full accreditation or failure to become accredited. Without accreditation, students will not receive a recognized diploma, which creates strong pressure on universities and the accreditation committee to accredit the program. Given the high-stakes nature of the decision, this leads to rampant corruption and pressures on all levels of the decision-making process. Partial or conditional accreditation, in which the HEI is given a period of time to address relevant issues, is a possible alternative which also supports improvement-oriented feedback and a more phased approach. Additionally, statistical information on accreditation should be collected and published, so the general public and prospective students can see the percentage of programs that were not fully accredited. C. Target resource flows to promote performance and sustainability Given macroeconomic vulnerabilities and fiscal pressures, and the fact that Ukraine already spends a high share of its GDP on education, there is a need to ensure that resource flows are made strategically to create the right incentives within the education system to achieve sustainable results. However, there is currently a disconnect—particularly in higher education—between resource flows which prioritize the status quo and the need to achieve strategic objectives in the sector. Ukraine’s input-based model of public funding for higher education has created perverse incentives in recent years for HEIs to lower admissions and quality standards, while maximizing the number of fee-paying students to compensate for declining public funds. Public funding is allocated through a system of quotas determined for each study field and level of education by the Ministry of Economic Development and Trade, similar to the system inherited from the USSR. Given the decline in student numbers over time, HEIs have become more dependent on public funds, creating at times perverse incentives for HEIs to lower entrance requirements to admit more and more students, including those with lower levels of preparation for higher education (on both state-funded and privately funded places). Today, 55–60 percent of current higher education students are paying fees (figure O.9). This is particularly the case in lower-cost fields such as social sciences and humanities. About half of those fee-paying students are enrolled in distance and evening programs, in which the quality and value of the education may be weaker. This dual-track funding system disadvantages poorer students and rural students who are less likely to achieve the high EIT scores needed to be admitted to state-funded places. Figure O.9 Enrolled students, by program, type of funding and form of study (2018) Specialist Master Bachelor Junior Specialist 0 100 200 300 400 500 600 700 800 Number of enrolled students (thousands) State-funded, intramural State-funded, extramural and evening Self-funded, intramural Self-funded, extramural and evening Source: Authors’ analysis of EDEBO database. 19 Despite several recent positive changes to improve transparency and competitiveness of funding, the public funding model for higher education remains unsustainable. Several mechanisms have been put in place in recent years to improve transparency in distribution of state-funded places and standardize EIT admissions requirements across HEIs offering the same field of study. However, these changes do not address fundamental concerns with the public funding model for higher education in Ukraine. First, despite the greater degree of autonomy afforded to HEIs, financial autonomy still remains quite limited. The amount of budget funding for higher education and the number of state-funded places are determined independently in practice, leading to persistent funding disparities even for the same study field. For example, with medicine, one state-funded place in 2016 afforded the average university under MOES 12 percent more funding than to a university under the Ministry of Health, despite having the same curricula. Without information on quality and relevance of education, higher education funding remains focused on quantity of admitted students rather than learning or graduates’ employment outcomes. Furthermore, the new mechanism for allocating budget seats at the bachel or’s level means that public funding depends on the number of applications. This creates a challenge for HEIs, which cannot adequately plan for the number of faculty they will need. This differs from good practice in higher education financing in other countries such as the Netherlands and Latvia, where differences in actual cost delivery are prioritized as a principle for ensuring financial sustainability of a given university. In secondary education, recent reforms to the financing formula provide strong economic incentives to improve efficiency, along with quality and equity, but the formula should be carefully monitored and adjusted. School funding according to a transparent formula that includes at least a component based on a per-student amount is considered good practice in the field, even though systems employ a wide range of different criteria in their funding formulas.36 Ukraine has had a per-student formula since 2014, when it was introduced along with a large budget decentralization reform, but it largely represented a de facto continuation of the previous financing system. However, in 2018, the formula included a crucial change by providing a hard budget constraint for local governments in which average class sizes were smaller than the norm in the formula. This has created a wedge between the actual and desired school network, in which local governments will be in a state of surplus or deficit vis-à-vis the subvention formula. At the same time, local governments have flexibility to reallocate resources across budget years and redeploy savings, for example, to purchase learning materials or provide preschool services. The possibility for an annual review represents a good opportunity to develop indicators on network efficiency and systematically revise the formula as needed. Three areas should be prioritized to improve targeting of resources: (a) reform of the public funding model in higher education, (b) financial incentives for efficiency, and (c) alignment of the secondary school financing formula and hub school program with the need to consolidate the school network. (a) Reform the public funding model for higher education to promote competition, performance and excellence and to consolidate resources Ukraine has an urgent need to move away from its input-based method of public funding for higher education, which has created strong incentives to lower quality standards while also contributing to funding disparities. At the same time, there is evidence that the network of HEIs is oversized relative to a shrinking student-age population. With the input-based method of funding, which does not rely on actual cost of delivery, many HEIs are in the position of having insufficient resources to deliver quality education that is relevant to the needs of the labor market. Furthermore, the current system does not incentivize excellence at the level of programs, faculties or institutions. Therefore, there is a need for systemic reform to the funding model to ensure transparency and efficiency, while also introducing a more differentiated approach based on indicators of quality. Special financial incentives may also help to encourage the development of centers of excellence within universities and stimulate top-performing programs. Systemic reform would 20 help to prioritize critical objectives of enhancing quality and improving links with the labor market, while also supporting consolidation in the sector and optimization of HEI finances. The concept of strategic financing in higher education, including performance-based funding mechanisms, have been a topic of discussion in Ukraine for several years now. A funding formula has been developed, along with draft legislative acts, which would allocate funds based on the adjusted number of students. However, the formula and its legal bases have not yet been formally approved or implemented. Moving forward, it will be important to improve the information base on which to institute a new funding model. This would include more information on the actual estimated cost of service delivery in different fields of study and formats, as well as more robust quality assurance information to mitigate the risk that formula- based funding exacerbates institutional stratification rather than rewards performance. 37 (b) Introduce additional incentives to reward or penalize HEIs on efficiency of resource use Ukraine could introduce additional mechanisms to incentivize the consolidation or merger of higher education programs and/or institutions. For example, the sector could establish an incentive program to create economies of scale and scope through voluntary strategic cooperation or mergers. A mix of top-down and bottom-up approaches may be suitable here, whereby the state provides incentives for consolidation, but the suggestions of where and what to consolidate are made by institutions, considering regional aspects and equity of access. For example, competitive funding could be provided as a top-down incentive to HEIs that have voluntary plans to merge, to build joint units or to collaborate to increase sector efficiency. A bottom-up development of models for collaboration and consolidation by HEIs would engender ownership on the part of HEIs and less political opposition.38 For example, Denmark presents a good practice example of comprehensive consolidation in higher education in which the government does not regulate which institutions should merge but supports the autonomy/ownership of HEIs and provides financial incentives to stimulate institutions’ participation in the process.39 At the same time, Ukraine could also consider additional financial penalties for HEIs that fail to improve efficiency of resource use and reduce waste, for example through the introduction of performance agreements. Unlike performance-based funding, performance agreements look at future performance, awarding institutions on the basis of expected performance rather than actual performance.40 Such agreements have been introduced in several European countries, including Croatia, Estonia, Finland, Germany, Latvia, and the Netherlands. They are individual agreements between an HEI and the funding authority, and the agreement usually includes a financial penalty or sanction if objectives are not achieved.41 (c) Monitor and adjust school financing formula, along with the hub school program, to incentivize optimization of school network and pedagogical workforce The education subvention formula for secondary education should be carefully monitored going forward and adjusted as needed to continue encouraging local governments to optimize their school network and consolidate resources. One option going forward is to gradually raise the desired class size goal specified in the formula, which is currently 13 students per class for rural areas. This would strengthen the economic incentive on the part of local governments to consolidate classes and schools. An additional consideration in the future is to create school size goals in the financing formula, which currently are not included. This adjustment of the formula needs to go hand-in-hand with a strategy for rationalizing the pedagogical workforce while making the teaching profession more attractive. The consolidation of classes and schools, leading to fewer but larger institutions with more efficient use of resources directly requires the rationalization of teachers and non-teaching staff in schools. This is particularly important to consider if teachers’ salaries continue to increase, either due to subsequent increases in the base pay or if more and 21 more teachers become certified and earn associated pay increases. Going forward, Ukraine needs to consolidate resources at the local level and raise student-teacher ratios while also rationalizing the workforce, perhaps through creating incentives for retired teachers to leave the workforce while introducing additional measures to improve the attractiveness of the profession. At the same time, the hub school program should be evaluated for implementation fidelity and revised accordingly. As mentioned, students in hub schools perform marginally worse on the EIT than other students, even though hub schools are supposed to provide more effective and efficient learning environments than comparator schools. Although this could be explained by the fact that the program is relatively new, the practical implementation of the program varies considerably, meaning that ‘hub school’ is just a title rather than a substantive indication of school conditions. The hub school criteria have changed over time as well, meaning that schools may meet different conditions for achieving hub school status. The hub school program, together with the education subvention formula, have great potential to improve efficiency of resource use along with equal access to quality learning environments in rural areas, but only if hub schools meet minimum quality conditions and represent a substantive change over their alternative. D. Align individual incentives and capabilities with learning goals The teaching load (Stavka) system for educators devalues the requirements of professional teachers and provides the wrong incentives for upgrading teachers’ capacity and the status of the profession. The teaching load system for organizing teaching and compensating teachers fragments their work into ‘piece-meal’ tasks, separating teaching hours from other important but non-teaching tasks. Because of this system, only 53-56 percent of take-home salary is from the base salary, while the remainder is tied to various top-ups. This system creates incentives that are misaligned with the expectations of teachers under the NUS, which requires a massive paradigm shift in terms of how teachers deliver instruction and interact with students. Voluntary teacher certification can be part of a strategy to upgrade teachers’ skills in line with the NUS vision, but it needs to be monitored carefully along with broader investments in teacher professional development opportunities. Substantial improvements are needed to upgrade the quality of training provided by in-service teacher training institutes while aligning both in-service and initial teacher education with the competency-based approaches to learning envisioned in the NUS. Furthermore, the professional status of teaching in Ukraine is relatively unappealing, further limiting incentives to enter the teaching profession. For example, the salary progression of a teacher’s career in Ukraine is relatively small, with those at the top of the scale making only 30 percent more than a new teacher; in OECD and EU countries, teachers at the top of the scale make 70–80 percent more than new teachers (figure O.10).42 Although teachers’ salaries were recently increased in an effort to improve the social status of the profession, they are lower than the salaries of other tertiary educated workers in Ukraine. Furthermore, the large share of top-ups reduces the transparency of the overall remuneration package for teachers. These conditions deter many bright students from considering teaching as a profession, unlike in top-performing education systems which consistently attract high-performing students into teacher preparation programs and the teaching profession.43 Students entering teacher preparation programs in Ukrainian pedagogical universities tend to have relatively lower scores on the EIT than those entering many other fields including sciences, health and welfare, and social sciences, and evidence suggests only a share of those entering teacher preparation programs will go on to become teachers. 22 Figure O.10 Teachers' salary progression: Ukraine vs. comparator countries 2.5 Ratio between starting and maximum base salary 2 1.5 1 0.5 0 Source: World Bank (2018c); OECD (2018a). There is a need to improve incentives for managing the overall teacher workforce. The teacher workforce in Ukraine is large and aging, with 25 percent over the age of 55 and 15 percent working while in retirement. While the teacher workforce has declined over time, it remains misaligned with the declining student-age population, leading to low student-teacher ratios and an inefficient use of funds. The hard budget constraint introduced in the education subvention formula will help local authorities to manage the size of the teacher workforce, but in general Ukraine needs to pursue a comprehensive approach that improves fiscal sustainability and raises the professional status and capabilities of the teaching profession. Both students and educators, particularly in higher education, have incentives to engage in corruption and academic misconduct which detracts from quality and diminishes the value of educational credentials. Estimates indicate that at least 25-30 percent of students have directly engaged in academic misconduct or bribery, with a larger share exposed to and familiar with such practices.44 Students face incentives to engage in academic misconduct, namely to compensate for an intense testing schedule (often in mandatory courses unrelated to a student’s specialization), living conditions in dormitories, obligations to work at least part-time while studying, and inadequate preparation for study at the higher education level.45 At the same time, declining public funding for universities has stretched the availability of resources, and salaries for faculty members are low relative to what they could get working in other fields, which further contributes to the corruption and integrity risks.46 There is also evidence in Ukraine that experience with bribery in HEIs is correlated with less valued diplomas on the part of employers, lower perceptions of institutional prestige, and lower perceptions that university study will help achieve a corresponding salary.47 This is particularly concerning given the global evidence showing that corruption in higher education normalizes corrupt practices among young people and increases social inequality.48 Moving forward, Ukraine needs to tackle priority areas to improve incentives: reform the career path for educators in schools and higher education institutions, harmonize the EIT to the NUS and vision for higher education, and introduce stronger tools to incentivize academic integrity and anti-corruption. (a) Reform teacher career path, including the teaching load (stavka) system for organizing and compensating teachers’ work and opportunities for professional development High-performing education systems around the world make teaching an attractive profession by improving its status, compensation policies and career progression structure, while also making good use of teachers’ time with students.49 However, the stavka system for organizing and compensating teachers’ work is poorly aligned with these objectives, as well as the new expectations of teachers under the New Ukrainian School. By fragmenting teachers’ work into teaching and non-teaching tasks, the result is that nearly half of 23 teachers’ take-home pay comes from various top-ups. This creates incentives for abuse and nontransparent allocation of teaching hours, while also devaluing the work of professional teachers. This stavka system is in contrast to the weekly workload system used in many OECD countries, where the income of all employees, including that of teachers, is based on 36–40 hours of work per week, of which 22–29 hours are allocated for teaching.50 Given that the New Ukrainian School reform expects teachers to tackle increasingly complex tasks associated with the new curriculum, this workload system of organizing and compensating teachers’ work may ultimately create disincentives for teachers and undermine reform. Teacher certification can play a significant role in supporting the NUS reform, but it must be part of a larger coordinated set of reforms aimed at making the teaching profession more attractive while also rationalizing the teacher workforce. High-performing education systems around the world attend to multiple teacher policy goals in a coherent manner, in order to ensure that policies like teacher certification are aligned with other aspects of teacher training, recruitment, and management. To function most successfully, teacher certification may need to be part of a comprehensive set of policies to reform the teacher career path and workforce that will include: (i) transparent remuneration packages for teachers; (ii) reform of workload system of organizing teachers’ work; (iii) dynamic opportunities for professional development, including coaching and mentoring opportunities for teachers to practice and demonstrate new teaching methods; (iv) rationalization of the teacher workforce through voluntary and involuntary attrition; and (v) optimization of schools and classes which drives the demand for teachers. (b) Reform career path for academic teaching staff in higher education institutions, including the teaching load system As with secondary schools, the teaching load (stavka) system is used to organize and compensate the work of teaching faculty and lecturers in HEIs. This system fragments their work and creates incentives for faculty members to collect as many teaching hours as possible, which may ultimately hinder their effectiveness as educators. The career path for academic teaching staff in HEIs needs to be reviewed as revised, putting remuneration within a wider framework of incentives. For example, more holistic approaches combine monetary and non-monetary rewards, including compensation, benefits, and opportunities for personal development.51 A more strategic approach is needed to improve the attractiveness of the academic profession and strengthen human resource management in HEIs. (c) Harmonize University Admissions Exam (EIT) to NUS and vision for higher education Given that the EIT plays such a crucial role in establishing quality of and regulating access to higher education, it is important that Ukraine continue to invest in and modernize the EIT to ensure it remains a state-of-the-art tool that is fit for purpose. On one hand, the EIT is used as a summative evaluation of secondary education (the state attestation exam) for all graduates to ensure that they pass a minimum competency threshold. Going forward, this will mean that the EIT content and test items will need to be adapted to reflect the new competency-based approach to learning that is envisioned in the New Ukrainian School curriculum. On the other hand, the EIT maintains its original purpose of providing transparency in regulating access to higher education for a subset of secondary school graduates who intend to continue their education in universities. The recognition that many students entering higher education today are not prepared for advanced studies has led some universities and programs to set minimum entrance thresholds on the EIT subtests for applicants. Given this trend, there is a clear argument to be made for reviewing the content of the EIT subtests, as well as their psychometric design, to ensure alignment with global best practices. Finally, the regular administration of EIT presents an opportunity to collect more information on students’ backgrounds, educational objectives, and pathways, so the incorporation of more survey questions could help to shed light on this and also track changes over time. 24 In addition, the introduction of more external exams for admissions to master’s degree programs could help to improve transparency while also controlling access to this level of education. The recent introduction of external exams have helped to control access for high-demand programs such as medicine and law, but this could be expanded to other regulated professions as well. (d) Strengthen incentives for academic integrity and anti-corruption and implement tools for oversight Strengthening the capacity of the MoES and the newly established National Agency for Quality Assurance in Higher Education (NAQAHE) to address issues of academic integrity is essential. However, Ukraine can seek to address this challenge on multiple fronts simultaneously. One important institutional reform that can help to strengthen incentives is to operationalize the Office of Education Ombudsman, as anticipated by the Law on Education.52 This office should create an institutional grievance redress mechanism for addressing instances of corruption or integrity violations, both in higher education and other elements of the education sector. The ombudsman position has recently been filled and the office is now in the process of being staffed. This could also be supplemented by support to establish ombudsman offices in HEIs, as well as a review and introduction of stronger legislation to penalize violations of academic integrity, including a retrospective check for plagiarism for persons who apply for high-level positions in HEIs. Additionally, Ukraine should seek to implement tools for combating plagiarism and academic misconduct in higher education, such as the National Repository of Academic Texts. This repository was designed to serve as a universal database of all academic work published in Ukraine. Having such a database would greatly facilitate the detection of plagiarism in students’ papers, theses, and dissertations. However, there has been little progress in this area since the resolution was passed by the Cabinet of Ministers in 2016. Ukraine could also disseminate information on academic integrity in universities based on student surveys. Such tools could identify high instances of integrity violations, while also encouraging and disseminating good practices among other HEIs. For example, in Romania, a coalition of education stakeholders developed detailed questionnaires which assessed university governance in several areas, including transparency and responsiveness, academic integrity, enforcement of rules, governance quality, and financial management. The exercise resulted in an immediate improvement in university transparency in procurement and recruitment, along with some significant improvements in awareness about integrity violations in universities at a high level.53 There is also a need to work directly with HEI students, faculty and administrators to expand awareness of academic integrity principles and raise awareness at higher levels. This is aligned with research on corruption which advises against fighting corruption in general, and instead focusing on specific malpractices.54 Ukraine can build on existing data sources and successful projects and programs, such as Profrights.org, a database containing information on violations of the rights of teachers and students in HEIs, as well as the Strengthening Academic Integrity in Ukraine Project (SAIUP). E. Provide effective feedback and information on systemic results The final imbalance relates to the availability of effective feedback and information to students, teachers, employers, and the system. As mentioned above, autonomy in education only works if also paired with accountability. However, through the provision of effective feedback and information, assessment is also required to hold actors accountable for improving service delivery and achieving results. 25 Assessment capacity to monitor learning should be improved, along with the EIT to better regulate access to higher education. Ukraine has initiated a sample-based external assessment of learning outcomes in grade 4, and this represents a positive development in the sector, since external assessment information on student learning is crucial to hold schools and the education system accountable. This brings Ukraine in line with most OECD countries, which have some form of a summative assessment at the primary level. However, Ukraine would also benefit from an external assessment in grade 9, before students transition into upper secondary school. The university admissions exam (EIT) has brought more transparency and trust to the admissions process, and it is now being used in place of the school leaving exam. However, it is not clear that the EIT in its current form is well designed to fulfill both functions of reform: (a) to assess that secondary school graduates have attained a minimum acceptable level of knowledge, and (b) to regulate the quality of higher education through a high-stakes examination. Furthermore, the EIT does not yet reflect or measure the competency-based approach to learning envisioned in the NUS. Limited access to academic and career guidance counseling in secondary schools complicates the choice of study field and institution. Ukraine does not have any national or large-scale programs on guidance counseling in secondary schools. While there are various nongovernmental organizations that aim to support students and their families in this area, the scale is limited. Youth who did not have access to academic and career guidance are more likely to randomly select study fields and occupations, according to the ILO School-to-Work Transition Survey. Currently, there is also very limited public information allowing young people to compare programs, fields of study, university rankings, or information on graduates’ employment. Although the government has identified professional counseling as a priority in its 2019 Priority Action Plan, the focus remains limited to vocational education, despite the fact that most students pursue general secondary education instead. Without information on the quality and relevance of individual degree programs or the skills of individual graduates, employers and education institutions become locked in a “credentialist equilibrium.” The supply and demand for educational credentials remains high in Ukraine. There is a strong preference for tertiary education as the means to enter and succeed in the competitive labor market, but there is also evidence that employers may over-emphasize educational credentials. 40 percent of young university graduates were working in jobs which did not require university-level education, as of 2013, compared to 29 percent for prime-age and older workers.55 A comparison of the structure of job vacancies and the required educational profiles seems to support this hypothesis.56 While employers highly demand57 cognitive, socioemotional and technical skills among new hires, more so than any education level, employers still preference ever higher credentials for at least two reasons: (i) they find that credentials do not accurately signal workers’ underlying skills, and (ii) employers do not know how to value or compare some credentials given changes in the structure of the education system over time.58 For example, there is evidence that employers have a fairly low demand for young bachelor’s degree holders because this is still perceived by many local employers as “incomplete” tertiary education compared to the Master or Specialist degrees.59 Without improving the flow of information between education institutions and employers on the supply and demand for skills, the credentialist equilibrium and associated education-labor market mismatches will persist. Moving forward, Ukraine needs to prioritize 3 areas: strengthening student assessment systems in basic education, developing a program for counseling and guidance services, and improving data collection systems and performance monitoring in higher education. (a) Strengthen systems for student assessment in basic education and disseminate results High-performing education systems in the EU and OECD have summative assessments of student learning. Twenty-nine education systems had such assessments at the primary education level, and 27 had them at the lower secondary education level.60 Ukraine has rolled out a grade 4 monitoring survey which is an 26 important step, but other reforms are needed to improve the student assessment system as well to track and improve learning outcomes in general secondary education. In particular, there is a need to introduce an external summative sample-based assessment at grade 9 to monitor the development of key NUS competencies and student readiness to progress into specialized upper secondary education. The system would also benefit from a more systematic approach to measuring quality in preschool education and student readiness for primary school. This could involve the integration of standardized assessments 61 of preschool quality into the quality assurance process managed by the SSEQ. UCEQA has developed a Strategy for Learning Assessments in General Secondary Education until 2030, which represents a strong step towards expanding and improving the student assessment system in Ukraine in line with the learning objectives of the NUS. Moving forward, this Strategy should be further developed and implemented. (b) Develop national program for counseling and guidance services as key pillar of upper secondary reform Educational and career guidance counseling plays an important role in motivating students and keeping them engaged in education by providing information on study options and work prospects and identifying careers that may interest them. Guidance staff also support young people in developing the skills they need to make smart decisions and take responsibility for personal growth and professional development. In many European countries, academic and career guidance is explicitly stated as a measure to facilitate the transition through secondary education and combat early school leaving. For example, some systems like Finland, Sweden, Norway, Spain and Italy specify that providing guidance, supporting students in their decision making, and preparing them to cope with real-world challenges are among the main tasks of all school staff, while in the United Kingdom, schools have a statutory duty to provide access to career advice.62 In Ukraine, students have little information on the labor market relevance of different higher education pathways, including labor market outcomes for graduates of vocational and higher education and different fields of study across higher education institutions. As a result, students end up making momentous educational and occupational choices based on anecdotal information from their peers and families, or based on random selection. A national program for counselling and career/educational guidance services at the upper secondary education level would be instrumental in helping to address this gap in the system. It could include more integrated site visits between schools and local employers, outreach efforts between schools and HEIs, and provision of current and relevant data on labor market outcomes. This should also be paired with transparent and accessible information about HEIs and possible outcomes based on graduate tracer studies, surveys of current students, and other similar sources. (c) Improve systems for data collection and monitoring performance of HEIs and higher education system, including through HEMIS, student surveys, and graduate tracer studies Current mechanisms for collecting data on HEIs and the wider HEI system as a whole should be strengthened in order to better inform policies. Although the EDEBO database contains a considerable amount of information, it is poorly suited for policy analysis purposes. Furthermore, there is a lack of basic data on a number of aspects, such as the number of personnel in HEIs and tuition fees across programs and HEIs. This could be linked to the EIT database and improved into a more effective higher education management information system (HEMIS). The establishment of a student experience and engagement survey could also improve quality assurance processes, accreditation, and monitoring. According to the European University Association (EUA), student experience and engagement surveys are the most common way for institutions to introduce quality assurance processes.63 There are several examples of such surveys that could inform Ukraine’s reform, such as the National Student Survey (NSS) in the United Kingdom, which is used for external quality assurance and is obligatory for publicly funded universities in the UK. Additionally, surveys such as the North 27 American National Survey of Student Engagement (NSSE) and the Student Experience in the Research University (SERU) survey are voluntary and used for institutional self-improvement and internal quality assurance efforts. Allowing Ukrainian universities to ask university-specific questions on such a survey may be an option to increase the response rate and buy-in. Other metrics of university performance and graduates’ employability would be helpful to moni tor quality. This could include the introduction of university rankings, graduate tracer studies, and other such mechanisms. One possible model is Poland’s Graduate Tracking System, which relies on data submitted by HEIs as required by the law, and is managed by the same agency that manages Poland’s HEMIS.64 *** Ukraine has embarked on an ambitious reform agenda with great potential to transform and reposition the education system as a driver for economic growth and social prosperity. However, without attending to the imbalances identified in this report, reforms may simply “tinker” with the status quo, rather than achieving the broad-based changes that Ukrainians expect and deserve. The experience of other high-performing education reformers has shown that reform can succeed if it is backed by political will, broadly supported through engagement with stakeholders, fiscally sustainable, and coherent within the education system. Addressing the imbalances and tensions that remain in the sector will help to secure these conditions, putting Ukraine’s education reform agenda on a path to success for the benefit of the next generation and the prosperity of the country. 28 Table O.1: Priorities for Action under Ukraine’s Education Reform Agenda Priority Areas Short-Term Medium-Term • Develop coherent and evidence- • Accelerate reform of upper secondary based sector strategy for higher education to introduce 3-year education, prioritizing labor curriculum structure and streamlined Vision market relevance, institutional educational pathways diversity, and sustainability • Maintain and expand support for inclusive education • Complete accreditation procedures • Build managerial and technical Institutional for higher education capacity for schools, local authorities, and HEIs on internal quality Governance assurance, planning, and resource Structures management • Reform public funding model for • Implement additional incentives to higher education to promote reward or penalize HEIs on efficiency strategic objectives and of resource use Resource consolidate resources Flows • Monitor and evaluate secondary school financing formula • Monitor and evaluate teacher • Restructure career path and Stavka certification program workload system for teachers and • Introduce more tools for detection faculty Incentives and oversight of academic • Modernize EIT to align with NUS and misconduct higher education strategic vision • Develop criteria for university • Develop counseling and guidance rankings services for secondary school students • Develop and implement graduate • Expand EMIS to other sub-sectors, tracer survey introduce capabilities for tracking Information individual students and staff, and link systems to broader investments in e- government and digital teaching and learning materials 29 Notes 1 World Bank (2019a). 2 World Bank (2018a). 3 World Bank (2019a). 4 World Bank (2019c). 5 KAS (2017). 6 Jorda and Alonso (2017). 7 Del Carpio and others (2017). 8 Kupets (2016). 9 World Bank (2017a). 10 Denisova-Shmidt and Prytula (2017); OECD (2017a); Osipian (2017, 2009). 11 Psacharopoulos and Patrinos (2018). 12 OECD (2017a). 13 EBRD (2016). 14 Kupets (2016). 15 OECD (2018b); Barber and Mourshed (2007). 16 Based on authors’ analysis of SSSU data from 2017. 17 UCEQA (2018) 18 Based on data for 2018 university admissions campaign. 19 World Bank (2018c). 20 World Bank (2019d). 21 World Bank (2018c). 22 Herczynski (2017). 23 World Bank (2019c). 24 MOES (2017b). 25 OECD (2011). 26 KAS (2017). 27 World Bank (2016). 28 KAS (2017). 29 OECD (2018b); Barber and Mourshed (2007). 30 For example, see OECD (2013). Synergies for Better Learning: An International Perspective on Evaluation and Assessment. OECD Reviews of Evaluation and Assessment in Education. 31 Sovsun (2019). 32 Paes de Barros, et al. (2019). 33 http://wiki.sklinternational.org.ua/ 34 KAS (2017). 35 Arnhold, Kivisto, Puttmann, Vossensteyn, and Ziegele (2018). 36 OECD (2017c); World Bank (2012). 37 World Bank (2016). 38 Arnhold and others (2018). 39 Pruvot, Estermann, and Mason (2015). 40 De Boer and Jongbloed (2015). 41 Curaj, Deca, and Pricopie, eds. (2018). 42 OECD (2018a). 43 World Bank (2019b). 44 DIF (2015); Denisova-Schmidt, Prytula, and Rmuyantseva (2018). 45 Denisova-Schmidt, Prytula, and Rmuyantseva (2018). 46 Osipian (2007); Klein (2012). 47 Authors’ analysis of Developers of Ukraine survey of 2,938 current students and recent graduates of information technology (IT) programs, conducted in May 2018. 48 Altbach (2013); De Waal (2016); Hallak and Poisson (2007). 49 World Bank (2019b); OECD (2018b). 50 Steiner-Khamsi (2016). 51 Arnhold, Pekkola, Puttmann, and Sursock (2018). 30 52 MOES announcement on education ombudsman office: https://mon.gov.ua/ua/news/z-1-sichnya-2019-roku-v- ukrayini-zyavitsya-osvitnij-ombudsmen-sho-zahishatime-prava-uchniv-studentiv-osvityan-i-naukovciv-uryad- prijnyav-vidpovidnu-postanovu 53 Transparency International (2013). 54 Shekshnia and Denisova-Schmidt (2017); Denisova-Schmidt (2018). 55 Kupets (2016). 56 Del Carpio and others (2017). 57 Muller and Safir (2019). 58 Kupets (2016). 59 Nikolaiev (2017). 60 OECD (2017). 61 Such as those developed under the Measuring Early Learning Quality and Outcomes (MELQO) initiative. 62 European Commission (2014). 63 Loukolla and Zhang (2010). 64 For reference, click here for the link to Poland’s graduate tracking system: http://ela.nauka.gov.pl/en/ 31 Introduction The life cycle of skills development forms the foundation for the World Bank’s analytical framework for this review. Broadly speaking, skills refer to a dynamic and interactive set of abilities that are required to apply knowledge in practice. There is substantial global evidence showing how different types of skills are developed throughout the life cycle, starting from early childhood through adulthood. Skills are multidimensional, including cognitive, socioemotional and technical skills, and they build on each other as they are developed through the life cycle.1 Cognitive skills refer to the ability to understand complex ideas, adapt effectively, learn from experiences, and engage in different forms of reasoning. They can be broken down into foundational skills, including basic literacy, numeracy, critical thinking and problem-solving, and higher-order skills. Socioemotional skills are the behaviors, attitudes and values a person needs to navigate interpersonal and social situations. Finally, technical skills are acquired knowledge, expertise and interactions needed for competent performance of duties associated with a specific job.2 The education system contributes to skills development at different stages: • Getting children off to the right start by supporting early learning through preschool education is critical to develop a strong foundation for high-development trajectories. Learning in school depends on students who are prepared and motivated. Moreover, high-quality preschool education, particularly for disadvantaged children, is needed to achieve this. • Ensuring that all students acquire foundational skills through general secondary education is needed to have a productive and informed citizenry. Foundational cognitive and socioemotional skills are needed to develop more advanced technical and professional skills in higher levels of education. • Building job-relevant skills through vocational education and training (VET) is needed to ensure that youth can successfully transition into the labor market, that employers have access to a supply of skilled labor, and that workers have opportunities for retraining later in life. • Finally, developing advanced skills through higher education is critical for employment in technical and professional occupations and innovation and knowledge transfer throughout the economy. This review considers three analytical dimensions: effectiveness, equity, and efficiency. These dimensions cut across the education system, reflecting the fact that they have cumulative effects as learners proceed through the system (figure I.1). For example, unequal access to quality education at lower levels of the system have implications for equity of access at higher levels of the system. Figure I.1 Analytical framework Source: Authors’ elaboration 32 Although the review examined multiple levels of education, this flagship report devotes substantial analysis and discussion to general secondary education (Chapter 3) and higher education (Chapter 4), for several reasons. In the development of this analysis, a broad review was first conducted across education sub-sectors in order to assess key constraints, reform momentum, data availability, and other considerations. Based on this, it was decided to conduct “deep dive” analyses in gen eral secondary and higher education. There are several key reasons for this. First, these two crucial segments of the education system have been most affected and targeted by reform efforts over the last five years, particularly with the passage of the 2014 Law on Higher Education and the 2017 framework Law on Education, which led to the New Ukrainian School. Second, these segments contain the clear majority of students and staff in the system, as well as the majority of spending (amounting to about 70 percent of total education spending).3 Finally, these segments have the best available sources of administrative and assessment data which are aggregated into various databases described in more detail below. It is also important to note that other education development partners actively engaged in Ukraine have been conducting analyses in complementary sub-sectors, such as analysis by UNICEF on preschool education access and quality and by the European Training Foundation (ETF) on vocational education. This review has been prepared based on an extensive analysis of primary and secondary data sources, interviews, and focus groups. First and foremost, this review makes use of available data sources produced by the Government of Ukraine, specifically the State Statistics Service of Ukraine (SSSU), the Institute for Education Analytics (IEA), and the Ukrainian Center for Education Quality Assessment (UCEQA), as well as the Ministry of Education and Science (MOES) and the Ministry of Finance (MOF). Two databases containing administrative data on education informed the analysis: the State Information System of Education (DISO) and the Unified State Electronic Education Database (EDEBO). Additional data sources included the Scopus database of academic publications, as well as large-scale surveys, such as the Ukrainian Household Living Conditions Survey (HLCS), ILO School-to-Work Transition Survey, the EBRD Life in Transition Survey, the Trends in International Mathematics and Science Survey (TIMSS) and the Skills Toward Employment and Productivity (STEP) survey of households and employers. The review has also been informed by secondary research and in-depth discussions with key MOES counterparts and education development partner agencies. Interviews and focus groups were also conducted with local authorities and universities in Kyiv, Lviv, Kharkiv, and Zaporizhia oblasts. The report that follows is comprised of five chapters with analysis and recommendations and is intended for key education stakeholders in Ukraine. Chapter 1 describes the context for education reform in Ukraine, including the current structure of the education system, major trends influencing the reform agenda, and main thrusts of reform since the Euromaidan. Chapter 2 assesses the latest available information on education and labor market outcomes in Ukraine, and the implications for the education system. Chapter 3 looks in detail at five important aspects for promoting learning under the New Ukrainian School reforms in general secondary education, namely readiness to learn, teachers’ incentives, spending and school network optimization, learning assessment and monitoring, and equal opportunities to prepare for higher education. Chapter 4 looks in detail at several aspects for strengthening the sustainability and transparency of higher education, including the higher education network, financing, access and admissions, governance, quality, relevance, and integrity. Finally, Chapter 5 discusses directions for the future of education in Ukraine, including the need to articulate a national vision for higher education and address key imbalances in the sector related to governance structures, resource flows, incentives, and feedback. The main audience for this report is education stakeholders, policymakers, and technical staff that are working throughout Ukraine’s education system. However, the report is also meant for the wider public, mainly those interested in a diagnosis of key sector challenges and in directions for strengthening Ukraine’s reform agenda in education. 33 Notes 1 World Bank (2018d). 2 World Bank (2018d). 3 World Bank (2018c). 34 Chapter 1: The Context for Education Reform in Ukraine Ukraine’s education system has been in a state of reform for many years. The history and context for education reform in Ukraine provides an important backdrop for understanding the level of ambition brought by the current reform agenda, as well as the potential roadblocks and implementation challenges. This chapter describes the current system structure, several major trends that have shaped the reform environment in Ukraine, the current post-Euromaidan reform agenda in the education sector, and public perceptions to the reform process. A key takeaway message is that Ukraine is facing a unique moment in terms of education reform, with an ambitious agenda that has been set forth. Given Ukraine’s history of reforms, including many that were not fully implemented or later reversed, Ukraine needs to pursue this agenda with a well-designed strategy to engage and communicate with the public. Articulating the vision for change while avoiding ‘reform fatigue’ will be critical to ensure success of the reform. Education System Structure Ukraine’s 2017 Law on Education guarantees every citizen the right to high-quality and affordable education. The law envisions equal opportunities for access to education, with the right to education guaranteed regardless of age, sex, race, health status, disability, nationality, ethnic origin, political, religious or other views, color, place of residence, language, origin, social and material position, and other circumstances and characteristics. The levels of education below correspond to the National Qualification Framework (NQF), which was initially established in 2011, codified into law in 2017, and updated in 2019. The latest updates to the NQF ensured the harmonization and alignment of the NQF with the European Qualifications Framework in the European Higher Education Area (EQF-EHEA) and the European Qualifications Framework for Lifelong Learning (EQF-LLL). Preschool education is mandatory and guaranteed free of charge. It can be obtained within the family setting, until the child reaches age five, and from qualified providers. When children reach five years of age, parents can choose a form of preschool education from among full-time preschool institutions, part- time groups, or special preprimary groups within primary schools. The Law on Preschool Education defines preschool age as from 3 years to 6–7 years. General secondary education is divided into three levels: primary (level I: grades 1–4), basic general secondary (level II: grades 5–9), and field-specific secondary education, or high school (level III: grades 10–11 currently, with grade 12 to be introduced in 2027. The new Law on Education extends complete general secondary education from 11 to 12 years in total. A certificate of completion of lower secondary education is issued after level II, and a certificate of completion of upper secondary general education is issued after level III. General secondary education is provided through a network of institutions of various types, including elementary schools (offering only level I), gymnasiums (level II), lyceums (level III), specialized boarding schools, vocational schools, and higher education institutions. Secondary Vocational Education and Training (VET). Students can enroll in secondary VET after completing basic secondary (after grade 9) or upper secondary general education (after grade 11). Those enrolling in VET after grade 9 can receive an upper secondary education certificate together with a ‘skilled worker diploma’ after two years of study. Those enrolling after grade 11 receive a ‘skilled worker diploma’ after one year of study. Short-cycle Professional Tertiary Education and Higher Education. Short-cycle professional tertiary education (referred to in Ukraine as “professional pre-higher education”) is provided in colleges and technical colleges (technikum). Junior specialist’s diplomas are awarded by colleges and technical colleges 35 after 2–3 or 3–4 years of education, depending on whether the student enters with a basic or complete secondary education. However, the Law on Higher Education eliminates the “junior specialist” educational level after 2019, and going forward this level of short-cycle professional tertiary education will result in the junior bachelor degree. Junior specialist degrees will be considered equivalent to junior bachelor degrees. Higher education is provided in universities, institutes, and academies. Junior bachelor’s (short -cycle) and bachelor’s (long-cycle) academic degrees are offered, as well as master’s and doctoral degrees (figure A1.1). Trends Shaping Education Reform in Ukraine The performance of Ukraine’s education sector and the forces underlying reform are deeply rooted in a broader transformation of Ukraine’s economy and society. There are several important historical and contextual factors shaping this process that must be clearly understood, including: (a) the historical roots of education reform, (b) the Euromaidan Revolution and ensuing conflict, (c) demographic pressures, and (d) economic competitiveness and human capital. Historical Roots of Education Reform In general secondary education, starting from 1930s, the Soviet Union created a highly centralized education system with unified teaching methods and curriculum across all states. Teachers did not have the freedom to express their opinion and the authority to engage in pedagogical experimentation; and parents could not get involved in school matters.1 After dissolution of USSR, the centralized system started changing significantly in Ukraine in several key areas: Decentralization. After independence, Ukraine introduced reforms to transform their education and training systems. These reforms granted autonomy to education institutions and included decentralization measures.2 Ukraine’s new state program after independence, ‘Education: Ukraine of the 21st Century’ is one of these reforms. The program aimed to eliminate uniformity in education and sweep away from the prevailing practices of authoritarian pedagogy.3 As a result of the reform agenda in the country, teachers today can integrate new teaching materials into classrooms alongside the recommended textbooks; and the influence of teachers and parents in terms of elective classes has increased in time. Curriculum and Language of Instruction. Fostering national identity by revising school curriculum and textbooks was common in many post-Soviet countries. Ukraine has brought two significant changes in curriculum: (a) the movement from Russian language and literature classes to world literature classes, in which Russian literature could take up a maximum of 25 percent of the content; and (b) the division of school curriculum into two parts: a nationally mandated component and a school-determined component. The school component includes subjects or extra hours chosen by pupil and school. This elective part of the school component grew significantly, signaling larger autonomy of schools and pupils when deciding on the curriculum. The curriculum in minority language schools changed in similar ways. An important change involved the reclassification of Russian language schools as ‘schools with non-Ukrainian language of instruction’ to emphasize that Russian is a minority language in Ukraine with the same status as other minority languages such as Bulgarian or Hungarian.4 Today, only 4.5 percent of schools teach in minority languages, enrolling about 10 percent of students in Ukraine.5 After the disintegration of the USSR, Ukraine initiated a higher education reform agenda which covered several areas that remain relevant to today’s reform context. 6 These reforms include the establishment and proliferation of a private (nonstate) sector in higher education; the introduction of tuition fees in the public sector to compensate for a decline in funding following the fall of the USSR; a shift from 36 a vocational workforce planning model to a market-driven orientation in higher education; the introduction of standardized admissions exams; and more recently, joining the Bologna Process, an intergovernmental cooperation of 48 countries in higher education, to ensure comparability in standards and quality of higher education qualifications. Ukraine’s path of reform in higher education has differed in several important ways from other post- Soviet countries. Its decisions on education reform has contributed to a diversification of its education system. It has continued maintenance of the Soviet 5-year specialist degree and 2 different doctorate degrees, as well as reformed its vocational and higher education system. For years, Ukraine maintained the 2 degrees of Candidate of Science (requiring 3+ years of research following a masters or specialist degree) and a Doctor of Science, the highest scientific degree which required an additional 2 years of research after the Candidate of Science degree. (This has since been changed with the introduction of the 2017 law). Additionally, Ukraine merged parts of vocational and higher education by integrating vocational schools (uchilischa and technikumy) into the higher education system by assigning level I and II accreditation to these schools, while keeping level III and IV accreditation for universities. In these ways, Ukraine has increased its institutional diversity of the higher education system and been able to attract students with different priorities.7 Euromaidan Revolution and Conflict The Euromaidan and subsequent events have dramatically changed Ukraine’s political context . In November 2013, the Government’s unexpected suspension of preparations to sign an Association Agreement with the European Union (EU), added to the population’s widespread frustration with high - level corruption, leading to the Euromaidan uprising in November 2013 which culminated in the ousting of President Viktor Yanukovych in February 2014. The country has witnessed several momentous events since then, including the developments in Crimea and Sevastopol in March 2014 (which led to the UN General Assembly resolution 68/262 affirming the territorial integrity of Ukraine) and the resulting conflict in eastern Ukraine.8 Since the Euromaidan uprising in 2014, the Government has been committed to broad-based reform and European integration across sectors, including in the education sector. The new Government had a mandate to pursue broad-based reforms but has faced formidable challenges, including containing the conflict and restoring peace in eastern Ukraine; ensuring macroeconomic stability and reducing the fiscal deficit during a recession without triggering social unrest; and reducing deep-rooted corruption while contending with powerful vested interests.9 However, the Eastern conflict remains a major challenge to ensuring educational access and continuity in service delivery. The conflict caused many teachers and school-aged children to leave their communities in eastern Ukraine. The internally displaced children constitute around 13 percent of 1.7 million internally displaced persons (IDP) population. According to the Ministry of Education and Science (MOES), 280 schools in Donetsk and Luhansk oblasts have been damaged because of the operations (UNICEF estimates that 218 schools were damaged in these regions, 136 in Donetsk and 82 in Luhansk). Although thousands of children moved to government-controlled regions and became internally displaced persons (IDPs), the Ukrainian government has been successful in registering most of these children into schools in host communities. The main problem that IDP children are experiencing is related to the trauma due to the experience of living in the zone of military operations. While 75 percent of school directors and teachers in the buffer zone noted striking behavioral changes in students, children who migrated to other regions find it difficult to adapt to their new social environment. Trainings for school psychologists and teachers are needed so that they can provide psychological support to these conflict-affected children.10 37 The conflict has also had a significant impact on the higher education sector as well, resulting in the introduction of a new term in the system: displaced higher education institutions. The Ukrainian government had to evacuate and move 18 state universities, 2 private universities and 11 research institutions in the temporarily occupied territories of Donetsk and Luhansk regions. It is estimated that around 40,000 students and 3,000 research and teaching faculty fled the occupied territories.11 Because most of their assets remained in the temporarily occupied territory, the displaced institutions have been suffering from major infrastructural, financial and organizational losses. Students of these institutions had issues with accessing their academic records because university archives may not have been transferred during the evacuation process. Furthermore, some of these institutions continued to operate in the conflict zones under the same names without any accreditation, creating confusion and a difficult situation for students whose degrees may be essentially worthless within Ukraine and the European zone.12 Despite these challenges, the displaced institutions managed to continue education services in their new locations and through distance learning. The government provided an opportunity to the students in displaced universities to change their university; and most of them accepted. After relocation of the universities, students had the chance to stay with their new university or rejoin the old one. Most students chose to stay at their new universities.13 As a result, the number of students and faculty members of the displaced universities have decreased. Demographics and Migration In 2018, the population of Ukraine was 42.4 million, down from 50 million in 1999. Like many other European countries, Ukraine faces depopulation; however, unlike other countries, Ukraine’s population decline is notable for its scale and rapid pace. Ukraine has one of the oldest populations of Europe. It has a percentage of people over 60 years that has been growing steadily in both urban and rural populations; in 2018, 23 percent of its population was over 60 years old according to the SSSU. Today, according to the State Statistics Service of Ukraine (SSSU), around 70 percent of the population lives in urban areas. The ratio of urban population has been almost stable since independence, with a gradual increase toward urban areas over time (figure 1.1). Figure 1.1 Demographic Trends in Ukraine (1991-2018) 60 80% 69% 70% Urban population (% of total) 50 Total population (millions) 60% 40 50% 30 40% 30% 20 20% 10 10% 0 0% 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 20021 Source: Authors’ analysis of ukrstat.org and World Development Indicators. 38 After independence in 1991, internal mobility in Ukraine started to decline mainly because of the disappearance of the organized movement and recruitment of the labor force typical in Soviet times, growth of housing prices, and challenges with finding jobs. Prior to the breakup of the Soviet Union, 3 million people migrated from one region to another internally14; and this number decreased to 622,000 people by 2013.15 The internal mobility of Ukrainians is only half of what would be expected in comparison with other countries. For instance, as of 2010, more than 7.0 percent of population migrated within the last 5 years in Latvia and more than 10.0 percent in Estonia. This ratio was only 4.5 percent for Ukraine.16 Internal mobility of labor force contributes into economic growth of countries by improving the distribution of labor, clustering skills and talents, and driving agglomeration spillovers. However, overall limited internal mobility in Ukraine also reflects on the low levels of labor mobility in the country. Compared to similar countries, relatively few people in Ukraine migrate with the motivation to find jobs or earn higher wages. Major barriers to internal labor mobility include challenges with the population registry system, lack of affordable housing and access to credit, frequency of informal jobs and weak labor institutions, and skills gaps. In terms of skills gaps, people in less developed regions often lack the necessary skills to find jobs in higher productivity sectors in more developed regions. For instance, in Ternopil oblast, the region with the lowest average wage in Ukraine, the share of unskilled jobs (elementary occupations) is over 45.0 percent, while the share of high-skilled professions including legislators, senior officials and managers and professionals is only 18.5 percent. On the other hand, in Kyiv City, the main destination for migrants, these shares are 4.7 percent and 50.6 percent, respectively. 17 In 2014, there were 700,000 Ukrainian citizens working abroad, mostly in the Russian Federation, Poland and the Czech Republic; and 47,000 Ukrainians studying abroad, mostly in Poland, Germany, and the Russian Federation. However, immediately after the 2014 crisis, the number of applications for asylum submitted by the Ukrainians in the EU countries exceeded 22,000, over 20 times more than in 2013. Most of the applications were submitted in Italy, Germany, Spain, and Poland.18 Labor migration also accelerated after the crisis, with Poland becoming the main destination for Ukrainian labor migrants. According to National Bank of Poland and National Bank of Ukraine, in 2016, around 1.4 million Ukrainians migrated to Poland for work; and this number increased to almost 2.0 million in 2017. However, the labor migration tends to be short-term and cyclical. The migrants to Poland contributed to Ukraine ’s economy almost US$1.3 billion in 2015; US$1.9 billion in 2016; and US$3.1 billion in 2017 through remittances.19 Economic Competitiveness and Human Capital Today, Ukraine’s economic transformation remains incomplete and the country’s growth trajectory has been volatile. Ukraine’s income per capita is equivalent to less than half of the world’s average. Excessive volatility of growth has reduced incentives to invest and accumulate human capital. Recent analysis by the World Bank indicates that the rate of growth of Ukraine’s modern economy has been too slow to absorb excess supply of workers and employees released by the old economy and the new entrants to the labor force. Because of this, many young Ukrainians have opted to emigrate, attracted by higher expected earnings in neighboring countries and elsewhere. At the same time, human capital skills demanded by expanding sectors are different than those supplied by workers in dying industries.20 Human capital is an essential ingredient for economic transformation and growth. Data from over 1,500 household surveys shows that human capital is the most important component of wealth globally. Human capital is defined as the combined health, skills, and knowledge of a population. In high- income economies such as the OECD member countries, human capital reaches 70 percent of wealth.21 In lower-middle-income countries like Ukraine, human capital comprises 51 percent of wealth as of 2014, but it is growing at an annual rate of 2.6 percent.22 39 Analysis on the changing nature of work and the impact of technological trends such as automation further underscores the importance of cognitive and noncognitive skill formation for resilience as well as growth. Trends in machine learning, artificial intelligence, and automation imply potentially major disruptions to certain occupations, particularly those involving low-skill repetitive tasks. On the other hand, such technological trends may augment high-skilled work. There is some evidence that the quality of education is a critical determinant of the extent to which a country will be negatively impacted by such technological trends. Higher levels of educational attainment are typically associated with greater resilience from job loss due to automation; however, low-quality education that does not improve the skills of lesser able students may be ineffective at improving resilience to job loss.23 Ukraine’s Ambitious Agenda for Education Reform Since 2014, Ukraine has been engaged in an ambitious and transformative reform of the education sector to promote human capital development as 1 of 5 strategic priorities of the Government. The Government’s Medium-Term Action Plan to 2020 and the annual Priority Action Plans identify human capital development and education system reform as a core priority for contributing to the goal of increasing living standards and quality of life through sustainable economic development.24 These reforms hold great promise to fundamentally transform the sector. 2014: Law on Higher Education The new Law on Higher Education is the first large systemic reform-oriented law adopted by the Verkhovna Rada after the Euromaidan Revolution. Intensive work on the law had been conducted for several years, with numerous attempts at reform initiated prior to 2014. The law was discussed at length within the higher education community and was prepared by experts representing leading universities in the country. The law defines several principles on which Ukraine’s public policy in the higher education sect or would be based, underpinned by an agenda of European integration. Principles include: • Promotion of sustainable development of society by producing competitive human capital and creating conditions for education throughout the entire lifetime. • Accessibility of higher education. • Independence of higher education from political parties and civil or religious institutions. • Integration of Ukraine’s higher education system into the European Higher Education Area (EHEA). • Government support to training of workforce with higher education for priority economy sectors, sectors of fundamental and applied research, pedagogical inquiry and teaching. • Government support for research and innovation, including preferential financial treatment for universities engaged in research. In passing the law, Ukraine took a remarkable step away from its post-communist heritage and toward a more modern higher education system. The law sets the stage for major reforms in higher education qualifications, university autonomy and management, and quality assurance. Key provisions of the law are briefly described below: • Definition of levels and qualifications. The law redefines the levels, degrees, and qualifications of higher education in relation to a National Qualifications Framework (NQF), moving away from the Soviet education system structure toward a model more aligned with European norms and the Bologna Process, which Ukraine joined in 2005. The Bologna Process is a pan-European process aimed at harmonizing higher education provision in Europe, including the support of enhanced quality of provision and greater mobility. Before the Law on Higher Education was passed in 2014, 40 Ukraine had four levels of higher education institutions (HEIs): technical colleges (level I), colleges (level II), institutes and conservatories (level III), and universities, academies, institutes, and conservatories (level IV). The 2014 law eliminated these distinctions. • Creation of higher education activity standards. The law specifies that higher education standards are to be developed for every level of higher education within every major according to the NQF. Standards would specify competencies of graduates, the number of European Credit Transfer System (ECTS) credits25 required for the degree, and other aspects. • Expansion of academic autonomy. The law specifies that higher education institutions (HEIs) will be autonomous and self-governing, managing their academic and day-to-day operations. In addition to the right to create educational specializations within licensed fields of study and independently fill these programs with subjects, Ukrainian HEIs gained other rights to increase their academic autonomy. • Establishment of democratic management structures in HEIs. The law specifies that HEI heads will manage all aspects of the HEI, including financial and business operations, structure and staffing, performance, and public monitoring. HEI heads will be elected for a five-year period by secret ballot. Those eligible to vote are all members of the teaching, academic and research staff, representatives of other staff members, and elected representatives of students. Democratic governing structures will be established in HEIs, including an academic council, supervisory board, working and advisory bodies, and a general assembly. • Provision of greater financial independence for HEIs. The law increases the degree of financial autonomy afforded to HEIs, though this has focused largely on technical barriers such as HEIs obtaining the right to open current accounts not only in the State Treasury but also in state banks. It is important to note that the law did not envision a chance in the funding principles in higher education, although the Government has taken steps in this direction following passage of the law. • Development of scientific research and innovation activities in HEIs. Finally, the law envisions a closer relationship between HEIs and the national academies of science. There remains a Soviet- era legacy that divides education from research, with education (teaching) occurring mainly in universities, and research in the National Academy of Sciences and the six sectoral national academies of science. The new law envisions more integration between research carried out in universities and the academies, thereby making better use of resources and potentially increasing Ukraine’s competitiveness in European and global research areas. For example, the law allows universities to be eligible to apply for designation as research universities, entitling them to additional research funding.26 2014: Budget Decentralization Decentralization reforms, started in 2014, aim to transfer a considerable extent of authority and responsibilities from the central government to intermediate, local levels of government .27 The Concept for Reforming Local Self-Government and Territorial Organization of Power launched the decentralization of local authorities in Ukraine in 2014. The main motivation behind this framework has been to address poor living standards, especially in rural areas; ineffective use of resources; and a lack of institutional capacity in providing public services.28 Sharing responsibilities with local authorities would both advance economic and social development in the country, while strengthening central government’s legitimacy and inclusivity. Ukraine’s decentralization strategy has three dimensions: political, administrative and fiscal . There are three levels of constitutionally guaranteed subnational governments in Ukraine: oblasts, rayons and hromadas. The smallest units, hromadas, have their own councils and consist of towns, smaller settlements and villages.29 The decentralization reform envisages amalgamation of smaller settlements into larger ones that can manage increased responsibilities and funds, especially education and health funds. Before the 41 reforms, 92 percent of villages had fewer than 3,000 residents, and 47 percent had fewer than even 1,000 residents. These small communities simply lacked sufficient human capital to drive economic growth.30 The decentralization plans for reform is centered on voluntarily31 combining smaller hromadas into larger amalgamated hromadas (AHs).32 Once amalgamated, hromadas are given additional public service responsibilities (including managing preschools and general secondary schools), access to increased financial resources, and an ability to negotiate their budgets directly with the oblast administration rather than depending on transfers from the rayon state administration. In 2015, 794 village, settlement and town councils voluntarily amalgamated into 159 hromadas. In 2016, another 946 village, settlement and town councils voluntarily amalgamated into 208 hromadas.33 Another aspect of Ukraine’s decentralization strategy includes a fiscal dimension, which aims to increase the capacity of AHs to provide better public services while incentivizing amalgamation. Because amalgamation is voluntary, legislative changes enhanced revenue capacity and greater autonomy only to the communities that chose to amalgamate. Once the consolidated local communities officially elect their mayor and municipal council, they can negotiate their budgets directly with their oblast administrations, rather than negotiating with and receiving funds from the rayon state administration. In addition to receiving funding support, AHs can now choose where to invest their money based on their own priorities. These conditions encouraged many local governments to engage in the amalgamation process.34 Furthermore, after amalgamation, all AHs receive a significant share of national tax revenue, additional funding in the form of grants (including a sectoral grant for education), and direct inter-budget relations with the state budget. This has led to an overall increase in per capita funding within AHs.35 Decentralization reform provides control of preschools and general secondary education institutions to AHs. As of July 2018, control of approximately 63 percent of schools in the amalgamated areas have yet to be transferred to AHs from rayon-level administrations, and less than 50 percent of AHs had taken full responsibility for their respective school systems.36 However, it is important to note that since the beginning of the creation of the AHs, it was not clear which functions would be transferred to local authorities regarding education and culture. The Law on Education of 2017 provided some clarity to this by specifying the responsibilities of local authorities in education as those related to serving as the “founder” of an education institution. The major tasks of a founder are to: • Plan and provide a network of institutions for preschool, general secondary and extracurricular education; • Plan and ensure the development of a network of specialized secondary education institutions (depending on size of local community); • Manage the network of educational institutions, including school openings, reorganization, and closure; • Determine the territory of service; • Provide transportation of students and teachers to educational institutions (as needed); and • Ensure transparency and openness of educational institutions for reporting purposes. The new budget and tax codes introduced in 2014 introduced the education subvention—a special purpose transfer for education—as a new resource allocation system.37 The 2014 budget code defines a modern system of local government finances, with specified revenue streams and expenditure responsibilities. Education features prominently on both sides of the budget. In terms of revenues, hromadas, rayons, and cities of oblast significance will receive an education subvention, calculated based on a per-student formula. On the expenditure side, these local governments will be responsible for managing and financing all secondary schools.38 However, in the case of suboptimal networks of education institutions, communities experience a shortage of funds transferred to the local budgets in the form of the education subvention (see Chapter 3). In general, the transfer of responsibilities (and hence funding) from 42 rayons to AHs remains a challenge, as many AHs are not ready to manage education service delivery with a view to issues of quality and equity. 2017: Framework Law on Education and the New Ukrainian School Three years of national dialogue took place with the aim to create a common understanding of the need for change in secondary education. Building a broad, shared understanding, across the political spectrum, on why reforms are needed is an important ingredient in sustaining reforms. On this front, the new law was underpinned by years of discussion, culminating in a set of reasons explaining why reforms were both needed and long overdue.39 These reasons included the following: • Outdated curriculum. Textbooks were too theoretical and overburdened with secondary factual materials. Students are only able to reproduce pieces of unstructured knowledge; however, they often do not know how to use this knowledge to solve everyday problems. Socioemotional skills that employers demands are not adequately addressed in the curricula. To explain the issue and vision in this area, a former advisor to the Minister of Education and Science in 2017 said, “If we define the goal of education as imparting knowledge, then education in Ukraine is quite good...The problem is that the Ukrainian students often do not know what to do with these facts. We would like to change the paradigm of education because we believe that skills —what you can do—are more important than how many facts you know.”40 • Eleven-year compulsory education. Today the 11-year school in Europe functions only in Ukraine, Russia and Belarus; this separates these countries from modern school systems around the world. • Poor performance in small schools. Students in small schools have the worst scores on the external independent testing. This is largely because small schools tend to have fewer teachers who have subject-specific or grade-level specializations. The clear majority of small schools are located in the rural areas.41 • Poor results of minority languages in university entrance exams. Students taught in minority languages are less likely to succeed on the External Independent Test (EIT) compared to those taught in Ukrainian. While over 90 percent of students taught in schools with the Ukrainian language of instruction pass the compulsory final exam in Ukrainian literature and language, in Transcarpathia, the ratio is only 23 percent of students in Hungarian schools, and 30 percent in Romanian schools. The New Ukrainian School and associated reforms mark an ambitious effort and high-level commitment to address these issues and improve the quality of general education. The framework Law on Education passed in 2017. However, as a framework law, this means that it defines only general principles and leaves enabling legislation to government subsectors. The law puts in motion some drastic changes to Ukraine’s Soviet-era education system, including codifying into law the main elements of the New Ukrainian School concept, to be implemented in three phases: phase I (2016–2018), phase II (2019– 2022), and phase III (2023–2029). Elements of the New Ukrainian School concept include modern approaches to: (a) school curricula, focused on 21st century skills and competencies; (b) teacher professional development, emphasizing student-centered learning; (c) system management and school administration, emphasizing greater local decision-making powers; and (d) a different role for the central government with a focus on setting and monitoring learning standards. The law sets the stage for much- needed structural reforms in the education sector, including: 43 • Reducing bureaucracy in the education sector by granting increased professional autonomy to teachers, schools, and local authorities; • Aligning the Ukrainian school system with European norms, including the transition to 12 years of schooling; • Introducing a national system of qualifications, including a National Qualifications Framework; and • Launching the State Service for Quality Education as a national agency for quality assurance in secondary education. The reform agenda of MOES is supported by a range of bilateral and multilateral donor partners, largely focused on basic and higher education though most are relatively small in terms of funding amount. A donor mapping analysis was conducted based on information from MOES regarding the programs of 16 external donor partners over the period 2012–2018. Although this mapping identified 72 registered projects, most of these projects were focused on technical assistance (with values of under US$500,000), with only 4 investment projects. More details can be found in annex 4. Public Perceptions of Education Reform Although education is not among the public’s top concerns, education is closely linked to those areas that are among top concerns: job creation, employment and corruption. In 2010, only 11 percent of the public thought that education should be the first sphere of priority for Ukrainian political leaders.42 Support for education was even lower in the Western region of the country (7 percent), versus closer to 12 percent in the other regions. Support for prioritizing education was also higher among women (13 percent) compared to men (8 percent) and among younger cohorts (15–16 percent among 18–29 and 30–44 age cohorts) compared to older cohorts (5–8 percent among ages 45+). This contrasted with much higher public support for creating jobs and lowering unemployment (63 percent) and reducing corruption (36 percent). Despite this relatively low level of support for prioritizing education reform, Ukraine remains an outlier among other countries on perceptions of corruption in education, with relatively low levels of satisfaction with public education compared to relatively high levels of experience with corruption in public education (figure 1.2). Even in 2016, after the Euromaidan Revolution and the initiation of large-scale education reforms, only about 20 percent of Ukrainians thought education was among the top challenges facing the country, and only 15 percent mentioned that education should be either the first or second priority for extra government spending (figures 1.3 and 1.4).43 Still, the support for the fight against corruption is strong, often reported as the most important focus area of the government by survey respondents. 44 Figure 1.2 Ukraine has low levels of satisfaction with public education and high experience of corruption Source: EBRD Life in Transition Survey (2016). Figure 1.3 Proportion of individuals who mentioned education among the three most important problems that should be addressed by the government (%), 2016 50 45 40 35 30 25 20 15 10 5 0 Germany Kyrgyz Rep. Cyprus Romania Moldova Estonia Russia Belarus Uzbekistan Italy FYR Macedonia Turkey Poland Hungary Kosovo Tajikistan Slovak Rep. Bulgaria Greece Georgia Kazakhstan Latvia Armenia Ukraine Albania Czech Rep. Mongolia Azerbaijan Montenegro Lithuania Serbia B&H Slovenia Croatia Source: Authors’ analysis of EBRD Life in Transition Survey (2016). Note: The question is “In your opinion, what are the three most important problems facing this country that government should address?”. Sample weights are applied. 45 Figure 1.4 Proportion of individuals who mentioned that education should be either the first or the second priority for extra government spending (%), 2016 45 40 35 30 25 20 15 10 5 0 Kyrgyz Rep. Cyprus Germany Romania Moldova Kosovo Belarus Poland Uzbekistan Estonia FYR Macedonia Greece Russia Italy Turkey Hungary Tajikistan Georgia Bulgaria Slovak Rep. B&H Ukraine Latvia Montenegro Armenia Lithuania Kazakhstan Croatia Czech Rep. Albania Mongolia Serbia Slovenia Azerbaijan First priority Second priority Source: Authors’ analysis of EBRD Life in Transition Survey -2016. Note: The question is “In your opinion, which of these fields 1st priority should be the first and second priorities for extra government spending?”. Sample weights are applied. Countries are ordered by the total proportion of individuals who mentio ned education either the first or the second priority. At the same time, in an environment of ‘reform fatigue,’ more efforts may be needed to communicate the objectives and means of reform and build support among stakeholders and the broader public. The Ilko Kucheriv Democratic Initiatives Foundation has been conducting annual public opinion surveys about reforms across all oblasts of Ukraine (except occupied territories) since 2015. These surveys confirm strong public support for the fight against corruption, reported by 70 percent of youth as the most important focus area of the government. At the same time, very few Ukrainians believe that education reform has been implemented successfully. A mere 3.8 percent of respondents think that the reform is successful as of May 2018, down from 4.7 percent in 2017. However, most respondents thought that no other areas of reform—anti-corruption, health care, pensions, law enforcement, or defense—have been successful either, so the education reform is not an outlier. Furthermore, at least 25 percent of those surveyed did not know enough about the reform to answer.44 This indicates the importance of concerted efforts on the part of the Government to clearly communicate and engage stakeholders in the reform process, build awareness of the reform and its objectives, and generate broader support among stakeholders in universities, schools, local governments, and communities. 46 Notes 1 UNESCO (2017). 2 Mercer and Weidman (2004) 3 Stepanenko (1999). 4 Janmaat (2000). 5 MOES (n.d.). 6 Huisman, Smolentseva, and Froumin (2018). 7 Huisman, Smolentseva, and Froumin (2018). 8 World Bank (2017b). 9 World Bank (2017b). 10 UNICEF (2016, 2017). 11 Furiv (2018). 12 Magaziner (2016). 13 O’Malley (2015). 14 IOM (2016). 15 Koettl and others (2014). 16 Koettl and others (2014). 17 Koettl and others (2014). 18 IOM (2016). 19 National Bank of Ukraine (2018). All dollar amounts are U.S. dollars unless otherwise indicated. 20 World Bank (2019a). 21 World Bank (2018a) 22 World Bank (2018a) 23 Bentaouet Kattan, Macdonald, and Patrinos (2018). 24 World Bank (2017b). 25 ECTS is an important element of the Bologna Process, meant to facilitate student mobility within the European Higher Education Area (EHEA). ECTS is a tool of the EHEA for making studies and courses more transparent and thus helping to enhance the quality of higher education. 26 KAS (2017); British Council (2015). 27 OECD (2018c). 28 Jarábik and Yesmukhanova (2017). 29 Jarábik and Waal (2018). 30 Sydorchuk (2015). 31 The law permits hromadas to amalgamate if they are contiguous, in the same oblast, and have historical, natural, ethnic, cultural or other unifying characteristics. Citizens can initiate the amalgamation process, which undergoes a public consultation and approval by each of the local councils involved. A public referendum is often held to make the final amalgamation decision, and the decision must be approved first by oblast council; and then the Cabinet of Ministers (OECD 2018c). 32 Rabinovych, Levitas, and Umland (2018). 33 Cabinet of Ministers of Ukraine (2018); OECD (2018c). 34 OECD (2018c). 35 Cabinet of Ministers of Ukraine (2018). 36 Rabinovych, Levitas, and Umland (2018). 37 Sydorchuk (2015); Herczynski (2018). 38 Herczynski (2017). 39 MOES (2017b). 40 Wynnyckyj (2017). 41 World Bank (2018c). 42 Bennich-Bjorkman, Kashyn, and Kurbatov (2019). 43 EBRD (2016). 44 DIF (2018). 47 Chapter 2: Assessing Education and Labor Market Outcomes in Ukraine Evidence suggests that Ukraine’s education sector is misaligned with the needs of the labor market, which creates the wrong incentives both for students and education institutions. Although there is limited international comparative data on learning outcomes, there is clear evidence on the importance of education for labor market outcomes in Ukraine. Tertiary education provides much better employment prospects as well as economic returns, which clearly drives the high demand for tertiary education, which represents the top objective for most young Ukrainians. However, many young Ukrainians appear to be increasingly “over-educated” relative to the types of jobs that are available in the labor market. Although economic returns vary by field of study, all forms of tertiary education provide higher returns than general secondary or vocational education. This creates incentives for students to pursue tertiary education at any cost, regardless of field or the quality or relevance of the program. Despite skills mismatches and evidence of diminishing quality discussed more in Chapter 4, the labor market still awards high returns to higher education. This apparent paradox can be explained by several factors, including selectivity by institution and program, variation over different cohorts of tertiary education graduates, and widespread credentialism and social preference for higher education. Without reliable information on the quality or value of individual tertiary education institutions or programs, and without support from academic and career guidance counselors, students end up making highly consequential decisions about their education in a manner that is misaligned with the needs of the labor market. At the same time, employers find that educational credentials do not accurately signal workers’ underlying skills, leading to skills mismatches and reliance on social connections and social capital as well as alternate forms of skills assessments. Going forward, it will be important for Ukraine to provide more support to students when facing such decisions and to improve the alignment and responsiveness of the education system to the needs of the labor market. More opportunities for feedback mechanisms and more information on labor market outcomes would help to improve students’ transition to higher education and shape their demand in line with labor market needs. Learning Outcomes and Skills Supply Results of the Trends in International Mathematics and Science Study (TIMSS) from 2011 shows Ukrainian students in 8th grade to be lagging their peers in other European and OECD countries. TIMSS 2011 was the last international student assessment in which Ukraine participated. TIMSS data indicate the average Ukrainian 8th grade student scored 490 points, compared to their peers in the European Union (524) and the OECD (527). Eighth grade students in the Republic of Korea performed on average a full 100 points better than students in Ukraine.1 Ukraine’s performance on TIMSS 2011 is driven largely by its relatively large share of ‘low performers’—those students who only achieve the low benchmark or who fail to meet that performance threshold. In Ukraine, 28 percent of students reached only the low benchmark for mathematics performance, and another 20 percent of students failed to reach the low benchmark. 2 This means that nearly 50 percent of students in total are at the lower end of the mathematics achievement distribution. In Hungary, Lithuania, and Finland, by comparison, only 12, 10, and 4 percent of students (respectively) failed to achieve the low benchmark. On the other extreme, only 22 percent of Ukrainian students were ‘high performers,’ achieving the high or advanced benchmarks compared with around 30 percent in Lithuania, Finland, and Hungary (or over 60 percent in high-performing education systems of East Asia) (figure 2.1). 48 Figure 2.1 Performance at international benchmarks for mathematics achievement: Ukraine and comparator countries 100% 90% 80% Percent fo students 70% 60% 50% 40% 30% 20% 10% 0% Failed to achieve low Low Intermediate Advanced High Source: TIMSS 2011. TIMSS measures content knowledge based on the curriculum taught in schools, whereas the OECD’s Programme for International Student Assessment (PISA), in which Ukraine participated in 2018, measures skills and application of content knowledge. The New Ukrainian School program focuses heavily on developing competencies such as mathematics, science, and digital literacy. It is also moving toward development of transversal skills such as problem solving. Given these education trends in Ukraine, PISA will provide a useful snapshot of Ukraine’s performance in terms of students’ ability to apply knowledge in mathematics, science, and reading.3 Although TIMSS and PISA achievement scores are highly correlated at the national level, the two tests measure different types of learning.4 Analysis of skills among the adult population indicates that tertiary education graduates have higher mean reading proficiency scores compared to people with lower levels of education, but there is wide dispersion both across and within educational levels.5 This dispersion is particularly apparent among workers with general upper secondary education and tertiary vocational (short-cycle tertiary education at colleges). For example, the top quartile of workers with general upper secondary education has higher scores than more than half of university graduates.6 Wide variations within education levels may reflect variation across cohorts due to inefficiencies in Ukraine’s education system in the 1990s, which were triggered by transition-specific circumstances. More specifically, variation in reading proficiency scores between college graduates (with short-cycle tertiary professional education) and university graduates (with long-cycle tertiary academic education) point to fundamental differences in the quality of education at the two levels, as well as the quality of learning outcomes (figure 2.2). Figure 2.2 Reading proficiency scores of urban residents by educational attainment, 2012 49 Source: Del Carpio and others (2017), based on STEP/ULMS-2012. Note: Sample includes urban residents aged 15–64 years. Ukraine performs worse than OECD countries in terms of the mean literacy proficiency score among the adult population with tertiary education. This may be due to inefficiencies in the tertiary education system in the 1990s and early 2000s. As a result, the mean literacy proficiency score among Ukrainians of all education levels is lower than in many other countries which have significantly lower shares of highly educated people among the adult population. Because the STEP survey for Ukraine only included the adult population in urban areas, it is expected that the observed performance gap would be substantially larger if literacy proficiency among the rural population of Ukraine were also included in the data (figure 2.3). Figure 2.3 Literacy proficiency score of adult population in 2011–2013 in Ukraine and selected countries 50 320 310 Literacy scale (range 0-500) 300 290 280 270 260 250 240 USA Armenia Canada Poland Slovak Rep. Estonia France Japan Finland Norway UK Italy Netherlands Sweden Austria Czech Rep. Spain Belgium South Korea Ukraine Georgia Germany Ireland Denmark Total adult population Adult population with tertiary education (ISCED-1997 5A/6) Source: Authors’ analysis based on PIAAC Survey of Adult Skills for OECD countries and Russian Federation (2011-2012); STEP Skills Measurement Surveys for Ukraine (2012), Armenia (2013) and Georgia (2013). Notes: Results from the literacy assessment are reported along a proficiency scale ranging from 0 to 500 with tasks at the lower end of the scale being easier than those at the higher end. Statistics based on the STEP survey for Ukraine, Armenia, and Georgia refer to urban population only; statistics based on the PIAAC survey used for the other countries refers to urban and rural population. Adult population is defined here as 25–64 (65) years for STEP (PIAAC)-based literacy proficiency. In terms of the applicability and relevance of skills, workers with tertiary education are significantly more likely to use their skills intensely, both at work and outside of work. For example, workers with tertiary education are much more likely to use numeracy and computer skills at work, as well as more analytical skills associated with advanced thinking and learning. The intensity of use of reading and numeracy at work increases with educational attainment. However, the intensity of use of numeracy is highest among people with lower secondary education, mainly because this group has a high share of inactive students who use numeracy skills during studies at school (for example, outside work) (figure 2.4). Figure 2.4 Cognitive and technical skills of urban residents by educational attainment, 2012 51 Reading at work 2.5 Reading outside Physical tasks work 2 Repetitiveness 1.5 Writing at work 1 Writing outside Autonomy 0.5 work Lower than upper 0 secondary Upper secondary Learning Numeracy at work Tertiary Numeracy outside Thinking work Use of computer at Interpersonal skills work Use of computer outside work Source: Authors’ analysis of STEP/ULMS-2012. Note: Sample includes urban residents aged 15–64 years. Noncognitive skills, increasingly viewed as crucial to social and economic development, do not appear to be affected by educational attainment. Although urban residents with tertiary education perform relatively better in terms of noncognitive skills than their less educated counterparts, especially in terms of openness, grit and decision making, the differences are not statistically significant (figure 2.5). This suggests that higher education does not provide significant opportunities to develop noncognitive skills, above and beyond those provided in lower levels of education. Figure 2.5 Noncognitive skills of urban residents by educational attainment, 2012 Openness 3.5 3 Decision making Conscientiousness 2.5 2 1.5 Lower than upper Hostile bias 1 Extraversion secondary Upper secondary Tertiary Grit Agreeableness Emotional stability Source: Authors’ calculations based on STEP/ULMS-2012. Note: Sample includes urban residents aged 15–64 years. Labor Market Outcomes 52 Ukrainians with tertiary education have much better employment prospects than their less educated peers. On average, 66 percent of 15–70-year-old7 Ukrainians who have a college or a university degree were employed in 2017, compared to about 52 percent for persons with an upper secondary or postsecondary nontertiary (vocational) qualification, and less than 20 percent for persons who have not completed upper secondary education (figure 2.6). Although higher educated workers are less likely to be unemployed, unemployment rates decrease only slightly with education. Unfortunately, SSSU does not provide information on differences in employment prospects by tertiary education fields of study. Furthermore, field of study in the SSSU labor force survey (LFS) is coded in line with the International Standard Classification of Occupations (ISCO), so even if individual-level data were available for 2017, labor market outcome indicators could not be estimated by field of study. Figure 2.6 Labor market outcomes, by broad educational groups (%), 2017 80 72.1 66.1 58.1 60 51.9 40 18.9 16.7 20 11.9 10.7 8.4 0 Labor force participation Employment Unemployment Primary or less (ISCED 0-1) Upper secondary and vocational education (ISCED 3-4) Tertiary (ISCED 5-8) Source: Authors’ calculations based on SSSU data for seven levels of education (LFS). Notes: “Tertiary” includes incomplete (short-cycle), basic (bachelor’s) and complete (baster’s or equivalent) higher education according to the classification used by SSSU. Holders of a bachelor’s degree seem to be penalized in the Ukrainian labor market because they have lower employment rates and higher unemployment rates than many other educational groups. However, this level of education is not yet widespread among the population age 15–70, given Ukraine’s history with the 5-year specialist degrees and the recent transition to the 3-cycle structure of degrees under the Bologna Process. There are almost no Ukrainians aged 45+ with a bachelor’s degree. According to the individual-level LFS data in 2013,8 over 77 percent of all individuals having a bachelor’s degree (basic higher education) were young people under 30 years old. Nearly 60 percent of them were classified as economically inactive, mainly due to further education, followed by those looking after family and home. (More detailed information by age and education are not available due to limitations with the SSSU data). At the same time, there is evidence to suggest that the labor market and general public do not yet consider the bachelor’s degree to represent ‘completed’ higher education.9 In fact, until 2014, the law stated explicitly that the bachelor’s degree represented incomplete higher education. This negative effect is particularly distinct among males having a bachelor’s degree, whose unemployment rate is several times higher than among women with the same educational attainment. However, this reflects differences in occupational structure. In 2017, the unemployment of men was at 21.2 percent compared to women’s unemployment of 6.1 percent. Moreover, the unemployment rate for males with a bachelor’s degree has increased from 16.2 percent in 2015 to 21.2 percent by 2017. Analysis of gender gaps in 2008–2009 shows that holders of a bachelor’s degree experienced the largest increase in unemployment between 2008 and 2009 (8.7 percentage points among men and 2.7 percentage points among women), facing a reversal in the gender gap in favor of men.10 These changes can be interpreted as the 53 devastating effect of the economic crisis on educated newcomers to the labor market, especially men, due to the differences in male and female employment by sector rather than gender as such. Specific features of the labor market in Ukraine that were identified in 2008–2009 are still relevant today in explaining the relatively high unemployment rate among males with a bachelor’s degree. Such features include a high demand for unskilled and semi-skilled labor, a fairly low demand for young professionals with a bachelor’s degree who are not perceived by local employers to have completed tertiary education, and low responsiveness of the Ukrainian education system to the rapid changes in the labor market. Upper secondary education is the minimum educational attainment level for successful integration to the Ukrainian labor market, but even the highest level of education does not guarantee finding a job. Completing general secondary education improves the employment rate by almost 25 percentage points, compared to adults with a lower secondary education, and getting vocational education increases employment rate by a further 19 percentage points (figure 2.7). Although having a master’s or equivalent level of education is associated with a higher employment rate and a lower unemployment rate, a large number of highly educated workers are unemployed (452,000 persons in 2017, or nearly 27 percent of the total unemployed population) (figure 2.8). Figure 2.7 Employment rates by educational attainment, gender and place of residence (%), 2017 80 70 % of population (15-70 years) 60 50 40 30 20 10 0 Total Master’s or Bachelor's Short-cycle Vocational Upper Lower Primary or equivalent level tertiary non-tertiary secondary secondary less and above Total Women Men Urban Rural Source: Authors’ analysis of SSSU, based on the LFS data 54 Figure 2.8 Unemployment rates by educational attainment, gender and place of residence (%), 2017 25 % of the labor force (15-70 years) 20 15 10 5 0 Total Master’s or Bachelor's Short-cycle Vocational Upper Lower equivalent level tertiary non-tertiary secondary secondary and above Total Women Men Urban Rural Source: Authors’ analysis of SSSU, based on the LFS data Surprisingly, having attained vocational education does not reduce the risk of unemployment. This is in contrast with persistent complaints of Ukrainian employers about the lack of blue-collar workers and a large and growing number of vacancies for blue-collar workers reported to the Public Employment Service (PES).11 Skills mismatch and/or mismatch between workers’ wage expectations and firms’ wage offers are among the most likely explanations for this inconsistency. Returns to Education This section provides World Bank estimates of economic returns to education in Ukraine using two data sources and two approaches, the first of which is based on aggregated data on average wages of staff employees across education levels. The primary source of data is a survey of firms with at least 10 employees on wages by occupation which is conducted by SSSU once in 4 years and available for 2 years, 2012 and 2016. Wage premiums are estimated for a certain level of education as a percentage change in average hourly wage between this level of education and the lowest one. For example, a premium for a bachelor’s degree is estimated as the difference in average hourly wage of employees with a bachelor’s degree compared to their peers with primary education or less, as a percentage of hourly average wage of the least educated workers. An alternative indicator based on the same data is a percentage change in average hourly wage between two adjacent levels of education. An important caveat, however, is that aggregate data do not allow for the control of other individual and job characteristics, so the estimated returns to education are unadjusted for these important characteristics. This analysis of wage premiums shows that there is a positive payoff for workers who invest in their education, but it is sizeable only for workers with a master’s degree or higher. The aggregate-level analysis of the SSSU data on average wages of staff employees across education groups and gender shows that workers with a relatively higher level of education on average have higher hourly wages compared to their lowest-educated peers, with the largest return to those with master’s degrees (completed higher education) (figure 2.9). However, a wage premium between two adjacent levels of education is not always positive. For example, average hourly wage of male workers with vocational (postsecondary, nontertiary) education is lower than wages of male workers with upper secondary education, suggesting that vocational 55 education brings a wage penalty relative to upper secondary education. Even though female workers with a bachelor’s degree get some wage premium compared to graduates of colleges (short -cycle tertiary education) they substantially lag females having a master’s degree or equivalent. Figure 2.9 Wage premiums in average hourly wages of staff employee (compared to primary education or less) (%), 2012 and 2016* Lower secondary Upper secondary Women Vocational non-tertiary Short-cycle tertiary Bachelor's level Master’s or equivalent and above Lower secondary Upper secondary Vocational non-tertiary Men Short-cycle tertiary Bachelor's level Master’s or equivalent and above Lower secondary Upper secondary Total Vocational non-tertiary Short-cycle tertiary Bachelor's level Master’s or equivalent and above 0 50 100 150 2012 2016 Source: Authors’ analysis, based on SSSU data on average hourly wage. Note: A wage premium for a certain level of education is estimated here as a percentage change in average hourly wage between this level of education and the lowest one. For example, premium for a bachelor’s degree is estimated as the difference in av erage hourly wage of employees with a bachelor’s degree compared to their peers with primary education or less, as a percentage of hourly average wage of the least educated workers. A second approach uses individual-level data in 2006 and 2016 from the Ukrainian Household Living Conditions Survey (HLCS) to estimate the returns to education. This approach is controlled for available individual characteristics in an extended Mincerian earnings function.12 The survey sample includes individuals of working age (16–64 years) who reported income from the main job, either money or in-kind, or both. Returns were estimated to an incremental year of education, and to a given level of education and/or field of study for higher educated individuals. The HLCS contains data on level of education with three categories: (a) no education, illiterate; (b) general secondary education, including primary, basic secondary and complete secondary; and (c) higher education, including incomplete, basic, and complete higher education. Additionally, the dataset includes information on vocational education and field of study for higher education (10 suggested options). With this data, two variables were constructed: • Level of education: 1 = Base category: no education, primary or general secondary education without vocational qualification; 2 = Vocational: general secondary education with vocational qualification; 3 = Tertiary: incomplete, basic and complete higher; • Level and field of studies: 1 = Base category: no education, primary or general secondary education without vocational qualification; 2 = Vocational: general secondary education with vocational qualification; from 3 to 13 = Tertiary: major in economics, law, medicine, technical sciences, natural sciences, humanities, education, arts, military and defense, agriculture, and other, respectively. 56 An extra year of education represents an increase of 6.3 percentage points in yearly earnings. This is after controlling for experience, gender, marital status and location, when only earnings of employees at their main job are considered. The effect of education is even larger (7.8 percent in 2016) when the sample is extended with self-employed individuals and earnings include income from all labor-related activities including entrepreneurial activity and freelancing.13 Figure 2.10 Return to an extra year of education or a given level of education Relative to upper secondary education and less on yearly earnings of employees at their main job ( %) Tertiary, Agriculture Tertiary, Military and defense Tertiary, Arts By field of study Tertiary, Education Tertiary, Humanities Tertiary, Natural sciences Tertiary, Technical sciences Tertiary, Medicine Tertiary, Law Tertiary, Economics Vocational Tertiary level By Vocational 0 10 20 30 40 50 60 70 2006 2016 Source: Authors’ calculations based on HLCS-2006 and 2016 (individual-level data for the fourth quarter). Note: Sample includes individuals of working age (16-64 years) who reported income from the main job (both money and in- kind). Bars show significant coefficients of education variables in models (1)-(3) multiplied by 100%. The reference category in models (2) and (3) is General secondary education and below. Alternative models—with the levels of education, instead of years of education—show that workers with tertiary education have much higher returns than graduates of secondary vocational schools as well as lower-educated workers with general secondary education or less. Besides, when tertiary education is included in the model together with the field of studies, the returns for college and university graduates compared to lower-educated workers with general secondary education or less differ depending on the field of studies (figure 2.10). Yearly earnings of workers with a tertiary degree in natural sciences do not significantly differ from earnings of secondary school graduates, whereas holders of degrees in military and defense experienced a wage premium of over 60 percent compared to low educated workers in 2016. Comparisons of wage premiums in 2006 and 2016 show that returns to vocational education almost did not change whereas wage premiums for tertiary education increased from about 27 percent to 30 percent. This is in-line with findings based on the aggregate-level analysis of average wages by education presented above. Hence, returns to tertiary education are relatively high and increasing over time. But an increase in estimated returns to education is observed only in half of fields of tertiary education, namely economics, technical sciences, medicine, arts, and military and defense. In the other five fields, particularly law and humanities, wage premiums compared to the least educated workers decreased between 2006 and 2016. Without comparable data on yearly earnings of employees by the field of study, it is difficult to determine with confidence whether these changes are in line with actual trends in the Ukrainian labor market. 57 Information on average monthly wages of staff employees in top occupational groups can be used to shed light on wage premiums to selected fields of study, but this is only by assuming a match between the field of study and current job. This assumption is often not the case, particularly for younger graduates. However, the exercise is useful to check analysis and triangulate the conclusions. Such data on average monthly wages in 2012 and 2016 are published by SSSU. This data show that estimated wage premiums for working in a higher skill job compared to elementary occupations increased in all occupational groups from 2012 to 2016, except for life science professionals and life science and health associate professionals, for example, occupations in which natural sciences graduates seek work. Hence, returns to education in natural sciences are relatively small and decreasing, most likely due to low labor demand in the private sector and low wages in the public sector. An increase in returns to the military and defense study field (figure 2.10) can be attributed to a substantial increase in compensation of military personnel since 2014, especially those serving on the front line in eastern Ukraine.14 This analysis points toward a paradox: how can economic returns to tertiary education be so high given the evidence of diminishing quality of tertiary education and education-job mismatches? This is likely driven by a confluence of factors that characterize both the education sector and labor market. First, there is a strong preference for tertiary education as the means to enter and success in the competitive labor market and to self-sufficiency. This has created a high level of credentialism in the labor market, where both students and employers put an over-emphasis on educational credentials. A comparison of the structure of job vacancies and the required educational profiles seems to support this hypothesis. 15 At the same time, while many students enter tertiary education, there is a high level of selectivity by program and institution, moderated by the university admissions exam and other factors. This had led to a high variation in quality, with an upper tier of highly selective and reputable institutions from which graduates get good- paying jobs, and a lower tier of less selective institutions that have graduated large numbers of students in recent years but with questionable quality and relevance. Additionally, the high returns to tertiary education also reflect different cohorts of tertiary graduates, including younger graduates who increasingly take jobs for which they are over-qualified, as well as older graduates who have years of work experience already. One final consideration is the emigration of highly skilled workers, which has been observed in larger cities and Western regions. This constrains the supply of highly educated workers and keeps returns higher than they might be otherwise. Employment Structure and Demand for Tertiary Education Due to a slow pace of technological progress and labor market rigidities, Ukraine still has a traditional production structure that requires many workers with low qualifications in less knowledge-intensive services, mining and construction, agriculture and low-tech industry. These four broad sectors accounted together for over 75 percent in total employment of business units, excluding budget organizations, between 2010 and 2017. At the same time, knowledge-intensive services and high- tech manufacturing have relatively small and almost persistent shares during 2010–2017 (table 2.1). Table 2.1 Sectoral and occupational structure of employment in Ukraine (% of total employment), 2010–2017* 2010 2011 2012 2013 2014 2015 2016 2017 Broad sector (firm-level data)** Less knowledge-intensive services 45.5 44.6 45.5 46.1 45.7 46.4 47.2 47.0 Knowledge-intensive services 12.4 12.7 13.1 13.3 13.7 13.5 13.7 13.6 Mining, utilities (electricity, gas, water 14.9 14.9 12.1 11.9 12.6 12.7 11.3 12.1 supply and sewerage) and construction 58 Low-tech manufacturing 7.6 7.8 8.2 8.1 7.7 7.8 8.0 8.2 Agriculture, forestry and fishing 7.4 7.2 7.6 7.5 8.1 7.7 8.1 7.7 Medium-low-tech manufacturing 6.7 6.7 6.9 6.6 6.3 6.3 6.1 5.9 Medium-high-tech manufacturing 4.8 5.3 5.9 5.8 5.2 5.0 4.9 4.8 High-tech manufacturing 0.7 0.7 0.7 0.7 0.7 0.6 0.7 0.7 Occupation (LFS) Elementary occupations 24.0 23.9 23.4 23.7 18.3 18.4 18.9 19.4 Professionals 13.9 14.5 14.8 14.9 17.1 17.7 17.9 17.9 Services and sales workers 14.6 15.0 15.1 15.3 15.9 16.4 16.8 16.9 Craft and related trades workers 11.7 11.8 12.0 12.0 12.8 12.3 12.0 12.0 Technicians and associate professionals 11.6 11.2 11.2 11.3 12.2 12.0 11.7 11.3 Plant and machine operators and 11.7 11.6 11.5 11.2 11.4 11.2 11.2 11.0 assemblers Managers 8.0 7.9 7.9 7.6 8.0 7.9 7.7 7.6 Clerical support workers 3.4 3.1 3.2 3.2 3.2 3.2 3.0 3.0 Skilled agricultural, forestry and fishery 1.0 1.0 0.9 0.9 1.0 0.9 0.9 0.8 workers Source: Authors’ analysis; Sector – calculations based on SSSU data on total employment at all business units, including banks and workers employed by private entrepreneurs but excluding activity of budget organizations (original source – firm-level survey); Occupation – calculations based on SSSU data on total employment of individuals aged 15–70 years (original source – LFS). Notes: *2014–2017 data for the territory of Ukraine excluding the Autonomous Republic of Crimea and temporarily occupied territories of Donetsk and Luhansk oblasts. **Eurostat approach to aggregation of manufacturing and services based on two-digit NACE Rev. 2 is used. Professionals represent a growing share of the occupational structure in Ukraine, while elementary occupations have declined over time, although they remain the largest occupational group. Despite a substantial decrease in the share of elementary occupations from 24.0 percent of total employment in 2010 to 19.4 percent in 2017, it remains the largest occupational group. A positive development is a substantial increase in the share of professionals—from 13.9 percent in 2010 to 17.9 percent in 2017, due to which this occupational group became the second largest occupational group in total employment. The other occupational groups with positive employment changes in Ukraine over 2010–2017 are services and sales workers, and craft and related trades workers (figure 2.11). Hence, an increase in relative employment of tertiary educated individuals is accompanied with a decrease in employment of the least educated workers (given correspondence between education and occupation). Aggregation of 9 ISCO occupational groups into 4 skill level groups, in line with the ILO 16 mapping, reveals an increase in the share of occupations at the higher end of the occupational skill distribution and a decline at the lower end of this distribution between 2013 and 2014. This can partly be attributed to the exclusion of Crimea and temporarily occupied territories of Donetsk and Luhansk regions from the employment statistics. However, observed changes in the occupational structure can also be interpreted as a sign of positive shift of employment toward higher-skilled jobs. Figure 2.11 Changes in employment shares of broad occupational groups (2010=100), 2010–2017 59 120 110 Skill level 4 100 Skill level 3 90 Skill level 2 80 Skill level 1 70 2010 2011 2012 2013 2014* 2015* 2016* 2017* Source: Authors’ analysis based on SSSU LFS-based data on total employment by occupation and education. Note: The skill level is based on the mapping by ILO (2012); see table below. *2014 –2017 data for the territory of Ukraine excluding the Autonomous Republic of Crimea and temporarily occupied territories of Donetsk and Luhansk oblasts. There is some evidence that the supply of tertiary educated workers, particularly those with university-level education, exceeds the relative demand in the economy for workers educated at this level. This conclusion is based on analysis using the methodology in which the composition of labor supply and demand by education are compared, to check if there is a relative oversupply of workers with a university or college diplomas in Ukraine that does not correspond to production needs.17 The LFS-based data on the composition of the labor force aged 15–70 years of education are used as a measure of relative supply. Then, the ratio of the labor force with tertiary education to the labor force with lower levels of education is calculated to estimate relative supply. Since there are no reliable country-level data on labor demand by education in Ukraine, this analysis instead uses the normative approach of categorizing major ISCO-based occupational groups by level of education (see table 2.2). Relative demand for tertiary educated workers is calculated then as the ratio of the number of jobs requiring tertiary education to the number of other jobs. An important caveat with this approach is that the observed employment by occupation is the result of realized matches between labor demand and supply, and that actual educational requirements might be different from those used in the ISCO-education mapping.18 Furthermore, this analysis does not capture likely differences in employment by field of study, given that such data are not available. Table 2.2 Mapping of skill levels and occupations to education levels Skill level ISCO-1988 major group ISCED-1997 education group Skill level 4 1. Senior officials and managers 6. Second stage of tertiary education (leading to 2. Professionals an advanced research qualification) 5A. First stage of tertiary education, 1st degree (medium duration) Skill level 3 3. Technicians and associate 5B. First stage of tertiary education (short-cycle professionals tertiary) Skill level 2 4. Clerks 4. Post-secondary, nontertiary education 5. Service workers and shop and 3. Upper secondary level of education market sales workers 2. Lower secondary level of education 6. Skilled agricultural and fishery workers 7. Craft and related trade workers 8. Plant and machine operators and assemblers Skill level 1 9. Elementary occupations 1. Primary level of education Source: International Labour Office (ILO), 2012. International Standard Classification of Occupations: ISCO-08. Vol.1, Structure, Group Definitions and Correspondence Tables. ILO, Geneva. 60 This exercise shows that the relative supply of university graduates has outpaced the relative demand, creating a surplus in the labor market. The gap between the relative demand and relative supply of all tertiary educated workers including college graduates is even larger due to a steady reduction of jobs for technicians and associate professionals that are presumed to require tertiary short-cycle college education (figure 2.12). The lack of relevant jobs has caused a considerable increase in the share of individuals with college or university diplomas among the unemployed and also to high incidence of education-job mismatch, especially among college graduates (figure 2.13).19 Figure 2.12 Relative supply and demand for tertiary educated workers, 2004–2017 1.2 Supply/demand at given level of education, relative 1.0 to other levels of education 0.8 0.6 0.4 0.2 0.0 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014* 2015* 2016* 2017* Relative supply, university only Relative supply, university AND college Relative demand, university only Relative demand, university AND college Source: Authors’ analysis based on SSSU LFS-based data on total employment by occupation and education. Note: Relative demand for tertiary education (university and college) is calculated as the ratio of the number of jobs filled by the employed aged 15–70 years requiring tertiary education (for example, skill level 4 and 3 jobs) to the number of other jobs (for example, skill level 2 and 1 jobs). Relative supply is calculated as the ratio of the labor force with tertiary education to the labor force with lower level of education. *Data for 2014–2017 for the territory excluding the Autonomous Republic of Crimea and temporarily occupied territories of Donetsk and Luhansk oblasts. Figure 2.13 Excess labor supply relative to demand, by education, 2005–2017 61 250% Supply as a share of demand 200% 150% 100% 50% 0% Tertiary university Short-cycle tertiary Secondary education Primary education education (ISCED 5A/6) education (ISCED 5B) (ISCED 2-4) (ISCED 1) 2005 2010 2015* 2017* Source: Authors’ analysis based on SSSU LFS-based data on total employment by occupation and education. Note: For estimating labor demand by education, authors applied the ILO (2012) normative approach as described above. Supply refers to the labor force with respective level of education. *Data for 2015 and 2017 excluding the Autonomous Republic of Crimea and temporarily occupied territories of Donetsk and Luhansk oblasts. School to Work Transition Higher education represents the top objective for most young Ukrainians, with the overwhelming majority of young people either having completed or currently pursuing tertiary education. According to the ILO School-to-Work Transition Survey (SWTS) conducted in 2015, 62.0 percent of Ukrainian youth aged 15–29 years have completed their studies; 36.0 percent were in education at the time of the survey; and about 2.0 percent have stopped their education before completion (compared to 2.3 percent in 2013).20 Among young people who have completed their education, 59.0 percent have tertiary education (predominantly a master’s level or equivalent), followed by 30.0 percent with completed vocational education and 9.6 percent with completed general secondary education (figure 2.14). Among youth in education at the time of the survey, 52.4 percent studied in higher educational institutions (includes 4 levels from short-cycle tertiary to PhD studies colored in blue); 22.7 percent were in general secondary educational institutions; and only 16.8 percent were in vocational schools. The share of young women and residents of large cities pursuing tertiary education or having completed it is much higher compared to men and residents of towns or villages (figure 2.15). Males and residents of towns are relatively more likely to choose vocational education. Figure 2.14 Youth who have completed education by highest level of education, age, gender and place of residence (%), 2015 62 100% 80% 60% 40% 20% 0% 15-19 20-24 25-29 Female Male City Town Rural area Age group Sex Area of residence Total Primary Lower secondary Upper secondary Vocational Short-cycle tertiary Bachelor's, specialist's or master's levels Post-graduate education PhD studies Professional courses, training, etc. Source: Libanova and others (2016), based on SWTS-2015. Notes: City is defined here as a center of oblast, town refers to other cities and towns. Figure 2.15 Youth currently in education by highest expected level of education, age, gender and place of residence (%), 2015 100% 80% 60% 40% 20% 0% 15-19 20-24 25-29 Female Male City Town Rural area Age group Sex Area of residence Total Primary Lower secondary Upper secondary Vocational Short-cycle tertiary Bachelor's level Master's or specialist's levels Post-graduate education PhD studies Source: Libanova and others (2016), based on SWTS-2015. Notes: City is defined here as a center of oblast, town refers to other cities and towns. The ILO School-to-Work Transition Survey (SWTS) reveals a low and decreasing attractiveness of vocational education among Ukrainian youth. Among current students at the time of the survey in 2015, only 13.7 percent had plans to undertake vocational education and training, whereas 84.6 percent of youth planned to obtain higher education.21 Approximately 75.0 percent young people who intended to complete higher education said they would not reconsider going through the vocational educational system even if they were told that they would have a better chance of finding a stable job at a higher wage if they pursued a vocational career (table 2.3). For comparison, the share of such individuals not willing to switch to vocational education increased since the prior survey in 2013, when the share was 62.3 percent.22 The most popular reason for rejecting vocational education is that higher non-vocational education is necessary for desired future employment (48.2 percent), followed by expected higher earnings from tertiary education 63 and better career prospects (42.5 percent), higher social status with a higher level degree (25.2 percent), family’s disapproval of vocational education (14.7 percent) and its bad reputation (11.7 percent). Table 2.3 Perceptions of possible vocational track among young people willing to complete higher education by age, gender and place of residence (%), 2015 Age group Gender Area of residence Total Female 15-19 20-24 25-29 Town Rural Male City Would you have considered taking the vocational track? * Yes 29.1 21.5 37.7 23.1 30.9 25.7 23.2 32.5 26.8 No 70.9 78.5 62.3 76.9 69.1 74.3 76.8 67.5 73.2 Reasons for rejecting vocational track (multiple responses) Vocational education has a bad 10.4 13.4 13.9 12.7 10.6 15.2 7.3 12.0 11.7 reputation My family would not approve 15.7 13.6 9.7 16.3 12.7 18.2 12.8 11.7 14.7 I will have a higher status with a higher- 25.6 25.1 20.1 24.5 26.0 25.7 23.3 26.9 25.2 level degree (non-vocational) The job that I really want to do requires a 46.8 50.5 46.6 49.7 46.4 47.9 51.9 43.8 48.2 higher-level degree (non-vocational) I believe that higher education in the 43.0 41.3 45.8 42.1 42.9 41.0 44.6 41.9 42.5 future will allow me to make a better career and earn more Other 0.8 0.5 3.7 0.6 1.0 0.2 1.6 0.5 0.8 Source: Libanova and others (2016), based on SWTS-2015. Notes: City is defined here as a center of oblast, town refers to other cities and towns. *The question is “You chose to pursue a higher educational programme rather than a vocational programme. If someone that you trust had told you that you would have a better chance of finding a stable job at a higher wage if you pursued a vocational career, would you have reconsidered going through the vocational educational system?”. The sample includes young people who were stud ying at the time of survey and who expected to complete some higher educational institution. Occupational choice by young people is mainly influenced by their personal preferences, family and friends, rather than teachers or career guidance activities (table 2.4). Strikingly, nearly 14.0 percent of young individuals said that the choice of profession was random, and this is despite the fact that 48.7 percent of all young respondents attended events where information on various career options was disseminated and professional tests conducted. Yet, a positive sign is that the proportion of youth who made the choice of profession randomly is significantly lower among those covered by career guidance services compared to their peers: 7.6 percent and 21.1 percent, respectively.23 Table 2.4 Main influence in the selection of current profession by youth (%), 2015 Age group Gender Area of residence Total Female 15–19 20–24 25–29 Town Rural Male City Myself (by books, television, seen from the 42.1 39.4 39.6 39.6 40.7 36.9 40.0 43.8 40.1 side, other activities) Stories and examples of parents/ relatives 26.2 22.4 22.2 25.0 21.6 24.4 23.7 21.6 23.3 Random 9.7 16.0 16.3 12.6 16.4 17.1 14.5 11.8 14.5 Friends’ stories and examples 8.1 10.2 11.5 9.7 10.7 12.1 9.2 9.3 10.2 School teachers influence 5.7 5.5 3.5 5.7 3.7 3.9 4.5 5.8 4.7 Career guidance activities 4.1 4.1 3.8 4.5 3.5 2.7 5.5 3.7 4.0 Other 3.1 1.6 1.7 1.8 2.2 2.0 1.4 2.5 2.0 64 Stories and examples of wife/ husband/ 1.1 0.9 1.5 1.1 1.2 1.0 1.1 1.5 1.2 partner Source: Libanova and others (2016), Table A.12, based on SWTS-2015. Notes: City is defined here as a center of oblast, town refers to other cities and towns. The question is “What did influence to you understanding that you want to choose your current profession?”. Approximately 20 percent of young people reported that the profession they had obtained or were studying at the time of the survey did not match their preferences . The main reasons for a mismatch between the chosen occupation and the ‘dream’ profession were those related to poor employment and income prospects at that ‘dream’ profession. At the same time, 20 percent of individuals with a mismatch said it was because the choice of their current profession was made randomly (table 2.5). Poorly considered choice of study field and subsequent occupation is caused by insufficient attention by the government, educational institutions and employers to career guidance activities, especially at secondary general schools. As a consequence, young people, especially from less well-off households, have more complicated pathways to higher education and a stable and secure job.24 Table 2.5 Share of youth having/obtaining profession that does not match personal preferences by educational status and the reason for mismatch (%), 2015 Currently in Complete Stopped education Total education d before completion education % with profession that does not match 10.3 24.4 41.7 19.7 personal preferences Reasons for mismatch (multiple answers allowed) It’s difficult to find a job 11.9 33.2 23.0 28.7 The choice was made randomly 16.5 20.5 26.2 20.0 The wage level is low 9.7 22.0 15.8 19.4 Parents have insisted on other profession 15.8 8.9 2.2 9.9 I do not remember 6.8 10.5 10.4 9.8 Other 13.8 5.6 10.8 7.4 Study of this profession is not financially 5.7 3.5 8.7 4.1 accessible for me It has a negative reputation (and / or low 0.0 2.0 0.0 1.6 social status) It is too difficult for studying 3.3 1.1 0.0 1.5 The educational institutions where this 1.5 0.5 0.0 0.7 profession is studied are too far away Wife / husband has insisted on other 0.0 0.1 0.0 0.1 profession Source: Libanova and others (2016), based on SWTS-2015. Notes: The questions are “Does the profession that you have obtained/ are studying now matches your preferences?” and “What were the reasons why you have chosen a profession that didn’t match your preferences?”. Limited access to academic and career guidance counselling in secondary schools complicates the choice of study field and higher education institution for those young people aiming to enter a university. Ukraine does not have any national or large-scale programs on guidance counselling in secondary schools, so there is no available information on the coverage or quality of such programs. There are various nongovernmental organizations (NGOs) that aim to inform youth and their families in this process. For example, MOES recently partnered with the CEDOS think tank to develop a checklist25 with tips and recommendations for young people making decisions about university options. However, information allowing young people to compare programs, fields of study, and universities, such as university rankings or information on graduates’ employment, remains very limited. 65 Although professional counselling remains limited, it has been identified as an objective of inter- agency government policy, though with a focus on professional education. The MOES has identified professional counseling as a critical area for support in vocational education and has enshrined it in the concept of implementation of the state policy in the field of vocational education through 2027. Furthermore, the MOES and other key ministries have signed a joint memorandum on cooperation in the field of professional counseling. The Government’s Priority Action Plan for 2019 includes a number of steps to create conditions for identifying young people’s aptitudes and abilities, aimed at introducing modern mechanisms for youth employment and disseminating information among young people about effective employment opportunities available in Ukraine. However, for the time being, focus remains limited to vocational education, with less attention paid to general secondary or higher education. Higher educated youth are more likely to complete the transition from school to work, but this can be partly due to differences in the age at which young people would have completed the various education levels. Larger shares of youth with upper secondary and vocational degrees remaining in transition compared to their higher educated peers can be a sign of dissatisfaction with blue-collar jobs or a longer period in unemployment in search of suitable jobs (table 2.6).26 Table 2.6 Youth by stages of transition from school to work and educational attainment (%), 2015 Transition not yet Transited In transition started 2015 Change 2013- 2015 Change 2013- 2015 Change 2013- (%) 2015 (pp) (%) 2015 (pp) (%) 2015 (pp) Total 41.3 3.9 31.3 2.7 27.4 -6.6 By educational attainment (completed education only) Primary and lower secondary 35.8 n.a. 51.1 n.a. 13.1 n.a. Upper secondary 46.2 3.5 49.1 -1.6 4.7 -2.0 Vocational 62.6 -0.1 36.1 1.3 1.3 -1.5 Tertiary 69.7 -1.6 29.0 2.8 1.3 -1.2 Source: Libanova and others (2016), Table 6.1, based on SWTS-2015. Notes: Transited refers to a young person who is not in school and is currently employed in a stable job, in a satisfactory temporary job or satisfactory self-employment. In transition refers to a young person who is currently an active student (employed or unemployed), unemployed, employed in a temporary and non-satisfactory job or in non-satisfactory self- employment or inactive and not in education or training, with the aim of looking for work later. Transition not yet started refers to a youth who is currently still in school and inactive (inactive student), or inactive and not in education or training (inactive non-student), with no intention of looking for work. Tertiary education includes incomplete higher education, basic higher education, complete higher education, postgraduate and PhD studies. Although tertiary education is not a guarantee against unemployment, it decreases the risk of being unemployed on the path from school to work and reduces the duration of transition . Young nonstudents, who are in transition and have tertiary education, are much less likely to be unemployed and more likely to be inactive with the intention to work in the future than less educated individuals (figure 2.16). Moreover, average duration of transition to a first stable/satisfactory job is substantially shorter among youth with a tertiary degree compared to their peer with secondary general or vocational education: 5.5 months and 7.8 months, respectively.27. Figure 2.16 Youth in transition and not in education by subcategories and educational attainment (%), 2015 66 100% 90% 28.2 80% 41.3 38.7 46.5 41.1 38.2 70% 55.4 54.4 60% 50% 40% 30% 52.2 20% 44.6 45.6 50.1 44.5 44.4 39.7 39.3 10% 0% 2013 2015 2013 2015 2013 2015 2013 2015 Primary and lower Upper secondary Vocational Tertiary secondary Unemployed youth (non-student, relaxed definition) Youth in temporary employment (who want to change their job) Self-employed youth (who want to change their job) Inactive youth not in education or training with intention to work later Source: Libanova and others (2016), based on SWTS-2015. Notes: Tertiary education includes incomplete higher education, basic higher education, complete higher education, postgraduate and PhD studies. Conclusion Moving forward, it will be important for Ukraine to focus on aligning education with the new and evolving needs of the economy, particularly given the declining population. This will mean ensuring that secondary and tertiary education both deliver the appropriate cognitive, technical, and socioemotional skills needed for the growth industries of tomorrow. Tertiary education clearly provides better employment prospects and high economic returns, though this also reflects the high demand on the part of students and employers for higher education and the prevalence of higher education credentials in the labor market. There appears to be a growing gap between supply and demand of university graduates, leading to education-job mismatch. Despite this, vocational education remains an unpopular choice, though young people have little information or guidance available to them on how to select an occupation or field of study in higher education. Improving the linkages between secondary and tertiary education and the labor market are critical to ensure that human capital contributes fully to delivering Ukraine’s aspirations for growth. 67 Notes 1 Mullis and others (2012). 2 The low benchmark for mathematics performance represents some knowledge of whole numbers and decimals, operations, and basic graphs. Higher benchmarks emphasize application of math knowledge and ability to reason and make generalizations based on such knowledge (Mullis and others 2012). 3 PISA 2018 results are anticipated in December 2019. 4 Rindermann and Ceci (2009); Altinok, Angrist and Patrinos (2018). 5 Proficiency in literacy (reading proficiency) is defined by the OECD (2013) as “the ability to understand, evaluate, use and engage with written texts to participate in society, achieve one’s goals, and develop one’s knowledge and potential.” 6 Del Carpio and others (2017). 7 SSSU defines economically active population in Ukraine as ages 15-70. 8 Individual-level LFS data is not available for years after 2013. 9 Nikolaiev (2017). 10 Kupets (2010). 11 According to SSSU, the number of vacancies for craftsmen and machine operators was about 20,700, or 41 percent of the total stock of vacancies in the PES dataset as of the end of 2017. By the end of 2018 their number increased to 23,600. 12 The main dependent variable is the logarithm of yearly wage at the main job trimmed at the 1st and 99th percentiles. Alternative dependent variables used in a sensitivity analysis are earnings of employees at all jobs, and earnings of self-employed and employees from labor-related activities including entrepreneurial activity and freelancing. 13 Estimation results for alternative dependent variables. These include (a) earnings of employees at all jobs and (b) earnings of self-employed and employees from labor-related activities including entrepreneurial activity and freelancing. 14 Ponomarenko (2017). 15 Del Carpio and others (2017). 16 ILO (2012). 17 Kupets (2016). 18 Kupets (2016). 19 Kupets (2016). 20 Libanova and others (2016). 21 Libanova and others (2016). 22 Libanova and others (2014). 23 Libanova and others (2016). 24 Libanova and others (2016). 25 See URL link: https://cedos.org.ua/vybir 26 Libanova and others (2016). 27 Libanova and others (2016). 68 Chapter 3: Ensuring that All Students Learn in the New Ukrainian School In assessing performance of secondary education in Ukraine and the New Ukrainian School (NUS) reform agenda, it is instructive to assess other high-performing education systems in the world and examine the key drivers of educational performance. High-performing education systems have several features and approaches to education that provide a useful benchmark for analyzing Ukraine’s education system and the direction of reform under the NUS. At the same time, global evidence highlighted in the 2018 World Development Report on Education indicates that these areas also represent factors which can break down in education systems, preventing learning from happening or preventing key actors in the education system from improving learning (figure 3.1). Figure 3.1 Learning for all in the New Ukrainian School Ensuring readiness to learn Providing equal Aligning opportunities to incentives for prepare for higher teachers education Learning for all in the New Ukrainian School Assessing learning Spending wisely and monitoring for capable school quality networks Source: Authors’ elaboration Ensuring Readiness to Learn Highly effective school systems around the world meet the needs of students by ensuring their readiness to learn. Strong international evidence indicates that investments in young children pay dividends, and similarly that low parental investments, malnutrition, poor stimulation, and the harsh environments associated with poverty undermine early childhood learning. Deprivation in terms of nutrition, unhealthy environments, or lack of nurture by caregivers have long-lasting effects because they impair infants’ brain development. Without sufficient developmental foundations and adequate availability of early learning opportunities through preprimary education, many children, particularly from vulnerable groups, arrive at school unprepared to benefit fully from it.1 69 Early childhood investments efficiently produce skills that are relevant to a child’s future and increased equity. Learning is cumulative; skills acquired at an earlier stage facilitate skills formation in subsequent stages. The returns to early investments are the highest of those made over a life span, and the advantages conferred by these investments grow with time. An additional dollar invested in quality early childhood programs yields a return of between US$6–$17 based on data from low- and middle-income countries and depending on the preschool enrollment rate and discount rate.2 In terms of equity, early childhood investments are also critical to ameliorate the effects of poverty and other adverse conditions. Evidence shows that quality early learning opportunities can offset such effects and ensure that children from vulnerable groups are able to benefit from schooling later in life. Ukraine does not have a system for assessing learning outcomes or school readiness at the preschool level. However, the recent Ukrainian Center for Education Quality Assessment (UCEQA) monitoring study of primary school graduates confirms the importance of preschool education for primary school performance. Without a standardized assessment of preschool quality, such as those developed under the Measuring Early Learning Quality and Outcomes (MELQO)3 initiative, it is difficult to know to what extent preschool education in Ukraine is contributing to school readiness. However, a recent study of grade 4 graduates’ achievement in mathematics and reading helps to confirm the importance of preschool education for learning outcomes (figure 3.2). The study found that students who had attended formal preschool performed statistically significantly better on the mathematics assessment than those who did not. This is consistent with international studies that found a strong positive relationship between student performance across subjects and preschool enrollment rates. For example, PISA 2015 results showed that in most countries, even after taking socioeconomic status into account, students who had attended at least a year of preschool were still less likely to be low performers on the PISA mathematics assessment compared to those who had not.4 Figure 3.2 Preschool education impacts primary school performance 215 210 Mathematics score 205 200 195 190 185 More than two Did not go to Attended Prepared for years kindergarten preschool 1st grade with courses to my family prepare for 1st grade Source: UCEQA Grade 4 monitoring survey (2018) Ukraine has high rates of enrollment in preprimary education by international standards and has achieved gender parity. Most children are enrolled in public kindergartens, and the net enrollment rate for children ages 3–5 in public institutions was 75 percent in 2017.5 However, including private institutions, the coverage rate is higher: 87.5 percent for children age 3-5 and 96 percent for children of pre-primary age (ages 5-6/7), according to the MOES. Furthermore, boys and girls are equally likely to enroll, with nearly 70 perfect gender parity in terms of preschool enrollment rates. Though children can start preschool as early as age 1, preschool education becomes obligatory once the child reaches age 5. Preschool can be done either at home or in preschool education facilities. The majority attend public institutions, established and owned by local governments; only 1–2 percent of preschool places are in private institutions, which are mostly in urban areas. Considering children ages 3–5, Ukraine’s gross enrollment rate in preprimary education is comparable to that of the average in Europe and North America.6 However, access has been unequal, particularly in rural areas and for poor families, which creates initial inequities in opportunities to learn. In urban areas, the net enrollment rate for children ages 3–5 is 85 percent on average nationwide, compared to 58 percent in rural areas. The rural enrollment rate drops even further in some oblasts, such as Kharkiv (54 percent), Lviv (44 percent), and Ivano-Frankivsk (39 percent) (figure 3.3).7 According to the 2012 Multiple Indicator Cluster Survey (MICS), enrollment also varies significantly by household income. Only 30 percent of the poorest quintile families had their children enrolled in preprimary education, compared with 70 percent for the highest quintile. Access is even more limited for children with special educational needs (see Box 3.1 on efforts to expand inclusive preschool education). Figure 3.3 Urban and rural net enrollment rates for preschool children ages 3-5, by region (2017) 100 90 80 m. Kyiv Kiev 70 Cherkassy Khmelnytsky Mykolaiv 60 Kirovohrad Vinnytsia Chernivtsi Percent Sums Herson Zakarpattya Zhytomyr Poltava 50 Ukraine Ternopil Chernihiv Zaporizhia Volyn Dnipropetrovsk Kharkiv Odessa Rivne 40 Lviv Ivano-Frankivsk 30 20 10 0 Urban Rural Source: Authors’ analysis of SSSU data. Ukraine has made progress in recent years on establishing preschool education places, although there remains significant unmet demand, with over 40,000 children currently on waiting lists. The Government wants to create new places for children in preschools and to streamline procedures for licensing private kindergartens in an effort to continue expanding access while reducing the number of children on waiting lists. Oblasts are developing plans for how they intend to deal with waiting lists in each year. Although a sizeable waiting list remains, progress has been made: between 2017 and 2018, local authorities created more than 51,000 new places in preschool education institutions, according to the MOES. Another 30,500 additional places are expected to be created in 2019. The scarcity of enrollment capacity, particularly in urban centers, has created overcrowding as well as opportunities for nontransparent procedures for accessing preschool. This disproportionately benefit better-off families. In major cities in Ukraine, there are on average 117 children of preschool age enrolled per 100 places. Limited enrollment capacity and overcrowding not only threaten quality of 71 services, but they also create opportunities for nontransparent processes for children to access preschool education institutions. For example, school directors may seek unauthorized contributions from parents in exchange for their child’s entry into the institution, which creates further disparities in access on the basis of socioeconomic status of families.8 Although MOES recommends the creation of electronic systems to manage applications and waiting lists for preschools, these systems have not been universally implemented in part because local governments can decide whether or not to utilize an electronic system.9 Most preschool teachers in Ukraine have completed some higher education in teaching, either at pedagogical colleges or universities. This reflects the priority placed on specialized education for preschool teachers, as well as efforts to ensure access to initial preparation programs. This is not the case in other countries, including the United States where only 45 percent of teachers, in center-based settings with children ages 3–5, have completed a bachelor’s degree.10 A recent study of preschool stakeholders including teachers, methodologists, school directors, and officials found that while initial training programs offer strong theoretical foundations, programs often lacked practical training components. This is related to the fact that many teacher educators themselves do not have practical experience, since university professors tend to have academic backgrounds rather than teaching backgrounds. In-service training opportunities available to staff of preschool education institutions are limited and often disconnected from their needs. Preschool education teachers are required to receive in-service training support every five years. Teachers attend regional training institutes funded by local governments, but these opportunities vary significantly in terms of quality. Teachers and school directors interviewed for a recent study11 on preschool staff noted that these training opportunities provided them with a chance to learn from peers, helping to reduce burnout. However, teachers also noted that offerings were limited, and courses were not all relevant to teachers’ needs. Teachers also feel that training should be offered more frequently and that they would benefit from more opportunities to learn practical skills and tips from other teachers. Box 3.1 The road toward inclusive preschool education in Ukraine Ukraine is moving toward inclusive preschool education, further advanced by the 2017 Law on Education. The concept of inclusive education, with children with special educational needs learning alongside peers instead of in segregated institutions, gained traction in Ukraine in the 1990s and obtained further prominence after Ukraine ratified the Convention on the Rights of Persons with Disabilities in 2009. The 2017 Law on Education broadens the definition of inclusion as the provision of additional support in education to all children, not only those with disabilities. The 2017 law, for the first time, introduced the term “person with special educational needs” and envisioned an individual approach to providing additional services. The 2017 law defines children to be supported by inclusive education as all children who require additional permanent or temporary support. This broader definition includes those children affected by conflict as well as those from diverse ethnic groups, including Roma populations.12 Ukraine is now in the process of ‘deinstitutionalization’ by expanding the number of inclusive classrooms in preschools and other education institutions. MOES is developing Inclusive Education Resource Centers (IERCs) to replace the previous system of psychological-medical-pedagogical consultations, which assessed children with special educational needs. It is hoped that this reform will help to ensure that all children are assessed for special educational needs and that education institutions adopt an inclusive approach. The IERCs are expected to: • Assess the special needs of children and develop recommendations about how to support them in school • Provide psychological and pedagogical services to children with special needs 72 • Develop a register of all children assessed • Develop a catalogue of all services, rehabilitation centers, and professionals that provide services for children with special needs within a given territory • Provide methodological support to teachers, preschools, schools, and vocational schools on inclusive education topics. IERCs began their operation in 2018, and 561 have been created as of May 2019. An additional 98 IERCs are to be established by the oblasts as well. The automation system for the IERCs has been created as well, allowing parents or their guardians to receive some electronic services through the website, e.g. submit applications or receive invitations for general assessments. At the same time, the number of students with special educational needs who receive inclusive general secondary education has increased by 4.4 times between the 2015-16 school year and the 2018-19 school year, according to the MOES. The number of inclusive classes and schools offering such classes have also increased by 2-2.5 times over the same period. The budget for 2019 includes a large amount of funding for supporting inclusive education. The 2019 budget contained 504.5 million UAH (approx. USD 18.9 million) for the development of inclusive education, with the majority directed toward supporting children with special educational needs at schools. However, this was also the first year in which the Government directed funds toward inclusive education in kindergartens and vocational schools as well. Funds were also directed for establishing and equipping the IERCs.13 In April 2019, the procedure for organization of operations of inclusive groups in preschool education institutions was approved, and the Government plans to approve similar regulations in the near future for professional/vocational and higher education. Ensuring the success of these IERCs within a decentralized context will be key to fostering inclusive education more broadly in the education system. Although the number of inclusive classes and the number of children in these classes has increased, the increase has been uneven across the country. Furthermore, there has not yet been a corresponding decrease in enrollments in special education schools, possibly because many students previously stayed home rather than being enrolled in special boarding schools. MOES is currently working to develop a strategy on inclusive education development in order to help standardized approaches across the country. Further teacher training will also be important: Grade 1 teachers under the New Ukrainian School professional development modules have been trained in inclusive education and special educational needs of students, but expanding this training to higher levels as the NUS implementation continues will be important. Source: Ministry of Education and Science Aligning Incentives for Teachers Teachers are at the core of any effort to improve the quality of education, and the quality of an education system cannot exceed the quality of its teachers. Top-performing school systems consistently attract more capable people into the teaching profession, leading to more effective teachers. Effective teachers are those who combine deep content knowledge, high-quality practices, creativity and empathy to improve student learning today and students’ long-term readiness to learn.14 For many students, teachers are the adults with whom they will interact the most. Successful teachers are likely to be the first role models that young people encounter outside the home. They teach content, make learning fun, shape students’ attitudes, exemplify empathy, teach teamwork and respect, and build student confidence. Effective teachers prepare students for a world where they must interact with others, 73 adapt quickly to change, and where success will hinge on knowledge as well as attitudes and behavior. Helping young people develop these skills is a complex task, especially when many come from deprived backgrounds. Quality teaching requires consistent human engagement with students so that teachers can convey their deep knowledge to students and their conviction that all students can succeed. Teachers are essential to any effort to improve learning because the most effective interventions rely upon teachers. For instance, in a review of interventions across low- and middle-income countries, teacher- driven interventions such as structured pedagogy programs were found to have large impacts on student learning for both language and mathematics.15 Such programs target the teacher directly by supporting lesson planning and training to deliver new content and materials, sometimes including mentoring and feedback. In contrast, computer-assisted learning programs has had a more negligible effect. Furthermore, a successful teacher can make a major difference to a student’s learning trajectory. Going from a low-performing teacher to a high-performing teacher increases student learning dramatically. The effect has been measured from more than 0.2 standard deviations in Ecuador to more than 0.9 standard deviations in India. This is equivalent to multiple years of schooling.16 Effective teachers also have a substantial impact on the long-term wellbeing of students, affecting not only their academic achievement and how far they will study, but also their income once they enter the labor market. 17 In terms of equity, teaching can also be effective in offsetting learning deficits of disadvantaged students.18 High-performing education systems prioritize teaching by designing and implementing policies that attract high-ability individuals into the profession and that prepare, support, and motivate them to become effective teachers. A handful of countries such as Finland, Japan, and Singapore boast a cadre of highly successful teachers. In many other countries, including countries across the income distribution, teacher policies are either ineffective or lack internal consistency. For example, good teacher performance is often not recognized or rewarded, while unprepared and poorly trained teachers are expected to teach a complex curriculum, which even they have a weak grasp on. Teacher Workforce Analysis of teacher characteristics is complicated by the form of data collection and reporting. The structure of statistical forms sent by schools into the State Information System of Education (DISO) makes it difficult to compare and analyze teacher data comprehensively across two or more dimensions. The forms do not contain separate entries for each individual teacher, but rather data that are already aggregated on the basis of a restricted set of variables. This limits the analysis of the teacher workforce. The teacher workforce is predominantly female, and approximately 25 percent are of retirement age, with many continuing to work after retirement. Fifty-eight percent of pedagogical employees19 have more than 20 years of teaching experience, and about 25 percent of them are age 55 or older, according to DISO data. In 2017, the retirement age in Ukraine was 55 years for women, and the vast majority (84 percent) of teachers in Ukraine are women. (However, female teachers comprise nearly 100 percent of teachers at the primary school level, 88 percent of secondary school teachers, and 68 percent of school principals). Teachers were given the right to an early retirement based on having a minimum of 25 years of teaching experience. However, many teachers continue to work after retirement, largely due to the low wages. Fifteen percent of teachers nationwide are working while in retirement, but this share rises to 18 percent for teachers in grades 5–11, and it is as high as 25 percent in Kyiv oblast for teachers in grades 5– 11 (figure 3.4). Figure 3.4 More retired than new teachers: new vs. retired teachers in grades 5-11, by region 74 Share of retired teachers Share of new teachers 30% 25% 20% 15% 10% 5% 0% Dnipropetrovsk Ivano-Frankivsk Khmelnytsky Odessa Lugansk Kherson Poltava Kirovograd Donetsk Zhytomyr Lviv Cherkasy m. Kyiv Kharkiv Zakarpattya Volyn Zaporozhye Rivne Chernihiv Ternopil Chernivtsi Kiev Sumy Mykolaiv Vinnytsia Source: Authors’ analysis of DISO data. At the national level, the size of the teacher workforce (teacher quantity) is misaligned with the declining student-age population. This leads to low student-teacher ratios and an inefficient use of public funds, though Ukraine has made progress. Ukraine has a relatively low number of students per teacher, at only 9, compared with an average of 13 for OECD countries.20 This is mostly an effect of the unreformed school network, in which rural schools with mere dozens of students are extremely hard to close or consolidate into bigger units. Experts have recommended both changes to school network and the general optimization in the employment of teachers. Ukraine has made some progress in this area. Although there are no recent data on teachers from Crimea and the temporarily occupied territories of Donetsk and Luhansk oblasts, a comparison of numbers for all other regions of Ukraine can be made. From 2010 to 2017, the number of students increased 5.0 percent (because of higher fertility in the 2000s), while the number of teachers declined by 4.7 percent.21 The number of students per teacher accordingly increased from 8 to 9, indicating that some steps have been taken toward the consolidation of the school network and addressing the demographic needs of the country. Professional Status of Teachers The teaching profession is an attractive option when it pays well, provides an environment conducive to work, and offers learning and career advancement opportunities. All of these factors can be addressed through policy. In addition, job prestige—the social status gains from being in a particular profession versus another—matters but can be more challenging to influence than policy. Even so, better pay, conducive working environments, intellectual rigor on the job, learning and career advancement opportunities may add to job prestige.22 Available evidence in Ukraine shows that teacher job satisfaction affects students learning outcomes, even after controlling for the socioeconomic status of students. Students of teachers with high job satisfaction performed nearly 20 points higher than students of teachers who demonstrated low job satisfaction according to TIMSS 2011. Although this does not indicate a causal relationship, it does suggest 75 that teachers’ satisfaction in their job and professional career have a n impact on their motivation and how they approach their daily work with students and peers. This is also supported by international research on how teacher motivation and satisfaction affect their practices at school and in the classroom. Although most students in Ukraine interact with teachers who are generally satisfied with their profession, at least 33 percent of students are in classrooms where teachers are unsatisfied or frustrated with their job. Teacher job satisfaction tends to reflect the teacher’s views of their working conditions and surrounding environment, whereas satisfaction with the teaching profession is related to their personal motivations and goals in becoming a teacher in the first place.23 Between 30–40 percent of students in Ukraine are taught by teachers who have a low level of job satisfaction, which likely has an impact on their motivation, especially in the context of implementing the New Ukrainian School curriculum (figure 3.5). Figure 3.5 Teacher satisfaction with profession and job conditions in Ukraine I do important work I am content with profession as a teacher I am frustrated as a teacher I am not satisfied with my job at this school 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent of students taught by teachers with stated responses Source: Authors’ analysis of TIMSS 2011. In general, teachers in Ukraine have a low social status with limited options for career progression. In Ukraine, there are three different ranks of teachers based on number of years in service, above and beyond the status of new unranked teachers (see table 3.1). Compared to other countries, the salary progression of a teacher’s career in Ukraine is relatively small, with those at the top of the scale making only 30 percent more than a new teacher. By comparison, in OECD and EU countries, teachers at the top of the career ladder earn 70–80 percent more than teachers just entering the profession (figure 3.6).24 Figure 3.6 Teachers' salary progression: Ukraine vs. comparator countries 2.5 Ratio between starting and maximum base salary 2 1.5 1 0.5 0 Source: World Bank (2018c); OECD (2018a). 76 Table 3.1 Ukraine teacher rankings Rank Years of Experience Salary Progression Unranked 0–3 years in service Base salary Rank I 3–10 years in service 10% increase over base salary Rank II 10–20 years in service 20% increase over base salary Highest rank More than 20 years in service 30% increase over base salary Source: World Bank (2018c). The low popularity of the teaching profession has become more pronounced since 2016, when higher education institutions transitioned to their new list of fields and subject areas, distinguishing between classical and pedagogical fields. Since 2016, many universities introduced parallel programs in which the profession of a subject teacher in general secondary school was defined as a separate subject area. This resulted in the establishment of classical and pedagogical options within a given field, such as ‘chemistry’ and ‘secondary education in chemistry’. Data suggest that for a number of fields, such as chemistry, physics, geography and history, there are many more bachelor’s degree students enrolled in the classical field compared to the pedagogical field (a four-fold difference for some fields).25 This demonstrates the relatively low attractiveness of the teacher profession. Though teachers’ salaries in Ukraine are low in absolute terms, they are on par with other countries when compared domestically against other professionals with equivalent levels of education. Teachers in Ukraine earn considerably less than peers in other EU and OECD countries in USD equivalent, even adjusting for purchasing power parity. However, this is not a useful comparison given that such countries have higher overall salaries and higher levels of income, allowing them to afford higher public sector wages. The more meaningful comparison is by comparing teachers’ salaries to other tertiary- educated workers in Ukraine and to GDP per capita. Teachers in upper secondary general education in Ukraine earn on average about 75 percent of the earnings of other tertiary-educated workers in the country, and 104 percent of GDP per capita (figure 3.7).26 This is roughly on par with other countries in the region: teachers, who tend to work fewer hours and enjoy more job security, usually receive 10-30 percent less than peers with similar education and experience.27 The figures are lower for teachers in primary education: 66 percent of the earnings of tertiary-educated workers and 91 percent of GDP per capita. Ukraine has taken steps recently to increase salaries in an effort to improve the social status of the profession. The 2017 Law on Education promised to increase the starting salary of teachers to four times the living wage by 2023. In January 2018, teacher salaries rose by 25 percent relative to 2017. No additional increase was planned or implemented in 2019, although further increases have been announced. However, the language of the law was unclear in specifying whether the fourfold increase applied to the base salary or take-home salary. Previous analysis by the World Bank showed that if the base salary becomes four times living wage, it would mean an almost 100 percent increase in teachers’ salaries. This would lead Ukrainian teachers to be among the highest paid teachers relative to OECD countries, while total education spending would increase to nearly 9 percent of GDP. This magnitude of an increase would be financially unsustainable.28 Figure 3.7 Teachers’ Salaries 77 Annual statutory teachers' salaries after Comparisons of teachers' salaries 15 years of experience 2 Teachers' salaries relative to comparison 1.8 90 1.6 USD equivalent (PPP, in thsd.) 80 1.4 70 1.2 60 1 0.8 50 0.6 40 group 0.4 30 0.2 20 0 Denmark Slovenia Austria Israel Germany Poland USA Hungary Greece New Zealand Ukraine Turkey Norway Slovak Rep. Portugal Ireland 10 0 Denmark Poland Israel Slovenia Germany Hungary Austria Greece New Zealand Ukraine Turkey USA Slovak Rep. Norway Ireland Portugal GDP per capita Earnings of Tertiary Educated Workers Source: Authors’ analysis using SSSU, OECD, and WDI data. Notes: Panel 1 (left) contains data for upper secondary general education teachers’ salaries in OECD countries (2017) and Ukraine (2016), expressed in equivalent USD converted using PPP for private consumption. In Panel 2 (right), all data is from 2016 and also for upper secondary general education teachers. The standard workload (stavka) system, used for governing and organizing teachers’ work in Ukraine, allocates time inefficiently and results in nontransparent compensation. Teachers’ work in Ukraine is organized and compensated on the basis of a standard teaching load (stavka), which comprises 18 hours. The base salary is set to one stavka. Teachers can work a maximum workload of 36 hours per week (two stavkas), though roughly half of these hours are spent teaching. The stavka system attaches teachers’ base salaries to teaching hours (time spent delivering lessons per week) and does not regulate or compensate teachers for nonteaching tasks, such as lesson preparation, formative student evaluation, or working with colleagues and parents. Teachers’ total salaries are composed of the base salary plus mark - ups for years in service, occupational prestige, and other markups for class supervision, oversight of classroom workshops, checking of notebooks, and other such tasks.29 The stavka workload system and piece-meal form of compensation means that nearly 50 percent of teachers’ take-home pay can come from various top-ups. This creates incentives for abuse while devaluing the work of professional teachers. Only 53–56 percent of take-home salary is from the base salary. As of January 2018, a teacher who was just starting in the profession earned on average 6,672 UAH per month (around US$ 238). On the other hand, the take-home salary is 12 percent, 19 percent and 28 percent higher than this amount for rank I, rank II, and highest rank teachers, respectively (figure 3.8). Even though teachers can work more than one stavka, most teachers do not have the maximum permissible workload of 36 teaching hours per week. For teaching-intensive subjects such as Ukrainian language, school principals commonly split the available teaching hours among several teachers.30 This arrangement makes teachers vulnerable and dependent on the goodwill of the school principal, which in effect promotes nonformal evaluation criteria and nonprofessional decisions in scheduling the teaching hours of staff.31 Additionally, this mechanism for organizing and compensating teachers in effect treats teaching as a series of ‘piece-meal’ tasks, equating teachers with production line workers rather than professional educators responsible for complex tasks, which would only become more complex in the context of the New Ukrainian School curriculum and associated reforms. Figure 3.8 Components of Teachers' Take-Home Salaries in Ukraine, by Teacher Rank (2018) 78 10,000 9,000 Other markups (class supervision, 8,000 checking of notebooks, etc.) Monthly salary (UAH) 7,000 Annual vacation (portion) 6,000 Bona fide work 5,000 Occupational prestige 4,000 3,000 Years in Service 2,000 Base salary (18 hours/week) 1,000 - No rank Rank I Rank II Highest Rank Source: World Bank (2018c). Teacher Quality: Training and Certification There is recognition that the system of teacher professional development needs to be modernized in line with the vision under the New Ukrainian School (NUS). Previously, teachers were subject to assessment and attestation once in five years, which was provided by 25 in-service teacher training institutions (ITTIs) in Ukraine. The new law makes this voluntary, and teachers will be able to accumulate 150 hours of in-service training during a period of 5 years. Accurate information about the quality of training provided by ITTIs does not exist, although there is wide recognition that the quality of teacher training programs varies considerably (box 3.2).32 The network of ITTIs was inherited, and it cannot keep up with demand. ITTIs are overloaded, with hundreds of teachers coming for training every week. In any case, preservice teacher training does not meet the needs and current reality in schools, where the NUS is being rolled out. Box 3.2 Teacher education in Ukraine Pedagogical colleges provide training programs for teachers in preschool education, primary school education, music, arts, technology, physical training, and vocational education. Pedagogical colleges award both the junior specialist’s diploma (or junior bachelor’s diploma) and the bachelor’s degree. Pedagogical universities train teachers for lower and upper secondary schools, and colleges. However, many pedagogical universities along with the pedagogical colleges also provide training programs for preschool and primary school teachers. Pedagogical universities award bachelor’s and master’s degrees (and formerly specialist’s degrees). Classical universities provide training for teachers within training programs titled ‘education.’ Additional pedagogical training includes training in psychology, pedagogy, methods of teaching, information technology and teaching practice. Source: MOES and EACEA The 2017 Law on Education introduces a dedicated mechanism of voluntary certification for teachers, linked to a salary increase, but it remains to be seen how this will fit into the broader teacher policy framework. Article 51 of the Law on Education describes the procedure for voluntary teacher certification, and the resolution of the Cabinet of Ministers on approval of the regulation on teacher 79 certification (dated December 2018) explains the types of assessments and modalities for certification.33 Applicants who succeed in passing the evaluations become ‘certified’ and will move up one qualification category and receive an extra 20 percent to their salary. Moreover, when certified, teachers will no longer have to go through the usual attestation exercise that is currently mandatory every five years, though this change remains to be mandated and enforced. Certification will entail assessment of a teacher’s general professional competencies, which includes content knowledge and pedagogical knowledge, as well as a teacher’s practical in-class teaching competencies. A teacher’s general competencies will be conducted in specialized testing premises at first, though it will eventually be assessed through online evaluations (ready by late 2019). A teacher’s in-class teaching competencies will be evaluated by two experts through a visit to the teacher in school. In total, certification will involve three assessments: an in-class observation performed by SSEQ, a self-assessment through the creation of a teacher portfolio, and an assessment of professional knowledge and skills through independent testing carried out by UCEQA.34 MOES is piloting the certification mechanism for 859 primary school teachers starting in 2019. So far certification is voluntary, and the objective of MOES is to identify those teachers who are already applying the new approaches that school reform requires. That is, at this stage it is not about spreading new approaches among teachers but to find those who already use them. On January 15, 2019, MOES announced the commencement of registration for those wishing to undergo a certification. Registration was open until February 1, 2019, after which the number of potential applicants for certification was divided in proportion to the number of primary school classes in each oblast,35 with MOES taking into account the distribution of teachers’ stavkas rather than head count of teachers. During the first week Kyiv, Volynska and Zaporizhzhia oblasts had already reached their limits, according to UCEQA. This could mean that demand for certification is higher in these regions of the country, but this is not yet clear. Expert assessment of professional competencies by SSEQ was completed by May 2019; the second stage will start in September 2019. It is expected that other categories of teachers will be able to get certified in 2020 and 2021. Teacher certification should be part of a systematic approach to the teaching profession that takes into consideration the whole continuum of teacher education and development. Teacher certification (or licensing, or licensure, as it is sometimes called) is designed to raise the quality of those entering, or already in, the teaching profession and to maintain, manage, and continually update that quality throughout the teacher’s working life. The certification or the recertification of active teachers is int roduced into systems when a quantitative ‘jump up’ in quality is desired and/or a new status is required. Ideally, this certification should not be a stand-alone option, seen as a simple one-time training program for upgrading skills. Rather, it should be part of a systematic strategy that takes into consideration the whole continuum of teacher education and development, including high quality preservice teacher education, induction, structured but flexible continuous professional development (CPD), and a motivational career path which links CPD to promotion. Teacher licensing underlies all these steps by regulating, controlling, and safeguarding teacher quality. Accreditation of training institutions is also a key component of a modern teacher licensing system. Finally, underlying the entire system should be clear, recognized, and agreed upon standards and competences for all teachers. An approach to certification will be adopted in the New Ukrainian School reform. This will include a process by which teachers volunteer to enter the licensing process, while offering certain advantages for teachers who may truly be interested and motivated to attain licenses but may confront certain challenges that need to be managed. Certification is meant to identify teachers who have the requisite competences to teach in a new kind of school. It is unclear whether there are currently enough teachers in the system who can do this or would be willing to do this. It is quite possible that the kind of motivated, competent teacher envisioned by MOES is looking for may have already self-selected out of the school system. The salary of a teacher is modest, the working conditions uninspiring, and the system is onerously bureaucratic. So much so that very few enter the teaching profession. It is true that some of those who remain in the system may 80 be interested to upgrade, and perhaps the possibility of a 20 percent increase in salary will incentivize them, but participation in licensing is expected to be voluntary. If 20 percent of teachers36 choose to go through certification, it is not at all clear that this number of newly certified teachers will make a significant enough impact toward promoting the NUS reform within schools, if 80 percent remain working in traditional ways. There is also the issue of self-selection bias at play: those teachers who choose to upgrade are those the Ministry want to be NUS teachers, but what happens with the rest will be important for the success of the NUS reform. In terms of the quality of teaching and teachers, it should be clear that both must evolve to ensure the transition from the post-Soviet education system to the New Ukrainian School model. This inevitably entails changes in teacher profiles: teachers must be ready to work differently, in line with the new NUS requirements, to be effective in this new paradigm. MOES has chosen to lead teacher reform through a combination of professional development and teacher certification, that is, the certification of the stock of existing practicing teachers so that they are in line with the NUS reform. Although the certification of new teachers is also a concern, certification of currently practicing teachers is a logical first-step choice because few new teachers are needed and relatively few are in fact entering the system.37 However, improving the capacity of teachers to deliver the new competency-based curriculum requires a comprehensive and coordinated approach that harmonizes preservice and in-service teacher education and professional development (PRESET AND INSET). Current international best practice suggests that doing this in a harmonized way requires a series of instruments and practices being put into place. For instance: • A teacher competency framework that spells out what teachers should be able to do in several domains (planning teaching, creating conducive learning environments, providing differentiated instruction, teaching the NUS curriculum, assessing learning, and so on) and at different levels of competence (novice, proficient, expert). By defining these abilities, this index of competences serves to define a teacher’s proficiencies across their career. • It is on the basis of this framework and index that preservice and in-service professional development (PD) modules can be developed to be in line with one another (though at different levels). • Good practice in preservice teacher education includes a set of practicums so that students can progressively learn how to teach in the classroom. It should also be followed by an induction period during which time there is a reduced teaching load and the new teacher has a mentor, usually an experienced teacher, who can coach the new teacher in the first year(s). • Best practice in in-service PD suggests that the PD must be content-specific, taught mostly in- school, and be relevant to the daily needs of the teacher. The evidence on what works for INSET is relatively clear, even if in practice the efforts to improve the quality and impact of teachers through professional development varies widely.38 A recent meta- analysis,39 essentially a study of studies, sheds light on what is working and identifies certain characteristics of INSET programs that have been most effective at improving student learning, described briefly below. Teacher licensing could include many of these features. Overarching Aspects. It is important to create incentives for teachers to learn the new competence-based NUS program. As the meta-analysis suggests, professional development programs linked to career advancement, those for instance that give points toward promotion or to a salary increase, were associated with improvements in student achievement of 14 percentage points. Therefore, the MOES decision to award career advancement as well as salary increases to teachers who have successfully finished the teacher licensing is a judicious policy choice. As the teacher licensing program advances in the next years, it may be necessary however to revisit this formula.40 81 Delivery of Professional Development. To date, the NUS professional development (PD) program has been delivered via blended learning, that is, a combination of face-to-face and online learning. The research evidence suggests that professional development held in specialized centers, like universities or teacher training colleges, leads to greater effectiveness. However, this point requires careful consideration. Very often, on its own, this kind of professional development is ineffective. When used carefully, it does offer some benefits: it allows teachers to get out of the confines of school and provides them some space to reflect on their practice without principals, other teachers, or the pressures of everyday obligations in school, weighing upon them. It is in this way that it could be beneficial, especially when combined with on-site, in-school professional development (see box 3.3). Box 3.3 EdEra support for NUS professional development of teachers The online portion of the NUS has been developed and implemented with the support of a local online education platform known as EdEra (www.ed-era.com). This platform is similar to Coursera, EdX, and other platforms that offer online courses to enable flexible and free access to learning materials. EdEra, in partnership with MOES, is currently offering an online course for primary school teachers who need to upgrade their qualifications. This started in 2016 and has garnered high interest, with 7000 –9000 online comments from participants at launch. The distance course41 aims to introduce teachers to the new State Standard of Elementary Education and competency-based training methods. This is one of the mandatory stages for the improvement of the qualifications of teachers who will teach grades 1–4 starting in September 2018. The course consists of six modules. Among the topics is the State Standard of Elementary Education, methods of competency-based training, integrated learning, neuropsychology and the science of learning, and inclusive education. Source: MOES Any center-based professional development should ideally be followed up with on-site, in-school professional development. In South Africa, teacher coaching has been one way to do this: it produces larger and more cost-effective gains than traditional center-based teacher professional development alone.42 One way to link center-based learning and/or distance learning to what teachers need to apply in school is through a contract. A contract can be written that stipulates the off-site professional obligations to be learned. Teachers can then follow through with the terms of the contract by applying any completed professional development in school. All teachers sign these contracts and are held accountable to them in their performance review. Distance Learning. There are many benefits to distance learning: it can help provide professional development to many teachers who are geographically dispersed. It can also allow teachers a flexibility in their learning: they may study when it works conveniently into their professional and personal schedule. However, a significant challenge arises with online learning. In comparison to traditional classes, research43 suggests there are considerable attrition rates with this form of learning. While figures may vary, typically between 40 percent to 80 percent of online students drop out of online classes. This is because the online format of these courses brings with it several social, technological, and motivational issues in both learners and the faculty responsible for creating and facilitating the courses. Even if the format is slightly different, lessons from studies on retention in massive open online courses (MOOCs) are relevant, such as the need for frequent engagement with instructors as well as other learners and the importance of pairing online participation with off-line physical interactions.44 82 Creating Professional Learning Communities. To improve the chances that this blended learning is effective, more effort could be made to develop learning communities. These can take many forms but could, above all, be designed to suit teachers’ needs. There is no one-size-fits-all solution. Possibilities include in-school collaboration, teacher coaching, or school-to-school collaboration. • In-school collaboration. Over the last two decades, in-school collaboration, that is, collaboration between teachers, has been leveraged in school systems as diverse as Canada, China, Singapore, Sweden, the United Kingdom, and the United States. Recent research has demonstrated how positive this policy option can be. Results from one of the most cited reviews45 indicate that students have higher achievement in mathematics and reading when they attend schools where there is higher levels of teacher collaboration. This means that there is a direct correlation between teachers having opportunities to collaborate on issues related to curriculum, instruction, and professional development, and their students achieving academic success. • Coaching. Teacher coaching can be carried out in many ways. Schools could create teacher learning circles and/or identify their own teacher coach. Visiting coaches and teaching assistants have also been shown to be helpful, too.46 • School-to-school collaboration. School-to-school collaboration can take many forms. These forms can be used as mechanisms to promote various, sometimes overlapping, objectives, though one important use is in the pursuit of school improvement, when high- and low-achieving schools are matched together. Several reviews have demonstrated the impact of such collaborations. High- performing schools in Ukraine could be matched with lower-performing schools to learn the New Ukrainian School model together. This final set of points is important to consider if teacher licensing is to have any impact. Teachers who do become licensed are expected to be resource people in their schools; they are expected to share best practices. In-school collaboration and coaching could be the mechanism through which this knowledge transfer can take place. Between-school collaboration would also make sense. Better schools, those that are more in line with the NUS reform (perhaps those in cities), could be matched with weaker schools in the regions so that they can work collaboratively and improve together. Hub schools in Ukraine are an opportunity to deliver high-quality schooling for students while also creating a learning organization for teachers and staff. In an ideal system, teachers, as professionals, remain motivated, competent, and well supported. The report Beyond PD: Teacher Professional Learning in High-Performing Systems47 looks at how Shanghai, British Columbia, Singapore, and Hong Kong offer professional learning to their teachers. These four high-performing systems all score near the top in mathematics, reading and science on the Programme for International Student Assessment (PISA). Although these systems differ in many ways, central to them all is collaborative professional learning that has become part of the daily lives of teachers and school leaders. In these systems, teachers engage in collaborative activities such as peer observation, lesson-planning, team-teaching, and mentoring, which provide them with opportunities to interact with more experienced and effective colleagues, allows them to experiment with new instructional approaches, and better understand policies and practices, all of which can inform and improve in turn their teaching practice. Ukraine’s hub school program could be seen in this light. Not only will the schools become modern establishments for students, they could also be learning organization which encourage and promote NUS ideals. Spending Wisely for Capable School Networks The need to adapt a general secondary network to the demographic reality has long been recognized in Ukraine. Demographic changes in fertility rate and migration have led to a steep drop in the general 83 secondary education student population in the past three decades, from around 7 million students in the 1990s to around 4 million students at the current time. While some urban student populations have increased as a result of rural to urban migration, many rural areas and small towns have faced dramatic reductions in population. Ukraine’s school network meanwhile has adjusted in size and form very slowly. Communities tend to hold on to their schools even as people leave, and school and class size become lower and lower. The average rural class size in Ukraine today is about 13 students, but there are quite a few classes that have fewer than 5 students. Year after year, as young families with children move away to larger towns and cities or emigrate to other countries in search of better livelihoods, there are fewer school age children to attend these rural schools. These schools were originally constructed with uniformly sized large classrooms to hold thirty or more children. As these classrooms become emptier every year, they do not do so in any systematic or orderly manner. Holes start to form when there are schools with zero students in a particular grade because there is not a single child of that age grade category in the settlement. Or there might be only one or two children. Sometimes a school might unofficially form a multi-grade classroom; sometimes there might be a single classroom of four students. However, decisions over school networks in Ukraine are the responsibility of local governments (referred to here as districts). The maps in figure 3.9 are based on a computation of the number of classrooms within the district boundaries shown. Panel (a) shows the location of 35,840 classes with less than 13 students, which covers almost the entire country. In the middle panel (b), one can see more districts in yellow, these are showing classrooms of less than 5 students—a total of 9,781 such classrooms existed in Ukraine in 2018. In panel (c), one can observe a handful of yellow marked districts with more than 200 classrooms of zero students, of which there were a total of 2,891 in the entire country. Unless effective and decisive policy action is taken to address the school network, school aged populations in rural areas will continue to dwindle over time. Figure 3.9 Snapshot of school network in Ukraine that shows a glimpse of the future Source: DISO (2018) Statistics for school data; Map from United Nations Humanitarian Data Exchange of Administration Level 2. The Government of Ukraine has been aware of the problems posed due to increasingly inefficient school networks, but serious attempts to solve the problem have been lacking. Small schools and small class sizes lead to deficient learning environments from a pedagogical point of view, and constitute a tremendous fiscal strain on the government, as the same amount of education expenditures generate ever lower levels of public service delivery. According to Herczyńki (2017), “the present highly undesirable state of local school networks in Ukraine is the result of 25 years of confused responsibilities in the education sector and of poor sectoral and budget management.” In summary, such evidence demonstrates that decisions on budgets are made by local councils that do not face hard budget constraints, and legislative norms are dictated to local village councils who have the responsibility of closing schools, a decision which they almost never make. These factors have perpetuated the inefficiencies in the school network. A deep reform of the fiscal regime in Ukraine was introduced in December 2014, which implemented modern rules of revenue allocation with attendant implications regarding accountability across levels 84 of government.48 However, for the next three years, the per-student subvention formula was used as a de facto continuation of the financing system that had been in place since 2000. The budget law since 2014 in Ukraine requires the subvention formula to be proposed by the Ministry of Education and Science (MOES), reviewed by the Ministry of Finance (MOF) and approved each year by the Parliament. The subvention forms a substantive part of government spending on education. The total amount of the subvention in the 2019 budget is Ukrainian hryvnia 69,623,581,400, equivalent to about USD 2.6 billion (at the official exchange rate), or 1 percent of Ukrainian GDP. The education subvention is a transfer to local governments for the payment of pedagogical staff in schools, as well as teachers and pedagogical support staff, such as psychologists or psycho-pedagogues. The subvention was not originally meant to pay for payment of nonteaching staff, for capital or maintenance expenditures or for other levels of education such as preschool. The local government was responsible for all other education expenditure than the pedagogical salaries. Since 2019 there has been an additional subvention for salary payment for staff of inclusive education centers. The formula in place since 2018 is essentially a per-student allocation mechanism with a standard allocation combined with modifying coefficients.49 It is useful and instructive to look at the base calculation of the formula allocation: () 1 1 0 = × 1−4 × 18 × 0 = Normative allocation of subvention for 1 week for 1 student in the base case = Normative class size for Metropolitan Kyiv/Oblast capitals, currently 27 1−4 = Number of lessons to be provided in the Teaching Plan for Grades 1-4 1 18 × = Per lesson teacher salary assuming 18 lessons per week The formula calculation for the base amount per student per week takes the total number of lessons to be provided for grades 1–4 and multiplies it by the cost of each lesson, under the assumption that the standard teacher work load is 18 lessons per week. The teacher salary (TS) in the formula is taken to be a typical teacher’s salary defined in a particular way.50 The base amount 0 is actually a minimum amount per week. The actual subvention is this minimum amount multiplied by the number of weeks in the month for the monthly transfer to local government accounts, and then multiplied by the number of students enrolled, and then multiplied by a series of coefficients greater than 1. These coefficients reflect the higher cost of teaching compared to the base or minimum case. For example, grades 5–9 and 10–11 envisage more teaching hours. Boarding school students need both teaching and accommodations. Since 2018, however, the formula has included a crucial change with potential to improve efficiency going forward. The subvention to local governments, from its introduction in 2014 and for the next three years, has effectively maintained its previous allocation mechanism.51 Though this method had the benefits of per capita allocation, it had some perverse incentives that exacerbated rather than ameliorated the problem of fiscally inefficient and pedagogically deficient school networks. In the period 2001 –2014, and the three following years of de facto identical allocation, the additional coefficient for rural areas depended on the actual average class size—the less the value, the higher the coefficient. The idea was to support rural areas which faced a higher per student cost of providing educational services. The size of the coefficient kept gradually increasing over the years, reflecting the decreasing size of classes in rural areas. Thus a perverse effect was generated; a local school with a bigger class size received less funds. Combining this 85 perverse effect with the prevailing context of low local accountability prior to the decentralization reforms, this created a ‘perfect storm’ of conditions for the deterioration of school networks. The effects of the 2018 formula subvention can be seen in rural locations. As seen in Equation (1) above, the formula with coefficients does not depend on actual class size. Rural locations currently take into consideration the share of rural population and density of school-age children per square kilometer— factors that can be considered as a given for a local government. The changed formula provides a positive incentive for network optimization because of inbuilt incentives in the formula, as well as certain other policy issues that have become salient, which have been generated by the changed formula. The new formula creates a wedge between the actual and desired school network, creating a powerful economic incentive for local governments. The coefficients in the new formula introduced desirable class- size goals (though not school-size goals, which may become a consideration in future years). Given the various considerations such as rural population density that alter the required unit cost per student, for every spending unit the formula includes what can be considered to be a target class size. Currently, for rural areas on average this class size is 13 students. The spending unit receives a subvention based on a computation that uses the desired or targeted number of students. If the local government has an actual average class size that is lower than its target, for instance because it has too many small schools, the actual expenditure will have to come from the local government’s own resources, outside of the subvention from the federal government. Under these conditions, a local government can be said to be in a deficit state with regard to the subvention funding. On the other hand, if a local government has optimized its network beyond the level targeted, it can be said to be in a surplus state with regard to the subvention. The introduction of the possibilities of a deficit and a surplus with the subvention financing creates a powerful economic incentive for the local government to get its act in order regarding network optimization. Local governments also have flexibility in reallocating resources across budget years, meaning that they would benefit from their savings. The flexibility of reallocating resources generates a strong incentive for local governments to save. Consolidating the school network is one example that the subvention can lead to productive uses and savings over time. If there are unused funds on the account of the spending unit by the end of the budget period (calendar year) and the spending unit does not have any debts in salary payments, the funds are kept on the account and can be used next year within the account of subvention designation. This provides material support of the learning process, including: procurement of school buses; equipment (computers, projectors) and appliances for hub and optimized schools, exact and natural science rooms, and primary school classes; procurement of textbooks and school connection to internet. Education spending tends to be the biggest category of expenditure for local governments in Ukraine. There are many requirements such as provision of preschool services that local governments have to provide from own resources. Fungibility of local spending across subvention covered expenses and those not covered by the subvention implies a beneficial effect for all educational services and indeed the overall functioning of the local government. The formula combined with the flexibility in reallocation of resources across local governments within an oblast creates additional incentives for efficiency. This flexibility has existed even before the 2018 change and becomes important only in the context of another policy issue: the pending implementation of the New Ukrainian School (NUS) at the high school level (grade 10–12, with extension to the 12th year of high school). Ongoing policy discussions are geared toward a decision for only oblasts and large cities of oblast significance to administer a network of high schools, so that local governments will be focused on preschool and grades 1–9. This mechanism will reduce a major source of school network dysfunction: the presence of a large number of small high schools, where the quality effects of small size are most pernicious as students in small schools lack choices attuned to their preferences or that properly prepare them for higher education. In the context of zero additional resources to an oblast, the flexibility produces an unwelcome choice: to take money away from local governments and reallocate those funds to 86 ones that are in deficit. However, this is clearly inadvisable because it establishes a perverse incentive and introduces a soft-budget constraint. However, if additional resources were to be made available to an oblast, for instance, to enable the implementation of the NUS, this situation would change. Oblasts would be able to use moral suasion to nudge the local governments within their jurisdictions to migrate from the deficit side to the surplus side or at least to break even. Often, the political solutions found by one local government are of relevance to other local governments, and oblast administrations can support capacity development with local government to emulate effective solutions.52 Finally, the annual review mechanism is an important feature of the formula allowing for regular revision and correction. The law that established the formula requires MOES and MOF to revise the coefficients every year; the coefficients were altered between 2018 and 2019. The built-in feedback mechanism is a very useful feature because changes can be made based on the analysis of results. The information collection mechanism is now much improved, though the collected data are not so far analyzed in a systematic manner. Systematic data collection can be utilized to generate indicators of network efficiency and network capability. These indicators can be used in a systematic revision of the formula based on empirical analysis of the effects of policy actions taken by local governments. Box 3.4 An issue beyond school financing: Education quality is a transport sector issue The incentive effects explained above can only deal with the political economy constraints that prevent effective and efficient school networks. The political economy constraints are often the binding constraints, for instance, when viable transportation options are available for students from outlying villages and small towns to be taken to a bigger school by school bus. Indeed, with prudent fiscal management, local governments can take care of the outlays required for the purchase of school buses through the savings generated through optimization. However, a big outstanding problem is the technical feasibility of transporting students due to the precarious condition of the small roads that typically connect communities. The World Bank (2018b) report Strategy for Prioritization of Investments, Funding and Modernization of Ukraine’s Road Sector reports a deterioration from 5 percent to 17 percent between 2011 to 2016 of all Ukraine roads deemed to be in poor condition. The number in 2019 and especially for loads linking rural or semi-urban communities is probably higher. Another measure of the quality of roads are accidents and deaths, with poor road surface conditions being a key determinant of safety. The report mentions loss due to accidents to the order of US$4–5 billion, which is incidentally about twice the total amount spent by the government in education subventions. While the federal government and oblast administrations prioritize the main national highways and roads that connect big cities within the oblast, and city administrations take care of internal city roads, the ‘in -between’ roads appear to be neglected. Revised formula financing combined with oblast level support as noted above will go a long way in jurisdictions where roads are passable. For localities where roads are terrible, it is hoped that the recently introduced Roads Fund will address the issue of feasible school transport. Source: World Bank (2018b) Based on an in-depth analysis of the first year of implementation of the revised funding formula, there is strong evidence that the natural incentives to move away from high deficits are sufficiently powerful. Figure 3.10 shows an excerpt of the data from local governments with up to 500 stavkas or full- time teacher appointments. The horizontal axis shows the ideal or expected number of stavkas, and the vertical axis shows the actual number of stavkas. The area of the dots represents the magnitude of surplus or deficit faced by the local government. The local governments represented by the red dots in the upper portion of the graphs face a pressure to move toward the diagonal line by optimizing their network, for example, by closing down small schools, and arranging for the remaining students to study in bigger, consolidated schools. 87 Figure 3.10 Natural incentive for movement away from high deficits from the subvention formula Source: Authors’ analysis of DISO statistics for school data and Ministry of Finance subvention data. There is clear negative internal pressure to find resources to pay teachers who are no longer supported by the federal subvention. This is combined with the positive demonstrational pressure of those local governments that have managed to earn a surplus from the subvention because they have optimized their networks beyond the standard expected by the formula for the current year. However, these ‘surplus’ spending units also face the risk of moving into deficit territory if the formula is revised upwards in a subsequent year, for instance, the standard of 13 students per class could be moved to 15 or 17. The surplus units have the advantage of being able to spend resources toward enhancing the quality of education, by paying and motivating the teachers better, or adding pedagogical staff, acquiring didactic equipment or internet connectivity, and so on. Local governments can also use the resources to enhance access to preschools. Data from 2018, the first year of implementation of the incentivized formula, was also analyzed to investigate preconditions for further consolidation, considering one measure of the quality of the learning environment. The local governments were divided into two groups: a group shown in figure 3.11 as cyan-colored dots, representing districts of at least 5 schools with classes of less than 5 students; another group shown as light green-colored dots, representing districts of less than 5 such schools. The graphs examine the relationship between the amount of deficit/surplus, the average quality of ICT facilities (based on an index of number of computers and internet connectivity), and the presence or absence of schools with small classes as defined with the arbitrary cut-off of five classes. This analysis suggests that the formula does not clearly designate winners and losers from the point of view of small schools, which may help to increase the likelihood of impact and avoid political opposition. Figure 3.11 shows the two groups of districts plotted separately and together in a superimposed map. Even though the number of cyan-colored dots on the right side is lower, there are still cyan-colored districts on the surplus side. Light-green-colored school districts appear to be similar in number on both left and sides of the vertical axis which separates deficit and surplus school districts. From a political economy consideration, this points to an important favorable condition for future reform. The subvention formula does not appear to clearly demarcate between ‘winners’ and ‘losers’ from the point of view of small schools that need to be consolidated. If only the deficit school networks possessed small schools, it might have led to entrenched positions and an ‘us’ vs. ‘them’ approach with regard to optimizing the school networks. With the current context, it would be easier for local governments on both sides of the deficit and surplus divide to work toward optimization of the network. 88 The preconditions for optimization of the school network become even more obvious when looking at the vertical dimension, which represents the quality of ICT facilities in schools. There is a slight tendency that quadrant I, the ‘desirable’ quadrant, has fewer inefficient cyan-colored networks and quadrant III, the least desirable one has more cyan-colored school networks. This can be determined by looking at the red-bordered boxes, indicating that inefficient networks (the cyan-colored dots) are less likely to have high-quality ICT facilities. Clearly there are surplus school networks with low quality on the ICT facilities index (below the horizontal axis, drawn at the mean value of the index). This means that they have room to invest their savings in things such as internet connectivity and computers for children, which would help improve the quality of the learning environment (figure 3.11). Figure 3.11 Investigating precondition for incentive-based reform to work Source: Authors’ analysis of DISO statistics for school data and Ministry of Finance subvention data. Box 3.5 Case study of Novokalinovsky, Lviv oblast Good quality of education is the future of children. The school building is not the future of the children. – Mayor Yuzvyak Bogdan Osipovich As part of a qualitative study that involved investigating approximately 10 school networks in 3 oblasts in different regions of the country (Kharkiv, Lviv and Zaporihizia), the World Bank study team travelled to Novokalinovsky, Lviv oblast that had been identified as one of the positive deviations in terms of capable school networks. The team interviewed the mayor and head of the education department, visited the 3 existing schools, 1 of which used to be a high school but was converted into a level I–II school, generating revenue savings for the community. The map of the community shows the presence of a single road, along which three main villages are located. Key findings from the visit are highlighted in this box. Leadership. The task of persuading parents and village leaders about closing down schools that have stood for many decades is not easy and requires dedicated and creative thinking. It is unlikely that there are common prescriptions for all regions, but communities can be encouraged to devise the solution that works for them and strong community leadership clearly helps in pointing toward the 89 required creative solutions. Novokalinovsky spends about half of its budget on education (see figure 3.12), which is congruent with the high level of attention toward the sector. Figure 3.12 Case of Novokalinovsky Amalgamated Hromada, Lviv oblast Population: 7,339 Local Government Expenditures: 2018 Schools: 3 [One High School and Two schools of Levels I-II] Area of the territory: 108 square km Total budget (2018): 49 million HR Education spending: 21 million HR Subvention: 12 million HR Breakeven from teacher salaries: Formula pays for the same number of teachers, 70 of whom are actually employed Source: openbudget.gov.ua Clever local solutions to general systemic problems. The Novokalinovsky administration of Mayor Osipovich adopted the novel and clever idea to make children the key agents of their own change. This was done by introducing classes in certain subjects, such as mathematics and physics, on a rotational basis across the 3 schools that form the school network and that cover 7 villages. The rotation was started toward the end of the 2017/2018 school year, when there were only a few months left of the school year, so the children from the remote school could attend the soon-to-be designated hub school. This allowed the remote school children to form friendships in the hub school, and for the hub school children, in turn, to treat them as part of the school, and not as outsiders. Children also got used to the bus ride from the village to the hub school. The timing of these rotational classes toward the end of the school year and the fact that it was not an attempt to transfer children to a new school together minimized and delayed anticipated 90 opposition from teachers, parents, and village leaders from the remote villages. After the summer holidays, when the new school year began, the administration announced the change that there would be no more 9th grade in the old school. Although parents and teachers did not like this, the children did because they had become familiar with these hub schools and the experience of taking the bus by themselves. They served as allies with the administration. The mayor noted that that in September 2018, at the beginning of the school year, while the parents had previously gathered for the school opening ceremony at the old school, parents refused to attend the opening ceremony at the hub school. When the ‘rebellious’ children took the school bus to the ceremony themselves, the parents decided they would attend and wanted the school bus to take them, but the mayor refused. This is a classic positive deviance story in which the administration set in motion conditions whereby the children modeled for the parents a change in and acceptance of new behaviors to improve local conditions related to school network reform. Pedagogical benefits of scale compared to disadvantages of remote schools. The mayor had many reasons why scale is important to school quality and why he is convinced about its utility. The idea is that if there are just 3 or 4 kids in a classroom (as often happens in remote schools), then they tend to become unmotivated and unfocused about their studies. He gave the example of the effect of one child being sick, another child not ready for the lesson, and a third child questioning the need to study when the two others were not. In contrast, when a class has 10 to 15 children, there is some measure of competition that keeps children alert and focused. There is a dispersion for different skill dimensions, and the stronger can help the weaker ones, while the weaker ones are inspired. The mayor also mentioned team-building activities, in which there is a needed number of children to form an effective team; and also how extracurricular activities such as dance groups help play a role in developing and exhibiting leadership, which is another reason to have classrooms and schools at scale. Benefits of an optimized network. There are key benefits to optimizing the school network, especially in terms of resources for the preschool and kindergarten, so that school coverage approaches 100 percent. The old school could eventually be converted into a health care and elder care center. Additional payments to teachers in the form of bonuses are also possible with an optimized network. Schools in this environment are more like to have access to lab equipment and biology labs. An optimized network also capitalizes on its favorable external environment, in this case two companies and an air force squadron which hire students’ parents and have various spillover effects for the local economy. Plans to further expand the network.. There are ongoing plans to incorporate or amalgamate surrounding communities, increasing from 7 to 15 villages. For example, the mayor hopes that transportation to schools will include not just the hub schools but also base schools that cannot accommodate all of its students. However, he acknowledges that change depends on public sentiment. While he hopes school competition will have a positive uplifting effect, he understands that there will be challenges ahead as he seeks to extend the experiment of 3 school networks to 8 schools, and further increase the school networks to 15 and then 40. This ongoing experiment in school network reform in Novokalinovsky bears close watching. 91 Assessing Learning and Monitoring Quality The main method for measuring learning levels of students in Ukraine has been the External Independent Test (EIT), although other forms of assessment exist. This is a national test for admission to higher education institutions, which was introduced in 2008 largely to replace the former tests that had been riddled with corruption and low transparency in access to higher education (see next section). In addition to the EIT, the education system uses school leaving examinations (state attestation exams) set by MOES. These state attestation exams are administered and evaluated by schools for students at the end of primary (grade 4), lower secondary (grade 9), and upper secondary (grade 11). After grades 9 and 11, successful completion of the exam results in a School Leaving Certificate (SLC).53 Grades received in the last years of school, particularly in the final year, are reflected in each student’s SLC, which is a requirement for progression to postsecondary, nontertiary, or tertiary education. However, because the school leaving exams are administered and evaluated by schools, their results are not directly comparable. This leaves authorities with little information on learning outcomes before students reach the end of upper secondary school. School leaving exams theoretically are high-stakes tests, since they directly impact the school career of students. However, in practice, students rarely ‘fail’ such exams. Central and regional authorities can also administer external diagnostic tests to monitor quality of the education system, but these do not have an impact on individual students. These circumstances mean that potential learning gaps between students from different background and schools go unnoticed and unmeasured. Furthermore, there are substantial disparities between the school-based assessments of student learning and the standardized and external independent test. This suggests a high prevalence of overmarking and insufficient accountability at the school level for learning. Research from 2016 in Khmelnytskyi oblast showed that students are consistently overmarked on the school leaving exam relative to their performance on the EIT. The researchers examined data for 26 rayons in the oblast covering the share of high-performing students according to the school leaving exam and the share of high-performing students according to the EIT. This research found that 46 percent of those marked as high-performers on the school leaving exam were also consistently marked as high-performers on the EIT (figure 3.13). In other words, the school-based assessment led to many more high scores than the EIT; this was the case in 25 of 26 rayons. At the same time, the school leaving exam led to far fewer poor results than did the EIT. This is driven in part by the fact that school marks count for up to 10 percent of university entrance scores.54 Figure 3.13 Correspondence between school leaving exam and EIT in Ukrainian language and literature, Khmelnytskyi oblast (2015) 92 Oblast Average Stara Syniava Derazhnia Horodok Krasyliv Vinkivtsi Dunaivtsi Kamianets-Podilskyi Volochysk Polonne Bilohiria Iziaslav Khmelnytskyi city Netishyn town Kamianets-Podilskyi city Slavuta town 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Low-performers consistently assessed by schools (SLC/EIT) (%) High-performers consistently assessed by schools (EIT/SLC) (%) Source: OECD (2017a), citing Fasolia, O. I. (2016). Presentation by the Education Department Director of Khmelnitsky Region. The Law on Education has established a more comprehensive performance assessment system. In particular, the law established (a) a new system for monitoring the quality of education, starting with a focus on measuring reading and math skills of primary students in line with the NUS curriculum; and (b) a national agency for quality assurance in secondary education, known as the State Service of Education Quality (SSEQ). This new agency will be established on the bases of the existing State Inspectorate of Educational Institutions. It will be responsible for the accreditation issues in general secondary education. Regional divisions of the agency will inspect the educational institutions. Additional external independent evaluation institutions can be set up to measure learning outcomes. Since 2016, the final high school leaving exams for grade 11 in Ukrainian language, which students must pass for state final attestation, are conducted in the form of the EIT. This expanded the number of students taking the EIT. Before 2016, the final high school leaving exams were separated from the EIT and organized by schools themselves. This meant that the EIT was only taken by those who planned to apply to higher education institutions. Today, all high school graduates take the EIT in Ukrainian language, as well as either mathematics or Ukrainian history, whichever they choose, as well as one additional EIT subject of the student’s choice (for example, biology, geography, chemistry, physics, or foreign languages). This allows students to pass the state final attestation to receive their School Leaving Certificate (SLC). To qualify for admission to HEIs, students need to exceed a designated cutoff score for each EIT test, whereas there is no such cutoff score for students to complete the state final attestation. In practice, this means that students who simply show up and participate in the attestation exam process (by taking the EIT) will be able to receive the SLC and ‘successfully’ graduate from high school. Therefore, while the EIT is used to regulate access to higher education, it is not effectively used as an instrument to ensure that high school graduates have attained a minimum acceptable level of knowledge. There is a need to hold oblasts, local authorities and schools more accountable for learning results, and better external assessments can help. The principal benefit from external and independent assessments is for earlier stages of schooling because the information provided by such assessments can help to ensure the integrity, reliability and comparability of school-based assessments and to determine which regions or schools are lagging behind. Teachers benefit from the assessments, since they provide 93 benchmarks on student progress within the curriculum. Although Ukraine is making progress by implementing the grade 4 monitoring assessment, it could be improved and further expanded in order to provide school-level performance measures. The majority of OECD countries have national summative assessments: 29 education systems had such assessments at the primary level, and 27 had them at the lower secondary level. UCEQA has developed an ambitious Strategy for Learning Assessments in General Secondary Education until 2030 which provides a good basis for improvements to the assessment system. The SSEQ is charged with developing a quality assurance system for secondary education, including audits of education institutions, supervision and monitoring compliance with requirements of the Law on Education, and monitoring education quality. The previous form of inspection, prior to the law, focused on a closed system with rigid management that could not adequately respond to the needs for continuous quality improvement. The prior agency was focused on detecting violations, rather than supporting education institutions to improve outcomes.55 With the decentralization reform and the introduction of the New Ukrainian School, the SSEQ will play a crucial role in supporting quality improvement in schools. Pilot institutional audits will start as early as September 2019 for schools which have requested such an audit, and regular audits will be carried out starting in 2020. However, the SSEQ is a new institution with weak capacity; 25 territorial divisions of SSEQ are being established. Additional support will be needed to ensure that this institution can fulfill its role and responsibilities. Providing Equal Opportunities to Prepare for Higher Education Many students in Ukraine face large inequities in acquiring foundational skills in secondary school, driven by income, location, and school segregation. TIMSS 2011 data show that students in the top 20 percent of the income distribution perform substantially better on the assessment than students in the bottom 20 percent. While this is in line with findings from other countries, it is troubling to note that the gains in performance between 2007 and 2011 were attained only by students in the top 20 percent, whereas the performance of students in the bottom 20 percent did not change over this same period. This could be driven in part by large differences in school quality by location and socioeconomic status. Figure 3.14 below shows that poorer students tend to be clustered in poorer schools, which further contributes to initial inequities. However, it should be noted that gender disparities in performance are small, at least in Grade 8. Boys slightly outperformed girls on the TIMSS 2011 mathematics assessment, but the difference was not statistically significant.56 Figure 3.14 School segregation by socioeconomic status in Ukraine 100 90 Schools in the highest Percentage of Students 80 quintile 70 Q4 60 50 Q3 40 30 20 Q2 10 0 Schools in the lowest Poorest Q2 Q3 Q4 Richest quintile Quintile Quintile Source: TIMSS (2011). 94 Ukraine’s recent sample-based assessment of grade 4 students in mathematics further confirms these inequities in learning based on school size, type, and location. This suggests that such inequities are persistent over time. In 2018 the Ukrainian Center for Education Quality Assessment (UCEQA) conducted a monitoring survey of primary school graduates (grade 4 students) in order to establish a baseline for learning according to the New Ukrainian School curriculum. Based on results of this assessment for mathematics, it is clear that learning inequities appear even at the primary school level. For example, students in small schools were less likely to meet the average and high thresholds for mathematics competency compared with students in regular schools. While nearly 60 percent of students in regular schools met the average threshold, only 36 percent of students in small schools met the same level of achievement. Similar results were found for schools based on geographic location (figure 3.15). However, as with the TIMSS results described above, results by gender on the Grade 4 math assessment were not statistically significant, indicating a high degree of gender parity in terms of learning outcomes. Figure 3.15 Disparities in Primary School Performance: Percent of primary school graduates achieving average threshold (2018) (%) 70 Percent of Grade 4 students 60 50 40 30 20 10 0 Small Regular Rural Village City in City of school school region oblast significance Source: UCEQA grade 4 monitoring study (2018) Recent in-depth analysis of the results of EIT confirms the nature of inequity in terms of learning opportunities and outcomes in secondary education. It also indicates the effects of preparation for higher education. The analysis uses the latest available EIT data from 2018, as well as EIT results from 2016 when information on students’ socioeconomic background was collected. The 2018 EIT data was also matched with school-level data collected in DISO for 2018. The analysis of the EIT test used data from the Ukrainian language and literature test because this is the only subject that all high school graduates had to take to receive their high school diploma. School-level variables are available for over 16,000 schools, and the EIT results are available for over 335,000 students. The data are described in more detail in annex 3. The majority of students continue their secondary education in grade 10 in general secondary schools, though a large share continue in professional tertiary-level colleges as well as secondary vocational schools. According to the State Statistics Service of Ukraine (SSSU), 60 percent of grade 10 students are in general secondary schools, which themselves can be further differentiated on the basis of school location and school profile. In urban areas, students can enter specialized elite schools (lyceums, gymnasiums, and other specialized schools) as well as regular nonselective schools. Fewer options exist for students in rural areas. In total, about 15 percent of general secondary schools nationwide are specialized, and they enroll about 22 percent of total general secondary education students.57 Additionally, students can continue their 95 secondary education in colleges and technical colleges offering both secondary and short-cycle professional tertiary education, and about 23 percent of grade 10 students choose this option. Finally, about 16 percent of grade 10 students go into secondary vocational schools. Unequal access to quality secondary education has at least two major forms, including on the bases of school location and school profile. First, there is an important inequality in education and thus learning outcomes between urban and rural schools. Learning outcomes, which are measured by students’ performance on the EIT, show the difference between students from different types of settlements. In 2016, the results of EIT in Ukrainian language and literature were more evenly distributed between different test scores for students from urban areas, while for the students from rural areas the results were consistently lower, concentrated around 20–30 test points out of the maximum score of 104.58 Second, there is a between-school inequality as well. In urban areas this can be seen in the results of the students from the different types of schools: selective elite schools (gymnasiums, lyceums, specialized schools) and regular nonselective schools. In 2018, EIT results indicate important differences in performance based on school profile. More specifically, in the 2018 EIT results for Ukrainian language and literature, students from selective elite schools had a mean score of 68 out of 104, compared to a mean score in urban regular schools of 60, and approximately 43 in rural schools.59 In rural areas the most disadvantaged schools are small schools with small-sized classes (malokomplektni). Usually these schools are very small rural schools, defined as those that have less than 100.0 students, though this criterion is not formally specified. For this analysis, malokomplektni schools are defined as those in which the average class size is less than 10.0 students. The reason for this definition of malokomplektni schools is derived from the calculation of the funding formula (known as the education subvention) on the allocation of funds between local budgets. At the same time, the average class size of all rural schools is 13.2 students, compared to 23.8 students in urban schools (table 3.2). Table 3.2 Equity of Performance and Opportunity in Secondary Schools Urban schools Rural schools All Schools Hub Elite Regular All rural schools urban with small- schools schools schools schools sized classes Number of students, thousands 1,706.8 556.1 1,150.6 737.6 112.8 197.0 Mean test score in EIT in Ukrainian 60.3 68.0 55.2 42.7 43.2 52.5 language and literature Mean test score in EIT in all the 44.5 49.7 41.1 33.4 33.4 39.4 subjects Average class size 23.8 25.6 23 13.2 8.28 16.8 Share of students that took EIT in 48.9 54.8 45.1 40.3 46.6 44.7 Mathematics,% Share of young teachers (up to 30 9.6 9.8 9.5 11.2 10.7 9.2 years old), % Share of teachers older than 60 years, 11.3 12.2 10.8 9.4 9.2 9.1 % Share of students learning second 70.4 78.8 64.9 55.5 41.8 67.5 foreign language, % Mean of ICT index 0.667 0.761 0.606 0.202 -0.065 0.643 Mean of material equipment index 0.928 0.964 0.905 0.333 0.014 0.942 Mean of teacher characteristics index 0.738 0.919 0.620 0.290 0.074 0.551 Source: Authors’ analysis of matched DISO-EIT database Note: Data describe the matched DISO-EIT database prepared for this analysis, which is a subset of the complete DISO database. 96 The EIT mathematics test is required for admission to a number of higher education study programs. This is because it is an optional test that reflects students’ level of preparation for higher education. The EIT in Ukrainian language and literature is mandatory for graduation, while the other EIT subjects required for different types of higher education fields of study are optional. Students choose additional EIT subjects based on the majors they are considering for application, and the subjects for which they are the best prepared. Mathematics is considered one of the most difficult subjects to take on the EIT, and it is the ‘main’ subject test considered for admissions to higher education study programs in science, technology, engineering and mathematics (STEM) fields, including computer science and computer engineering in which Ukraine has a comparative advantage. Therefore, it is an important finding that there is a difference in the percentage of students that took the EIT in mathematics from urban and rural schools: 40 percent of students from rural areas chose to take the EIT in mathematics, compared to 50 percent in urban schools and 55 percent in elite specialized schools. The unequal access to quality secondary education can be partially explained by the unequal distribution of resources across schools, including teachers, material equipment, and ICT. For example, younger teachers who are new to the profession and less experienced tend to be somewhat more concentrated in rural schools, whereas the share of teachers over age 60 is highest in elite urban schools. At the same time, rural schools with small-sized classes are at the greatest disadvantage in terms of the availability of qualified teachers of different subjects. Of all school types, these small rural schools have the highest percentages of three or more subject teachers (table 3.3). Table 3.3 Availability of subject teachers in different types of schools Urban schools Rural schools Schools All urban Elite Regular All rural with small- Hub schools Percent of schools that schools schools schools schools sized have a shortage of: classrooms 1 subject teacher 65.9 72.6 63.2 19.9 4.1 60.6 2 subject teachers 19.1 15.6 20.5 26.4 19.2 24.0 3 subject teachers 7.9 4.9 9.2 26.7 31.6 10.6 4 subject teachers 3.1 2.7 3.2 16.9 27.0 2.4 5 subject teachers 1.0 1.4 0.9 6.5 12.7 0.2 more than 5 subject 1.6 1.4 1.7 2.7 5.2 0.6 teachers Source: Source: Authors’ analysis of DISO data Another indicator of unequal learning conditions is the availability of second foreign language to students. English is a mandatory foreign language to learn, but a second foreign language is an optional subject, so schools themselves can decide whether to include it in the curriculum and which language should be taught. The statistics show that the lowest percentage of students that study a second foreign language are those from the rural schools with small-sized classes, while the highest percentage can be found in urban elite schools.60 The establishment of hub schools and enrollment policies are two efforts intended to increase quality and equity. In 2016, hub schools in rural areas were formed as an instrument to increase the quality of education in rural areas and settlements, while also addressing the inequities in Ukrainian secondary education described above. Hub schools are supposed to have an effect on the learning outcomes of the student from rural areas by giving them more opportunities to study and learn well compared to small-sized schools in which resources are stretched too thin to offer a sufficiently high-quality education. Additionally, in 2018, there was also a modification to the enrollment policy for first graders. Currently, all the schools, including those that were selective ones, have their own catchment areas and are admitting all students that 97 live in these catchment areas. Although this policy was also not implemented as a direct mechanism for battling inequality, but rather as an anti-corruption instrument, it is also indirectly affecting inequality. Schools will no longer select students in the first grade and will provide education to children with different socioeconomic backgrounds and advantages, which may in time result in a decrease of the between-school inequality in learning outcomes that comes from selective admissions processes.61 Although it may be too early to see the impact of these policies on education outcomes, multivariate regression analysis using DISO and EIT results provide a preliminary assessment of the relationship between specific school characteristics and student outcomes. This analysis attempts to see what are the other factors that play a role in the differences of student learning outcomes. As mentioned above, the school network policy of creating and developing hub schools in rural areas has indirectly affected the inequality in secondary education. The reason for this is that inequality in secondary education is mostly seen in terms of differences between schools in urban and rural areas. Hub schools are supposed to reach the goal by being an instrument to improve the quality of education. However, they are not supposed to improve the quality of education comparing just to any school, but rather comparing to the other rural schools, and especially the rural schools with small-sized classes in which the average class size is less than 10 students (malokomplekny). Since the two policies to increase quality and equity have only recently been initiated, it may be too early to observe much of a measurable effect of the change in the data on student outcomes and school conditions. Even so, the data shed light on the factors that affect student learning outcomes in Ukrainian schools. The regression analysis confirms that students from hub schools score only marginally lower on Ukrainian language and literature exam than students from other schools, whereas students from elite schools score around 9.6 points higher than students from ordinary schools. After accounting for the schools’ material equipment and teacher qualifications, the effect of elite schools became smaller and the negative effect of hub schools increased somewhat for both EIT tests in Ukrainian language and mathematics (see full results in annex 3). This likely indicates that the difference in the test scores between ordinary and elite schools can be explained by positive selection of students into elite schools rather than better learning environment and better teaching. At the same time, better school equipment, ICT availability, teaching staff and extracurricular activities, all together positively influence the difference in student performance between hub and non-hub schools. It might signify that hub schools are put in place where they are in need, and better school environments in such schools positively influence the test score partly by compensating for the lower levels of socioeconomic status in rural areas. Results in hub schools raises some important questions regarding the implementation of the hub schools program in practice. As mentioned, hub schools are intended to give students in rural areas (rayons or amalgamated hromadas) access to better quality learning environments with more resources while also enhancing the capacity of local authorities to manage their school networks. However, average performance in hub schools is worse than in other schools. Furthermore, some of the effects of the hub school model could be reflected in the school size coefficient, which is positive. This would make sense since hub schools are supposed to be larger than comparator schools. That said, school size results are also somewhat unclear since the first-order coefficient is positive while the quadratic term is negative (but small and close to zero). Therefore it is not clear if the effect of school size on quality depends on a particular size range. In any case, there is some evidence that the hub school program so far has not been implemented with strict fidelity in terms of downgrading the affiliated branch schools and the criteria for ‘hub’ status. The negative effect of hub schools on learning outcomes likely reflects the mixed implementation and the fact that the program is still very new, so the effect has yet to materialize in test scores of beneficiary students. The effect of socio-economic background was studied with the help of a survey conducted in parallel with EIT, and this confirmed that parental occupation and educational attainment affect student 98 performance, which is consistent with the literature on determinants of student achievement. However, only a share of students took part in the survey, so the results are not fully representative of all EIT test takers. Occupational status of parents had a negative effect on both EIT scores of students in rural area, while the opposite was true for students from urban schools. All the specifications showed a positive relation between one of the parents working in the IT sector and students’ EIT scores. This effect might be due to higher income in this sector and higher motivation of students from such families to continue with higher education. Also, there is some effect of parents’ education on EIT scores of their children. If one of the student’s parents has attained higher education, the student taking both EITs, on average, will score more than if the student’s parents has attained education at only the secondary level. Also, having paid lessons with a tutor had a significant effect only for EIT score in Ukrainian language and literature, on average, with a score increase of one point. An interesting finding here is that differences in performance by gender appear in the EIT, whereas earlier assessments of student learning show gender parity. Table A2.4 and A2.5 show that male students perform statistically significantly worse than female students across the board on both the Ukrainian language and literature exam and the mathematics exam, although the size of the effect is much larger on the language exam. In fact, for language, gender is a main determinant of performance on the EIT, whereas the effect is more modest for mathematics. Region and language of instruction also influence test performance. The average score of a student from Uzhgorod region on the EIT in Ukrainian language and literature is up to 12 points lower than that of the average student in Kyiv, and the average score of a student from Lviv region is 6 points more than the average student in Kyiv. This difference is further amplified if the language taught in the class is not Ukrainian. On average, those students who were taught in minority languages (excluding Russian) scored up to 17 points lower than those taught in Ukrainian.62 As expected, the location of the school in the urban area has a significant positive effect on the students’ score on the EIT in Ukrainian language and literature. Expected mean score for a student from rural area is about 8 points less on Ukrainian language EIT and around 4 points less on mathematics EIT than of student from urban area. The smaller effect of rural area on mathematics is probably due to students’ self-selection. EIT in Ukrainian language and literature was compulsory for all graduates and students could choose either mathematics or history as a second EIT. As previously mentioned, fewer students from rural areas (40 percent) choose to take the EIT in mathematics, compared to students from urban areas (50 percent) and elite schools in urban areas (55 percent). The school size has a positive although diminishing effect on EIT score both for urban and rural schools. This indicates the economies of scale that can be achieved when resources are concentrated in larger schools, at least within a range. However, the number of students of 11th grade has a negative effect on the EIT test scores (both in Ukrainian language and mathematics) for all schools while the quadratic term of number of the students in 11th grade has a positive sign. It seems that the relation between class size and EIT score is negative for smaller class sizes and positive for larger class sizes. As the average class size for urban schools is 23.8 students and for rural is 13.2 students, a separate analysis was done for urban and rural schools. For urban schools the increase of 11th class size by 1 person is associated with the increase of the EIT score in Ukrainian language and literature by up to 0.2 points. For rural schools the effect is not statistically significant. The same effects are present for mathematic EIT score although of a different size. This suggests that school size in rural areas is too small to generate the positive effects of interaction with peers that is found in larger classes and schools in urban areas. In terms of the learning environment, teacher qualifications and ICT equipment appear to have a positive relationship with EIT scores for both mathematics and Ukrainian language. For all model specifications, the results show that the ICT index for the school, created based on DISO data, is positively 99 related to students’ performance on both EIT tests. This could reflect the underlying resources of the school because schools with higher socioeconomic status also have better learning environments in terms of access to ICT for students and teachers. On the other hand, the effect of school equipment with subject classes and learning materials had a negative effect on scores, which is likely due to measurement error. Teacher qualifications were also positively related to student performance in urban schools, though the effect is not significant for rural schools. There are also signs of within-school inequality, based on the students’ class profile which ultimately reflects the curriculum available to students. Students who studied in classes with designated profiles in mathematics or humanities scored better on the Ukrainian language and literature EIT than those who studied in ‘universal’ profile classes, which are the default option available in most schools. On the other hand, students in other types of profile classes (arts, sports, or manufacturing) scored on average 5–6 points lower for both Ukrainian language and literature and mathematics compared to the universal profile. Ultimately, students’ motivations and expectations, shaped through their experiences in school, have a major effect on their performance and educational trajectory. Analysis from the TIMSS 2011 survey indicates that students who have the expectation to finish university perform substantially better in terms of math achievement (520 points) than students who do not expect to finish university (459 points).63 Schools, and the quality of education they provide, contribute to setting such expectations for students. Limited access to relevant curricula and guidance counselling for students has the effect of diminishing students’ expectations for their future and their performance while in school. The results of this analysis paint a clear picture: unequal access to quality education at the secondary level contributes to a growing equity gap in terms of graduates’ preparation for higher education and the labor market. Data on admission to higher education institutions clearly show that school location as well as school profile affect the patterns of demand for higher education and student performance on the EIT, which regulates access to higher education. Rural students are less likely than urban students to pass the EIT cutoff thresholds and are considerably less likely to apply and ultimately enroll in higher education. Whereas nearly 70 percent of urban students passed, applied and ultimately enrolled in HEIs, this figure drops to 40 percent for rural students.64 Furthermore, rural students are less likely to achieve the high levels of EIT performance required to access state-funded places in HEIs; only 17 percent of state-funded places for bachelor’s programs in 2018 went to students from rural areas (figure 3.16). Figure 3.16 Higher education admissions by student origin and EIT performance (2018) 70 175 Average EIT score (Ukrainian Admitted students (thousands) 170 60 165 50 160 language) 40 155 150 30 145 20 140 135 10 130 0 125 Urban Rural Urban Rural State-funded places Privately funded contract places Number Average EIT score Source: Authors’ analysis of EDEBO database. 100 Conclusion The New Ukrainian School vision for general secondary education is an important step forward in modernizing Ukraine’s education system and preparing students with the skills to succeed in the new economy and in higher education. The NUS vision is well-defined, combining several key elements including educational content, motivated teachers, greater decentralization and autonomy, child-centered approaches to teaching, new schooling structure, and contemporary educational environments. However, the success of the NUS clearly relies on a number of interlocking parts that must work together. Students must enter primary school ready to learn, so they can capture the benefits of the NUS competency-based curriculum. Schools and the wider education system must be equipped with the appropriate assessment and monitoring tools to ensure that students are developing the skills envisioned in the curriculum. Particularly in the context of the declining student-age population, the system must prioritize equity of opportunity to ensure that all students benefit from the NUS and are adequately prepared to enter higher education or the labor market. Only in this case will secondary education reform provide the transformative paradigm shift needed to position Ukraine’s human capital for inclusive growth. Two critical dimensions of the success of the NUS are teacher commitment and effectiveness, and the sustainability of school networks. The analysis above describes key considerations in both of these areas that must be addressed to ensure that NUS achieves its intended results. The capabilities and motivations of teachers are crucial for success at the classroom level, and this depends largely on teachers’ incentives to change their teaching and behaviors in line with the reform. Incentives for teachers, including their workloads, salaries, recognition for performance, and opportunities for professional development, must all be well-aligned to empower teachers as agents of change and “front-line” supporters of the reform. At the same time, improving incentives for teachers and the attractiveness of the teaching profession must be done hand-in-hand with efforts to rationalize the teacher workforce and school network in line with needs and demographic trends. Leaving school networks unchanged and oversized will harm sustainability and threaten success of the reform. Close monitoring and adjustment of the secondary education financing formula as a tool to drive sustainability will be important. 101 Notes 1 World Bank (2018d). 2 World Bank (2019c). 3 The MELQO initiative began in 2014 under the leadership of UNESCO, the World Bank, the Center for Universal Education at the Brookings Institution, and UNICEF. The initiative aims to promote feasible, accurate and useful measurement of children’s development and learning at the start of primary school, and of the qual ity of their pre- primary learning environments. 4 World Bank analysis of PISA 2015 results. 5 SSSU (2017). 6 UNESCO Institute for Statistics (UIS). 7 Based on authors’ analysis of SSSU data from 2017. 8 OECD (2017a). 9 OECD (2017a). 10 Putcha and others (2018). 11 Putcha and others (2018). 12 Putcha and others (2018). 13 According to the MOES website: https://mon.gov.ua/eng/news/inclusion-2019-5045-million-uah-has-been- included-budget-and-it-first-time-money-directed-towards-support-inclusion-kindergartens-and-vocational- institutions 14 World Bank (2019b). 15 Snilstveit and others (2015). 16 Rockoff (2004); Evans and Yuan (2018). 17 Chetty, Friedman, and Rockoff (2014). 18 Hanushek and Rivkin (2010); Rivkin, Hanushek, and Kain (2005); Rockoff (2004). 19 Pedagogical employees include teachers, principals and their deputies, psychologists, librarians, and other similar types of staff. 20 World Bank (2018c). 21 SSSU reports on schools for years 2010-2017. 22 World Bank (2019b). 23 Mostafa and Pal (2018). 24 OECD (2018a). 25 Ministry of Foreign Affairs, Finland (2017); authors’ analysis of EDEBO database. 26 Authors’ analysis of SSSU data for 2017. 27 World Bank (2018c). 28 World Bank (2018c). 29 World Bank (2018c). 30 OECD (2017a). 31 UNICEF (2011). 32 Ministry of Foreign Affairs, Finland (2017). 33 The reasons for having voluntary certification are complex. It is mostly to encourage those teachers who really wish to participate in the reform, but it is also to avoid certification becoming a mandatory, onerous (and so possibly gameable) procedure. 34 The criteria for each aspect of the evaluation are under development but will be promulgated before the certification procedure starts. The external independent testing for (a) is being developed based on requirements set in the standards for primary school teachers and will be executed by UCEQA, while criteria for classroom observations (b) will be provided by the SSEQ. The testing program will also be based on distance courses for primary school teachers that are being developed for this purpose. 35 According to UCEQA website: http://testportal.gov.ua/wp-content/uploads/2019/01/Nakaz- 1486.pdf?fbclid=IwAR2zOvYyhz5nLxXH2HQTLHtcOT82ZVdBVSounFdSm0Bl90brAzagP7sgAFo 36 This may be very generous. Certification in the United States under the National Board of Professional Teaching Standards (NBPTS) is only achieved by 4 percent, 1 in 25 teachers. 102 37 The number of new teachers entering the system is relatively minor, approximately 5,000 per year, representing less than 3.5 percent of the total teaching force of 141,000 (World Bank 2018c) and new teachers being usually not effective until their fifth year, the reform needs to find the teachers it requires in the current teaching force. 38 Hanushek and Rivkin (2010). 39 Arancibia, Evans, and Popova (2016). 40 That said, there are also other ways to incentivize teachers: in Chile, teachers can be recognized for excellence by voluntarily signing up and then passing a series of assessments of their teaching. Having a peer get the award almost doubles another teacher’s likelihood of applying to do the same. A recent OECD (2018b) report shows that teachers who work together through collaboration, report greater levels of job satisfaction and efficacy, which is in turn correlated with better student learning outcomes. 41 Link to course: https://courses.ed-era.com/courses/course-v1:MON-EDERA-OSVITORIA+ST101+st101/about 42 Cilliers and Taylor (2017). 43 Bawa (2016). 44 Hone and El Said (2016); Evans, Baker, and Dee (2016). 45 Goddard, Goddard, and Tschannen-Moran (2007). 46 In Mexico, the objective of the program Mobile Pedagogical Tutors, (Avitabile C. and others, 2016) was to improve the quality of service delivery in rural schools. Young professionals with relevant experience serve as mobile pedagogical tutors alternating their time on two-week intervals in two nearby school communities during the academic year. A successful program in India where female high-school graduates, called Balsakhis, were offered positions as assistants in primary schools. 47 Jensen (2016). 48 Herczyńki (2017). 49 The currently applicable formula, based on Decree 65, dated February 6, 2019 of the Cabinet of Ministers of Ukraine can be accessed from: https://zakon.rada.gov.ua/laws/show/65-2019-%D0%BF 50 The standard salary of a teacher of higher qualification category (14th wage grade) is used, with basic and additional allowances for long service, prestige of teacher’s profession, class management, marking of papers, and other allowances. Maximum statutory values are taken for basic allowances paid to all teachers, and average values for additional allowances. In 2019, the resulting value used is UAH 136,000 per year, approximately USD 17,000 in PPP terms, a little over USD 5,000 at the official exchange rate, about twice the per capita GDP of the country. 51 Herczyńki (2017). 52 Such a dynamic was catalogued many years ago by Judith Tendler in her MIT book “Good Government in the Tropics” that described State-Municipal government interactions in the Brazilian state of Ceará. Tendler explained how effective health services provision practices, encouraged by the State government, spread across neighboring municipalities (Tendler 1998). 53 OECD (2017a). 54 OECD (2017a). 55 SSEQ (2018). 56 Mullis and others (2012). 57 Based on authors’ analysis of DISO data. 58 CEDOS (2016). 59 This analysis uses raw test score points rather than the 100–200 scale used for university admissions because the university admissions scale eliminates low-performing students who do not pass the cutoff score. Raw test score points capture the full range of student performance. See annex 3 for more information. 60 Russian language is excluded from this analysis as being considered a second foreign language. 61 However, this policy does not address territorial segregation in cities, whereby families in selected districts or areas of the city have higher socioeconomic status, as do the schools in those areas. 62 Students in Russian language classes also scored lower on the EIT in Ukrainian language and literature, but the difference is negligible. 63 Mullis and others (2012). 64 Based on data for 2018 university admissions campaign. 103 Chapter 4: Strengthening the Sustainability and Transparency of Higher Education Higher education develops the advanced skills needed for modern economies. Through the development of technical, professional, and specialized knowledge and skills, graduates from higher education can actively contribute to their society and economy as informed citizens and skilled workers. At the same time, productivity and economic growth are driven by innovation which relies on human capital and research. Higher education significantly contributes to the development of both. Additionally, higher education plays a critical role in fostering social cohesion through its contribution to social and cultural development. There is also strong evidence that higher education and HEIs are drivers of economic growth. For example, there is evidence that HEIs affect growth through at least four paths: the provision of human capital, the contribution to innovation and diffusion of productivity-enhancing practices, support for democratic values and fostering pro-growth institutions, and by generating demand for related goods and services on the part of administrators, faculty and students. These economic impacts of HEIs across countries and regions are well summarized in the research literature.1 Ukraine has a long and proud history of higher education, although performance of the system has weakened over time. Ukraine became one of the top countries by higher education coverage starting in the early 1970s, when the USSR implemented its policy of higher education for its citizens. Over time, and particularly in the years after independence, Ukraine has become a leader in terms of high rates of emigration. In the first 10 years after independence, lower skilled workers tended to comprise the majority of emigrants, while this has shifted in the last decade to include more highly skilled professionals. 2 For example, 43 percent of university professors and researchers expressed an interest in moving abroad for studies, temporary employment or permanent residence according to the International Labor Organization (ILO) mobility study conducted in 2013.3 This reflects challenges of employment and wages in the local economy, as well as the high share of skilled labor produced by Ukrai ne’s higher education system. However, performance of the system has weakened while at the same time the economy has been in a state of transition, leading to declining relevance and increasingly inefficient organization of training. In spite of these changes, economic returns to tertiary education remain high. As discussed in Chapter 2, the returns to tertiary education have remained high over time, in spite of the weakening performance in the system. While this seems to be a paradox, there are several important factors that explain this, including credentialism in the labor market, selectivity in higher education admissions by institution and program, effects of different cohorts of tertiary education graduates in the labor market, and increasing emigration of skilled workers. The confluence of these factors have kept tertiary education returns high, even while quality and relevance have weakened over time. However, resting on the historical strengths of the system will not provide Ukraine with the human capital it needs for the future. The Law on Higher Education was the first large systematic reform measure adopted by Ukraine after the Euromaidan Revolution. It represented a major step forward after years without a clear development strategy for higher education. However, more work is needed in higher education to articulate a vision. To this day, there is no clear or coherent vision for the development of the higher education sector, which is a fundamental problem: higher education in Ukraine cannot serve the needs of the people and the economy without clear objectives and a strategy for how to achieve them. 104 Massification and Inefficiency of the Higher Education System Ukraine has an extensive and diverse higher education system, composed of universities, academies, institutes, and colleges. According to the Law on Education, universities are multisectoral or sectoral higher education institutions which carry out innovative educational activities at different levels of higher education, including the doctoral level. Academies and institutes are sectoral higher education institutions which carry out educational activities related to particular specialties. Colleges are tertiary pre-higher education institutions or structural units (branches) of universities, academies or institutes which perform educational activities related to obtaining a bachelor’s (long-cycle) and/or junior bachelor’s degree (short- cycle), conduct applied scientific research, and/or creative artistic activity. Colleges are also entitled to provide short-cycle field-specific VET and short-cycle professional tertiary education (see Box 4.1). This distinction between academic HEIs (universities, academies and institutes) and professional tertiary education institutions (colleges) corresponds to previous accreditation levels, which were introduced originally in 1996.4 Institutions of accreditation levels I and II included colleges and technical colleges, while institutions accredited at levels III and IV were institutes, conservatories, academies, and universities that trained students at bachelor’s, specialist’s, master’s and doctoral degree levels. As of 2019, according to the Unified State Electronic Education Database (EDEBO), there are 541 public HEIs and 135 private HEIs, for a total of 676 institutions.5 Out of the 350 state institutions in Ukraine, the majority (281) are subordinated to the Ministry of Education and Science (MOES). These figures include only main legal entity institutions, excluding their branches. According to the law, branches are territorially separated structural divisions of a HEI established to meet the demands of regional labor markets in particular professions and provide proximity of a place of learning to the learners’ place of residence. A branch is not a legal entity and acts on the basis of regulation approved by a HEI and in compliance with the license obtained for educational activity. In total, there are about 600 branch institutions, including both regional branches of universities as well as colleges that have been legally subordinated to universities. Including branches, the total number of HEIs is 1,282 (table 4.1 and table 4.2). Table 4.1 The number of higher education institutions by type and ownership (2019) Public Private Grand State Municipal Total Total Total Universities, academies and institutes 209 22 231 96 327 Colleges 141 169 310 39 349 Total 350 191 541 135 676 Source: Authors’ analysis of EDEBO database. Note: Excludes branches of HEIs.6 Table 4.2 The number of higher education institutions by type and subordination (2019) Universities, Academies Colleges Total and Institutes State 209 141 350 Ministry of Education and Science 142 139 281 Ministry of Health 16 16 Ministry of Culture 10 2 12 Ministry of Internal Affairs 8 8 Ministry of Defense 8 8 105 Other State Agencies 25 25 Municipal Authority 22 169 191 Private 96 39 135 Grand Total 327 349 676 Source: Authors’ analysis of EDEBO database. Note: Excludes branch institutions. Box 4.1 Secondary vocational education and professional tertiary education Vocational education is delivered in several different institutions, which students can access at different points in their educational path. Students can enter secondary VET schools (vocational lyceums or professional-technical schools) as well as professional tertiary education institutions, including the colleges and technical colleges formerly at level I–II accreditation. Students can enter either secondary VET or professional tertiary education after 9th grade as well as after 11th. Only the duration of the educational program will differ. For example, if one enters the vocational lyceum to get a qualification of ‘qualified worker’ as a plumber after 9th grade, they would study for 3 years, and after 11th grade they would study for only 1.5 additional years. With colleges, one can enter after 9th grade, and after a 3- or 4-year program in the college, the individual would receive a junior specialist degree (junior bachelor’s degree going forward). Historically, this individual could then enter a university without taking the EIT and after a shortened program complete a bachelor’s degree. However, starting in 2019, college graduates wanting to enter a bachelor’s programs would also be expected to successfully complete the EIT in Ukrainian language and literature and either mathematics or history.7 Secondary VET in Ukraine has faced years of declining popularity, driven by low investment in the sector and relatively easier access to higher education institutions over the past 20 years. Only 14 percent of secondary school students reported that they strived for VET as the highest level of educational attainment, while 85 percent plan to undertake higher education.8 Furthermore, 73 percent of students who plan to complete higher education would not enter VET even despite evidence of equal if not higher wage potential and employment stability. Many students report that higher education is necessary for desired future employment (48 percent) and that it will bring greater earnings in the future (43 percent). About 25 percent of students similarly recognize that VET has a low social status. This low level of popularity has led to a relatively small share of students who enter secondary VET schools, with only about 16 percent of grade 10 students enrolling, although another 23 percent of grade 10 students continue in colleges. There has been a contraction in both the number of teaching staff in VET schools and the number of students. This is largely due to the demographic trends that face the education sector as a whole. However, unattractive working conditions and poor coordination and feedback between employers and education institutions are contributing factors to the challenge of enrollment in VET schools. Lower salaries for VET teachers relative to those in industry is another contributing factor. This has led to an ageing teacher workforce, low professional mobility, and poor motivation to master innovative technologies. A mismatch between employers’ expectations of student skills and VET institutions in teaching skills, as well as limited incentives for establishing better coordination, further limits the relevance of VET education. Ukraine is attempting to address skills mismatches between VET and employers’ needs through the introduction of the dual model approach. The MOES is actively introducing elements of the dual model approach to VET in accordance with the ‘Concept of Training Specialists using the Dual Form of Education’ approved by the Government in September 2018. Dual education is now being offered in around 250 VET institutions for 160 blue-collar professions with the involvement of approximately 106 10,000 students and about 1,000 employers. Although the labor market effects of the dual model approach in Ukraine are not yet known, the efforts to link VET institutions with employers are good practice. Currently, the VET sector is undergoing a large-scale decentralization process. The objective of this process is to move away from the centralized model of management, build the capacity of local self- governments, and develop an efficient system of territorial power. VET decentralization started in earnest in 2016, following the adoption of the state budget, but the process faced serious challenges from unclear legal and regulatory provisions and a shortage of financial resources. Ultimately, additional VET subsidies were allocated to close financing gaps. In 2017-2019, funding for VET included the educational subvention for students completing general secondary school, funds for training blue-collar workers in professions of national significance, and a subvention for modernization and updating of the material and technical base of VET institutions. The draft Law on Vocational Education has been developed and is expected to be approved by the Government in the near future The concept for VET envisages a phased transfer of authority for management of VET institutions and their financing to the oblast level, though the plan for the first stage (2019-2021) is a partial transfer of financing of VET institutions located on the territory of cities of oblast status. They would be financed from oblast budgets through the implementation of a pilot project in selected oblasts. Eventually, regions must ensure the financing of VET institutions from regional budgets and thus define trends in regional VET development using the regional order for skills training.9 However, the challenges regarding legal transfer of ownership continue, meaning that oblasts are facing delays to merge and/or close VET schools in line with demographic trends. VET transformation is proceeding with external support. VET decentralization is proceeding, but challenges of low relevance of VET content and poor quality of provision remain. To support the VET reform and improve quality, Ukraine and the European Commission recently signed a EUR 58 million project (EU4Skills) to modernize VET in Ukraine. The project would support the development of VET Centers of Excellence, development of content and new educational standards for VET programs, the training of master trainers and teachers, and the modernization of infrastructure and equipment. Additionally, Ukraine is embarking on the Torino Process 2018–2020 with the European Training Foundation (ETF).10 The Torino Process is an evidence-based approach to analyzing human capital development related to the VET sector, as well as VET policies and responses. Through this process, MOES will produce an analysis culminating in a national report, and ETF will also conduct an assessment and in-depth analysis of the VET sector.11 Labor market information exists but is fragmented across different institutes, which creates challenges for decision-making bodies particularly in a context of decentralization. According to the European Training Foundation (ETF), Ukraine possesses the capacity and robust labor market statistics needed for analysis on the types of skills mismatch and their incidence at the regional and local levels. However, such research is fragmented across various institutes and research organizations, meaning that decision makers often cannot benefit from the newest labor market research and skills forecasting data. The VET sector faces some key questions as a result of ongoing reform initiatives, namely the extension of secondary education to 12 years and the status of colleges and technical colleges.12 The new Law on Education has initiated the transformation of secondary education. One of the major changes is lengthening the duration of the secondary education to 12 years. According to the plan, separate high schools are to be established, that can offer academic and/or vocational tracks for their students. This could significantly change the network of the existing educational institutions that offer education for the relevant age-group. The further development of the new network of the high schools is still unclear. It is possible that some high schools will offer academic and vocational education, in which case the 107 existing network of vocational schools may change considerably. Another issue is the demarcation between the secondary vocational schools and colleges offering professional tertiary education. This issue has been one of the most significant impediments in the development of the new Law on Education. While the colleges are treated as a completely separate educational sector, such demarcation in general distinguishes the future of such educational institutions, versus the future of the education programs they deliver. In debating the future of reform, the quality of education and how it can be best adjusted to increase further life opportunities of the young people should be the primary issue. Ukraine’s higher education sector expanded substantially from the early 1990s through its peak in 2007/2008. The table below shows the comparison in number of HEIs, students, postgraduates and PhD candidates, educators and academics in 1992/93 compared to 2007/08 (table 4.3). The number of institutions more than doubled, while the number of privately owned HEIs increased by over nine times. At its peak in 2007, this sector covered almost three times more secondary school graduates than when Ukraine gained its independence (see table 4.3). This expansion was driven largely by several factors: • The appearance and development of private HEIs • Upgrading the status of former technical schools to colleges • Upgrading the status of institutes to academies and universities • Opening of numerous branches and separate structural divisions of HEIs. 13 Table 4.3 Key quantitative indicators of the higher education system in Ukraine (universities, academies and institutes only) Growth in 2017/18 vs Indicators 1992/93 2007/08 2007/08 vs 2017/18 2007/08 (%) 1992/93, times Universities, academies and 158 351 2.2 289 82 institutes, including: Public (state and municipal) 146 242 1.7 212 88 Private 12 109 9.1 77 71 Number of students, thousand 855.9 2372.5 2.8 1329.9 56 persons, including: Students of public HEIs N/A 1995.1 – 1224.5 61 Students of private HEIs N/A 377.4 – 105.4 28 Number of students per 10 163.8 511.6 3.1 313.7 61 thousand of population Number of teachers, thousand 71.8 159.1 2.2 129.4 81 persons, including: Holding an academic degree, 40.4 74.4 1.8 76.0 102 persons Holding an academic degree 56 47 – 59 – (%) Number of students per 1 11.9 14.9 1.3 10.3 69 teacher Higher education coverage of 82.3 45.4 78.9 1.7 104 population (%) (2016) Share of students enrolling at universities, academies and 24.5 75.7 3.1 78.7 104 institutes compared to the number of grade 9 school 108 graduates two years earlier Source: Authors’ calculations of SSSU data. Although private HEIs offered more flexibility and a response to the needs of a changing society, most private institutions could not compete with budget-funded seats in public institutions. In the early 1990s, the development of the private higher education sector was considered to be positive, with such institutions offering a larger flexibility to meet the needs of society in studying relevant and pressing professions at a time when public HEIs were not meeting such needs. Private HEIs benefited from the period of active higher education sector expansion, but they also experienced the greatest problems in attracting students during the period of demographic decline. The availability of budget-funded seats at public HEIs is an important competitive advantage, and many private HEIs shrank or disappeared because they could not find other funding sources except tuition fees paid by students. At the same time, the network of public HEIs has undergone several transformations through changes to the status of institutions. For example, in the 1990s, dozens of institutions obtained the level IV accreditation and changed their names to universities and academies; institute branches obtained independent status and later became universities.14 Later in the 1990s and 2000s, a second wave of status improvements for universities and academies occurred, when they obtained the status of a national institution. This stipulated a slight increase of guaranteed governmental funding and a right to improve teachers’ salaries compared to the salaries of the Uniform Wage Scale.15 Today, over 100 Ukrainian universities and academies have the word ‘national’ in their names, which indicates the respective status. A third wave occurred in 2010–11, when several dozen universities acquired the status of ‘autonomous (self-governed) research university,’ although this was unsuccessful and ultimately cancelled. There was also a transformation of vocational schools (through mergers, reorganization, or change of specialization), which contributed to a further growth of academic HEIs at the expense of vocational HEIs. The growth in the higher education system, particularly in HEIs offering academic degrees, was accompanied by a large increase in the higher education coverage rate. The increase was from approximately 45 percent at the time of independence in 1993 to 82 percent as of 2018. Ukraine has had relatively high rates of higher education coverage for the population for many years, starting in the early 1970s when the USSR implemented its policy of higher education for its citizens. The higher education coverage index includes students in both colleges and universities. It grew steadily from 1993 until 2007, mirroring the increasing demand for higher education and expansion in the system. After 2007 –2008, the coverage rate dipped somewhat due to the economic crisis of 2008. However, coverage has remained high, and is substantially higher than the average in the Europe and Central Asia (ECA) region, as well as the EU and OECD. This coverage has also led to a large increase in the number of students graduating from long- cycle programs. University graduates now comprise over 80 percent of all tertiary education graduates (see figures 4.1, 4.2 and 4.3). Figure 4.1 Tertiary education gross enrollment ratio 109 90 80 70 60 Percent 50 40 30 20 10 0 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Ukraine ECA EU OECD Source: Authors’ analysis of UNESCO Institute for Statistics, World Bank EdStats, and SSSU data Figure 4.2 Population age 25+ with tertiary education 45 40 Ukraine 35 Percent of population age 25+ 30 OECD Sweden 25 Germany Poland 20 Hungary Chile 15 Portugal Argentina 10 5 0 1980 1985 1990 1995 2000 2005 2010 Source: Authors’ analysis of UNESCO Institute for Statistics, World Bank EdStats, and SSSU data. Figure 4.3 Increasing demand for long-cycle over short-cycle tertiary education 110 600 90% Percent of total tertiary education graduates Number of people (in thousands) 80% 500 70% 400 60% 50% 300 40% 200 30% 20% 100 10% 0 0% 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 Graduates of tertiary education (short-cycle) Graduates of tertiary education (long-cycle) University graduates as a share of total tertiary education graduates (right axis) Source: Authors’ analysis of data from SSSU. Access to higher education continued to expand for several years, even in spite of a shrinking student- age population. If demographics were the only factor of change of the rate of admittance to HEIs, the rate would have begun to decline back in 2004, which is the point at which the number of secondary school graduates began to decline in line with demographic trends. In fact, accelerative expansion of access to higher education had been stronger than the demographic factor for at least three years. Only after 2007 did the negative demographic dynamics exceeded the wider access of higher education, and the number of students began to decrease. Private HEIs have been more impacted than public HEIs by negative demographic dynamics: the private higher education sector shrank by almost 4 times within 10 years, while the public sector shrank only 2 times. The share of students in private universities peaked in 2006/2007 and 2007/2008 academic years (16 percent), and since then it has reduced to 8 percent in early 2017/2018 academic year, comprising only about 105,000 students (figure 4.4). Figure 4.4 Enrollment trends in academic HEIs, by type of ownership (2005–2017) 111 98% 100% 100% 97% 93% 93% 98% 100% 97% 87% 82% 91% 78% 83% 66% 63% 63% 61% 70% 61% 50% 42% 34% 31% 29% 28% 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 state and communal HEIs private HEIs all HEIs Source: Authors’ analysis of SSSU data. Note: 2007 = 100 percent. The network of public HEIs remains oversized relative to the total population, which continues to decline. The total population of Ukraine is approximately 42.4 million, and Ukraine has a network of 327 universities, academies and institutes, of which 231 are public. This amounts to 7.7 HEIs per 1 million population, or 5.4 public HEIs per 1 million.16 While the total number of public and private HEIs varies significantly across countries, even after standardizing by population size, Ukraine appears to have a relatively high number of public HEIs relative to its population in comparison with other countries in Europe (figure 4.5). Figure 4.5 Density of Higher Education Institutions Relative to Population Size in Ukraine and Selected Countries (2017–2018) 12.0 HEIs per 1 million population 10.0 8.0 6.0 4.0 2.0 - Ukraine Poland Germany Hungary Romania Spain All HEIs Public HEIs Source: Authors’ analysis of EDEBO for Ukraine, excluding colleges; EACEA for selected EU countries. 112 Given the declining population, many HEIs in Ukraine enroll relatively few students which may pose challenges both for financial sustainability and quality of service delivery. The network of colleges and technical colleges in particular appears to be oversized relative to student enrollment and the broader population. There are over 300 public colleges and technical colleges in Ukraine, with an average enrollment size of approximately 550 students. Although the majority of students go into long-cycle degree programs in universities, academies and institutes, rather than colleges, even these institutions enroll relatively few students. Around 60 percent of academic HEIs enroll fewer than 5,000 students, particularly academies and institutes where the average enrollment size is around 680 students for institutes and 1,800 students for academies. The average university enrolls around 6,200 students, although there is wide dispersion due to some outliers: 4 universities enroll over 20,000 students each, while another 15 universities enroll fewer than 2,000 students (figure 4.6). Figure 4.6 Enrollment size of public higher education institutions, by type 70 60 Number of institutions 50 40 30 20 10 0 1,000 or 1,000-3,000 3,000-5,000 5,000-10,000 Over 10,000 fewer Academy Institute University Source: Authors’ analysis of EDEBO data. Even though the number of HEIs has decreased over time, along with the number of students, the number of university lecturers has decreased much more slowly, creating an inefficient network with low student-faculty ratios. As of the 2017/2018 academic year, there are approximately 10 students per every 1 university teacher in Ukraine, which is low by international standards. The typical ratio in European countries ranges from 12 to 20 students per teacher (see figure 4.7). This low ratio reflects a clear failure of the system to adjust to the declining student-age population, particularly in terms of staffing. Figure 4.7 Student-Faculty Ratio in Tertiary Education, 2012–2017 113 Romania 24 22 Student-faculty ratio (headcount basis) Poland 20 Finland UK 18 Moldova 16 Hungary Sweden 14 Belarus Ukraine 12 10 2012 2013 2014 2015 2016 2017 Source: Authors’ analysis of SSSU and UIS data. Today, in terms of student enrollment, the majority (75 percent) of the 1.57 million higher education students in Ukraine are enrolled in long-cycle degree programs in universities, academies and institutes. Today, about 47 percent of students in HEIs are in bachelor’s degree programs and another 24 percent are in master’s degree programs in universities, academies and institutes. This reflects how the composition of the higher education system has changed over time, with a large increase in the number of universities and a large reduction in the number of colleges between independence and the mid-2000s (see figure 4.8). Even so, about 25 percent of higher education students are enrolled in short-cycle junior specialist programs in colleges and branch colleges. Figure 4.8 Composition of higher education system (1991–2018) Universities, academies and institutes Colleges 900 800 400 2500 800 700 350 2000 700 300 600 People (thousands) People (thousands) 600 Institutions Institutions 250 500 1500 500 200 400 400 150 1000 300 300 100 200 500 200 50 100 100 0 0 0 0 1991 1994 1997 2000 2003 2006 2009 2012 2015 2018 1991 1994 1997 2000 2003 2006 2009 2012 2015 2018 Academic HEIs Students Colleges Students Entrants Graduates Entrants Graduates Source: Authors’ analysis of SSSU. 114 Note: Data since 2014/15 exclude the Autonomous Republic of Crimea and temporarily occupied territories in the Donetsk and Luhansk regions. About 44 percent of students studying in HEIs are studying on state-funded places, while the remaining 56 percent are paying tuition fees, with a large share enrolled in extramural and evening programs.17 The highest share of state-funded students (67 percent) are studying toward junior specialist’s degrees, compared to 38 percent for bachelor’s, 35 percent for master’s and 26 percent for specialist’s degrees. Although nearly all publicly funded students are enrolled in regular intramural programs, self- funded students are split more evenly between intramural programs and extramural/evening programs (56 percent and 44 percent, respectively). Over 60 percent of students enrolled at the bachelor’s level are pursuing their education on a self-funded contractual basis (figure 4.9). Figure 4.9 Enrolled students, by program, type of funding, and form of study Specialist Master Bachelor Junior Specialist 0 100000 200000 300000 400000 500000 600000 700000 800000 Number of enrolled students State-funded, intramural State-funded, extramural and evening Self-funded, intramural Self-funded, extramural and evening Source: Authors’ analysis of EDEBO database. Over 70 percent of students continue from bachelor’s into master’s degree programs, reflecting a tradition from Soviet times when students obtained ‘specialist’ degrees that were roughly equivalent to master’s degrees. The former specialist degrees took 5–6 years to acquire after completing secondary education. Because this degree was so common, and the transition to the three-cycle degree structure in line with the Bologna Process has been recent, many students, parents and employers continue to perceive bachelor’s as ‘incomplete’ higher education. Until 2014, this was explicitly stated in the law. Furthermore, many professions (for example, school and university teachers, doctors, lawyers, and judges, among others) require at least a specialist’s or master’s degree. This contributes to the situation in which approximately 72 percent of bachelor’s degree graduates entered master’s programs in 2017 (and 74 percent in 2018). 18 However, this is gradually changing, in part due to opportunity costs of continued higher education. Given Ukraine’s weak economic position, which has not yet returned to the pre-crisis levels, many young people cannot afford to continue higher education as they might have done in the past. Improving Efficiency and Transparency of Financing Although spending has come down in recent years, Ukraine continues to spend a high share of its GDP on tertiary education, higher than in neighboring countries and the OECD average. At 1.6 percent of GDP, the fraction of spending on higher education as a percentage of GDP is twice as high in Ukraine as in other countries of the former Eastern Bloc, or compared to the average OECD of 1.1 percent. In part, the difference is explained by the lower fraction of private funding in higher education in some countries of the former Eastern Bloc, and the lower number of years required to obtain a higher education degree (3 years of bachelor’s studies, while in Ukraine obtaining a bachelor’s degree takes 4 years), as well as more years of study in secondary education. Spending on tertiary education in Ukraine has come down 115 from 2.1 percent in 2012, though it remains high, comparable to levels of spending in high-income countries with robust higher education systems such as Australia, Republic of Korea and the United Kingdom (figure 4.10). Figure 4.10 Tertiary education spending in Ukraine Total expenditure on tertiary Government spending on tertiary education as % educational institutions as a percentage of GDP 2.5 of GDP (2015) 1.8 1.6 2 1.4 Finland Percent of GDP 1.2 Percent of GDP 1 1.5 Ukraine 0.8 U.K. Germany 0.6 Poland 1 South Korea 0.4 0.2 Japan 0 0.5 Germany South Korea UK Australia Poland EU avg. Hungary Lithuania Ukraine Estonia OECD avg. Finland Czech Republic 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Sources: World Bank (2017d); OECD (2018a); EdStats.19 Historically, tertiary education spending in Ukraine has been higher than spending on the other educational levels. In part, the reason for this is that household funding is involved in higher education, while at the other education levels official private funding is practically nonexistent. (However, parents do make sizeable unofficial payments to secondary schools). At the same time, in past years the share of higher education spending has been falling gradually. In 2007, higher education took 42 percent of the total education spending (both public and private), while in 2016 its share was 36 percent.20 The share of private spending for higher education in Ukraine is higher than the EU and OECD average. In Ukraine, 34 percent of spending for higher education comes from private sources, compared to 31 percent in the OECD, 22 percent in the EU, and 16 percent in Poland.21 In Ukraine, almost all private spending comes from households, while in the Russian Federation or Hungary, for example, where private spending is also high, approximately 33 percent of spending comes from other private sources, mostly businesses. At the same time, public HEIs in Ukraine have grown more dependent on public funds over the last 10 years. The share of private income in the total income of public HEIs has decreased slightly, from 39 percent in 2007 to 36 percent, though this decrease could have been larger given the overall decline in the number of students during this same period. An increase of tuition fees in 2016 also led to a modest increase in the share of private income. The system of public funding of universities in Ukraine was inherited from the USSR and modified only partially since Ukraine became independent. The funding is allocated through the system of public procurement of the training of professionals in higher education institutions, which is described in the 116 relevant law and two government decrees.22 In this process, ministries which oversee universities— primarily MOES, Ministry of Health, Culture, and Internal Affairs—transmit suggestions to the Ministry of Economic Development and Trade (MOEDT) about the number of publicly funded study places for each major.23 The MOEDT aggregates this information and coordinates them with the so-called mid-term forecast of labor market needs. After this, the MOEDT develops a draft of the number of publicly funded study places per major, which is approved by the Government (table 4.4). Typically, this decree includes more than 200 pages of information on the number of state-funded places for each major and level of tertiary education, for both full-time and part-time programs.24 After the number of publicly funded study places was determined, each ministry that oversaw HEIs distributed the study places between its institutions: a process that is disconnected from a strategic vision for the sector or specific needs of HEIs. To distribute the places across HEIs, the ministries formed competition committees that determined how many publicly funded places should go to each university at different education levels and in different majors. This process was at times politically motivated, with HEIs loyal to the ministry receiving more study places and hence more public funds. The committees did not use objective performance indicators in their decision making, and they had no evidence of which HEIs ‘deserved’ to receive more or fewer study places. In its report, the Accounting Chamber of Ukraine analyzed the process of public procurement distribution and concluded that the allocation of publicly funded places to HEIs under MOES is done in a manual way, under the guise of a formal competition procedure.25 Table 4.4 Number of state-funded study places, by level of education (2018) Level Full-time Full-time,% Part-time Part-time,% Junior specialist 35881 91.5% 3329 8.5% BA 72073 91.3% 6882 8.7% MA 53520 86.0% 8683 14.0% Merged MA (health care programs) 5855 100% 0 PhD 3548 80.7% 851 19.3% Doctor of sciences 478 100% 0 Total 171355 89.7% 19745 10.3% Source: https://zakon.rada.gov.ua/laws/file/text/64/f476391n51.doc. Nominally, according to the law, the forecast made by the MOEDT must be based on economic and labor market data, but in practice the forecast is not well oriented to the needs of the labor market. The challenges faced by the MOEDT in developing a rigorous forecast that guides the higher education sector are well-recognized.26 Because the MOEDT is supposed to conduct this forecast, which theoretically inform its decree on the number of publicly funded places, universities rarely examine the needs of the labor market and the economy themselves. However, even if the forecasts of labor market needs did not have any shortcomings, their results would still be difficult to map on the list of majors in higher education, which is used to determine the numbers of publicly funded study places. The Government’s policy for higher education funding means that at least half of potential higher education students will be funded by the state, regardless of overall demand or level of performance. Traditionally, the Government assigns more state-funded places to full-time programs and to the bachelor’s degree level, and a lower number of places to the junior specialist’s and master’s levels. However, the current legislation requires that the Government maintains certain minimal numbers. 27 The number of places for bachelor’s and junior bachelor’s28 levels cannot be lower than 51 percent of the number of high 117 school graduates in the corresponding year. The number of places in master’s programs cannot be lower than 50 percent of the publicly-funded bachelor’s graduates in the corresponding year. In terms of majors, in the last several years the MOEDT has been increasing or maintaining the number of public procurement places for engineering and science majors, while decreasing the number of places in social sciences. To some extent, this move balances out the recruitment of students for different majors because the students that select the popular majors in the social sciences typically have to pay tuition fees. Majors in engineering and sciences, however, which tend to be less popular among prospective students, are more likely to be publicly funded. Although the mechanism for determining the number of state-funded study places remained unchanged for many years, the allocation mechanism for distributing those places between HEIs was radically changed in 2016 in an effort to improve transparency in the system. In 2016, the regulations of the new 2014 Law on Higher Education came into effect and radically changed the system of distributing state-funded places between universities at the bachelor’s level. The competition system was replaced with an algorithm.29 This algorithm forms a national ranking of university applicants (based on their EIT results) who applied for each particular major in all universities. Each applicant can submit up to 7 applications for up to 4 majors in up to 7 universities. The applicants mark each of their applications with priority indicators from 1 to 7, in which 1 is their most desired choice and 7 is their least desired choice. After the call for applications ends, the algorithm checks each of the applications, from the most desired to the least desired, to determine whether the specific applicant is eligible for a publicly funded place in the university to which he or she has applied. When the algorithm finds an eligible application, it ignores all the other applications from the same applicant. The procedure is applied to all the applications of all the applicants who have applied for public procurement places. Each major has a fixed number of public places (determined by the MOEDT), so when the algorithm fills all the places with student applications, it does not consider that particular major as an option for any additional applicants. When the algorithm finishes its work, each applicant that was found eligible for a certain publicly procured place has to submit the original copy of their application to that university before a certain deadline.30 The algorithm has improved transparency in the distribution of publicly funded places, and MOES has taken additional steps to improve transparency at other levels as well. The algorithm helps to ensure that publicly funded study places follow the choices of applicants themselves. This has eliminated the influence of the ministries on the allocation of publicly funded places at the bachelor’s level. The algorithm was only introduced at the bachelor’s level because the EIT only exists at this level, allowing for the formation of the national ranking of applicants for each major. However, in 2017, MOES introduced a university ranking for allocating publicly funded seats for master’s programs according to certain objective criteria, such as the number of bachelor’s graduates, the number of foreign students, the number of professors who have published their work in journals indexed in Scopus and the Web of Science, and so forth. The same practice was used in 2018. MOES also wants to extend the use of this practice to the junior specialist’s level. While these reforms have improved transparency in the distribution of publicly funded places, funds are still allocated according to MOEDT quotas across study fields. This has led to very uneven per- student expenditures across institutions. For the HEIs under MOES, the 10 biggest universities in terms of public funding together receive more than 30 percent of the total funding that MOES allocates for education of university students. Among these 10 universities, 2 HEIs are the most prominent —the Igor Sikorsky Kyiv Polytechnic Institute and the Taras Shevchenko Kyiv National University—which together receive almost 13 percent of public funds distributed by MOES, although they educate less than 5 percent of all state-funded students. In general, some universities receive more funding not only because they are large in size, but also because they leverage considerable political influence, which has resulted in higher rates 118 of funding. For example, 5 of these 10 universities have special coefficients (provided by presidential decrees) which have increased their faculty’s base salaries by 2 times (see annex 6). Public funding for higher education is not based on the cost of service delivery, which contributes further to funding distortions across HEIs and majors. The indirect distribution of funding through the distribution of publicly funded places should theoretically ensure that the Government provides each public place with the funding it requires based on the cost of delivering education. For example, the cost of higher education in engineering is obviously not equivalent to the costs of education in law or business. In practice, Ukraine does not use any financial norm that must cover all the university’s costs of teaching each student. In addition, the universities themselves cannot send the government a bill to cover their expenses. This means that each year, MOES arithmetically divides the amount of total education funding by the number of students who occupy publicly funded places. A student who studies philosophy and a student who studies aviation and rocket technology ‘cost’ the same to MOES, although the ‘cost’ of state -funded places only exists as a matter of analysis, not for operational purposes of managing funding. Differences may occur in cases in which a university has higher salaries or other additional coefficients, but not between different majors. There are also differences between ministries. MOES and the Ministry of Health (MOH) have universities that offer the same health care curriculums, and these ministries distribute publicly funded places across their respective universities. At MOES, one publicly funded place affords the university a higher amount of funding than at MOH—29,141.6 UAH31 versus 25,820.9 UAH in 2016.32 Therefore, the amount of public funding is neither based on the cost of training students nor on specific national norms. The recently introduced mechanism for allocating budget seats, combined with the fact that funding is not based on the cost of service delivery, creates significant challenges for the financial planning and sustainability of universities. The new mechanism for allocating budget seats means that public funding for universities depends on the number of the applications. This creates the situation in which universities cannot adequately plan for the number of faculty members they will need. HEIs prepare budgets based on planned salaries, utilities, and other expenses, which are then submitted to MOES. MOES either approves or rejects different quantities of staff workloads (stavkas), including additional payments. However, the funding in the state budget does not consider specific requests of HEIs. In practice, the funding for higher education in the state budget and the number of state-funded places (which are distributed to separate HEIs) are determined independently. Private tuition fees paid for by students studying on a contract basis are substantially lower than the per-student funding received by the state, meaning that HEIs use public funds to subsidize tuition- paying students. Out-of-pocket payment for higher education became very popular in the 1990s, and remains in practice today. This means that the Government does not cover all the expenses of universities and provided insufficient funding for teaching in publicly funded places. As a result, HEIs are given permission to offer services for a fee, with the funding from tuition fees serving to compensate for public underfunding. This has led to several challenges. First, HEIs has tried to accept as many students as possible, even students with poor EIT results, thereby prioritizing a student’s ability to pay the tuition fees over their preparedness for higher education.33 This contributes to the deterioration of higher education quality. Second, HEIs are conservative about the tuition fees they charge, with a few exceptions.34 The majority of HEIs charge fees that are two to three times lower than the amount of public funding they receive for one publicly funded place.35 Because HEIs do not want to deter potential students by raising tuition fees, most HEIs try to keep them low, meaning that the education of tuition-paying students is, to a certain extent, subsidized from the public budget.36 This resulted in very different levels of per-student spending: for publicly funded students, the average per-student expenditure across both full-time and part- time programs is about 31,000 UAH, compared to about 16,700 per self-funded student.37 There has also been a sizeable increase in the enrollment of international students over time, as this has been a mechanism for HEIs to attract more fee-paying students. Over time, the number of HEIs 119 with international students has increased from 185 to 443 over the last 3 years, according to MOES Ukrainian State Center for International Education.38 The total number of students has increased from about 53,000 in 2011 to over 75,000 as of 2018, with a large share of students coming from countries such as India (19.8 percent), Morocco (9.8 percent), Azerbaijan (8.2 percent), Turkmenistan (6.7 percent) Nigeria (4.7 percent), Egypt (4.5 percent), and Turkey (4.3 percent). Medical universities are the top choice for these international students. However, international students are largely attracted by lower entry requirements, rather than high quality of education, and HEIs have strong incentives to cover funding shortages by admitting more fee-paying students. However, given the strong element of corruption in higher education, particularly in medical higher education, and the concerns over the quality of medical graduates, several countries, such as Saudi Arabia and Kuwait, no longer recognize Ukrainian medical diplomas.39 Private HEIs are restricted from accessing public funding. Although the 2014 Law on Higher Education technically allows private education institutions to receive public funds, it is still prohibited by the budget code. This puts private HEIs in a less competitive position relative to state and municipal institutions, creating the incentive for private HEIs to admit students with low scores on the EIT indicating that such students are not prepared for the rigors of higher education. A final but crucial shortcoming of the higher education funding system in Ukraine is that it prioritizes quantity over quality. The funding system does not take into account the quality of curricula or instruction in HEIs. In general, these institutions receive funding for the number of students they have accepted rather than for the quality of education they provide. The more students enrolled in a university, the more funding it receives. With the newly introduced algorithm at the bachelor’s level, funding follows student applications, but students have little information on the quality of education in different HEIs (for example, graduate employment rates, employer feedback, and so forth). This means that students’ choices are often based predominantly on stereotypes about the prestige and popularity of a particular major and institution. At the master’s level, MOES has started to form a ranking of HEIs based on certain criteria in order to distribute publicly funded places, but these criteria do not always definitively reflect the quality of education programs. For example, the number of foreign students in a Ukrainian university does not necessarily indicate a high level of quality, since universities have a strong financial incentive to attract and retain students, both domestic and foreign, regardless of quality or the student’s performance in their program. Higher education financing reform was excluded from the 2014 Law on Higher Education, but a requirement to allocate public funding for higher education by formula was included in the budget code in 2017. The concept of strategic funding for higher education, including performance-based funding, has been under discussion for at least the last several years. The budget code in 2017 included the direct requirement that the Government allocate public funding for higher education according to a formula that takes into account the number of students, the EIT, and performance of HEIs. MOES is now charged with developing such a formula based on adequate performance criteria for HEIs. Draft legislative acts on improving the system of higher education financing have been developed and are now being negotiated within MOES. A funding formula has also been developed with extensive local expertise, and modeling of the formula and related calculations have been carried out. However, necessary legal actions have been delayed. In general, progress in this area has been slow; there are strong vested interests resisting change. Ensuring Transparent and Equitable Access to Higher Education Until 2008, admissions to higher education institutions for all levels of tertiary education were organized through separate internal entrance exams in each university, reflecting a high level of autonomy in admissions. These were mostly written exams in 2 or 3 separate disciplines, carried out by examination commissions. In other cases, entrance exams took the form of oral examinations or interviews. For some majors which required it, there were also creative or athletic tests. All the regulations for entrance 120 exams and the rules of admission were determined by the higher education institutions themselves.40 That is, universities had wide autonomy in terms of selection and admission of applicants. It allowed them to select those students whose knowledge and motivation best matched the demands of the particular university’s curriculum. Applicants’ chances to be accepted to universities were also affected by two other factors: the logistics of admission campaigns and the system of distributing publicly funded places. Prospective students had to submit applications to the universities they selected in person, the applications had to be in a written form, and the application process lasted only 10 days. This created many inconveniences for applicants and their parents, who were physically unable to cover more than a few institutions, because sometimes people had to wait to submit their applications in a live queue for 1 –2 days. The manual distribution of publicly funded places, which was then managed by MOES, artificially made the competition in some universities excessively intensive or lax. Given that the distribution did not necessarily take into account the popularity of universities among applicants, it resulted in situations in which the universities that received a higher number of applications had proportionally lower number of publicly funded places, and vice versa. This system created many opportunities for corruption, bribery, and abuse, which undermined social trust in higher education. There is extensive research in Ukraine describing the scope for corrupt access to higher education and the history of abuse in university admissions. Prior to 2008, when EIT was introduced, an estimated 33 percent of HEI students provided bribes to be admitted to HEIs.41 Even MOES acknowledged that 15–20 percent of students in 2005 and 2006 were admitted to HEIs through bribes.42 Complex logistics and inappropriate levels of competition contributed to the corruption risk.43 The introduction of the EIT is widely recognized as a major success in curbing corruption in university admissions and increasing public trust in higher education. Since 2008, MOES has made it mandatory for universities to use EIT certificates for selecting applicants. In 2014, this decision was made official in the Law on Higher Education. Today, the law states that each applicant’s entrance grade is determined on the basis of the EIT certificate (each certificate makes up at least 20 percent of the competition grade) and the average grade in their Certificates of Secondary Education (up to 10 percent of the competition grade). Surveys show that the majority of Ukrainians (57 percent in 2016) support the introduction of the EIT (table 4.6). Table 4.6 Public support for external testing for higher education admissions October December December December 2013 2014 2015 2016 Definitely yes 17.8% 23.5% 18.5% 22.6% Rather yes 36.1% 36.0% 33.5% 34.0% Rather no 16.1% 15.6% 17.7% 12.8% Definitely no 9.6% 6.7% 7.3% 9.0% Difficult to say 20.9% 18.2% 23.1% 21.6% Source: https://dif.org.ua/article/vishcha-osvita-v-umovakh-reformi-zmini-gromadskoi-dumki_2016. Note: Question posed was as follows: “Since 2008, the main criterion of student admission to higher education institutions (HEIs) are the results of external testing. Do you support this system of selection to HEIs?” 121 In 2015–2017, universities were allowed to decide for themselves which EIT subjects to require for admission to each major, though this created perverse incentives for HEIs. MOES has since moved away from this policy.44 As a result, some universities, trying to recruit as many students as possible, required EIT certificates which were not necessarily relevant for the major itself, but which were more likely to provide students with higher incoming grades. For example, engineering majors often dropped the requirement for physics EIT certificates and required geography instead, and medical majors required Ukrainian history or geography instead of chemistry. When the algorithm for distributing publicly funded places was introduced in 2016 (see financing section), this created a situation when the same national ranking for certain majors, such as medicine, included applicants whose EIT scores had different components. Given these perverse incentives, in 2018, MOES started to mandate the specific EIT certificates that universities had to require from applicants for each major.45 Until 2018, students in colleges were allowed to bypass the EIT when entering universities; in 2019, this policy is changed. Until 2019, students in colleges or technical colleges were allowed to bypass the EIT when entering higher education at the university level, because students that received junior specialist’s diplomas from colleges could apply directly to the second year of a bachelor’s program without EIT test results, using only traditional internal entrance exams. It is important to remember here that secondary school students mostly enter junior specialist’s programs after grade 9 and study the 10th and 11th grade curriculum at the same time as the junior specialist’s curriculum, followed by taking final high school exams and receiving Certificates of Secondary Education. Since 2018, the final high school exams in Ukrainian language for students in colleges and technical colleges have been merged with the EIT test.46 This means that all high school students as well as those students who left in 9th grade to study in colleges now have to take the EIT test in order to access university education. From 2019, they will also take EIT tests in either mathematics or Ukrainian history, depending on the student’s choice.47 The procedures of admission to master’s programs are still perceived as untrustworthy by the society. In 2016, 44 percent of the population believed that corruption in education is the highest at the stage of admission to master’s programs.48 Trying to eliminate opportunities for abuse during admissions to master’s programs, MOES started to introduce EIT tests for entering master’s programs in 2017. As of 2019, admissions to law and international law majors are completely independent from universities and organized through EIT tests. Applicants take tests in a foreign language, as well as complex tests consisting of a subtest of general educational legal competencies and a subtest of eight basic legal disciplines. In addition, the foreign language EIT test is taken by those who apply for master’s programs in social sciences and humanities, as well as in IT. In the future, MOES plans to introduce foreign language EIT tests for admissions to all master’s majors. However, the professional knowledge for each major (except for law and international law) will still be tested by universities themselves at their own internal exams. In 2016–2018, MOES considerably expanded the use of EIT tests in admissions campaigns at different levels of higher education. Today, universities have their own internal exams only for specific exceptional categories of bachelor’s degree applicants (which will be discussed below), and only in the case of professional exams (without the foreign language tests) for master’s applicants. This policy is likely to improve the society’s trust in admission procedures but limit university autonomy and reduce universities’ ability to find students with the required knowledge, skills and motivation. The EIT has come to be used as an instrument for addressing problems of higher education quality for which it was not designed. In 2018, MOH and MOES made a joint decision to use the EIT as an instrument to set a quality threshold on incoming students who have low EIT test results but can pay tuition fees. For a long time, medical schools accepted tuition-paying students despite their possible lack of knowledge needed to master the higher medical education curriculum. Since universities did not want to give up this practice, because they used it to compensate for the lack of public funding, MOH and MOES introduced a requirement of at least 150 points in each EIT subject for students admitted to medical majors. 122 In 2019, these limitations still apply, and a similar practice (at least 130 points) will be used for law, international law and public administration majors; the Ministry of Justice wants to use the same approach to reduce the number of bachelor’s students in law. Although the establishment of minimum entrance requirements is a common approach to control access to higher education programs, the setting of such requirements does not address underlying issues of quality in HEIs related to inefficient funding or academic integrity violations, which influence the quality and capabilities of higher education graduates. The expanding use of the EIT has also been accompanied by an increasing number of exceptions and special coefficients used to facilitate access for priority groups or study fields. For example, there are exceptional rights not to take the EIT, afforded as a form of social aid given to certain categories of applicants, such as orphans, students with disabilities, applicants living in temporarily occupied territories in eastern Ukraine, and other special protected classes of applicants. There is also a system of special coefficients that increase the competition grades of certain categories of applicants. The rural coefficient increases the competition grades of applicants from rural areas by 2 percent and the grades of applicants from rural areas applying for agricultural majors by 5 percent. The industry coefficient increases the competition grades of applicants applying for engineering majors and giving these majors first or second priority by 2 percent. The regional coefficient helps regional higher education institutions rather than prospective students. If students apply to these universities, their competition grades increase by 2 –4 percent depending on the region, which means that regional universities have higher chances of enrolling these applicants. However, these measures are not part of a systematic affirmative action policy aimed at reducing inequalities in access to higher education, but rather it is a piecemeal approach to adjusting the competition grades for priority groups of students or study fields. The EIT needs to be reviewed and modernized in light of the New Ukrainian School reform, its objectives, and global best practices for test design. There is a growing need to review and update the EIT itself, as well as its design and methods. For example, the content of the test does not allow for a proper assessment of the level of systematic knowledge or the ability to think logically, and consequently the results reflect poorly the applicants’ real level of knowledge. This becomes even more urgent in the context of the New Ukrainian School competency-based curriculum. Furthermore, there is a view among students and parents that there is a mismatch between the EIT and the school curriculum. This can be disadvantageous for secondary and tertiary education, since it contributes to a perception among students that they do not need to study school subjects systematically or learn to think and perform creative tasks. They may think it is sufficient just to study hard (often with tutors) who will train them successfully to solve the EIT tasks.49 Governing the Higher Education System Governance of the higher education system refers to the regulatory framework and associated structures, policies and processes that enable HEIs to operate. In terms of the governance of the higher education system, several structures are important, including adequate coordination among the various components of a diversified system, as well as adequate institutional autonomy and accountability measures. Together, these components help a tertiary education system function more effectively to meet national and local needs.50 A clear division of labor that recognizes the unique role of different types of institutions in a higher education system is important for sound governance. Different institutional missions entail different goals and expected results. At the same time, a differentiated system requires an enabling governance structure that facilitates articulation, or the transition of students between different types of institutions. The passage of the 2014 higher education law advanced and improved higher education governance in Ukraine, but important challenges remain and the country lacks the means in terms of information and 123 steering mechanisms to orient the sector toward competitiveness and performance and to steer it based on agreed outputs. Although the governance of higher education seems relatively simple according to legislation, the process in reality is complicated because of the incoherent transformations of the system structure over the past 20 years, poor quality of public administration, corruption, and international isolation. In Ukraine, the parliament sets the legislation, the cabinet (particularly MOES) develops secondary legislation and implements policy, and other ministries and agencies play a role in overseeing the universities under them. Additionally, the National Agency for Quality Assurance in Higher Education (NAQAHE) works to accredit universities, certifying the quality of their programs. However, there are numerous, practical barriers to the governance, not the least of which are the limitations on information and insufficient mechanisms for oversight. There are influences from outside the education policy sphere that affect how HEIs operate. There are at least three key examples of this. The first is that the funding allocation and use is mostly subject to general budgetary legislation with little specification to account for specifics of organization of the education sector. This means that universities have little financial autonomy because they must follow detailed regulations as budgetary organizations. Second, universities are subject to general legislation on licensing, as well as specific procedures for licensing educational institutions. Finally, regarding the funding of colleges, the responsibility of funding occurs at the regional level but without clear provisions or consideration of potential risks associated with or specific to HEIs. Because governance within the system is fragmented, the Cabinet of Ministers is required to make many decisions which diminishes responsibility of MOES over higher education policy and makes the policymaking process and regulatory framework more cumbersome. Many of the Cabinet of Ministers (CMU) decrees are within the realm of the education policy and should logically be adopted by MOES. However, the Law on Higher Education dictates they should be adopted on the level of CMU. There are two reasons for this. First, according to legislation and practice of the governmental process in Ukraine, any decisions that lead to changes in funding (amount or allocation mechanism) should be taken by CMU, not individual ministries, and many decisions can be argued to lead to changes in funding. Second, since many governmental agencies have subordinate universities under them, they request to have input in the decision-making process. This request creates its own challenges, since higher education is low on the agenda for these governmental agencies and they tend to have lower expertise in higher education policy.51 It contributes to the challenge because there is a lack of clarity as to the roles and mandates of other ministries and agencies when CMU decrees are up for adoption. For example, the Ministry of Justice sometimes takes a broad approach to its mandate, at times making significant changes to documentation despite being focused on conducting a legal review. There is no direct mechanism for bringing different parties to the table and encouraging agreement, contributing to delays and inefficiencies. Limited data and weak collection and analytical capacity in the higher education sector further constrains evidence-based policy making and governance in higher education. While MOES has the EDEBO database, this collects only limited information on student applications, enrollment, and graduation, but the data is not used much for analysis and policy making. MOES has limited capacity to collect additional data, and other existing data sources are not well integrated, which further limits analytical potential. For example, greater integration of data sources on higher education financing, employment and labor markets, university admissions and external independent testing, as well as student enrollment data, can greatly expand the range of analytical information available for policy making and governance in the sector more broadly.52 124 Strengthening Quality, Relevance, and Integrity Despite the high coverage in higher education, there is little information available on the quality and relevance of education delivered by individual HEIs. Currently, there are no systematic rankings of HEIs in Ukraine or valid measures of the quality of teaching and learning in HEIs. The STEP skills survey (described above) measures adults’ proficiency in literacy, numeracy and problem solving, and takes int o consideration differences by level of education. This is one of the few measures of learning that exists, though the sample is relatively small and it is not representative of individual HEIs. In terms of relevance, there are also no graduate tracer studies or other similar information indicating the relevance of particular study fields or education provided by particular HEIs. However, Ukraine recognizes the need to create a unified system for monitoring employment of graduates of higher education and VET institutions, and this is noted in the 2019 State Budget. In terms of the most influential global academic rankings, very few universities in Ukraine are included. The top global academic rankings of universities include the Shanghai Academic Rankings of World Universities (ARWU), the Times Higher Education (THE) World University Rankings, and QS Top University Rankings.53 In the THE World University Ranking for 2018, only one university in Ukraine— Taras Shevchenko National University of Kyiv—was among the world’s top 1,000 universities. However, it has been falling in the rankings in recent years, and as of the 2019 rankings, it is no longer among the top 1,000 universities.54 The QS Top University Ranking55 includes six universities in Ukraine, with their respective rankings out of 1,000 included in parentheses: • V. N. Karazin Kharkiv National University (481) • Taras Shevchenko National University of Kyiv (531–540) • National Technical University of Ukraine “Igor Sikorsky” Kyiv Polytechnic Institute (601–650) • National Technical University “Kharkiv Polytechnic Institute” (701–750) • Lviv Polytechnic National University (751–800) • Sumy State University (751–800) However, these rankings consider fewer than 5 percent of the more than 25,000 academic institutions worldwide, and they focus primarily on research productivity. It is worth noting that 57 Ukrainian HEIs participate in U-Multirank, the European Commission supported multi-dimensional international ranking. Results show that participating Ukrainian universities tend to perform well in the area of teaching and learning compared to research or knowledge transfer activities which can partially be explained by the current system-level organization of research and the fact that the vast amount of research funding goes to the national and specialized academies of sciences (see section below on research activity). Ukrainian universities also are ranked highly in terms of regional cooperation. In addition, two Ukrainian universities are among the top 25 according to U-Multirank 2019: Kharkiv National University of Economics named after Semen Kuznets (in the category of communication with the professional environment) and Ternopil National Technical University named after Ivan Puluj (in the category of international focus of degree programs). Some segments of the higher education sector, particularly in well-organized fields like information technology, have developed their own measures of HEI quality and relevance. There are several unofficial rankings of universities in Ukraine, such as the consolidating rating of Ukrainian universities compiled by the informational education resource Osvita.ua, the national rating ‘Top 200 Ukraine’, and international rankings from Scopus and Webometrics. Additionally, sectors and industry groups have developed their own rankings. For example, the web platform forum for IT specialists in Ukraine, Developers of Ukraine, conducted an extensive survey of nearly 3,000 current students and recent graduates who studied IT in 50 universities in Ukraine. Although this survey is not representative, it demonstrates a clear desire among the public and in the labor market to better measure the quality and educational value 125 of specific HEIs. This survey collected information on the extent to which employers value diplomas from a given institution, the level of institutional prestige, the relevance of knowledge for work in the IT sector, the modernity of the curriculum and correspondence with needs of the IT sector, as well as experience with bribery and corruption. This study field reflects a wide range of HEI quality and relevance as perceived by students (see figure 4.11). Figure 4.11 University Ratings for IT Programs in Selected Higher Education Institutions (2018) 10 Average rating (1 = low, 10 = high) 9 8 7 6 5 4 3 2 1 Prestige of university Employer value placed on diploma from university Relevance of knowledge acquired in university to IT field Source: Developers of Ukraine survey of 2,938 current students and recent graduates of IT programs, conducted May 2018. A recent diagnostic on Ukraine’s innovation and entrepreneurship ecosystem found that the quality and relevance of university training is in need of improvement, particularly in order to support entrepreneurship in Ukraine. Although some universities are very strong and closely linked to industry, many use out-of-date curriculums that are not well-aligned to the needs of the economy. Applied business, financial, and management skills are in short supply, and only a few universities actively promote entrepreneurship training and programs.56 Relevance of Study Fields in Higher Education Since Ukraine became independent, there has been a transformation of the study fields available in higher education. Research in individual social sciences and training in specific professions (for example, law, sociology, political science, psychology) in the USSR were significantly limited due to ideological considerations. At the same time, the sectoral structure of the Soviet economy was substantially different than the present structure of the economy of modern Ukraine. The USSR economy had high industrial production specific gravity, and the Ukrainian SSR was an agricultural ‘granary’ of the country, in addition to industry. Today, the largest number of workers in Ukraine is present in low-skilled services (see chapter 2). The list of study fields (majors) grew considerably along with the growth of the sector, reaching 500– 600 fields, and enrollment in social sciences fields also grew significantly. This was driven by several factors, including Ukraine’s transition to a market economy and growing demand for higher education in fields of relatively lower cost. At the same time, influential universities secured high-level resolutions from the government that certain majors would be protected from rapid changes.57 There has been a growth in the number of students studying social science fields like political science, public administration, law, journalism and international relations. There has been some decline in the popularity of engineering 126 sciences, an area in which Ukraine has traditionally excelled; however, this change has not been too significant. A number of majors (natural sciences, medicine, metallurgy) have maintained the absolute numbers of admitted students at the stage of higher education sector expansion through the mid-2000s, but those numbers have declined since the 2010s. Majors such as economics, human and computer sciences were sufficiently popular both in the USSR and in independent Ukraine (figure 4.12, table 4.6). Figure 4.12 Number of students of Ukrainian HEIs by Study Field and Major Agriculture Services Health and Welfare Education Social Sciences and Humanities STEM 0 5000 10000 15000 20000 25000 30000 35000 40000 45000 Students admitted to BA programs (2018) Privately funded State-funded Source: Authors’ analysis based on EDEBO data Table 4.6 The number of students of Ukrainian HEIs by study field and major Major Number of students, persons 1989 2005 2010 2017 Law 13135 176363 194099 118392 Human sciences 100971 101226 123529 83909 International relations 63 n/a * 13266** 14670 Journalism 1889 n/a 11358** 14461 History 20564 n/a ≈8000 n/a Pedagogy/education 88216 237191 154681 126826 Health care 50890 67801 74826 94681 Social sciences, except economics ≈2000 54070 79310 25391** Economics 100807 744010 605439 217997** Sociology 173 n/a ≈10000 n/a Public administration 0 2129 3356 8009 Natural sciences 71047 75424 99413 40041 Physics and Mathematics majors 35699 n/a 18385 4782** Chemistry 4611 n/a ≈1300 n/a Biology 13000 n/a ≈5000 n/a Engineering 309435 486440 403668 206977 Metallurgy and materials sciences 12740 n/a 11087** 2165** Computer sciences 16625 n/a 53933** 79881 Source: Authors’ analysis of SSSU data. Notes: Changes of the lists of majors in which specialists are trained caused various grouping of statistical information regarding majors in various years: for some years, the SSSU provides information on the number of students broken down by majors; for some others there is only generalized information on the number of students broken down by knowledge spheres, each of which includes several majors. * n/a — no data available. 127 ** — minimal assessment of the number of students. The SSSU submits the number of first-year students for a certain major, as specified in this Table. Also, in the respective year, there is a certain number of older students studying the given major, with the available statistics by knowledge spheres but not by specific majors of these knowledge spheres, so this data is excluded. The process of consolidating the selection of study fields and majors started in 2005, but it was not completed until the implementation of the 2014 Law on Higher Education. Initially, the process faced a strong degree of political opposition, initially leading to an increase in the number of bachelor’s degree programs to over 140 and majors to about 500. Following the 2014 Law on Higher Education, the Cabinet of Ministers also profoundly changed and classified Ukrainian study fields and majors.58 The list was substantially shortened to 125 majors. The list became unified for all higher education levels (from short- cycle to postgraduate PhD programs). Finally, there is now correspondence between 85 –90 percent of Ukrainian majors with ISCED,59 classification, which will facilitate cross-national comparisons. 60 Research Activity and Faculty Research activity is often used as an indicator of a higher education performance, though in Ukraine, research activities in universities are not very strong. Only 50 percent of HEIs perform any kind of research and development (R&D), receiving only 7 percent of state budget. Research is mostly carried out in HEIs that were formerly accredited at level IV.61 Ukraine has around 1,000 institutions which carry out scientific R&D activities, according to the SSSU. Universities represent a small share of funding, while the majority of R&D funding goes to the National Academy of Sciences of Ukraine (NASU), the largest scientific research performer and recipient of 50 percent of the state budget allocated for R&D. The second category is the five state-sponsored, specialized academies of science related to medicine, agriculture, pedagogy, arts, and justice. Ukrainian researchers are generally poorly represented on the international academic scene, such as international peer-reviewed journals. This is due to historical isolation dating back to the Soviet period and is reinforced now by limited use of English and other foreign languages in university throughout Ukraine. There is also evidence that research quality is often poor, particularly in certain fields, such as humanities and social sciences. An analysis of research output of Ukrainian universities shows that that majority of publications are concentrated in STEM fields. Publications of researchers of Ukrainian universities that were indexed by the Scopus database were analyzed in order to quantify the types of research output that are produced in Ukraine. The majority of publications—74.0 percent—were prepared by Ukrainian researchers in six fields: engineering (17.7 percent), physics and astronomy (17.4 percent), materials science (13.6 percent), computer science (9.3 percent), mathematics (8.5 percent), and chemistry (7.4 percent). Together, research output across all STEM fields accounts for nearly 90.0 percent of total research output, which reflects the historical legacy of fields that were given top priority in the Soviet period. However, it is notable that the fields favored by university students do not produce internationally visible research. STEM reflects the vast majority of research output, even though it only represents a small share of student enrollment. In comparison, while fields such as social sciences, economics, and humanities have higher student enrollment than STEM fields, they produce less than 5 percent of the research output. While it is common for international peer-reviewed journals globally to focus on STEM research, it is particularly the case for published works by researchers in Ukraine (figure 4.13). Figure 4.13 Research output of Ukrainian universities, by field 128 1% 2% Agriculture 6% Humanities and Social Sciences Natural Sciences 40% Economics, Business and 48% Management Technical and Applied Sciences Medicine and Health Care 3% Source: Authors’ analysis of Scopus database Research activity is highly concentrated in several leading universities, those which also receive the largest share of public funding and have large enrollments. The top five universities in terms of research activity account for 41.0 percent of all Scopus-indexed publications; these include the Taras Shevchenko National University of Kyiv (13.9 percent), V. N. Karazin Kharkiv National University (7.8 percent), National Technical University of Ukraine ‘Igor Sikorsky Kyiv Polytechnic Institute’ (6.9 percent), Lviv Polytechnic National University (6.3 percent), and Ivan Franko National University of Lviv (5.9 percent). The top 20 universities account for 69.0 percent of all research activity. However, it is important to account for the size of the university in terms of the number of faculty, which presents a different picture of which universities are most productive in terms of research per faculty member. Data on faculty were collected from the websites of 136 universities. Although legislation obliges universities to publish staffing lists, many do not do this, or they do not clearly distinguish between ‘education’ and ‘research’ staff. However, most staff are involved in education in one way or another, so combined staff lists were used for this analysis. Taras Shevchenko National University of Kyiv remains a leader in research activity, even after accounting for the number of faculty. However, other universities appear more efficient at research production on a per-faculty member basis. For example, the small private university Kyiv School of Economics ranks third in the number of publications per faculty member, even though this is a small institution with relatively few faculty members. Although research in universities is not strong due to limited budget, around 70 percent of Candidates of Sciences and Doctor of Sciences are working in the higher education sector, mainly involved in teaching rather than research. Currently, there are two kinds of doctoral degrees in Ukraine, considered as research and development personnel involved in scientific R&D: Candidate of Science (corresponding to a PhD) and Doctor of Science, the highest scientific degree in Ukraine, which requires an additional two years of research after Candidate of Science degrees. One of the main reasons why the majority of these researchers are working as university teachers rather than researchers is the low level of salaries for researchers.62 According to MOES, although absolute salaries of university scientists increased in UAH, the real purchasing power has decreased by 30 percent over the last 10 years. This helps to explain the preference for scientific personnel to work in teaching positions rather than as researchers (figure 4.14).63 Figure 4.14 Persons with science degrees in Ukraine 129 Source: Authors’ analysis of SSSU data from ukrstat.org (data in 2014 excludes Crimea, Sevastopol, and part of the anti-terrorist operation zone). The imbalance between the expenditure allocation on science, technology and innovation (STI) to NASU and the sectoral academies of science vis-à-vis universities affects Ukraine’s competitiveness negatively. The existing funding structure supports state-owned NASU and specialized academies with block funding which consumes around 75 percent of total budget, leaving limited resources for competitive funding mechanisms for researchers in universities and private sector commercial projects. This allocation is not well-suited to Ukraine’s future competitiveness, given that R&D outputs of NASU and the specialized academies of science tend to benefit traditional sectors rather than emerging, technology-focused and innovative fields.64 However, it should be noted that the role and mission of individual HEIs is also an important consideration in determining their performance and quality. Although a large number of HEIs do not produce any research, they may be performing other core functions, namely teaching and learning, and should be supported in that core mission. Furthermore, some smaller institutions have potentially greater research capacity on the basis of per-faculty research output, and they could be supported toward research as part of their core mission, even if they produce relatively little research in an absolute sense. Corruption and Academic Integrity in Higher Education Corruption and academic integrity violations in higher education remain endemic and a major concern. These violations inhibit opportunities for knowledge acquisition and training, which are essential factors for social mobility and economic development.65 Many researchers and observers have identified Ukrainian higher education as having significant challenges with academic misconduct and integrity violations.66 Integrity is vital to good governance and institutional trust, and a high-performing education system must have policies and structures in place to safeguard integrity. Integrity is one of the key pillars of political, economic and social structures. It is essential to the economic and social wellbeing and prosperity of individuals and society as a whole.67 At the higher education level, corruption and academic violations have the additional harmful effect of normalizing corrupt practices among young people.68 Corruption in higher education also increases social inequality, because people from poor families cannot pay bribes to get into prestigious universities or pass exams, and hence they miss their opportunity to use education for social mobility.69 The dominant problems of Ukrainian HEIs include a mass disregard for academic integrity and a 130 high tolerance for academic violations. Integrity violations can take many forms, ranging from misuse of resources, assets and authority, to bribery, cheating, plagiarism, and undue recognition of academic achievement.70 Tolerating mediocrity seems common, and systemic violations of academic integrity principles (for example, cheating during exams, or plagiarizing master’s and/or doctorate theses) are not perceived as abuses.71 Different estimates indicate that at least 25–30 percent of students have directly engaged in academic misconduct or bribery, with a larger share exposed to and familiar with such practices. According to the 2011 study by the Democratic Initiatives Foundation, 24 percent of students paid for exams, though by 2015 the subsequent survey showed that this figure had decreased to 18 percent. An even larger share of students confessed that they had given presents to their professors (32 percent in 2011 and 29 percent in 2015) or provided some kind of service to their professors, for example, buying the books they published (27 percent in 2011, 22 percent in 2015).72 Another survey by the Sociological Association of Ukraine reported that 23 percent of students claimed that students pay for grades in their departments. Faculty reported that 11 percent of students pay for grades. A 2018 recent study covering several public universities in Lviv found consistent evidence of both personal experience and general perceptions of corruption, particularly in the form of non-monetary academic misconduct (see figure 4.15).73 Figure 4.15 Experience and perception of academic misconduct Asking a professor for an indivdiual approach Buying a paper from companies or classmates Encountered bribes at universities Downloading a paper from the internet Deceiving professor while explaining study problems Writing a paper, but with extensive plagiarism Copying somebody's work during exams or tests Using cheat sheets during exams 1 2 3 4 5 From 1 (never) to 5 (systematically) General perception Personal experience Source: Denisova-Schmidt, Prytula, and Rmuyantseva (2018). While bribes from students to professors are the most reported type of corruption, they are by far not the only type. Most recently several large scandals have shaken the system of medical education, when the Ministry of Health dismissed rectors of two major universities (in Kyiv and Odessa). This has led to a broad discussion of the level of corruption in medical education. Yet, the ongoing debate often lacks data and proper analysis which could lead to proper policy interventions on different levels. Corruption in higher education is sensitive, and although the Ministry of Internal Affairs tracks criminal offenses, most forms remain hidden. Corruption is a strong motive behind many decisions at the national and institutional levels. While it is rarely spoken of publicly and by the university leaders, it remains a quiet force shaping universities’ interests and positions. Data are limited because many students are disinclined to report corruption abuse; they have been threatened by university faculty or administration to fail exams or be expelled from the university.74 Many students also believe that reporting on corruption cases will not yield any results, given the broader shortcomings in the justice system in Ukraine. Students face incentives to engage in academic misconduct, namely to compensate for an intense testing schedule and conditions which are not conducive to self-study. In Ukrainian HEIs, students have 131 to take close to a dozen exams every semester, often in mandatory subjects that are not related to the student’s specialization. For instance, engineers often have to pass exams in the history of Ukraine in their first year.75 Hence, students, particularly those that were less well-prepared in the first place, often feel overwhelmed, and with little motivation in the subject, may seek alternative means to get good grades. Additionally, students who live in dormitories are more likely to resort to cheating.76 Many dormitories do not guarantee quiet study places, and they cultivate a culture of sharing in the context of communal living. This appears to contribute to the normalization of academic misconduct in Ukraine. Furthermore, students who work part-time are also more likely to engage in nonmonetary academic misconduct, and previous research also documents the negative impact of part-time work on academic performance in higher education. There is also evidence that declining funding for universities and low salaries has created an environment tolerant to corruption and bribery. After the breakdown of the USSR, Ukraine and other post-communist countries experienced a significant reduction in funding for HEI budgets.77 Although the Ukrainian government funds 51 percent of university spots by law, it is not enough to cover university budget deficits.78 The cost of corruption in HEIs is also quite high, which may have the effect of further exacerbating equity concerns in higher education. Analysis from Transparency International (2015) shows a range of costs for different types of bribes in universities, from EUR 5–12 for passing a module in a regional university to EUR 30–35 for buying a thesis to over EUR 1000 in cases of admissions to master’s programs. Bribery costs are also higher in bigger cities, particularly in Kyiv given the higher costs of living, and in prestigious universities. These amounts add up, constituting a large financial barrier for some students and a massive waste of resources in general, especially compared to the minimum salary of approximately EUR 110 (according to State Treasury of Ukraine for 2017).79 There is also some evidence in Ukraine that bribery in HEIs has a corrosive effect on the perceived value that employers place on diplomas from those institutions. Although data are scarce, the afore- mentioned survey of students and recent graduates in IT fields collected information on students’ experience with bribery (figure 4.16). This survey indicated that for those universities in which more students have personal experience with bribery, their associated diplomas also tend to hold less perceived value on the part of employers. Although this survey is not fully representative, it does align with other research on the impact of corruption on the value of educational credentials. Figure 4.16 Bribery and perceived value of diploma by employers 3 Experience with bribery (1 = personal experience) 2 1 1 2 3 4 5 6 7 8 9 10 Rating of value employers place on diploma from given institution (1 = no value) 132 Source: Developers of Ukraine survey of 2,938 current students and recent graduates of IT programs, conducted May 2018. Quality Assurance Ukraine’s licensing and accreditation model of higher education quality assurance began as early as 1992. It was defined that licensing grants the right to a higher education institution to perform educational activity. Attestation was developed to determine the level of conformity of an HEI’s educa tional activity with the governmental requirements to education quality of a respective level, and it also granted the right to issue state-recognized diplomas.80 With only slight changes concerning details of licensing and accreditation procedure implementations, as well as changes of approaches to understanding the contents and procedures of higher education state standards preparation, this quality assurance model with the help of licenses and state-recognized diplomas is still in effect today. Educational activity in Ukraine is licensed, though licensing was historically not a serious barrier to entering the Ukrainian higher education market. To educate and train students, a university should obtain a state license to perform educational activity and train specialists at a certain educational level and major. While a license to educate is required, it has historically neither been a serious barrier to entering the Ukrainian higher education market nor a determinate of education quality. Licensing has not produced a serious effort at controlling quantity or quality since the number of public and private HEIs grew from 149 (including 3 private) to 351 (including 115 private) during the period from 1990 through 2007.81 The previous accreditation and licensing system had a number of challenges, including cumbersome and time- consuming procedures, monopolization of the licensing process for particular majors by specialist councils, insufficient expertise, and unclear differences between procedures of attestation and accreditation of HEIs.82 However, changes have been made to licensing. New requirements for licensees in higher education have been approved following the passage of the 2014 Law on Higher Education. These requirements were brought into line with the Law of Ukraine ‘On Licensing of Business Activity Types’ in the part where licenses became permanent, without time limit. Qualitative changes of requirements to HEIs concerned the appearance of such ‘international’ criteria as publications of teachers in scientific journals indexed by Scopus and Web of Science systems, participation in international scientific projects, delivering lectures in foreign languages, membership in editorial staff of a foreign reviewed scientific periodical, and possession of a certificate on foreign language command at least at B2 level.83 These requirements were formulated as ‘optional upon teacher’s discretion’, but one could assume that with time, they will become mandatory. Licensing of training programs for higher education recipients seeking a PhD at postgraduate courses was launched. Postgraduate course licensing led to complaints from scientific institutions which had difficulty meeting individual requirements to licensees, such as availability of a first-aid post and training premises with the area of no less than 2,000 m2. Significant consulting support provided by MOES to licensees helped all applying HEIs and scientific institutions to meet licensing requirements or substantiate cases when meeting such requirements was impossible or inexpedient. Accreditation is a more significant tool to ensure education quality. Current legislation understands accreditation of an HEI as state recognition of its status (accreditation level). Accreditation of majors or specialization in an educational institution by a certain educational and qualification level means state recognition of conformity of specialist training (retraining) level with the governmental requirements set for such major, specialization or HEI, as approved by MOES.84 After 2014, new approaches to the development of higher education standards were approved by law, including a competence-based approach to standards, defining skills and knowledge to be mastered by the student. To move away from previous practice, it is currently prohibited to establish fixed names of training disciplines in the standard: HEIs form their list of disciplines independently.85 133 Since 2014, Ukraine has legislatively established an obligation to rebuild the accreditation system for HEIs on the basis of principles of the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG).86 As of 2018, the implementation of this obligation is at its preparatory stage. The National Agency for Quality Assurance in Higher Education (NAQAHE), a newly established government agency, finally launched its new accreditation system in early 2019, after some delays. NAQAHE is charged with developing accreditation procedures and accrediting educational programs every five years. It will also support institutional accreditation, the establishment of a university ranking mechanism over the next 1–2 years, a system of enforcement of academic integrity, and regulation of the PhD and DSc granting system. Through this work, it is expected that NAQAHE would also support HEIs to develop their own internal quality assurance systems, although this is not the direct responsibility of NAQAHE. However, accreditation procedures are only now being developed, and work is underway to develop the regulation on accreditation of educational programs, though there is substantial shortage of trained experts to support accreditation once regulations are approved. Several projects and organizations (for instance, the Quality Assurance System in Ukraine Project (QUAERE) and the Institute of Higher Education of the National Academy of Pedagogic Sciences) are participating in the development of new procedures and training of experts, but this work is still far from being completed.87 Still, this process represents a positive development for the agenda of improving quality assurance and oversight in the higher education sector. As Ukraine’s higher education systems moves toward greater decentralization and institutional autonomy, monitoring progress in meeting institutional and systemwide goals is essential for quality assurance. The implementation of a Higher Education Management Information System (HEMIS) can be an important tool in this regard. Systematically gathering information about proxies for educational quality can help policy makers and other stakeholders evaluate progress and plan ahead. Such indicators may include student enrollment, retention, and graduation rates; related entry and exit tests; graduates’ transition to the labor market; and the value-added of attending a particularly HEI. Monitoring graduates’ transition to the workforce is essential to evaluate the relevance of offerings. Measures such as exit surveys and tracer studies are critical for HEIs to accurately assess their own performance or respond to the changing demand for skills.88 Conclusion The vision for higher education in Ukraine needs to be defined while advancing core reforms to improve quality, relevance, efficiency, and transparency. The lack of a clear or coherent vision for the higher education sector in Ukraine represents a fundamental challenge: higher education cannot serve the needs of the people and the economy without clear objectives and a strategy for how to achieve them. In this context, individual HEIs use their autonomy to achieve individual goals rather than working to achieve a broader goal for the system and nation. At the same time, there is a clear need to modernize teaching and learning, bring higher education programs more in line with the needs of the labor market, increase efficiency and sustainability of resources, and improve governance and transparency throughout the system. Without such reforms, the system will become increasingly disconnected from the needs of the people and the country, particularly given demographic, economic and technological trends. 134 Notes 1 For example, see Valero and Reenen (2019). 2 Hofmann and Reichel (2011). 3 Kupets (2013). 4 Sovsun (2019). 5 These figures differ somewhat from those provided by the SSSU, given that SSSU data reflects the start of the 2018/2019 academic year. EDEBO data was extracted in February–March 2019. 6 According to the law, “branches are territorially separated structur al divisions of a higher education institution established to meet the demands of regional labor markets in particular professions and provide proximity of a place of learning to the learners’ place of residence. A branch is not a legal entity and acts on the basis of regulation approved by a HEI and in compliance with the license obtained for educational activity.” 7 Students entering bachelor’s programs after completing general secondary education need to successfully complete the EIT for three school subjects: Ukrainian language and literature, mathematics or history, and an elective third option. College graduates would also be expected to successfully complete the test in Ukrainian language and literature, and one in either mathematics or history, but the third entrance exam would be a professional exam administered by the university. 8 Livanova and others (2016). 9 ETF (2017); MOES website on transfer of higher education institutions of levels I-II accreditation from state into communal ownership: https://mon.gov.ua/ua/osvita/visha-osvita/shodo-peredachi-vishih-zakladiv-osviti-i-ii-rivniv- akreditaciyi-derzhavnoyi-vlasnosti-z-derzhavnoyi-u-komunalnu-vlasnist 10 ETF Flash Report on launch of 2018-2020 Torino Process: https://www.etf.europa.eu/sites/default/files/2019- 03/Flash%20report_TRP%201_Ukraine%2021-22%20March%202019.pdf 11 ETF (2018). 12 There is a new draft Law On Vocational Education under discussion: https://zakon.rada.gov.ua/laws/show/103/98- %D0%B2%D1%80 13 According to Sichkarenko (2014), in early 2002, there were over 410 separate divisions, 80 percent of which had been opened in the previous four years. 14 Sichkarenko (2014). 15 Regulation No. 1298 of the Cabinet of Ministers of Ukraine of August 30, 2002 “On Wages of Employees Based on the Uniform Wage Scale of Ranks and Factors of Wages for Employees of Institutions, Establishments and Organizations of Individual Budget Sphere Branches”. 16 Taking into consideration the colleges and technical colleges, the total number of institutions rises to 676, in which case Ukraine has over 15 HEIs per 1 million population. However, colleges are excluded here because they offer short-cycle professional tertiary education. 17 MOES uses a coefficient of 0.1 for extramural students when calculating the so-called ‘adjusted number of students.’ This means that one intramural student is considered equivalent to 10 extramural students for the purposes of calculating the workload for HEI teachers. 18 Based on data from EDEBO. 19 Here and after information for the years 2014 –2016 is given excluding the Autonomous Republic of Crimea and the temporarily occupied territories of Donetsk and Luhansk regions. 20 According to SSSU as of 2018. http://www.ukrstat.gov.ua/druk/publicat/kat_u/2018/bl/02/bl_NRO16_pdf.zip, page 6. 21 OECD (2018a); Treasury data for Ukraine 22 These regulations include: the Law of Ukraine on formulation and placement of state order (https://zakon.rada.gov.ua/laws/show/5499-17); the CMU Resolution on approval of the procedures for forming the state order (https://zakon.rada.gov.ua/laws/show/306-2013-%D0%BF#n12); and the CMU Resolution on approval of the procedures for placing the state order (https://zakon.rada.gov.ua/laws/show/363-2013-%D0%BF#n7). 23 Currently there are more than 120 official majors. 24 https://zakon.rada.gov.ua/laws/file/text/64/f476391n51.doc 25 http://www.ac-rada.gov.ua/doccatalog/document/16746176/ZVIT_20150811_No_1_5.pdf > page 26 26 For example, see the link: https://www.epravda.com.ua/columns/2018/08/13/639582/ 27 https://zakon.rada.gov.ua/laws/show/1556-18 > Article 72 28 The junior bachelor’s degree will replace the junior specialist’s degree. 135 29 The algorithm, based on the Gale-Shapley algorithm, is designed to match students and higher education institutions such that no pair of the two would prefer each other more than their current matches. This algorithm is commonly used in school and university admissions processes. 30 The law defines a series of exceptional circumstances in which the algorithm does not apply, such as in the case of higher military educational institutions, education in accordance with admissions procedures for persons residing in the temporarily occupied territories of Ukraine, or in the case of foreigners and non-citizens, including persons permanently residing in Ukraine, persons who have been granted refugee status, and those who need additional temporary protection. 31 Based on MOES budget data, available at https://mon.gov.ua/storage/app/media/budzhet/pasport_budzh_program_2017.rar 32 Based on MOH budget data, available at: http://moz.gov.ua/uploads/0/2037- zvit_pro_vikonanna_pasporta_budzetnoi_programi_za_2016_rik_za_kpkvk_2301070.pdf page 6 33 CEDOS (2019). 34 In general, tuition fees are low with a few exceptions: (a) 2 –3 HEIs (e.g. Kyiv-Mohyla Academy, Kyiv National University, and Ukrainian Catholic University); (b) some popular majors such as law and economics in prestigious HEIs; and (c) medicine. 35 CEDOS (2019). 36 MOES has proposed the introduction of indicative costs for fee-based higher education, particularly for 12 high- demand study fields, but for now this is still only a draft proposal. 37 Based on authors’ analysis of MOF and EDEBO data for HEIs with available data on funding. 38 http://studyinukraine.gov.ua/life-in-ukraine/international-students-in-ukraine/ 39 Osipian (2018). 40 https://zakon.rada.gov.ua/laws/show/z0321-05 41 Osipian (2009). 42 For example, see interview with former Minister of Education Stanislav Nikolayenko, available at: http://www.osvita.org.ua/articles/218.html 43 Some institutions at the time focused on creating high academic integrity standards and introduced transparent conditions of admission. For example, the Kyiv-Mohyla Academy conducted anonymous and automated testing which created a transparent and just-in-time ranking of applicants. https://www.ukma.edu.ua/index.php/news/1207- archive-2004-2012?news_show=item&news_id=1044 44 https://zakon.rada.gov.ua/laws/show/z1351-15 > Chapter 7 45 https://zakon.rada.gov.ua/laws/show/z1397-17 > Annex 4 46 MOES Decree No. 1224 (2017) https://zakon.rada.gov.ua/laws/show/z1138-17#n28 and MOES Decree No. 1433 (2017) https://zakon.rada.gov.ua/laws/show/z1426-17 47 MOES Order No. 931 (2018) on “Issue in the 2019 External Independent Test of Learning Outcomes Obtained on the Basis of Complete General Secondary Education.” https://zakon.rada.gov.ua/laws/show/z1030-18 48 https://dif.org.ua/article/vishcha-osvita-v-umovakh-reformi-zmini-gromadskoi-dumki_2016 49 OECD (2017a); Kholin and Kalugin (2013); Osipian (2009). 50 World Bank (2016). 51 There are some exceptions, however, such as the Ministry of Health which has taken a strong position on issues of medical higher education. 52 World Bank (2017c). 53 Altbach and Hazelkorn (2018) – “Measuring Education Quality in Global Rankings: What’s the Likelihood?” International Higher Education. No. 95. 54 https://www.timeshighereducation.com/world-university-rankings/taras-shevchenko-national-university-kyiv 55 The QS World University Rankings is based on six metrics: (i) academic reputation; (ii) employer reputation; (iii) faculty/student ratio; (iv) citations per faculty; (v) international faculty ratio; and (vi) international student ratio. 56 World Bank (2017a). 57 Sovsun (2019). 58 Order of the Cabinet of Ministers of Ukraine of 29.04.2015 No. 266. 59 http://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-fields-of- education-and-training-2013-detailed-field-descriptions-2015-en.pdf 60 Nikolaiev (2017) 61 Schuch et al. (2016) 62 Yegorov (2012). 63 Ministry of Education and Science (2017a). 136 64 World Bank (2017e). 65 Altbach (2013). 66 Denisova-Shimidt and Prytula (2017); OECD (2017a); Osipian 2017; Sherstjuk and Zharikova (2017); KAS (2017); Artiukhov and Liuta (2017). 67 OECD (2017b). 68 De Waal (2016) 69 Hallak and Poisson (2007). 70 OECD (2017a). 71 Osipian (2009, 2017); Sondergaard and others (2012); KAS (2017). 72 DIF (2015). 73 Denisova-Schmidt, Prytula, and Rmuyantseva (2018). 74 Osipian (2007) 75 Denisova-Schmidt, Prytula, and Rmuyantseva (2018). 76 Denisova-Schmidt, Prytula, and Rmuyantseva (2018). 77 Osipian (2007). 78 Klein (2012). 79 Vasylyeva and Merkle (2018). 80 Sovsun (2019) 81 Data from SSSU 82 Sichkarenko (2014). 83 For levels of commanding a foreign language, please refer to the Common European Framework of Reference for Languages. 84 Provision on accreditation of higher education institutions and majors in them, as well as higher professional schools. Approved by Resolution No. 978 of the Cabinet of Ministers of Ukraine of August 09, 2001. 85 Due to the peculiarities of departmental regulatory frameworks, for instance, medical HEIs (controlled by the Ministry of Health Care) worked by curriculums and lists of training disciplines approved by MOES until the end of 2018. 86 For reference, please see link: http://www.britishcouncil.org.ua/sites/default/files/standards-and- guidelines_for_qa_in_the_ehea_2015.pdf 87 Sovsun (2019). 88 World Bank (2016); Porta and Arcia (2011); Sondergaard and others (2012). 137 Chapter 5: Directions for the Future of Education in Ukraine Throughout history, the purpose of education has been to develop students’ desire and ability to think and learn about the world around them. While there are competing visions of what knowledge and skills will be needed in the future, it is clear that education and the systems that educate the next generation must constantly evolve and adapt to a fast-changing world. The increasing role of technology in economic activities and everyday life has already led to significant changes in the demand for skills, with a greater need for advanced skills in all types of work.1 However, upgrading cognitive skills alone is not sufficient: ‘soft’ skills are increasingly important given that interpersonal relations between humans cannot (yet) be replaced by the intervention of technology. Adaptability requires a strong and balanced toolkit of skills, which means that the dividing lines between academic and technical disciplines will likely diminish over time. The defining goal of all education systems is to improve the skills of the population. The highest- performing and rapidly improving education systems around the world have a clear vision of the knowledge and skills they want students to acquire to participate actively as capable workers and engaged citizens. They structure themselves accordingly, putting learning at the center of all they do to achieve this. They align their laws, institutions, resources, and people toward that objective. But this takes time, effort, action, and reform on the part of many people and institutions. Ukrainians want a modern education system for their futures and the future of their children. They want a system that facilitates Ukraine’s integration into Europe and the wider world. They recognize that change is needed. The Euromaidan revolution in 2014 and the excitement and optimism that it produced led to the introduction of bold and ambitious reforms across the education sector. They aimed to decentralize and democratize education, transitioning the system away from its Soviet past and toward a vision for the future. However, this vision for the future is not coherent across the education sector, and reforms have proceeded at different paces. There is a need to better articulate and align the vision for reform across the education sector as a whole. Articulating a Vision for Improving Education Quality and Relevance The education sector is an open and organic system, with different elements interacting and reacting to each other through dynamic processes. With 8 out of 10 secondary school graduates continuing to colleges and universities, it is clear that the higher education segment of the sector is crucial. However, to reach higher education, a student must pass through all the prior levels which depend and build on each other, imparting key knowledge and skills at a given moment in the student’s human development. Furthermore, the higher education system produces the teachers, pedagogues, psychologists, and administrators who work in public and private schools at all levels, as well as the researchers that work in universities and the national academies of science. Although the education system in Ukraine is extensive, general secondary and higher education represent the lion’s share of students, staff, and resources that feed into the education system at the national level. These core segments of the system also represent targets for reform on the part of the Government. The vision for reform in general secondary education—articulated in the New Ukrainian School—is a positive step in the right direction. This vision is rooted in a clear argument for change: today’s Ukrainian school should better equip pupils with the skills needed to learn throughout life, think critically, set and achieve goals, work in teams, and communicate in a multicultural environment. As noted in the NUS vision document: 138 Today, just as 10, 20, or even 50 years ago, an average Ukrainian pupil receives a given amount of out-of-date knowledge at school. This amount has increased considerably in recent years, in line with an increase in the general information flow around the world. However, pupils are only able to reproduce pieces of unstructured knowledge; however, they often do not know how to use it to solve everyday problems. There are also structural challenges in how education is organized and provided. To start, many children, particularly from vulnerable groups or areas, do not have access to opportunities to learn. Ukraine maintains a large network of small schools, which do not have the staff or resources to provide high-quality instruction and specialized curriculum for all their students. Given that so many students continue to higher education, this puts a segment of them at a disadvantage. Furthermore, teachers lack motivation and opportunities for professional development, and teaching methods are outdated. The attempts to modernize Ukraine’s general secondary school system to address these fundamental challenges are ambitious and badly needed. The Law on Education and the New Ukrainian School will prepare students for the 21st century through a combination of several elements: new educational content, more motivated teachers, greater decentralization and autonomy, child-centered approaches to teaching, a new schooling structure, fair allocation of public funds, and contemporary educational environments.2 The current structure of upper secondary education represents an important challenge to achieving the vision of this reform. The structure is fragmented, with upper secondary education offered in general secondary schools, vocational schools, colleges, and technical colleges. This level also represents the crucial link between schools and universities. It is a key phase in one’s educational trajectory where decisions about higher education are informed and shaped. The movement toward a student-centered and competency-based approach to learning in line with the NUS vision will be challenging at the upper secondary level without advancing reform to concentrate resources —schools, teachers, funding, and instructional time—on what matters most. Going forward, it will be important for Ukraine to accelerate the expected reform of upper secondary education to introduce the new three-year curriculum structure in a consolidated network of specialized high schools. This reform will increase the duration of upper secondary education from 2 to 3 years, creating more time for students to gain exposure to the curriculum at a critical transition point in their educational trajectories. It will also concentrate resources in a smaller network of specialized institutions, allowing schools to offer a higher quality of education with more elective subjects for students while also using fiscal resources and educational facilities more efficiently. The Law on Education requires grade 12 to be introduced by 2027, but accelerating the reform would ensure the transition to three-year upper secondary education sooner. This reform also presents an opportunity to accelerate optimization of the school network in upper secondary education; underutilized vocational schools and colleges could be merged or closed as part of a consolidated network. It is also important to maintain a focus on Ukraine’s vision for equitable and inclusive education. High-performing education systems prioritize equity and inclusion of all learners, maintaining the vision that all students are capable of high achievement with the right level of support. Ensuring that every child benefits from high-quality instruction is not only an important end in itself. The evidence from international assessments suggests that strong performance for the system as a whole is dependent on the need to deliver for every child. Top-performing systems show a low correlation between learning outcomes and the home background of the individual student, meaning that these systems have produced mechanisms and approaches to ensure that the school can compensate for the disadvantages that result from the student’s home environment.3 Maintaining a focus on equity and inclusion should be a key aspect of Ukraine’s vision for education going forward. In particular, priority should be given to (a) expanding access to quality 139 preschool education with a focus on vulnerable groups, and (b) strengthening the capacity of Inclusive Education Resource Centers to support the transition to inclusive education. In higher education, the challenge is greater and increasingly urgent. There is no vision that links higher education to the positive developments for reform in secondary education or to the skills requirements of the labor market. The Law on Higher Education was the first large systematic reform measure adopted by Ukraine’s parliament in the immediate aftermath of the Euromaidan Revolution. It represented a compromise on the part of many different interest groups, coming on top of years of absence of a clear development strategy for higher education: a policy of nonpolicy.4 However, it did not address this challenge. To this day, there is no clear or coherent vision for the development of the higher education sector or individual universities. This is a fundamental problem: higher education in Ukraine cannot serve the needs of the people and the economy without clear objectives and a strategy for how to achieve them. At the same time, policymaking becomes an effort in ‘tinkering’ on the margins rather than addressing core challenges. Although the 2014 Law on Higher Education made a major step toward dismantling the centralized structures of the past, it provided more autonomy without the attending accountability mechanisms or financial flows. Without a strategy for higher education development, individual HEIs use their autonomy to achieve individual goals rather than working to achieve a broader goal for the system and nation. In terms of academic matters, teaching and learning in universities remain focused on knowledge acquisition, often with outdated content that is not adjusted to the needs of modern students or new trends, research, or technology. In this sense, there is a critical imbalance within the education system. The NUS reflects a progressive move toward competency-based learning, but there is no vision to match this in higher education. As such, there is a clear need to modernize curriculum and pedagogical teaching methods. Globally, higher education is trending toward multidisciplinary study programs that emphasize experiential learning and project-based approaches to solve complex problems. Ukraine would benefit from more support for problem-based and project-based learning, thematic teaching, entrepreneurship, and student-centered approaches to learning. Involving employers in study program revisions would also help to improve the employability of graduates, for example, through the introduction of new courses and restructuring existing course contents and curriculum in response to market demands. A greater focus on civic education and ethics in the curriculum could also play an important role in reducing the distortive impact of corruption while enshrining support for academic integrity in the educational process. In terms of structure, Ukraine’s higher education system is expansive, with a large number of specialized HEIs that are relatively small in size and scope. This has caused the system to become oversized and inefficient, especially in relation to the shrinking student population. Resources are still allocated according to quotas, as in the Soviet era. Despite spending a relatively high share of public resources, the existing model scatters funds across a large number of institutions and staff. This contributes to the incoherence: although the system spends a lot, many rectors and faculty members believe that funding is insufficient. Successful modern mass higher education systems are characterized by a high level of institutional diversity in which individual institutions have different missions and profiles. Ukraine’s higher education system would benefit from moving toward one with larger and more comprehensive institutions. What needs to be done to achieve that vision? Ukraine is committed to developing a modern education for the 21st century, and it has taken a number of key steps to make this happen. However, education reform has been uneven, and in some areas, it is incoherent. There is a critical balance that needs to be struck between the modern approach to competency- 140 based teaching and learning in the New Ukrainian School. To achieve this balance, more attention needs to be given to position the higher education sector to contribute more effectively to Ukraine’s economy and society going forward. The reforms introduced following the Euromaidan Revolution have generated great optimism by decentralizing and democratizing the education system. However, imbalances persist between an optimistic vision for the future (if poorly articulated in higher education) and the inertia of history. This review has identified five priority areas for reform going forward in order to address systemic imbalances which undermine this reform agenda. The first imbalance related to the strategic vision for the sector, as described above. This is at the center of the reform agenda and is directly connected with the other imbalances pertaining to governance, financing, incentives, and information. Figure 5.1 below summarizes the priority areas for strengthening Ukraine’s education reform agenda and addressing the identified imbalances which threaten progress. Figure 5.1 Priority areas for strengthening Ukraine’s education reform agenda Source: Authors’ elaboration. The first imbalance is related to institutional governance structures within the system. On one hand, reforms have greatly expanded the autonomy of HEIs, local governments, schools, and teachers. This reflects a major departure from the centralized direction of the system in the past. However, accountability and capacity to operate in a decentralized approach remain weak, particularly for internal governance and quality assurance. The second imbalance relates to the targeting of resource flows. There is a disconnect, particularly in higher education, between resource flows which prioritize the status quo and the need to achieve strategic 141 objectives in the sector, including sustainability. In secondary education, the introduction of a funding formula is a welcome mechanism to provide a strong economic incentive to local governments to effectively manage their school networks. But, for the time being, the oversized network of small schools and large number of teachers relative to the declining student-age population represents an imbalanced use of resources that maintains the status quo while preventing investments needed for the future. The third imbalance relates to the incentives faced by individual actors and institutions in the education system. Recent education reforms have increased the level of autonomy and thereby personal responsibilities for service delivery, but in some cases incentives are not aligned with assuming those responsibilities. The New Ukrainian School implies a significant change in the responsibilities of teachers and school directors, yet their career path, professional motivations, and opportunities for development remain limited. The teaching load (stavka) system reflects a legacy approach to workforce management, but it provides the wrong incentives for the type of modern professional teacher envisioned under the NUS. The EIT, given its high-stakes importance in the university admissions process, also creates incentives for teachers, students, and universities, which may not be well harmonized with NUS objectives or with the vision for higher education in Ukrainian society. Finally, the teaching load system in universities combined with low salaries creates incentives for corruption and dishonest practices, which detract from quality and diminish the value of educational credentials in the labor force. Finally, the fourth imbalance relates to effective feedback and information available to students, teachers, and the system as a whole. As mentioned above, autonomy in education only works if also paired with accountability. However, through the provision of effective feedback and information, assessment is also required to hold actors accountable for results. In secondary education, students, teachers and parents have little information currently on school and student performance. Students and their families have little information on how to select a study field for higher education, plan an educational path in relation to occupational objectives, or select the best fit university for their interests. Employers and policy makers have little information on the performance of HEIs and the relevance of their offerings to the labor market. Closing these gaps in feedback and information is important to ensure accountability and that autonomy leads to improved results. Strengthen institutional governance structures Strengthen capacity for decentralized service delivery at regional, local, and school levels A key area for accelerating education reforms both in secondary and higher education is the development of managerial capacity for decentralized service delivery. Reforms have provided local authorities and schools with more autonomy in how they use their budgets and organize curricula, but they need to have proper levels of capacity in order to manage this autonomy. This is a long-term objective, but the Government can support this through the State Service of Education Quality(SSEQ) and through the dissemination of information needed to build managerial capacity. Improving communication, outreach and stakeholder engagement is also key to building this capacity and generating support at the local level. For example, in Brazil, educational authorities have implemented the successful Jovem de Futuro project5 for several years and across multiple states. This project has had a significant impact on strengthening educational management capacity toward three goals: improving learning, increasing the number of high school graduates, and reducing educational inequalities. The project emphasizes a structured management decision-making process (the management circuit), which strengthens the 142 autonomy, co-responsibility and managerial capacity of education managers at the school, regional and state levels, all of which participate in a coordinated planning process informed by data on student learning and school performance. The project was rigorously evaluated and found to have positive effects on various managerial practices, such as adoption of internal assessment indicators, existence of management-focused targets, and identification of leaders by the school principal and attribution of responsibilities. With the availability of more data on resources and learning outcomes, Ukraine has the potential to adopt a similar approach to managerial capacity development with the involvement of the SSEQ, oblast education departments, local authorities and schools. Additionally, the Government can build on good practice examples from across the country, including those supported by partners, NGOs, and donor agencies. For example, the Swedish project supporting decentralization in Ukraine has developed and implemented a simulation game, a creative interactive tool to prepare communities for the network optimization process. This allows local authorities to experiment with the impact in a classroom setting before launching it in real life. Additionally, they have developed a database6 of good practices from newly amalgamated communities which could be distributed on a wider basis. Good practice examples cover various aspects of the quality of education management at the local level, such as the organization of methodological work supporting teachers, the selection procedures for school directors, and examples of collaborative and transparent approaches to school management with the engagement of parents, pupils, teachers, and local government management. Strengthen internal governance capacity in HEIs As in secondary schools, HEIs also face challenges in managing the increased levels of autonomy provided to them. HEIs have demonstrated insufficient capacity for democratic decision making aimed at improving quality of teaching and learning. Proper training and support systems should be developed to support internal transformation and build the managerial capacities needed in HEIs. Strengthen systems for accreditation and quality assurance There is an urgent need to strengthen internal and external quality assurance (QA) functions in the higher education system, including through improved capacity of the newly established National Agency for Quality Assurance in Higher Education (NAQAHE), as well as in individual HEIs. In particular, there is a need to accelerate QA reforms in line with the Standards and Guidelines for Quality Assurance in European Higher Education Area (ESG). In terms of accreditation, Ukraine should explore the potential for partial or conditional accreditation approaches. Currently, there are two potential outcomes of accreditation: full accreditation or failure to become accredited. Without accreditation, students will not receive a recognized diploma. This creates strong pressure on the universities and accreditation committee to accredit the program. Given the high-stakes nature of the decision, this leads to rampant corruption and pressures on all levels of the decision-making process. In order to ensure that it does not happen under the new accreditation model, several other options should be added, including the option of partial or conditional accreditation with an obligation that the HEI correct relevant issues within a given period of time (e.g. as in the case of Poland). Additionally, statistical information on accreditation should be collected and published, so the general public and prospective students can see the percentage of programs that were not fully accredited. Target resource flows toward sustainable growth 143 Reform the public funding model for higher education to consolidate resources while incentivizing competition, performance, and excellence Ukraine has an urgent need to move away from its input-based method of public funding for higher education, which has created strong incentives to lower quality standards while also contributing to funding disparities. At the same time, there is evidence that the network of HEIs is oversized relative to a shrinking student-age population. With the input-based method of funding, which does not rely on actual cost of delivery, many HEIs are in the position of having insufficient resources to deliver quality education that is relevant to the needs of the labor market. Therefore, there is a need to reform the funding model while also introducing a more differentiated approach based on indicators of quality. This would help to prioritize critical objectives of enhancing quality and improving links with the labor market, while also supporting consolidation in the sector and optimization of HEI finances. At the same time, the current system of funding and quality assurance does not incentivize excellence at the level of individual faculty or institutions and programs. While systemic reform is needed to ensure transparency and efficiency, special financial incentives may also help to encourage the development of centers of excellence within universities and stimulate top-performing programs. The concept of strategic financing in higher education, including performance-based funding mechanisms, have been a topic of discussion in Ukraine for several years now. A funding formula has been developed, along with draft legislative acts, which would allocate funds based on the adjusted number of students. However, the formula and its legal bases have not yet been formally approved or implemented. Furthermore, the formula could be highly sensitive to the coefficients that are included and the availability of data, including on cost of delivery. For example, a background paper on performance- based funding for higher education in Ukraine that was prepared in support of this report identified this challenge, noting how the lack of an indicator affects the evaluation of a univers ity’s performance. The simulation prepared for this background paper used existing data, including EIT scores of applicants and the impact factor of professors’ research papers (based on the Scopus database), to assess performance, though arguably there are better measures of performance such as graduates’ employability. Additionally, the cost-adjusted coefficients for education in different fields of study and formats need to be based on the actual estimated cost of delivery, which is currently not known in Ukraine. More in-depth information across study fields is needed to determine the share of state funding for higher education by field, the legal implications and potential cancellation of financial regulations, and the impact on staffing. It should be noted that a robust quality assurance system is necessary as a precondition to mitigate risks. To the extent that the funding formula will increase competition between universities, it could have the effect of incentivizing universities to improve the quality and labor market relevance of their offerings in order to attract students. However, if the formula rewards funding to HEIs, which will increase student enrollment, this could lead to further concerns about quality unless a robust quality assurance system is in place to mitigate such risk.7 Introduce additional incentives to reward or penalize HEIs on efficiency of resource use Ukraine could introduce additional mechanisms to incentivize the consolidation or merger of higher education programs and/or institutions. For example, the sector could establish an incentive program to create economies of scale and scope through voluntary strategic cooperation or mergers. A mix of top-down and bottom-up approaches may be suitable here, whereby the state provides incentives for consolidation, but the suggestions of where and what to consolidate are made by institutions, considering regional aspects and equity of access. For example, competitive funding could be provided as a top-down incentive to HEIs 144 that have voluntary plans to merge, to build joint units or to collaborate to increase sector efficiency. A bottom-up development of models for collaboration and consolidation by HEIs would engender ownership on the part of HEIs and less political opposition.8 For example, Denmark presents a good practice example of comprehensive consolidation in higher education in which the government does not regulate which institutions should merge but supports the autonomy/ownership of HEIs and provides financial incentives to stimulate institutions’ participation in the process.9 At the same time, Ukraine could also consider additional financial penalties for HEIs that fail to improve efficiency of resource use and reduce waste, for example through the introduction of performance agreements. Unlike performance-based funding, performance agreements look at future performance, awarding institutions on the basis of expected performance rather than actual performance.10 Such agreements have been introduced in several European countries, including Croatia, Estonia, Finland, Germany, Latvia, and the Netherlands. They are individual agreements between an HEI and the funding authority, and the agreement usually includes a financial penalty or sanction if objectives are not achieved.11 Maintain and monitor school financing formula and hub school program to incentivize school network optimization, teacher workforce rationalization, and improved efficiency and transparency of resource use in secondary education The formula for the education subvention for secondary education must continue to be carefully monitored going forward and adjusted as needed to continue encouraging local governments to optimize their school network and consolidate resources. One option going forward is to gradually raise the desired class size goal specified in the formula, which is currently 13 students per class for rural areas. The spending unit receives a subvention based on a computation that uses this desired or targeted number of students, and in the event that the local government has an actual average class size that is lower than its target (for example, due to too many small schools and classes), the actual expenditure will have to come from the local government’s own resources. This creates the powerful economic incentive to consolidate classes and schools. However, going forward, this desired class size goal could be gradually raised in order to keep the pressure on local governments to economize on resources when possible. An additional consideration in the future is to create school size goals in the financing formula, which currently are not included in the calculation of the subvention. This adjustment of the formula needs to go hand-in-hand with a strategy for rationalizing the teacher workforce while making the profession more attractive. The consolidation of classes and schools, leading to fewer but larger institutions with more efficient use of resources directly requires the rationalization of teachers and non-teaching staff in schools. This is particularly important to consider to the extent that the policy of teacher certification will increase costs, as more and more teachers become certified and earn associated pay increases. Going forward, Ukraine needs to consolidate resources at the local level and raise student-teacher ratios while also rationalizing the workforce, perhaps through creating incentives for retired teachers to leave the workforce while introducing additional measures to improve the attractiveness of the profession (see below on teacher career path). At the same time, the hub school program in rural rayons and hromadas needs to be evaluated for implementation fidelity and revised accordingly. As mentioned above, analysis suggests that students in hub schools perform marginally worse than other students, despite the fact that hub schools are supposed to provide better learning environments and better resource use compared to their comparator schools. Although this could be explained by the fact that the program is relatively new, the practical implementation of the program varies considerably, meaning that ‘hub school’ is just a title rather than a substantive indication of school conditions. The hub school criteria have changed over time as well, meaning that schools may meet different conditions for achieving hub school status. The hub school program, together with the education subvention formula, have great potential to improve efficiency of resource use along 145 with equal access to quality learning environments in rural areas, but only if hub schools meet minimum conditions and represent a substantive change over their alternative. The program should be reviewed and revised with this in mind. Align incentives and capabilities (beyond finance) Reform teacher career path, including the teaching load (stavka) system for organizing and compensating teachers’ work and opportunities for professional development High-performing education systems around the world make teaching an attractive profession by improving its status, compensation policies and career progression structure, while also making good use of teachers’ time with students.12 However, the stavka system for organizing and compensating teachers’ work is poorly aligned with these objectives, as well as the new expectations of teachers under the New Ukrainian School. By fragmenting teachers’ work into teaching and non-teaching hours, the result is that nearly half of teachers’ take-home pay comes from various top-ups. This creates incentives for abuse and nontransparent allocation of teaching hours, while also devaluing the work of professional teachers. This stavka system is in contrast to the weekly workload system used in many OECD countries, where the income of all employees, including that of teachers, is based on 36–40 hours of work per week, of which 22–29 hours are allocated for teaching.13 Given that the New Ukrainian School reform expects teachers to tackle increasingly complex tasks associated with the new curriculum, this workload system of organizing and compensating teachers’ work may ultimately create disincentives for teachers and undermine reform. A more structured and coherent approach to the policies governing teachers’ work, compensation, and professional development are needed to ensure that teachers have the right incentives to become key change agents in the implementation of the New Ukrainian School reform. Teacher certification can play a significant role in supporting the NUS reform, but it must be part of a larger coordinated set of reforms aimed at making the teaching profession more attractive while also rationalizing the teacher workforce. High-performing education systems around the world attend to multiple teacher policy goals in a coherent manner, in order to ensure that policies like teacher licensing are aligned with other aspects of teacher training, recruitment, and management. It is certain that teacher certification is needed. However, to function most successfully, teacher certification may need to be part of a comprehensive set of policies that will include (a) rationalizing the teacher work-force through voluntary and involuntary attrition; (b) optimizing the network of schools and classes; and (c) creating a dynamic professional development network. To ensure the success of teacher certification as a means to improve quality of teaching, there are several important elements that must be considered. Preparation for teacher certification in the form of professional development opportunities, both online and off-line, should be based on NUS teacher competencies as a starting point. The requirements to become certified (both the procedure and the content) should be clearly communicated to all stakeholders. This includes clearly communicating to stakeholders what general professional competencies (knowledge) and practical competencies (praxis) are expected. Opportunities to learn and practice these two sets of competencies should be available. Ideally, there should be practice modules for the general competencies and video examples of teaching that demonstrate the practical competencies.14 Coaching has been shown to be an important mechanism for teachers to assimilate new practices. Collaboration between teachers has also been shown to help improve professional development, which could be encouraged through social media or other mechanisms. Finally, hub schools 146 can become the nexus of this learning, and they should support the development of professional learning communities for teachers. Harmonize University Admissions Exam (EIT) to NUS and vision for higher education Given that the EIT plays such a crucial role in establishing quality of and regulating access to higher education, it is important that the Government continue to invest in and modernize the EIT to ensure it remains a state-of-the-art tool that is fit for purpose. On one hand, the EIT is used as a summative evaluation of secondary education (the state attestation exam) for all graduates to ensure that they pass a minimum competency threshold. Going forward, this will mean that the EIT content and test items will need to be adapted to reflect the new competency-based approach to learning that is envisioned in the New Ukrainian School curriculum. On the other hand, the EIT maintains its original purpose of providing transparency in regulating access to higher education for a subset of secondary school graduates who intend to continue their education in universities. Many students entering higher education today are not prepared for advanced studies. This has led some universities and programs (for example, medicine, law) to set minimum entrance thresholds on the EIT subtests for applicants. Given this trend, there is a clear argument to be made for reviewing the content of the EIT subtests, as well as their psychometric design, to ensure alignment with global best practices. Finally, the regular administration of EIT presents an opportunity to collect more information on students’ backgrounds, educational objectives, and pathways, so the incorporation of more survey questions could help to shed light on this and also track changes over time. In addition, the introduction of more external exams for admissions to master’s degree programs could help to improve transparency while also controlling access to this level of education. A high share of students continue from bachelor’s to master’s programs, and the recent introduction of external exams have helped to c ontrol access for high- demand programs such as medicine and law. However, this could be expanded to other regulated professions as well. Reform teaching load system for organizing work of university faculty As with secondary schools, the teaching load (stavka) system is used to organize and compensate the work of teaching faculty and lecturers in HEIs. This system fragments their work and creates incentives for faculty members to collect as many teaching hours as possible, which may ultimately impact their ability to facilitate learning of their students. Strengthen incentives for academic integrity and anti-corruption and implement tools for oversight Strengthening the capacity of the MOES and the newly established National Agency for Quality Assurance in Higher Education is essential to raise awareness and to manage a national program on academic integrity in higher education. However, Ukraine can seek to address this challenge on multiple fronts simultaneously. Another institutional reform that can help to strengthen incentives is to operationalize the Office of Education Ombudsman, as required by the Law on Education.15 This office should create an institutional grievance redress mechanism for addressing instances of corruption or integrity violations, both in higher education and other elements of the education sector. The legislation is available and the ombudsman position has recently been filled, with the office now in the process of being fully staffed. This could also be supplemented by support to establish ombudsman offices in HEIs, as well as a review and introduction of stronger legislation to penalize violations of academic integrity, including a retrospective check for plagiarism for persons who apply for high-level positions in HEIs. 147 Additionally, Ukraine should seek to implement tools for combating plagiarism and academic misconduct in higher education, such as the National Repository of Academic Texts. This repository was designed to serve as a universal database of all academic work published in Ukraine. Having such a database would greatly facilitate the detection of plagiarism in students’ papers, theses, and dissertations. However, there has been little progress in this area since the resolution was passed by the Cabinet of Ministers in 2016. Ukraine could disseminate information on academic integrity in universities based on student surveys. Such tools could identify high instances of integrity violations, while also encouraging and disseminating good practices among other HEIs. For example, in Romania, an education think tank developed a coalition of stakeholders that included students and teachers, along with professional associations and education journalists, in order to combat issues in university governance and corruption. Freedom of information provisions in Romania’s legislation compel public institutions, incl uding universities, to share information related to governance practices. Along with a detailed questionnaire for each university, this allowed for the assessment of university governance in several areas, including transparency and responsiveness, academic integrity, enforcement of rules, governance quality, and financial management. The exercise resulted in an immediate improvement in university transparency in procurement and recruitment, along with some significant improvements in awareness about the poor integrity of universities at a high level.16 There is also a need to work directly with HEI students, faculty and administrators to expand awareness of academic integrity principles and raise awareness at higher levels. This is aligned with research on corruption which advises against fighting corruption in general, and instead focusing on specific malpractices.17 Ukraine can build on existing data sources and successful projects and programs, such as Profrights.org, a database containing information on violations of the rights of teachers and students in HEIs, as well as the Strengthening Academic Integrity in Ukraine Project (SAIUP). Provide effective feedback and information on systemic results Strengthen systems for student assessment in basic education High-performing education systems in the EU and OECD have summative assessments of student learning. Twenty-nine education systems had such assessments at the primary education level, and 27 had them at the lower secondary education level.18 Ukraine has rolled out a grade 4 monitoring survey which is an important step, but other reforms are needed to improve the student assessment system as well to track and improve learning outcomes in general secondary education. In particular, there is a need to introduce a summative assessment at grade 9, before students move into specialized upper secondary education. The system would also benefit from a more systematic approach to measuring quality in preschool education and student readiness for primary school. This could involve the integration of standardized assessments19 of preschool quality into the quality assurance process managed by the SSEQ. UCEQA has developed a Strategy for Learning Assessments in General Secondary Education until 2030, which represents a strong step towards expanding and improving the student assessment system in Ukraine in line with the learning objectives of the NUS. Moving forward, this Strategy could be further developed and implemented. Develop national program for counseling and guidance services as key pillar of upper secondary reform 148 Educational and career guidance counseling plays an important role in motivating students and keeping them engaged in education by providing information on study options and work prospects and identifying careers that may interest them. Guidance staff also support young people in developing the skills they need to make smart decisions and take responsibility for personal growth and professional development. In many European countries, education and career guidance is explicitly stated as a measure to facilitate the transition through secondary education and combat early school leaving. For example, some systems like Finland, Sweden, Norway, Spain and Italy specify that providing guidance, supporting students in their decision making, and preparing them to cope with challenges of the real world are among the main tasks of all school staff, while in the United Kingdom, schools have a statutory duty to provide access to career advice.20 In Ukraine, students have little information on the labor market relevance of different higher education pathways, including labor market outcomes for graduates of vocational and higher education and different fields of study across higher education institutions. As a result, students end up making momentous educational and occupational choices based on anecdotal information from their peers and families, or based on random selection. A national program for counselling and career/educational guidance services at the upper secondary education level would be instrumental in helping to address this gap in the system. It could include more integrated site visits between schools and local employers, outreach efforts between schools and HEIs, and provision of current and relevant data on labor market outcomes. This should also be paired with transparent and accessible information about HEIs and possible outcomes based on graduate tracer studies, surveys of current students, and other similar sources. Strengthen systems for data collection and monitoring performance of HEIs and higher education system Current mechanisms for collecting data on HEIs and the wider HEI system as a whole should be strengthened in order to better inform policies. Although the EDEBO database contains a considerable amount of information, it is poorly suited for policy analysis purposes. Furthermore, there is a lack of basic data on a number of aspects, such as the number of personnel in HEIs and tuition fees across programs and HEIs. The establishment of a student experience and engagement survey could also improve quality assurance processes, accreditation, and monitoring. According to the European University Association (EUA), student experience and engagement surveys are the most common way for institutions to introduce quality assurance processes.21 There are several examples of such surveys that could inform Ukraine’s reform, such as the National Student Survey (NSS) in the United Kingdom, which is used for external quality assurance and is obligatory for publicly funded universities in the UK. Additionally, surveys such as the North American National Survey of Student Engagement (NSSE) and the Student Experience in the Research University (SERU) survey are voluntary and used for institutional self-improvement and internal quality assurance efforts. Allowing Ukrainian universities to ask university-specific questions on such a survey may be an option to increase the response rate and buy-in. Other metrics of university performance and graduates’ employability would be helpful to monitor quality. This could include the introduction of university rankings, graduate tracer studies, and other such mechanisms. One possible model is Poland’s Graduate Tracking System, which relies on data submitted by HEIs as required by the law, and is managed by the same agency that manages Poland’s higher education information system.22 149 Notes 1 World Bank (2019c). 2 MOES (2017b). 3 OECD (2018b); Barber and Mourshed (2007). 4 KAS (2017). 5 https://jovemdefuturo.org.br/ 6 http://wiki.sklinternational.org.ua/ 7 World Bank (2016). 8 Arnhold and others (2018). 9 Pruvot, Estermann, and Mason (2015). 10 De Boer and Jongbloed (2015). 11 Curaj, Deca, and Pricopie, eds. (2018). 12 World Bank (2019b); OECD (2018b). 13 Steiner-Khamsi (2016). 14 Example videos could be placed online of ‘unacceptable’, ‘acceptable’, ‘good’, and ‘great’ teaching with each example video accompanied by a scoring rubric that explains why they are scored in that way. Only then can teachers interested in taking the certification be in a position to understand what is expected. There are teacher classroom observation tools developed for this purpose. 15 MOES announcement on education ombudsman office: https://mon.gov.ua/ua/news/z-1-sichnya-2019-roku-v- ukrayini-zyavitsya-osvitnij-ombudsmen-sho-zahishatime-prava-uchniv-studentiv-osvityan-i-naukovciv-uryad- prijnyav-vidpovidnu-postanovu 16 Transparency International (2013). 17 Shekshnia and Denisova-Schmidt (2017); Denisova-Schmidt (2018). 18 OECD (2017). 19 Such as those developed under the Measuring Early Learning Quality and Outcomes (MELQO) initiative. 20 European Commission (2014). 21 Loukolla and Zhang (2010). 22 For reference, click here for the link to Poland’s graduate tracking system: http://ela.nauka.gov.pl/en/ 150 References Altbach, P. G. 2013. Academic Salaries, Academic Corruption, and the Academic Career. In the International Imperative in Higher Education. Altinok, N., N. Angrist, and H. A. Patrinos. 2018. “Global Data Set on Education Quality (1965 -2015).” Policy Research Working Paper No. 8314. World Bank. Arnhold, N., J. Kivisto, H. Vossensteyn, J. Weaver, and F. Ziegele. 2018. Focus on Performance: World Bank Support to Higher Education in Latvia. Volume 1: System-Level Funding. Washington, DC: World Bank. Arnhold, N., J. Kivisto, V. Puttmann, H. Vossensteyn, and F. Ziegele. 2018. Focus on Performance: World Bank Support to Higher Education in Latvia. Volume 2: Internal Funding and Governance. Washington, DC: World Bank. Arnhold, N., Pekkola, E., Puttmann, V. and A. Sursock. 2018. Focus on Performance: World Bank Support to Higher Education in Latvia. Volume 3: Academic Careers. Washington, DC: World Bank. Artiukhov, A. Y. and O. V. Liuta. 2017. “Academic integrity in Ukrainian higher education: values, skills, actions.” Business Ethics and Leadership, 1(1), 34-39. Barber, M., and M. Mourshed. 2007. “How the World’s Best -Performing School Systems Come Out on Top.” September. New York: McKinsey. Bau, N., and J. Das. (forthcoming) “Teacher Value -Added in a Low-income Country”. American Economic Association: Economic Policy. Bawa, P. 2016. Retention in Online Courses: Exploring Issues and Solutions —A Literature Review. SAGE Open January–March 2016: 1–11. Bennich-Bjorkman, L., A. Kashyn, and S. Kurbatov. 2019. “The Election of a Kleptocrat: Viktor Ianukovych and the Ukrainian Presidential Elections in 2010.” East/West: Journal of Ukrainian Studies (1): 91–124. Bentaouet Kattan, R., K. A. D. Macdonald, and H. A. Patrinos. 2018. “Automation and Labor Market Outcomes: The Pivotal Role of High-Quality Education.” Policy Research Working Paper 8474, World Bank, Washington, DC. British Council. 2015. “Higher Education in Ukraine: Briefing Paper.” December 2015. https://www.britishcouncil.org/sites/default/files/ukraine_he_briefing_paper.pdf Buhl-Wiggers J., J.T. Kerwin, J.A. Smith, and R. Thornton. 2017. “The Impact of Teacher Effectiveness on Student Learning in South Africa”. Paper presented at RISE Annual Conference 2017, Center for Global Development, Washington, DC, June 15–16. Cabinet of Ministers of Ukraine. 2018. Fiscal Decentralisation in Ukraine: First Successes. Kyiv, Ukraine. CEDOS. 2016. Inequality of educational achievements in Ukrainian schools by the results of EIT 2016. https://cedos.org.ua/uk/articles/nerivnist-navchalnykh-dosiahnen-v-ukrainskykh-shkolakh-za-rezultatamy-zno- 2016. CEDOS. 2019. Background paper on higher education financing prepared for World Bank. Unpublished manuscript. 151 Chetty, R., J. Friedman, and J. Rockoff. 2014. “Measuring the Impacts of Teachers II: Teacher Value -Added and Student Outcomes in Adulthood.” American Economic Review 104 (9): 2633–79. Cilliers, J., and S. Taylor. 2017. “Monitoring Teachers and Changing T eaching Practice: Evidence from a Field Experiment.” https://sites.tufts.edu/neudc2017/files/2017/10/paper_110.pdf. CGD (Commission on Growth and Development). 2008. The Growth Report: Strategies for Sustained Growth and Inclusive Development. CGD Report, Center for Global Development, Washington, DC. Coupé, T. and H. Vakhitova. 2011. “Recent Dynamics of Returns to Education in Transition Countries.” Discussion Papers 39, Kyiv School of Economics and Kyiv Economics Institute. http://repec.kse.org.ua/pdf/KSE_dp39.pdf Cunningham, W. and P. Villaseñor. 2016. “Employer Voices, Employer Demands, and Implications for Public Skills Development Policy Connecting the Labor and Education Sectors.” World Bank Policy Research Working Paper No. 7582. Washingon, DC: World Bank. Curaj, A., Deca, L., and Pricopie, R., eds. 2018. European Higher Education Area: The Impact of Past and Future Policies. Cham, Switzerland: Springer International Publishing. De Boer, H., and B. Jongbloed. 2015. “Reflections on performance agreements in higher education. Practical considerations in delivering system-wide improvements.” CHEPS Working Paper 09/2015. Center for Higher Education Policy Studies, University of Twente, Enschede, the Netherlands. De Waal, T. 2016. “Fighting a Culture of Corruption in Ukraine.” Carnegie Europe. April 18. https://carnegieeurope.eu/2016/04/18/fighting-culture-of-corruption-in-ukraine-pub-63364. Del Carpio, X., O. Kupets, N. Muller, and A. Olefir. 2017. Skills for a Modern Ukraine. Directions in Development: Human Development. Washington, DC: World Bank. doi:10.1596/978-1-4648-0890-6. License: Creative Commons Attribution CC BY 3.0 IGO Denisova-Schmidt, E. 2018. “Corruption, the Lack of Academic Integrity and Other Ethical Issues in Higher Education: What Can Be Done within the Bologna Process?” In The European Higher Education Area, Part III, 61–75. Cham, Switzerland: Springer. Denisova-Schmidt, E., Y. Prytula, and N. L. Rumyantseva. 2018. Beg, borrow, and steal: determinants of student academic misconduct in Ukrainian higher education, Policy Reviews in Higher Education, DOI: 10.1080/23322969.2018.1518726. Denisova-Schmidt, E., and Y. Prytula. 2017. Ukraine: Endemic Higher Education Corruption. International Higher Education (90): 16–18. DIF (The Ilko Kucheriv Democratic Initiatives Foundation). 2017. Higher Education in Conditions of Reform: Changes in Public Opinion. https://dif.org.ua/article/vishcha-osvita-v-umovakh-reformi-zmini-gromadskoi- dumki_2016. Kyiv, Ukraine: DIF. ———. 2018. Reforms in Ukraine: Public Opinion. Kyiv, Ukraine: DIF. Dobko, T., I. Zolotariova, S. Kalashnikova. 2015. Development of the Quality Assurance System for Higher Education of Ukraine: Informational and Analytical Survey. Kyiv: Priorytety Scientific and Production Center Subsidiary. EBRD (European Bank for Reconstruction and Development). 2016. Life in Transition Survey. London: EBRD. ETF (European Training Foundation). 2017. “Decentralizing Vocational Education and Training in Ukraine: Momentum for Action.” Green Paper to lead discussions on reforms and capacities needed for an attractive high - quality VET system in Ukraine. Kyiv: European Training Foundation. 152 ______. 2018. Torino Process 2018-2020: Guidelines. European Training Foundation. European Commission. 2014. Tackling Early Learning from Education and Training in Europe: Strategies, Policies and Measures. Eurydice and Cedefop Report. Luxembourg: Publications Office of the European Union. Evans, D. K., and F. Yuan. 2018. “Working Conditions of Teachers in Low and Middle -income Countries”. WDR 2018 background paper. World Bank, Washington, DC. Evans, R. J., R. B. Baker, and T . S. Dee. 2016. “Persistence Patterns in Massive Open Online Courses (MOOCs).” The Journal of Higher Education 87 (2): 206–242. Furiv, U. 2018. Ukrainian Higher Education During Armed Conflict: Perspectives of Crisis Management. (Master's Thesis). University of Tampere, Finland. Goddard, Y. L., and Goddard, R. D., and M. Tschannen-Moran. 2007. “A Theoretical and Empirical Investigation of Teacher Collaboration for School Improvement and Student Achievement in Public Elementary Schools.” Teachers College Record 109 (4): 877–896. Government of Alberta, International Qualifications Assessment Service. 2008. International Education Guide for the Assessment of Education from the Former USSR and the Russian Federation. Alberta, Canada. Guerra, N., K. Modecki, and W. Cunningham, W. 2014. “Social -Emotional Skills Development across the Life Span: PRACTICE.” Policy Research Working Paper 7123, World Bank, Washington, DC. Hallak, J., and M. Poisson. 2007. Corrupt Schools, Corrupt Universities: What Can Be Done? Paris: International Institute for Education Planning (IIEP), United Nations Education, Scientific, and Cultural Organization. Hanushek, E. A., and L. Woessman. 2007. The Role of Education Quality for Economic Growth. Policy Research Working Paper, no. 4122. World Bank, Washington DC. Hanushek, E. A., and S. G. Rivkin. 2010. “Generalizations about Using Value -Added Measures of Teacher Quality.” American Economic Review 100 (2): 267–71. Herczynski, J. 2017. Education Finance in Ukraine: Selected Strategic Issues. Stockholm: SKL International. Herczynski, J. 2018. Description of Allocation Formula for Education Subvention in 2018. Unpublished manuscript. Hofmann, M., and Reichel, D. 2011. Ukrainian Migration: An Analysis of Migration Movements To, Through, and From Ukraine. International Centre for Migration Policy Development (ICMPD), Vienna, Austria. Hone, K. S., and G. R. El Said. 2016. “Exploring the factors affecting MOOC retention: A survey study.” Computers & Education (98): 157–168. https://doi.org/10.1016/j.compedu.2016.03.016. Huisman, J., A. Smolentseva, and I. Froumin. 2018. 25 Years of Transformations of Higher Education Systems in Post-Soviet Countries. Cham, Switzerland: Springer. IED (Institute for Education Development). 2015. Secondary Education in Ukraine: Attitudes of Teachers and Parents. Institute for Education Development (IED), Kyiv. http://www.iro.org.ua/ua/main/research/23 ILO (International Labour Office). 2012. International Standard Classification of Occupations: ISCO-08. Vol.1, Structure, Group Definitions and Correspondence Tables. ILO, Geneva. IOM (International Organization for Migration). 2016. Migration in Ukraine: Facts and Figures. Kyiv, Ukraine. Janmaat, J. G. 2000. Nation Building in Post-Soviet Ukraine: Educational Policy and the Response of the Russian Speaking Population . Utrecht, Amsterdam: KNAG. 153 Jarábik, B., and Y. Yesmukhanova. 2017. Ukraine’s Slow Struggle for Decentralization. Carnegie Endowment for International Peace, Washington, DC. Jarábik, B., and T. D. Waal, T. D. 2018. Ukraine Reform Monitor: March 2018. Carnegie Endowment for International Peace, Washington, DC. Jensen, B., J. Sonnemann, K. Roberts-Hull, and A. Hunter. 2016. Beyond PD: Teacher Professional Learning in High- Performing Systems. Washington, DC: National Center on Education and the Economy Jorda, V. and J. M. Alonso. 2017. “New Estimates on Educational Attainment Using a Continuous Approach (1970- 2010). World Development, Vol. 90, Issue C, 281-293. KAS (Konrad-Adenauer-Stiftung). 2017. Higher Education in Ukraine: Agenda for Reforms. KAS Policy Paper, edited by Yevhen Nikolaiev. Kyiv: KAS. http://www.kas.de/wf/doc/kas_49848-1522-2-30.pdf?170816114818. Kholin, Y. and O. Kalugin. 2013. “Training of specialists in chemistry at classical universities in the context of higher education reformation in Ukraine.” Russian Journal of General Chemistry, 83(3), 586-593. Klein, E. 2012. Academic Corruption and Reform in Russia and Ukraine . In Governance Failure and Reform Attempts after the Global Economic Crisis of 2008/09, edited by L. Kosals, and H. Pleines. Case Studies from Central and Eastern Europe, 173–190. Koettl, J., O. Kupets, A. Olefir, and I. Santos. 2014. In Search of Opportunities? The Barriers to More Efficient Internal Labor Mobility in Ukraine. Bonn, Germany: The Institute for the Study of Labor. Kupets, O. 2010. Gender Mainstreaming at the Labor Market of Ukraine and Role of the Public Employment Service. International Labor Office, ILO Decent Work Technical Support Team and Country Office for Central and Eastern Europe, Kyiv: ILO. Kupets, O. 2013. International Mobility of Ukrainian Teaching and Research Professionals . ILO Decent Work Technical Support Team and Country Office for Central and Eastern Europe. Budapest: ILO, 2013. Kupets, O. 2016. “Education-job mismatch in Ukraine: Too many people with tertiary education or too many jobs for low-skilled.” Journal of Comparative Economics, 44(2016): 125-147. Libanova, E., A. Cymbal, L. Lisogor, I. Marchenko, and O. Iarosh. 2014. Labour Market Transitions of Young Women and Men in Ukraine. Geneva: ILO. Libanova, E., A. Cymbal, L. Lisogor, and O. Iarosh. 2016. Labour Market Transitions of Young Women and Men in Ukraine: Results of the 2013 and 2015 School-to-Work Transition Surveys. Geneva: ILO. Loukolla, T., and T. Zhang, T. 2010. Examining Quality Culture: Part I – Quality Assurance Processes in Higher Education Institutions. Brussels: European University Association. Magaziner, J. 2016. Barriers to Progress: Documentation Challenges in Ukraine and Crimea . New York: World Education Services. Mercer, M., and J. C. Weidman. 2004. Education Reforms in Countries in Transition: Policies and Processes. Manila, Philippines: Asian Development Bank. Ministry of Foreign Affairs, Finland. 2017. “Finland’s Support to the New Ukrainian School Reform Project Document”. http://www.fcg.fi/eng/projects?RefID=1533. MOES (Ministry of Education and Science). 2017a. Innovative Development of Universities and Scientific Institutions of the Ministry of Education and Science of Ukraine. Kyiv, Ukraine. ———. 2017b. “The New Ukrainian School. Conceptual Princip les of Secondary School Reform.” Kyiv, Ukraine. 154 ———. n.d. Law on Education: School Education for National Minorities and Indigenous Peoples . Kyiv, Ukraine. Mostafa, T., and J. Pal, J. 2018. “Science Teacher’s Satisfaction: Evidence from the PSIA 2015 Teacher Survey.” Education Working Paper No. 168, OECD, Paris. Muller, N. and A. Safir. 2019. “What employers actually want: skills in demand in online job vacancies in Ukraine.” Social Projection & Jobs Discussion Paper No. 1932, May 2019. World Bank Group. Mullis, I. V., M.O. Martin, P. Foy, and A. Arora. 2012. “TIMSS 2011 International Results in Mathematics.” TIMSS and PIRLS International Association for the Evaluation of Educational Achievement, Boston College, Chestnut Hill, MA. National Bank of Ukraine, Statistics and Reporting Department. 2018. Personal Remittances Data Revision for 2015-2017. Kyiv, Ukraine. Nikolaiev, Y. 2017. “What has been achieved by the third anniversary of the new Law On Higher Education?” in Higher Education in Ukraine: Agenda for Reforms. KAS Policy Paper, KAS, Kyiv, Ukraine. http://www.kas.de/wf/doc/kas_49848-1522-2-30.pdf?170816114818. O’Malley, B. 2015. “A struggle to shake off the Soviet era mindset.” December 11. http://www.universityworldnews.com/article.php?story=20151211024813514. OECD (Organisation for Economic Co-operation and Development). 2011. Strong Performers and Successful Reformers in Education: Lessons from PISA for the United States. Paris: OECD Publishing. ______. 2013. Skilled for Life? Key Findings from the Survey of Adult Skills. Paris: OECD. https://www.oecd.org/skills/piaac/SkillsOutlook_2013_ebook.pdf ______. 2017a. OECD Reviews of Integrity in Education: Ukraine 2017. Paris: OECD. http://dx.doi.org/10.1787/9789264270664-en. ———. 2017b. Public Integrity. Paris: OECD. ———. 2017c. Funding of School Education: Connecting Resources and Learning. Paris: OECD. ———. 2018a. Education at a Glance. Paris: OECD. ———. 2018b. World Class: How to Build a 21st Century School System. Paris: OECD. ———. 2018c. OECD Multi-level Governance Studies: Maintaining the Momentum of Decentralisation in Ukraine. Paris: OECD. Osipian, A. 2007. “Corruption in Higher Education: Conceptual Approaches and Measurement Techniques.” Research in Comparative and International Education 2 (4): 313–332. ———. 2009. “Corruption and reform in higher education in Ukraine.” Comparative and International Education/Éducation Comparée et Internationale 38 (2): 104–122. ———. 2017. “University Autonomy in Ukraine: Higher Education Corruption and the State.” Communist and Post-Communist Studies 50 (3): 233–243. doi:https://doi.org/10.1016/j.postcomstud.2017.06.004. ______. 2018. “Corruption in Ukraine’s Medical Universities.” Inside Higher Ed. Published on July 30, 2018. Available at https://www.insidehighered.com/blogs/world-view/corruption-ukraine%E2%80%99s-medical- universities. 155 Paes de Barros, R., M. Carvalho, S. Franco, B. Garcia, R. Henriques, and L. Machado. 2019. “Assessment of the Impact of the Jovem de Futuro Program on Learning.” World Bank Report No. 138507. Polese, A. 2010. “The Formal and the informal: exploring ‘Ukrainian’ education in Ukraine, scenes from Odessa.” Comparative Education 46 (1): 47–62. doi:10.1080/03050060903538673. Ponomarenko, I. “Ukraine’s military ups pay in 2017.” Kyiv Post, published Sept. 12, 2017. https://www.kyivpost.com/ukraine-politics/ukraines-military-ups-pay-2017.html Porta, E., and G. Arcia. 2011. Improving Information Systems for Planning and Policy Dialogue: The SABER EMIS Assessment Tool. World Bank: Washington, DC. Pruvot, E. B., T. Estermann, and P. Mason. 2015. DEFINE Thematic Report: University Mergers in Europe. Brussels: European University Association. Psacharopoulos, G. and H. A. Patrinos. 2018. “Returns to investment in education: a decennial review of the global literature.” Education Economics, 26(5): 445-458. Putcha, V., M. Neuman, O. Zaplotynska, and N. Sofiy. 2018. Supporting the Early Childhood Workforce at Scale: Preschool Education in Ukraine. Washington, DC: Results for Development. Rabinovych, M., A. Levitas, and A. Umland. 2018. Revisiting Decentralization After Maidan: Achievements and Challenges of Ukraine’s Local Governance Reform. Washington, DC: Wilson Center. Rindermann, H. and S. J. Ceci. 2009. “Educational policy and country outcomes in international cognitive competence studies.” Perspectives on Psychological Science, 4(6), 551-568. Rockoff, J. E. 2004. “The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data.” American Economic Review 94 (2): 247–52. Rivkin, S. G., E. A. Hanushek, and J. F. Kain. 2005. “Teachers, Schools, and Academic Achievement.” Econometrica 73 (2): 417–58. Schuch, K., G. Weiss, P. Brugner, and K. Buesel. 2016. Background Report – Peer Review of Ukrainian Research and Innovation System, Horizon 2020 Policy Support Facility, Center for Social Innovation (ZSI), Vienna, Austria. Shekshnia, S., A. Ledeneva and E. Denisova-Schmidt. 2017. “Managing Business Corruption: Targeting Non- Compliant Practices in Systematically Corrupt Environments.” Slavonic & East European Review, 95 (1): 151– 174. Sherstjuk, V. and M. Zharikova, M. 2017. Academic Integrity Support System for Ukrainian Universities Science and Transport Progress. Bulletin of Dnipropetrovsk National University of Railway Transport (2-68), 101-109. Sichkarenko, H.H. 2014. Historical experience of rebuilding higher education in Ukraine (1985–2005): monograph. Nizhyn; Kyiv: P.P. Lysenko. Snilstveit, B., J. Stevenson, D. Phillips, M. Vojtkova, E. Gallagher, T. Schmidt, H. Jobse, et. al. 2015. Interventions for Improving Learning Outcomes and Access to Education in Low- and Middle-Income Countries – A Systematic Review. 3ie Systemic Review 24, London: International Initiative for Impact Evaluation. Sondergaard, L., M. Murthi, D. Abu-Ghaida, C. Bodewig, and J. Rutkowski. 2012. Skills, Not Just Diplomas: Managing Education Results in Eastern Europe and Central Asia. World Bank, Washington DC. Sovsun, I. 2019. Higher education background paper prepared for World Bank. Unpublished manuscript. 156 SSEQ (State Service of Education Quality of Ukraine). 2018. Presentation on State Service of Education Quality of Ukraine: Questions and Answers. Prepared by Mr. Ruslan Gurak, Head of the SSEQ. Kyiv, Ukraine: State Service for Education Quality of Ukraine. State Statistics Service and Ukrainian Center for Social Reforms. 2013. Ukraine Multiple Indicator Cluster Survey 2012, Final Report. Kyiv, Ukraine: State Statistics Committee and the Ukrainian Center for Social Reforms. Steiner-Khamsi, G. 2016. “Teach or Perish: The Stavka System and its Impact on the Quality of Instruction.” Voprosy obrazovaniya / Educational Studies: (2): 14–39. Stepanenko, V. 1999. The Construction of Identity and School Policy in Ukraine. Commack, NY: Nova Science Publishers. Sydorchuk, O. 2015. Decentralization Reform in Ukraine: Prospects and Challenges. Ilko Kucheriv Democratic Initiatives Charitable Foundation. Tendler, J. 1998. Good Government in the Tropics. Baltimore, MD: Johns Hopkins University Press. Transparency International. 2013. Global Corruption Report: Education. New York, NY: Routledge Publishing. Ukraine, Ministry of Education and Sciences. 2017. Law on Education. Kyiv, Ukraine. UNESCO (United Nations Educational, Scientific, and Cultural Organization). 2017. Student accountability in Post- soviet countries. Background paper prepared for the 2017/8 Global Education Monitoring Report. Paris: UNESCO. UNICEF (United Nations Children’s Fund). 2011. Teachers: A Regional Study on Recruitment, Development and Salaries of Teachers in the CEECIS Region. Geneva: UNICEF Regional Office for Central and Eastern Europe and the Commonwealth of Independent States (CEECIS). ———. 2016. Out of School Children in Ukraine: A Study on the Scope and Dimensions of the Problem with Recommendations for Action. UNICEF, New York. ———. 2017. The Children of the Contact Line in East Ukraine: An Assessment of the Situation of Children and Their Families Living in Government-Controlled Areas along the Contact Line in the East Ukraine Conflict Zone. UNICEF, New York. Valero, A., and J. Van Reenen. 2019. “The Economic Impact of Universities: Evidence from Across the Globe.” Economics of Education Review 68: 53–67. Vasylyeva, A. and O. Merkle. 2018. “Combatting corruption in higher education in Ukraine.” Working Paper Series, #2018-021. United Nations University UNU-MERIT. World Bank. 2012. Skills, Not Just Diplomas: Managing Education for Results in Eastern Europe and Central Asia. Washington, DC: World Bank. ______. 2016. What Matters Most for Tertiary Education Systems: A Framework Paper. SABER Working Paper Series, Number 11, World Bank, Washington, DC. ———. 2017a. Innovation and Entrepreneurship Ecosystem Diagnostic: Ukraine . Washington, DC: World Bank. ———. 2017b. World Bank Group: Country Partnership Framework (CPF) for Ukraine for the Period FY17-FY21. Report No. 114516-UA. Washington, DC: World Bank. 157 ———. 2017c. SABER Education Management Information Systems Country Report: Ukraine 2017. Systems Approach for Better Education Results (SABER) country report: Education Management Information Systems. Washington, DC: World Bank. ______. 2017d. Ukraine: Public Finance Review 2017. Washington, DC: World Bank. ______. 2017e. Ukraine Science, Technology and Innovation Public Expenditure Analysis. Washington, DC: World Bank. ———. 2018a. The Changing Wealth of Nations 2018: Building a Sustainable Future. Washington, DC: World Bank. ———. 2018b. “Strategy for Prioritization of Investments, Funding and Modernization of Ukraine’s Road Sector ”. Washington, DC: World Bank. ———. 2018c. Ukraine Education Policy Note: Introducing the New Ukrainian School in a Fiscally Sustainable Manner, World Bank, Washington, DC. ———. 2018d. World Development Report 2018. Washington, DC: World Bank. ———. 2019a. Drivers of Growth in Ukraine. Washington, DC: World Bank. ———. 2019b. Successful Teachers, Successful Students: Recruiting and Supporting Society’s Most Crucial Profession. World Bank Policy Approach to Teachers. Washington, DC: World Bank. ———. 2019c. World Development Report 2019: The Changing Nature of Work. Washington, DC: World Bank. ———. 2019d. “Ukraine Economic Update: Spring 2019.” Washington, DC: World Bank Group. Available at http://pubdocs.worldbank.org/en/677701558601578072/Ukraine-Economic-Update-Spring-2019-en.pdf Wynnyckyj, M. 2017. “Remaking Ukraine’s Education System.” New Eastern Europe. Interview transcript, November 16, 2017. http://neweasterneurope.eu/2017/11/16/remaking-ukraines-education-system/ Yegorov, I. 2012. ERAWATCH Country Reports 2011: Ukraine. European Commission. 158 Annex 1: Education System Structure EDUCATION SYSTEM STRUCTURE IN UKRAINE (based on Law on Education, 2017) Figure A1.1 Education system structure in Ukraine based on Law on Education, 2017 Level of Access to ISCED NQF Cycle Degree Period of study Ages Education next level Level Level Doctor of Postdoctoral Sciences, Usually 2 years 10 Diploma Doctor of III cycle Philosophy, PhD Diploma 4 years 8 9 Higher III cycle 3 years (30-60 credits) Doctor of Arts 8 9 Education 1.5-2 years (90-120 II cycle Master, Diploma ECTS)* Age 21-22/23 7 8 Bachelor, 3-4 years (180-240 I cycle Diploma ECTS) Age 17-21 6 7 Junior Bachelor, 1.5-2 years (120-150 Short-cycle Diploma ECTS) Age 17-18/19 5 6 2-3 years (based on 11 years of complete Junior Specialist secondary education) Age 18-20/21 Higher Professional Pre-Higher Education Diploma 3-4 years (based on 9 education years of basic secondary education) Age 15-17/18 4 5 Higher Third (upper) level Certificate education 4 4 Vocational 1-1.5 years (based on Education and 11 years of complete EIT Training (VET) secondary education) Age 18-19 Higher Second (basic) level Certificate 3 years (based on 9 education years of basic secondary education) Age 15-17 3 3 First (initial) level Certificate 2 Field-specific 2 years, moving to 3 Higher Complete Certificate (profile) (Level III) years in 2027 Age 15-17 education 3 3 EIT General Secondary Basic secondary Certificate VET (Level II) 5 years Age 10-14 2 2 Education Primary (Level I) 4 years Age 6-9 1 1 Preschool education Age 3-6/7 0 0 * 300-360 ECTS for programs in Medicine, Veterinary, Pharmacy Source: Authors’ elaboration based on information received from MOES 159 Annex 2: Literature Review on Returns to Education in Ukraine The empirical evidence on returns to education in Ukraine is relatively outdated and scarce, partly due to data limitations. The Ukrainian Labor Force Survey (LFS) conducted on a regular basis in Ukraine since 1999 (in 1999–2003 quarterly, and since 2004 monthly) does not include information about workers’ earnings. The Household Living Conditions Survey (HLCS) conducted quarterly since 1999 lacks many important variables such as working hours and job characteristics. For this reason, the major source of data used for estimation of returns to education in a Mincerian earnings function framework has been the Ukrainian Longitudinal Monitoring Survey (ULMS) conducted in 2003, 2004, 2007 and 2012, and its 2012 STEP module. Although some studies are not focused on returns to education, they are included in this review as they provide the estimates of returns to education from Mincerian earnings functions. Gorodnichenko and Sabirianova (2005) is the first and most rigorous study of returns to education in Ukraine. Using the 2003 Ukrainian Longitudinal Monitoring Survey (ULMS) and its retrospective part for employed workers aged 15 to 59 years, the authors estimated returns to adjusted years of schooling for 1986, 1991, 1997, 1998, 2000 and 2002. Controlling for worker’s experience and job tenure (and their squared terms), gender, living in a capital city location, firm size and its ownership type, they found that returns to an additional year of schooling increased in Ukraine from 3.6 percent in 1997 to 4.8 percent in 2002, which are lower than in Russia (7.9 percent in 1996 and 9.7 percent in 2002). The authors conducted a careful sensitivity analysis of these estimates by relaxing restrictions on age and not using sample weights, employing actual years of schooling instead of adjusted ones, using alternative measures of earnings, controlling for parents’ background, and applying the IV estimation method. They came to the conclusion that the overall finding of the divergence in rates of returns between Russia and Ukraine is not affected. After applying semiparametric methods to construct counterfactual wage distributions for workers having university and general secondary education, the authors concluded that cross-country differences in unobservable characteristics did not contribute significantly to explaining the differences in returns to schooling between Russia and Ukraine, whereas the differences in prices of observable characteristics played a critical role. Ganguli and Terrell (2006) also used ULMS-2003 but they included dummies for different levels of education with years of schooling, which is used in a typical Mincerian earnings equation, and they estimated the model separately for men and women. After controlling for age, nationality, living in capital city, enterprise’s economic activity and month of the survey, the returns to different levels of education are as follows: • High school (compared to less than high school): for males the coefficient is insignificant in both 1986 and 2003, while for females it is significant and increases from 0.079 in 1986 to 0.183 in 2003. • Vocational (compared to less than high school): for males the coefficient in 1986 was 0.111 and became insignificant in 2003, while for females it increases from 0.133 in 1986 to 0.184 in 2003. • Secondary professional (two additional years after high school, compared to less than high school): for males the coefficient decreases from 0.194 in 1986 to 0.173 in 2003, while for females it again increases from 0.133 in 1986 to 0.267 in 2003. • University and higher (bachelor’s/specialist’s/master’s/doctorate, compared to less than high school): For males the coefficient is highly significant and increasing from 0.246 in 1986 to 0.388 in 2003; for females it is even higher at 0.396 in 1986 and 0.507 in 2003. The authors focused on explaining changes in wage inequality of men and women between 1986 and 2003, and argue that the returns to human capital do not appear to be an important factor in explaining the increase in gender inequality in Ukraine. The 1986–2003 difference in the returns to education is statistically significant only for women with secondary professional education. 160 Constant and others (2012) estimated determinants of earning by the nationality, language and gender and found that returns to education differ not only between women and men but also between the language-based population groups. Similar to Ganguli and Terrell (2006), the authors captured the effects of education by five levels of schooling, with less than high school used as a benchmark category. The positive and significant effect of complete higher education (bachelor’s or master’s degree, diploma, or doctorate equivalent from an institute, a university, or an academy) is found in Ukrainian and Russian language groups but not in the one called Surzhyk (an amalgam of the Russian and Ukrainian languages widespread in many regions of Ukraine). Coupé and Vakhitova (2011) used the same approach as Gorodnichenko and Sabirianova (2005), but their resesearch included the second round of the ULMS data (ULMS-2007). They found that additional year of adjusted schooling increases monthly salary by 5.7 percent for the whole sample (for men by 5.4 percent, for women by 5.9 percent) in 2003 and by to 6.1 percent in 2007 (for men by 4.5 percent, for women by 7.4 percent). Besides, the analysis of returns to years of education by its level shows that in 2003 having a secondary education increased one’s monthly wage by about 5.5 percent, compared to 2.6 percent for vocational education, 4.4 percent for secondary professional education and 5.8 percent for tertiary education. Meanwhile, in 2007 having secondary education was no longer a differentiating factor, and only tertiary education had a significant return of 5.6 percent per additional year. The authors interpreted this finding as a changing wage distribution, with fewer and fewer low-skilled people who were paid very low wages in 2007. Zhou and Nelson (2017) used the STEP module of ULMS-2012 covering only urban working-age population. They found the significant return to education for females only. It varies from 7.78 percent per year in the basic Mincerian regression estimated with OLS to 11.43 percent in the Heckman selection model. Kupets (2015) also used the STEP module of ULMS-2012 covering only urban working-age population. She found that years of education have a significant direct effect on the logarithm of hourly earnings (about 4 percent per year) only in the basic Mincerian model that includes education, experience and experience squared. Once occupation, sector, job and demographic characteristics and some set of cognitive and non-cognitive skills are added into the model, the coefficient of education becomes insignificant and sometimes negative. This can be interpreted in a way that individual’s occupation and skills as well as his/her mother’s education capture the effect of the years of education on wages. In turn, not accounting for these variables may be biasing the estimates of the impact of education upwards due to the omitted variable bias. 161 Annex 3: Methodology and Estimation Results for Analysis of Secondary School Learning Outcomes The data used for this analysis comes from the External Independent Testing (EIT) in 2018, DISO data 2018 and the student survey from 2016. Data about the schools comes from the statistical reports all of the schools have to submit about their students, teachers, classes, equipment and activities. The data are gathered and filled in the statistical form by the school itself. Upon gathering the data, the schools report submit it to the Department of Education. The Department of Education is by law responsible for the gathering, control and processing of this data that they then have to submit to the Institute of Educational Analytics of the Ministry of Education and Science of Ukraine (table A3.1).1 Table A3.1 School data variables full sample Variable n min max mean sd School size 16529 0 2933 245.68 292.16 Average class size of 11th grade 8880 1 41 15.4 8.05 Mean class size 16140 0 39.94 15.3 7.57 Hub schools 16529 0 1 0.11 0.31 Source: Authors’ analysis of DISO data Data on the results of the EIT come from the Ukrainian Centre for Educational Quality Assessment and are gathered automatically. The results of the EIT are also distributed automatically among the participants using the personal electronic cabinets (table A2.2). Table A3.2 EIT data variables full sample Variable n min max mean sd Type of school 335687 0 1 0.14 0.35 Type of settlement 335687 0 1 0.81 0.39 Age 335687 15 69 18.01 2.58 Gender 335687 0 1 0.50 0.50 Universal Profile 335687 0 1 0.65 0.48 Humanities Profile 335687 0 1 0.18 0.39 Mathematics Profile 335687 0 1 0.09 0.28 Science Profile 335687 0 1 0.03 0.17 Other Profile 335687 0 1 0.05 0.22 Ukrainian language class 335687 0 1 0.49 0.50 Russian language class 335687 0 1 0.05 0.22 Other language class 335687 0 1 0.45 0.50 Test scores for EIT in Ukrainian 323031 0 104 46.85 23.04 Test scores for EIT in Math 106373 0 62 23.29 14.48 Source: Authors’ analysis of EIT data Data on the survey of the EIT participants in 2016 comes from the Institute of Educational Analytics. The survey was integrated into the electronic cabinets of the EIT participants. It was conducted after the EIT was taken and before the participants learned their EIT scores. They had to fill out the survey form answering questions before they could get to the cabinet with their scores. The data were merged based on the school name and school ID. The name of the school could have different spelling in different databases. The first step was to match the names of the schools from different 162 databases—the one with the results of the EIT and the one with the statistical data on schools. The matching was done through a fuzzy matching algorithm and during this procedure, some of the schools were eliminated. Some of them were eliminated because they provide only the primary level of education, so there were no graduates taking the EIT there; others were eliminated because they were from vocational schools that were not present in the database with the statistical reports; and still others were not picked up by the algorithm. Out of 16,529 schools in the DISO database, there were 7,255 picked up by the algorithm. Additionally, out of 323,510 students that took the EIT in Ukrainian language and literature, this analysis contains a sample of 132,960 students. After the names of the schools from EIT database and statistical reports database were matched according to the algorithm, the necessary variables were merged by the variable ‘school ID’, a unique identifier for each school. The final dataset contains the observations that were matched and have an identifier for each school. Additionally, the subset of this dataset was created which contained only the variables needed for the regression analysis. This means that the ones that were identified as possible factors could affect or explain the differences in the student learning outcomes. Statistical analysis To measure learning outcomes, the raw EIT scores in Ukrainian language and literature and mathematics were used as dependent variables. The EIT test used for the analysis is Ukrainian language and literature because it is the only subject that all the high school graduates have to take in order to get their high school diploma. All graduates have to take the test on Ukrainian language and literature whether or not they intend to continue their education; and not all graduates choose mathematics, 55 percent in urban elite schools, 45 percent in regular urban schools, and 40 percent in rural schools. For the other two subject test for the EIT, students can choose their subjects. That is, for the second test, they can take either mathematics or Ukrainian history, and for the third test, they can choose Ukrainian history, mathematics, biology, geography, chemistry, or a foreign language. Notes 1 https://zakon.rada.gov.ua/laws/show/z1000-18 Box A3.1 Description of external independent test scores EIT scores are assigned on the scale from 100 to 200 score points and the same scale is applied to all the different subjects of EIT. These scores are based on the number of test points students earned on their EIT. The 200 points scale is rating-based. This means that it shows the placement results of all students taking one of the EIT subject tests, for example, in Ukrainian language and literature, indicating how one student test-taker has scored in comparison to other student test-takers. Each year a pool of experts determines the number of test points the students have to get to pass the test. This number is different for each of the subjects as it depends on the difficulty of the test itself. In 2018 students had to get at least 23 test points out of 104 to pass the test for Ukrainian language and literature. This means that everybody that did not pass the test got less than 23 test points. They were not given a score on EIT and were not included in the ranking system and in the database their EIT score is 0. This means that 100 to 200 score points in EIT are given to those that had already passed the test and got more than 23 test points. In this analysis, the raw data on EIT results were used, meaning test scores and not rating scores. The reason for using the raw test scores instead of the rating scores from 100 to 200 that are assigned by the 163 algorithm is that the rating scale has a cut-off score and the students that scored below do not pass the test and are not included in the rating scale. This skews the distribution of the rating scores (see figure A2.1) and does not create a clear picture. The second reason is that the algorithm assigning rating scores can level some of the test scores out that in reality are different (figure A2.2). Figure A2.1 Distribution of Rating Scores in EIT in Ukrainian language and literature (passing scores only) Figure A2.2 Distribution of Test Scores in EIT in Ukrainian language and literature Higher education institutions (HEIs) also have the right to set their own cut-off scores. In 2018 there was a special case for medical universities and medical faculties in regular universities: the minimum score for applicants was set at 150 score points. In line with the literature, results were controlled for available demographic characteristics of graduates, including the age of the students, which was calculated based on the year of birth that was listed; and the gender of the applicants (female is a basic category). Regional differences were also controlled for by including dummy variables for oblasts and Kyiv. The following class characteristics were included in analysis: 164 ● Class profile. The class profile describes the specialized curricular profile of the class from which the student graduated: a universal profile (no curricular specialization), humanities profile (foreign philology, Ukrainian philology, history, law, philosophy), mathematics profile (physics, mathematics, economics, information and technology), science profile (biology, physics, chemistry, biotechnology, ecology, agrarian technologies, biology, geography), and other profiles (technological, sports, military and sports, arts). A general hypothesis is that students from the subject profiled classes (humanities, mathematics, science) should have better EIT results due to selectivity of better students to these classes. A universal class profile is considered to be the base category. ● Class language. The language of studies in the class the student graduated from may be Ukrainian, Russian, or another language (Hungarian, Polish, Moldovan, Romanian). While the official language of studying is Ukrainian, there are other languages of study which include the study of Ukrainian as a subject. These other languages of study, such as Russian, are the language of studying of national minorities. The language of study may have an effect on the EIT scores especially in the EIT in Ukrainian language and literature because students that study all of the subjects in Ukrainian should have better knowledge of it. Ukrainian is a basic category. ● Average class size of the 11th grade in the school. The class size number was calculated based on the number of 11th grade students in the school to the number of the 11th grade classes in the school. The class size reflects the influence of the learning environment of a school on student learning outcomes. On one hand, small class size can mean more attention from the teacher during preparation for IET and higher selectivity of graduates (as some part of students could choose to pursue with a professional caretaker in 9th grade). On the other hand, smaller classes can mean less learning from peers and less competition for higher grades. School-specific characteristics were also included in the model in order to capture the effect of school type and size: ● Elite school. This variable denotes whether the school is elite school, meaning selective schools. These schools include gymnasiums, lyceums and specialized schools with extensive learning of a specific subject (regular nonselective school is a basic category). Students from elite schools are expected to have better EIT results due to positive selectivity of these students to such schools and a good learning environment. ● Urban. This type of the settlement refers to where the applicants’ school is located; an urban area is 1, while a rural area is 0. Students from rural schools are expected to have lower EIT results due to a lower quality teaching and learning environment. ● Hub school. This is an indicator of the school being a hub (regular, non-hub school is a basic category). The hub schools have been set up to improve the quality of education of children from rural areas. ● Size of the school. This reflects the total number of students in the school. The larger the school the more money they get from the local budget and the more subject teachers they can hire, ensuring better quality of teaching. The model also includes a squared (quadratic) term of this variable to account for the non-linear effect, as too many students in school can negatively affect the learning outcomes of the students. To account for various indicators of school equipment, the authors composed indices of the teaching staff and conditions for extracurricular activities: ● Index for material and technical equipment of the school. This index measures what schools provide to students in terms of material and technical equipment. For the most part the index focuses on the equipment for the specific subjects: biology, chemistry, physics, and informatics. The more well-equipped classrooms and working places there are, the better quality of education the students can receive. 165 ● Index for the teacher qualifications. This index measures the qualifications and characteristics of the teachers of the school that can affect the quality of the education provided in the school and thus on the learning outcomes of the students. The rank of the teachers is connected to their education, but it also shows the qualifications the teacher attained on the job. This means that the higher the rank of the teachers in the school, the better results students can be expected to achieve. ● Index for informational and communicational technologies (ICT) used in the school . This index measures informational and communicational capabilities and technologies of the school. The better the school is equipped in this field, and the more access there is for the students (for example, the internet or Wi-Fi availability in the school), the more opportunities students have to use the technology during their studies to enhance their knowledge. ● Index for the extracurricular activities. This index includes data on the number of extracurricular clubs of different interests and the share of the student body of the school that participates in clubs of different interests. It measures whether the school gives students the opportunity not only to have extracurriculars but also to learn through activities, which can have an impact on their learning outcomes. ● Share of teachers of age under 30 years old. The age of the teachers is an important factor. Younger teachers may have less experience than older teachers who have established specific methods of teaching. The share is calculated to the total number of teachers in the school. ● Share of teachers older than 60 years old. The age of the teachers is an important factor. Older teachers may not be as flexible in adjusting to new methods of teaching and new material. The share is calculated to the total number of teachers in the school. A separate analysis was also constructed for a subsample of students that took part in the surveys to control for the socio-economic characteristics of student’s background. This analysis is the Index for parents’ occupational status. The index is based on the occupational status of the mother, father and the highest occupational status out of the two of them. However, in Ukraine, higher occupational status does not always mean higher income or equivalent education due to high incidence of skill mismatch and different returns to skills in sectors. As such, this analysis includes other variables to control for other socio-economic characteristics: • Home possessions. This index is on the wealth of the students’ families. • Employment status of mother and father. Dummy variables are used for working in Ukraine, with working abroad, not working, as a basic category. • Highest education level of parents. Dummy variables are used for secondary, vocational, unfinished higher, higher education, have a scientific degree, with secondary education as a basic variable. • Sector of parents’ employment. Dummy variables are used for public service and local government; industry; construction; agriculture, forestry, fisheries; wholesale and retail trade, hotels and restaurants; transport and communications; information and telecommunications, such as journalism and advertising; programming and system administration; financial and insurance activities; education; health care and social assistance; culture, sports, public organizations, with industry as a basic category. Additionally, a variable was included to denote whether students took tutoring lessons from their own teachers or private teachers either in Ukrainian language or Ukrainian literature or both. In this case tutoring means taking paid lessons from any teacher. This includes students who have better socioeconomic status, and who are more likely to be able to pay for tutoring to prepare for the EIT; this does not include students taking free classes at school after hours or prepared for EIT during lessons on the initiative of the teacher. The results on the EIT testing should depend on future plans of the student. If students plan to continue higher education in Ukraine they will be more motivated to get higher scores on EIT because it is both a graduation exam and entrance exam to HEI. Therefore, dummy variables were included if the student plans 166 to continue to vocational education, higher education, and higher education abroad. Not planning to continue education after graduating from high school is a basic category. The survey also includes information that assesses the influence of territorial accessibility of school on learning outcomes. Therefore, a variable was included to indicate whether a school is located in the same settlement where the students live. Having to commute to another settlement may have a negative effect on the learning outcomes. At the same time commuting to school (both in another settlement or within the same settlement) may mean that students are studying in a selective school, usually meaning the school with higher quality of education. Although the time spent for commuting to school is important in view of the latest changes in the law, the World Bank does not include it in the model because this question is answered only by those who study in the settlement other than they live in (table A3.3). Table A3.3 Subsamples of merged data on EIT, DISO and survey used for the regression models mean min max sd mean min max sd Age 17.34 15.00 64.00 0.96 17.37 16.00 46.00 1.22 Gender 0.47 0.00 1.00 0.50 0.45 0.00 1.00 0.50 Class profile Universal 0.37 0.00 1.00 0.48 0.33 0.00 1.00 0.47 Humanities 0.33 0.00 1.00 0.47 0.37 0.00 1.00 0.48 Mathematics 0.15 0.00 1.00 0.36 0.18 0.00 1.00 0.38 Science 0.06 0.00 1.00 0.23 0.05 0.00 1.00 0.22 Other 0.09 0.00 1.00 0.29 0.07 0.00 1.00 0.25 Ukrainian language class 0.89 0.00 1.00 0.32 0.89 0.00 1.00 0.31 Russian language class 0.08 0.00 1.00 0.27 0.10 0.00 1.00 0.30 Other language class 0.03 0.00 1.00 0.18 0.01 0.00 1.00 0.07 Average class size in 11th grade 19.72 1.00 41.00 7.62 21.13 1.00 41.00 7.45 Type of school 0.29 0.00 1.00 0.45 0.36 0.00 1.00 0.48 Type of settlement 0.70 0.00 1.00 0.46 0.80 0.00 1.00 0.40 Hub school 0.07 0.00 1.00 0.26 0.03 0.00 1.00 0.18 Mean class size 22.72 2.33 36.19 6.54 24.29 2.86 36.19 6.14 Size of school 563.35 14.00 2933.00 378.32 629.26 20.00 2933.00 385.86 Index on material equipment 0.25 -4.00 3.00 0.68 0.24 -4.00 3.00 0.91 Index on teachers qualifications 0.59 -3.00 4.00 0.69 0.69 -3.00 2.00 0.90 Index on ICT 0.08 -3.00 9.00 0.92 0.07 -3.00 7.00 0.98 Index on extracurricular activities 0.15 -2.00 2.00 0.94 0.09 -2.00 2.00 0.80 Share of teachers up to 30 years old 0.12 0.00 0.82 0.08 0.11 0.00 0.75 0.07 Share of teachers older than 60 years old 0.12 0.00 0.71 0.09 0.12 0.00 0.50 0.08 Index on home possessions -0.10 -5.00 4.00 0.98 Index on parents' occupation status 0.07 -2.00 3.00 1.00 Mothers working in Ukraine 0.73 0.00 1.00 0.44 Fathers working in Ukraine 0.63 0.00 1.00 0.48 Mothers working abroad 0.02 0.00 1.00 0.13 Fathers working abroad 0.04 0.00 1.00 0.19 Mothers not working 0.19 0.00 1.00 0.39 Fathers not working 0.09 0.00 1.00 0.29 Sector of employment Public administration 0.15 0.00 1.00 0.36 Manufacturing 0.17 0.00 1.00 0.37 Construction 0.11 0.00 1.00 0.32 Agriculture 0.07 0.00 1.00 0.26 Services 0.14 0.00 1.00 0.34 Transport 0.10 0.00 1.00 0.29 Information and telecommunications 0.01 0.00 1.00 0.10 Programming and system administration 0.02 0.00 1.00 0.14 Education 0.13 0.00 1.00 0.33 Finance 0.07 0.00 1.00 0.25 Healthcare 0.10 0.00 1.00 0.31 Culture 0.03 0.00 1.00 0.16 167 Highest level of parents' education Secondary 0.16 0.00 1.00 0.37 Vocational 0.25 0.00 1.00 0.44 Unfinished higher 0.24 0.00 1.00 0.42 Higher 0.54 0.00 1.00 0.50 Scientific degree 0.03 0.00 1.00 0.16 Tutoring in Ukrainian language and/or 0.48 0.00 1.00 0.50 literature Student's future plans not to continue education 0.01 0.00 1.00 0.11 continue vocational education 0.09 0.00 1.00 0.28 continue higher education 0.85 0.00 1.00 0.35 continue education abroad 0.04 0.00 1.00 0.20 School in the same location 0.88 0.00 1.00 0.33 School in different location 0.12 0.00 1.00 0.33 Source: Authors’ analysis of DISO, EIT, and survey data Note: The number of observations is 125,405 for DISO and EIT merged data, and the number of observations is 24,367 for DISO, EIT and survey merged data. Estimation results To assess the joint effect of various school characteristics and socioeconomic background on the learning outcomes of students we used three specification of OLS model. Each next specification is the extension of previous one. Because there are significant differences in learning outcomes between students from rural and urban schools and in school characteristics in urban and rural area, the analysis was replicated with disaggregation by area. The results of the estimations are in table A3.4 for Ukrainian language and literature and table A3.5 for mathematics. Models 1, 4, 7 are basic specifications in which students’ learning outcomes are explained by students’ demographic characteristics, class specific characteristics, type of the school, and school size. Models 2, 5, 8 include more school specific characteristics of the learning environment summarized with the help of four indices: material equipment, availability of information and computer technologies, quality of teaching staff and availability of extracurricular activities in school. The last set of models 3, 6, 9 accounts for the differences in socioeconomic background, future student’s plans, and the need to commute to schools in another settlement. Table A3.4 Estimation results for EIT in Ukrainian language and literature All Urban Rural 1 2 3 4 5 6 7 8 9 Age -1.56 -1.55 -1.85 -1.87 -1.85 -1.96 -1.11 -1.12 -1.82 (0.06) (0.06) (0.09) (0.08) (0.08) (0.10) (0.08) (0.08) (0.20) Male -11.89 -11.90 -12.47 -10.92 -10.94 -12.31 -13.87 -13.87 -14.09 (0.11) (0.11) (0.25) (0.14) (0.14) (0.28) (0.20) (0.20) (0.56) Elite school 9.62 8.32 9.15 9.66 8.18 9.28 6.47 6.33 -2.18 (0.15) (0.15) (0.33) (0.16) (0.17) (0.35) (0.59) (0.60) (1.90) Urban 8.74 7.66 5.73 (0.18) (0.18) (0.50) Humanities profile 2.19 2.21 2.10 1.29 1.31 1.41 3.61 3.62 4.06 (0.15) (0.15) (0.34) (0.19) (0.19) (0.38) (0.27) (0.27) (0.73) Mathematics profile 5.36 4.90 2.88 4.89 4.36 1.98 2.61 2.47 4.88 (0.19) (0.19) (0.40) (0.22) (0.22) (0.45) (0.50) (0.49) (1.20) Science profile -0.05 -0.28 2.22 -0.49 -0.77 3.01 -0.19 -0.37 -3.48 (0.26) (0.26) (0.60) (0.31) (0.31) (0.66) (0.52) (0.52) (1.32) Other profile -6.42 -5.94 -7.09 -9.49 -8.90 -9.98 -2.33 -2.06 -0.82 (0.21) (0.21) (0.59) (0.29) (0.29) (0.69) (0.30) (0.30) (1.07) Russian language -0.59 -0.59 0.97 -0.73 -0.80 0.78 0.44 0.32 3.17 (0.23) (0.23) (0.46) (0.24) (0.24) (0.48) (0.67) (0.67) (1.91) 168 Other language -15.86 -17.10 -8.32 -17.88 -19.56 0.74 -11.80 -12.18 -11.60 (0.25) (0.26) (1.69) (0.35) (0.36) (3.14) (0.38) (0.39) (1.84) class size of 11th grade -0.22 -0.29 0.01 0.20 0.01 0.37 -0.12 -0.17 -0.14 (0.04) (0.04) (0.10) (0.07) (0.07) (0.16) (0.08) (0.08) (0.21) class size of 11th grade 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 squared (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.01) School size 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.01) School size squared -0.00 -0.00 -0.00 -0.00 -0.00 -0.00 -0.00 -0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) Hub school -2.14 -2.98 -3.83 -3.02 -3.63 -4.55 -0.16 -1.14 0.82 (0.24) (0.24) (0.68) (0.29) (0.30) (0.76) (0.41) (0.42) (1.45) Index on material -0.18 0.47 -0.40 -0.06 -0.11 0.95 equipment (0.09) (0.23) (0.11) (0.27) (0.16) (0.43) Index on ICT 1.63 2.10 1.62 2.00 1.30 1.64 (0.07) (0.16) (0.08) (0.18) (0.12) (0.33) Index on teachers 1.91 0.48 2.25 0.71 1.13 -0.02 qualifications (0.10) (0.22) (0.12) (0.25) (0.16) (0.47) Index on extracurricular -0.31 -0.06 -0.42 0.00 0.02 0.10 activities (0.07) (0.15) (0.08) (0.17) (0.12) (0.34) Share of teachers under 30 0.08 -8.51 6.15 -7.84 -7.54 -6.01 years (0.80) (1.91) (1.08) (2.34) (1.19) (3.36) Share of teachers above 60 4.79 2.36 8.89 6.17 -3.20 -7.69 years (0.74) (1.70) (0.94) (2.04) (1.24) (3.18) Index on home -0.08 -0.13 -0.04 possessions (0.13) (0.14) (0.30) Index on parents' 0.38 0.77 -1.23 occupation status (0.12) (0.14) (0.28) Public 0.29 0.13 1.22 (0.37) (0.40) (0.91) Construction 0.02 0.40 -1.02 (0.43) (0.49) (0.87) Agriculture -0.34 -0.50 -0.04 (0.53) (0.74) (0.77) Services 0.28 0.08 1.22 (0.38) (0.41) (1.04) Transport 0.11 -0.02 0.76 (0.44) (0.48) (1.10) Information and 0.47 0.35 1.41 telecommunications (1.20) (1.23) (4.80) Programming and system 3.18 3.70 -4.51 administration (0.81) (0.84) (3.01) Finance 0.68 0.76 0.07 (0.49) (0.51) (1.57) Education -0.06 -0.15 0.75 (0.42) (0.46) (0.94) Health 0.57 0.96 -1.43 (0.42) (0.46) (1.00) Culture -0.11 0.23 -2.06 (0.77) (0.85) (1.79) Vocational -0.16 -0.48 0.97 (0.33) (0.37) (0.65) Unfinished higher 0.27 0.06 1.16 (0.32) (0.36) (0.67) Higher 1.28 1.09 1.28 (0.31) (0.35) (0.71) Scientific degree 1.43 0.74 7.21 (0.78) (0.81) (2.68) Tutoring 0.88 0.88 0.64 (0.26) (0.28) (0.61) School in the same -0.10 0.35 -1.04 location (0.38) (0.45) (0.73) 169 Const 79.19 78.69 82.93 88.14 86.00 84.82 68.68 70.35 82.55 (1.11) (1.13) (2.09) (1.62) (1.64) (2.77) (1.59) (1.60) (4.88) N 125405 125405 24367 87136 87136 19567 38269 38269 4800 R2 0.31 0.31 0.29 0.23 0.24 0.23 0.20 0.20 0.21 Source: Authors’ analysis of EIT, DISO, and survey data Note: Heteroskedasticity robust standard errors in parentheses. OLS, dummies for oblast, employment status of mother and father, students’ further plans are included. Table A3.5 Estimation results for EIT in Mathematics All Urban Rural 1 2 3 4 5 6 7 8 9 Age -0.87 -0.86 -1.16 -0.95 -0.95 -1.03 -0.65 -0.63 -3.18 (0.06) (0.06) (0.22) (0.08) (0.08) (0.20) (0.09) (0.09) (0.48) Male -1.31 -1.36 -3.28 -0.96 -0.99 -3.60 -2.20 -2.25 -2.57 (0.11) (0.11) (0.23) (0.13) (0.13) (0.26) (0.18) (0.18) (0.52) Elite school 6.30 5.31 6.22 6.24 5.12 6.15 5.85 5.80 -1.17 (0.14) (0.15) (0.32) (0.15) (0.16) (0.35) (0.58) (0.58) (1.81) Urban 4.31 3.67 3.22 (0.17) (0.18) (0.46) Humanities profile -1.82 -1.80 -1.15 -2.33 -2.24 -1.39 -0.60 -0.65 -0.69 (0.14) (0.14) (0.32) (0.18) (0.18) (0.38) (0.23) (0.24) (0.68) Mathematics profile 6.19 5.81 3.89 6.18 5.80 3.61 2.39 2.28 2.95 (0.17) (0.17) (0.37) (0.20) (0.20) (0.41) (0.42) (0.41) (1.08) Science profile -1.10 -1.35 -1.83 -1.19 -1.45 -1.29 -1.54 -1.71 -6.22 (0.24) (0.24) (0.52) (0.29) (0.29) (0.56) (0.43) (0.43) (1.40) Other profile -4.89 -4.69 -3.31 -6.92 -6.64 -6.08 -1.40 -1.30 3.72 (0.20) (0.20) (0.56) (0.27) (0.28) (0.61) (0.29) (0.29) (1.08) Russian language 1.03 1.03 3.72 0.88 0.87 3.66 1.26 1.22 3.74 (0.20) (0.20) (0.45) (0.22) (0.22) (0.47) (0.52) (0.53) (1.72) Other language -5.21 -5.43 2.22 -5.42 -6.08 8.34 -3.59 -3.25 -3.86 (0.44) (0.45) (1.44) (0.74) (0.77) (1.28) (0.55) (0.56) (3.28) class size of 11th grade -0.10 -0.15 -0.01 0.13 0.00 0.35 -0.04 -0.07 -0.14 (0.04) (0.04) (0.09) (0.06) (0.06) (0.14) (0.07) (0.07) (0.19) class size of 11th grade 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 squared (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.01) School size 0.01 0.01 0.01 0.00 0.01 0.01 0.01 0.01 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.01) School size squared -0.00 -0.00 0.00 -0.00 -0.00 -0.00 -0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) Hub school -2.30 -2.86 -4.42 -3.03 -3.54 -4.98 -0.58 -1.05 0.76 (0.22) (0.23) (0.64) (0.28) (0.28) (0.79) (0.36) (0.38) (1.18) -0.12 -0.16 -0.21 -0.12 -0.20 -1.09 Index on material equipment (0.09) (0.20) (0.11) (0.24) (0.14) (0.37) Index on ICT 1.12 1.80 1.22 1.87 0.70 1.22 (0.06) (0.14) (0.08) (0.16) (0.11) (0.30) Index on teachers 1.13 0.82 1.20 0.62 0.63 0.96 qualifications (0.09) (0.21) (0.12) (0.25) (0.14) (0.42) Index on extracurricular -0.11 0.37 -0.21 0.26 0.24 0.73 activities (0.06) (0.14) (0.08) (0.17) (0.10) (0.28) Share of teachers under 30 3.44 2.00 8.16 8.36 -4.63 -9.27 years (0.75) (1.85) (1.04) (2.33) (1.06) (3.16) Share of teachers above 60 6.29 6.00 8.90 11.35 0.00 -6.69 years (0.70) (1.56) (0.89) (1.88) (1.07) (2.89) -0.52 -0.63 0.02 Index on home possessions (0.13) (0.14) (0.29) Index on parents' occupation -0.03 0.23 -1.08 status (0.11) (0.13) (0.29) Public -0.24 -0.37 0.29 (0.35) (0.38) (0.85) 170 Construction 0.22 0.56 -0.80 (0.40) (0.45) (0.81) Agriculture -0.82 -0.87 -0.98 (0.46) (0.65) (0.67) Services 0.22 0.25 -0.13 (0.35) (0.38) (0.86) Transport 0.22 0.20 -0.16 (0.40) (0.44) (0.98) Information and 1.83 1.56 4.68 telecommunications (1.14) (1.21) (3.29) Programming and system 1.70 2.03 -2.48 administration (0.75) (0.79) (1.83) Finance 0.50 0.59 0.23 (0.45) (0.47) (1.39) Education 0.18 0.02 1.34 (0.39) (0.43) (0.86) Health -0.11 -0.07 -0.77 (0.38) (0.43) (0.85) Culture 0.06 0.35 -2.80 (0.71) (0.77) (1.75) Vocational -0.40 -0.83 1.28 (0.30) (0.34) (0.58) Unfinished higher -0.29 -0.32 0.20 (0.29) (0.33) (0.58) Higher 0.86 0.71 0.36 (0.29) (0.32) (0.62) Scientific degree 0.47 0.21 3.97 (0.77) (0.80) (2.42) Tutoring 0.14 0.13 -0.03 (0.24) (0.26) (0.54) 0.01 0.24 -0.25 School in the same location (0.35) (0.41) (0.64) Const 36.12 34.73 36.70 39.22 36.98 31.17 30.34 30.71 75.67 (1.15) (1.17) (4.12) (1.52) (1.54) (4.07) (1.79) (1.80) (8.69) N 58209 58209 12092 42901 42901 9959 15308 15308 2133 R2 0.25 0.25 0.27 0.19 0.20 0.24 0.06 0.07 0.17 Source: Authors’ analysis of EIT, DISO, and survey data Note: Heteroskedasticity robust standard errors in parentheses. OLS, dummies for oblast, employment status of mother and father, students further plans are included. 171 Annex 4: Donor Mapping The donor mapping analysis is based on data collected by the Ministry of Education and Science of Ukraine from 16 external partners and conforms to a standard template. Relevant information includes external partner, name of project, description, expected results, indicators of project performance, type of assistance (for example, technical assistance, investments, or combined), initialization and finalization dates, financial instrument, level of education supported, year(s), and contact information. Data indicate that during the period of 2012–2018, the education system of Ukraine benefited from support of 16 bilateral or multilateral development partners agencies.1 In total, there are 72 registered projects, in the amount of US$143.2 million. In terms of type of projects, largest number (60 projects) focuses on technical assistance, although, in terms of money, the majority is for combined types including technical assistance and investments. Only four projects contained purely investments activities. All the projects are funded through a grant mechanism. The largest number of projects are focused on basic education, but they vary significantly in size and scale from small pilot projects to multi-million-dollar programs. There is also a large amount of support for the higher education sector, with the least support in preschool, lifelong learning, and special education. Level Development Partner Name of the Project Ensuring access to education in emergency Preschool UNICEF (2) Capacity building of preschool personnel and parents on child- friendly approach Special UNICEF (1) education Support to Inclusive Education Model School pilot British Council Ukraine (3) Change Agent training and NUS English reform Teacher Attestation and CPD Framework Deregulation of schools Development of the system of quality assurance for school Canada (3) education: standards and institutional audit procedure Development and launch of the National Educational Digital Platform (e-platform) Switzerland Swiss-Ukrainian Project Development of Citizenship Competences in Ukraine (DOCCU) Strengthening the protection of national minorities in Ukraine European Union Basic (Joint EU and CoE Programme; PGG) Primary School Pilot Network, New Ukrainian School Exams at schools (officially recognized German language Germany (4) certificates) Content and Language Integrated Learning (CLIL) Online teacher training courses on the job Consolidation of bilingual education Promotion of the French language and short mobility to France French reader France (6) Training in Ukraine PROFLE New Ukrainian School pilot project Regional registry 2.0 172 Norway Supporting Educational Reforms in Ukraine Programme USA Peace Corps Teaching English as Foreign Language (TEFL) project Sweden Swedish-Ukrainian Project, Support to Decentralization in Ukraine World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis Fostering Civic Activism via Social Project Management Charitable Organization, Teachers for Democracy and Partnership Developing civic competencies: practical cases and courses for educational programs (5–9 grades) Civic education for responsible citizenship. Introducing civic education into curricula Development of civic competence of students of the New Ukrainian School: The First Cycle of Elementary School NGO, Ukrainian Step by Step Foundation Promoting civic education in primary schools within new educational standards; NGO, Ukrainian Step by Step Foundation USAID (12) Development of civic activism in Ukraine through implementation of the civic education course, Culture of Good Neighborhood Development of civic education in Ukrainian schools through the design and piloting of special courses and modules Сampaign to promote civic education 1 Сampaign to promote civic education 2 Remote training course for elementary school teachers, Gender Sensitive and Non-Discriminative Approach in Education Pedagogy of Partnership as Foundation for New Ukrainian School Online course for primary school teachers on inclusive education Support and introducing reforms and increasing expertise of public administration in the field of education in Ukraine Czech Republic (3) Support for development of education in Ukraine Promotion of education quality assurance system in Ukraine School Entrepreneurship Academy Poland (3) Support to the reform of education system in Ukraine New Ukrainian School Safe and supportive learning environment: UNICEF Life skills based education (LSE) and ‘Safe Schools’ modeling Finland Finland's Support to the Ukrainian School Reform Germany Dual education (vocational training) EU4Skills Project European Union U-LEAD with Europe VET Active youth in the labor market Good Energy School Poland (4) Support the development of training nurses in palliative care Transformation of a vocational school in Chervonohrad British Council Ukraine (3) Ukraine Higher Education Leadership Development Program 173 English for Universities PRESETT reform Germany Learning how to Teach German (DLL) Provision of material for schools and universities France (3) Modernization in higher education Scholarships for educational stays in France Civic education for responsible citizenship. Introducing civic education into curricula Higher Implementing the course, Civic Education and Methods of Education Teaching It at the teacher training universities USAID (3) The Ukrainian Association of Teachers of Civic Education and Socio-political Disciplines Preparation of pedagogical university students for teaching a course on civic education in schools Network of higher education quality assurance experts Czech Republic (3) Support to the Ukrainian universities in their transition to European standards Material support to displaced educational institutions Poland Innovative university and leadership France Movie education Support to the Youth Entrepreneurship Incubator in Lviv Lifelong Young people with the initiative Learning Poland (4) Kharkiv business incubator, the chance for professional courses graduates The lessons with entrepreneurship in the background World Bank Ukraine Comprehensive Education Sector Analysis System Support to data collection and analysis (PISA, HBSC, SCORE, UNICEF and so on) Notes 1 Two agencies from the European Union, the European Council and a group of countries and EU that funded the project U-LEAD were combined under the EU umbrella. 174 Annex 5: Stakeholder Mapping for Higher Education Governance and Policymaking Body Roles and Responsibilities in Higher Education Policy Parliament • Sets the general legislative framework for higher education. Committees on • Decides on public funding. Science and • Provides general oversight over policy implementation and administration. Education and on • Represents the interests of constituencies. the Budget Cabinet of • Takes decisions over specific areas of higher education policy, such as Ministers of regulation of PhD programs, classification of fields of study, and regulation Ukraine of unified state qualifications exam for MA graduates. MOES • Directorate for Higher Education is the main department responsible for higher education policy, though many functions are controlled by other departments. • Department of Economics and Finance is responsible for funding, although Directorate of HEI is responsible for allocating public-funded seats. • HR department is responsible for the appointment of rectors. • A separate department is responsible for licensing, accreditation and attestation. Inforesurs • Operates EDEBO, crucial for the higher education admissions campaign, the collection of basic statistics on HEI and state registration of university diplomas, though capacity is limited. IEA • Established in 2015 to collect statistics and provide analytical support to the MES, though capacity is limited. NAQAHE • NAQAHE became operational in early 2019 and capacity is low. • Agency is legally independent from MOES, and comprises 23 members selected by a selection committee of international and local experts. • Responsible for accreditation of all higher education programs every five years. • Institutional accreditation, eventually leading to self-accreditation for universities with multiple accredited programs. • Accreditation of subject-specific agencies. • Establishment of university ranking mechanism over next 1–2 years (criteria to be determined). • Institute nationwide system for enforcement of academic integrity. • Regulation of PhD and DSc granting system. • Support HEIs to implement internal quality assurance systems via enforcement of new external quality assurance requirements. UCEQA • Develops and implements External Independent Test (EIT) and facilitates university admissions campaign. SSEQ • Controls for the compliance of educational institutions with educational legislature National Council • Permanent consultative body under the CMU, established in 2017 with the of Ukraine for goal to provide strategic coordination of research policy. The Council is Science and composed of the Administrative and Scientific Committee. 175 Technology Development Ministry of • Determines the amount of money available for higher education. Finance • Formally approves many other decisions in higher education policy, including all CMU decrees and funding related decisions and regulations. Ministry of • Prepares annual governmental decree on the number of the publicly funded Economic seats in universities (the so-called ‘state order’), which tends to be a Development and relatively low-priority issue for MOEDT. Trade Ministry of Justice • Provides approval to all documents that need to be approved by the CMU. • Registers decrees adopted by all ministries. • Supports reform of legal education over the past 3–4 years, representing broad interests of the legal community. State Regulatory • Occasionally reviews decrees on an ad hoc basis, such as the university Service admissions decree in 2018, despite never having been involved in adopting such a decree in the past. Many universities were strongly opposed to the introduction of minimum entrance requirements for law programs, because it was done for medical programs the year before. As a result, the SRS was requested to review the decree by Ministry of Justice, which contributed to significant delays in adopting the decree due to lengthy negotiations. State Accounting • Produces regular reports on spending in higher education, often arguing that Chamber the money is not spent properly. While there are multiple legitimate concerns on the efficiency of spending in higher education, the SAC is focused on the concern that some students do not graduate, which amounts to a waste of public resources and inefficient use of spending. This contributes to the pressure on universities to graduate students, even if they are not qualified. State Audit Service • Regularly reviews universities’ spending, though reviews are of mixed quality. • Requests extensive documentation and paperwork to conduct spending analysis, creating compliance burdens for universities. Higher education • HEIs are central components of the higher education landscape, though their institutions (HEIs) role in governance of the system is limited. HEIs tend to view policy changes as happening to them, rather than coming from them. • Several HEIs in particular are more active and vocal on policy and governance issues, particularly the Kyiv-Mohyla Academy and Ukrainian Catholic University. • In the policy making and governance process, HEIs are typically represented by their rectors. Student organizations are occasionally active in policy debates, but faculty are rarely involved. National • While not being an active player in higher education policy in general, Academies of NASU is involved in debates around recognition of scientific results and Science of Ukraine attestation policy in general. (NASU) 176 Annex 6: Universities with the Largest Share of Public Funding Table A6.1 Ten Universities with the Largest Share of Public Funding for Teaching Activities (excluding capital expenditure) in 2017, UAH thousands Number of full-time state-funded Public funding Share of full- students (all per state- time state- Amount of levels) in funded student Share of amount funded students public funding. October (2017, thsd of public funding (all levels) in University UAH thousand 2017 UAH) within MOES. % October 2017. % National Technical University of Ukraine “Igor Sikorsky Kyiv 907,822.30 17179 52.8 6.86% 2.57% Polytechnic Institute” Kyiv Taras Shevchenko National 807,559.80 14757 54.7 6.10% 2.21% University Lviv Polytechnic National University 562,631.00 12480 45.1 4.25% 1.87% National University of Life and 343,621.40 7889 43.6 2.59% 1.18% Environmental Sciences of Ukraine National Aviation University 311,285.70 8165 38.1 2.35% 1.22% Ivan Franko National University of 285,856.40 9962 28.7 2.16% 1.49% Lviv The National Technical University 284,608.50 7231 39.4 2.15% 1.08% "Kharkiv Polytechnic Institute" Oles Honchar Dnipro National 221,367.40 5805 38.1 1.67% 0.87% University V. N. Karazin Kharkiv National 210,747.00 6309 33.4 1.59% 0.94% University The Yaroslav Mudryi National Law 203,915.80 4679 43.6 1.54% 0.70% University Total 13,228,936.00 668191 100% 100% Source: Authors’ analysis of data from Ministry of Finance and EDEBO database. 177