Hungary: Measuring Inclusive Growth for Enhanced Development Impact Targeting and Monitoring of EU Co-funded Social Inclusion Activities at the Subregional Level Disclaimer This report is a product of the International Bank for Reconstruction and Development / the World Bank. The findings, interpretation, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. This report does not necessarily represent the position of the European Union or the Hungarian Government. Copyright Statement The material in this publication is copyrighted. Copying and/or transmitting portions of this work without permission may be a violation of applicable laws. For permission to photocopy or reprint any part of this work, please send a request with the complete information to either: (i) Ministry of Human Capacities (Arany János u. 6-8, 1051, Budapest, Hungary); or (ii) the World Bank Group Romania (Vasile Lascăr Street, No 31, Et 6, Sector 2, Bucharest, Romania) This report was prepared under the Advisory Services Agreement for Policy Advice to Support the Implementation of the National Social Inclusion Strategy of Hungary signed between the World Bank and the Ministry of Human Capacities on March 17, 2015. 1 Acknowledgments The team would like to express its gratitude to peer reviewers Joost de Laat, Katalin Szatmári, Maria Beatriz Orlando, and Andor Ürmös for their technical inputs and comments during the review process, as well as to Christian Bodewig, Alina Nona Petric, Manuel Salazar, Kenneth Simler, and Rob Swinkels for their helpful comments. The team is particularly grateful to counterparts from the Government of Hungary, in particular Csaba Andor and László Ulicska from the Ministry of Human Capacities, and András Kezán from the Hungarian Central Statistical Office, for their guidance, support, and helpful inputs throughout the preparation of this report. The team was supported by András Tamás Torkos, Isadora Nouel, and Andrei Zambor. Editorial work for this report was provided by Lauri Scherer. This report was prepared by a World Bank team composed of Céline Ferré, Sándor Karácsony, Ádám Kullmann, Valerie Morrica, and Nóra Teller, under the guidance and supervision of practice managers Nina Bhatt and Andrew D. Mason. 2 3 Table of Contents Acknowledgments ............................................................................................................ 2 Abbreviations and acronyms ............................................................................................. 6 Executive summary ........................................................................................................... 8 1. Introduction............................................................................................................. 10 2. Social exclusion: Frozen in a state of lagging behind ................................................. 12 2.1. Monetary poverty and social exclusion ........................................................................ 12 2.2. Capturing the dimensions of social exclusion ................................................................ 13 BOX 1. Using a multidimensional approach to compare social inclusion outcomes between Roma and non-Roma in Central Europe ................................................................................. 13 A recent report published by UNDP (Ivanov and Kagin, 2014) analyzes the difference in outcomes between the Regional Roma Survey (RRS) and the UNDP vulnerable groups survey of 2004. Instead of using a single indicator of poverty (such as income or consumption), the report uses a multidimensional approach to poverty (the MPI). Among Roma, the MPI decreased substantially between 2004 and 2011 in Bulgaria and Romania, primarily due to a decline in the number of “poor” households (that is, those with 5 to 7 deprivations in the MPI). “Severe poverty” (more than 7 deprivations) decreased significantly only in Romania................................................... 13 Looking at the different dimensions of poverty allowed Ivanov and Kagin to identify the key drivers of poverty reduction between 2004 and 2011; education and living conditions have improved, while fundamental rights and access to the labor market have worsened. The analysis of multidimensional versus monetary poverty metrics also reveals that the decline is similar in both metrics in Bulgaria, while the decline in monetary poverty in Romania is stronger than in multidimensional poverty. ...................................................................................................... 13 Source: Ivanov and Kagin (2014). ............................................................................................ 14 2.3. What are these dimensions? ........................................................................................ 14 2.4. Can the “depth” of social exclusion be measured? ........................................................ 16 3. Measuring social exclusion: Global and local lessons for Hungary ............................. 17 3.1. What has been done in Hungary so far? ....................................................................... 17 3.1.1. The HCSO used census-based indicators on education and unemployment to map marginalized communities ............................................................................................................ 17 3.1.2. Targeting tool for complex social inclusion interventions designed by HCSO .............. 18 BOX 2. Identifying pockets of poverty at the subregional level: Mapping Romania’s marginalized communities .................................................................................................... 22 3.1.3. Monitoring indicators from NSIS ................................................................................... 23 BOX 3. Europe-wide efforts to monitor national Roma inclusion strategies .......................... 24 3.1.4. Best predictors of AROP ................................................................................................ 25 3.2. International experience with measuring social inclusion outcomes at the national level 28 3.2.1. Choice of dimensions .................................................................................................... 29 3.2.2. Number of indicators..................................................................................................... 29 3.2.3. Composite versus non-unified indicators...................................................................... 33 3.2.4. Disaggregation level and frequency at which indicators are collected......................... 33 4 4. What indicators should be used in Hungary? ............................................................ 34 4.1. Indicators that are suited to track progress toward social inclusion ............................... 34 4.2. Issues and dilemmas regarding collecting and tracking indicators to monitor progress on social inclusion in Hungary ...................................................................................................... 35 4.3. Policy objectives and considerations for developing an indicator set ............................. 40 4.4. The proposed indicator set for Hungary........................................................................ 41 4.5. Confirming the validity of indicators............................................................................. 43 5. How to design a tracking tool that combines social inclusion indicators with project data? .............................................................................................................................. 45 5.1. Proposal for a tool to track development policy interventions in a social inclusion context 45 5.2. Topics and hypothesized impact mechanism of selected social inclusion measures ....... 46 5.3. Next steps toward an improved tracking tool ............................................................... 53 5.4. Recommendations for targeting and monitoring EU–co-funded social inclusion investments in the 2014–2020 period...................................................................................... 55 6. References ............................................................................................................... 57 Annex I. Summary of TÁMOP 5 measures and hypothesized impact mechanisms ............ 59 Annex II. Relative variability of selected indicators .......................................................... 67 Annex III. Mapping for results in Hungary........................................................................ 69 5 Abbreviations and acronyms AROP At risk of poverty AROPE At risk of poverty and social exclusion CLLD community-led local development ECA Europe and Central Asia ECD early childhood development ESF European Social Fund ESIF European Structural and Investment Funds EU European Union EU-SILC EU Survey for Income and Living Conditions FRA Fundamental Rights Agency GIS geographic information system HCSO Hungarian Central Statistical Office HDI Human Development Index LEP Local Equal Opportunity Plan M&E monitoring and evaluation NGO nongovernmental organization NRIS National Roma Inclusion Strategy NSIS National Social Inclusion Strategy OECD Organisation for Economic Co-operation and Development OP Operational Program OPHI Oxford Policy and Human Development Initiative PISA Programme for International Student Assessment TÁMOP Társadalmi Megújulás Operatív Program (Social Renewal Operative Program) TEIR Területfejlesztési és Területrendezési Információs Rendszer (Territorial Development and Spatial Planning Information System) 6 UNDP United Nations Development Programme 7 Executive summary Social exclusion is the process by which individuals or entire communities of people are systematically blocked from or denied full access to various rights, opportunities, and resources that are normally available to members of a different group, and which are fundamental to being socially integrated into that particular group—housing, employment, education, health care, civil participation, and due process. It includes (but is not limited to) poverty and material deprivation, and can be reflected in multiple dimensions of everyday life. There have been many attempts to measure the extent to which a community or an individual is socially excluded, both in Hungary and in other countries. For example, the Central Statistical Office of Hungary (HCSO) used the 2001 and 2011 population censuses to produce a map of segregated census blocks using two indicators related to unemployment and education. The Hungarian government defined the 33 most disadvantaged microregions so as to facilitate the targeting of European funds; the government has also introduced a monitoring framework of the National Social Inclusion Strategy (NSIS). In addition, HCSO and the World Bank have jointly identified the 2005 at-risk-of-poverty (AROP) rates at the microregional level. There have also been numerous global attempts that provide lessons on (i) how to choose which dimensions of social exclusion to measure; (ii) the number of indicators to collect; (iii) whether to use composite versus non-unified indicators; and (iv) the level of disaggregation and frequency of measurement. Against this background, this report offers an approach to developing a set of indicators that should be collected to track progress on social inclusion at the subregional level. These indicators should (i) be available at a highly disaggregated geographical level (ideally the microregion or district); (ii) be affected relatively quickly by changes in social inclusion/exclusion; and (iii) be available on a regular basis (ideally annually). The report identifies 22 such indicators that cover the thematic areas of (i) monetary poverty and material resources; (ii) employment/labor; (iii) education and health; and (iv) housing and living conditions. The report also explores whether there is an opportunity to leverage this approach to develop a tracking/monitoring tool that could combine locally available social exclusion data with project information from EU co-funded social inclusion projects. Such a tool could serve as a feedback mechanism for policy makers on whether funds are spent in the highest-need areas. The underlying analysis—which is based on the evaluation of impact mechanisms of selected EU co-funded measures within the 5th Chapter of the Social Renewal Operational Program (TÁMOP 5) between 2007 and 2012—demonstrates that it is possible to develop this feedback mechanism based on currently available data in the case of social inclusion challenges and investments aimed at responding to them. However, the analysis also finds that such tracking opportunities could be substantially improved by regularly collecting project information on (i) the social challenge that the call for proposal or measure intends 8 to address; (ii) the time frame in which the call for proposal is launched and the projects are financed; (iii) number of beneficiaries in the project; and (iv) the exact location of the implementation. The report encourages policy makers to add additional dimensions to routinely collected data when designing the calls for proposals for future EU co-funded social inclusion measures. These dimensions should correspond to the proposed indicator set and should also be linked to EU 2020 targets. At a minimum, these dimensions should include: (i) constrained school careers and/or low education levels; (ii) child poverty; (iii) crime or deviance; (iv) low employment or activity levels; (v) gaps in/lack of (selected) quality social/human service delivery for various target groups (for example, at the local level, home care, social, child protection, youth welfare service, services for people with disabilities); (vi) indebtedness or excessive housing costs; (vii) housing segregation in Roma/poor neighborhoods; and (viii) discrimination of vulnerable groups. 9 1. Introduction The objective of this paper is to develop a way to monitor and track progress on social inclusion of vulnerable groups in Hungary, particularly among marginalized Roma communities. This approach will enable stakeholders to track the status of social inclusion at the subregional level, and can serve as a feedback mechanism on whether projects cosponsored by the European Structural and Investment Funds (ESIF) are sufficiently targeted to disadvantaged areas. The paper builds on various Hungarian attempts to draft indicator sets to find and subsequently gear EU-funded projects toward areas with the poorest social inclusion outcomes. The Central Statistical Office of Hungary (HCSO) used the 2001 and 2011 population censuses to produce a map of segregated census blocks using a combination of two indicators related to unemployment and education. In 2007 the Hungarian government defined the 33 most depressed microregions based on a set of approximately 30 economic, social, and infrastructure indicators, and used a similar approach, with minor changes, for the 2014–2020 period as well. Finally, after determining the best set of poverty rate predictors, World Bank and HCSO experts have jointly developed a poverty map at the microregional level using 2005 data. While these modeling exercises all tried to link poverty, segregation, or regional disparities with a set of predictors so as to identify factors that influence these outcomes, none were aimed at tracking progress toward poverty reduction or social inclusion at a subregional level. The second part of this report takes stock of different exercises undertaken with Hungarian data to map, target, track, and monitor some aspects of social exclusion at different levels of disaggregation. We present four such attempts to: (i) map marginalized communities; (ii) target the most disadvantaged microregions; (iii) track selected social inclusion goals; and (iv) model at risk of poverty (AROP) rates at the microregional level. The report then examines what has been done in international practice in terms of selecting and collecting indicators that measure social inclusion. This part relies heavily on Labonté et al. (2011), who conducted a meta-analysis of such attempts in the United Kingdom, France, Australia, and across Europe. These examples from around the world focused primarily on collecting indicators of social exclusion at the national level, and not at a highly disaggregated subnational level (which is the objective of this exercise). In addition, international attempts to measure progress toward social inclusion did not attempt to track progress on a regular basis, be it annually or even every few years. Finally, the report describes the method and process of indicator selection for Hungary. We also identify gaps and articulate remaining research needs. In particular, this report aims to design the best possible set of indicators to be collected on a regular basis (ideally annually) at the lowest geographical level to monitor progress toward social inclusion. The optimal set of indicators should (i) accurately identify the different dimensions of social exclusion; (ii) be available at a geographically disaggregated level; (iii) 10 be collected regularly, preferably on an annual basis; and (iv) be “dynamic”—that is, respond to local development dynamics as demonstrated by relative variability. The concluding section summarizes the dilemmas associated with dynamically measuring social change in the Hungarian context, and proposes development project parameters that should be continuously followed in order to enable tracking and (limited) monitoring. In addition to providing inputs and making suggestions for future discussions on how to match local level information on social development and development projects co-funded by the European Union (EU) in the 2014–2020 period, the report also outlines a logical approach and proposes a tool to enable the parallel tracking of development activities in conjunction with social inclusion trends at the local level through a geographic information system (GIS). Such a tool can provide continuous feedback for policy makers on whether investments are geared toward the highest need areas. The approach underlying the proposed tool draws on the recently completed evaluation of social inclusion investments conducted under the TÁMOP 5. 11 2. Social exclusion: Frozen in a state of lagging behind Social exclusion is the process by which individuals or entire communities are systematically blocked from or denied full access to various rights, opportunities, and resources that are normally available to members of a different group, and which are fundamental to being socially integrated into that particular group—housing, employment, education, health care, civil participation, and due process (Levitas et al., 2007). The term “social exclusion,” which describes social disadvantage and being relegated to the fringes of society, first arose in France in the 1970s in a context of radical economic restructuring and subsequent concerns over risks to social cohesion and stability (Silver, 1994). Social inclusion, on the other hand, consists of improving the ways individuals and groups can take part in society, and especially pertains to improving the ability, opportunity, and dignity of people who are disadvantaged on the basis of their identity (for example, ethnic affiliation). Social inclusion and exclusion require an analytical framework that uncovers the underlying causes of poverty and deprivation. It exposes the multidimensional nature of deprivation and scrutinizes the correlates of poverty, be they lack of schooling, constrained labor market participation, poor health, or residential segregation. It underscores that deprivation arising from social exclusion tends to occur along multiple axes at once, so that policies that improve just one of these axes of deprivation—such as improved access to education—will not release the grip of others (World Bank, 2013). 2.1. Monetary poverty and social exclusion It is important to note that while social inclusion may well be about reducing poverty, it is often about much more; in some cases, it is not even about poverty at all. For example, the Middle East protest movements that have been grouped together as the Arab Spring have been fueled in part by middle-class citizens’ demands for greater inclusion in public decision making and that political leaders be held more accountable (World Bank, 2013). Nevertheless, given the fact that extreme and chronic poverty often overlap with social exclusion, monetary poverty has long been used as a proxy for social exclusion, and there has been much work done to standardize and improve poverty measures, mainly driven by work conducted by the World Bank (see Deaton, 1997; Ravallion and Bidani, 1994; and Ravallion, 1992). Poverty in most cases is considered in absolute terms: the share of people living below a set threshold that would ensure households have access to basic minimums—one- and two-dollar-a-day poverty lines, for instance. In other cases, including for Hungary and all other EU member states, poverty is considered in “relative terms”—that is, the share of people living with less than a percentage of the mean or medium income. According to the EU 2020 1 indicators on poverty and social 1 Europe 2020 is the EU’s growth strategy. It is based on three priorities: employment, productivity, and social cohesion. The EU has set five objectives related to employment, innovation, education, social inclusion, and 12 exclusion, EU member states are required to measure a headline indicator of people at risk of poverty and social exclusion (AROPE) (EUROSTAT, 2015). This composite indicator shows the number of people affected by at least one of three forms of poverty: monetary poverty (inadequate income); material deprivation; or low work intensity (limited participation in the labor market). People are AROPE if they suffer from at least one of these. Monetary poverty is defined as earning less than 60 percent of the median income in the country of residence. The material deprivation rate is an indicator that expresses the inability to afford items considered by most people to be desirable or even necessary for an adequate life. Finally, low work intensity is defined as the number of persons living in a household with a work intensity below a threshold set at 20 percent. 2.2. Capturing the dimensions of social exclusion More recently, additional measures of social inclusion have been developed to include dimensions beyond economic status. Amartya Sen has developed the Human Development Index (HDI), which gives equal weight to three non-monetary indicators (life expectancy, literacy, and infant mortality). The HDI was later modified to include additional aspects of poverty, in particular standards of living, which led to the Multidimensional Poverty Index (MPI) that was developed by the Oxford Policy and Human Development Initiative (OPHI) and the United Nations Development Programme (UNDP). Like the HDI, the MPI divides poverty along three dimensions—health, education, and living standards—but uses 10 indicators instead of 3. Health is measured by child mortality and nutrition; education is measured by years of schooling and school attendance; and living standards are measured through access to cooking fuel, toilets, water, electricity, floors, and asset ownership. The MPI also gives equal weight to the three main dimensions (health, education, and living standards), which means that each of the two indicators for health have a weight of 1/6; each of the two indicators for education have a weight of 1/6; and each of the six indicators for living standards have a weight of 1/18. BOX 1. Using a multidimensional approach to compare social inclusion outcomes between Roma and non-Roma in Central Europe A recent report published by UNDP (Ivanov and Kagin, 2014) analyzes the difference in outcomes between the Regional Roma Survey (RRS) and the UNDP vulnerable groups survey of 2004. Instead of using a single indicator of poverty (such as income or consumption), the report uses a multidimensional approach to poverty (the MPI). Among Roma, the MPI decreased substantially between 2004 and 2011 in Bulgaria and Romania, primarily due to a decline in the number of “poor” households (that is, those with 5 to 7 deprivations in the MPI). “Severe poverty” (more than 7 deprivations) decreased significantly only in Romania. Looking at the different dimensions of poverty allowed Ivanov and Kagin to identify the key drivers of poverty reduction between 2004 and 2011; education and living conditions have improved, while climate/energy to be reached by 2020. Each member state has adopted its own national targets in each of these areas. The strategy is underpinned by concrete actions at EU and national levels. 13 fundamental rights and access to the labor market have worsened. The analysis of multidimensional versus monetary poverty metrics also reveals that the decline is similar in both metrics in Bulgaria, while the decline in monetary poverty in Romania is stronger than in multidimensional poverty. Source: Ivanov and Kagin (2014). 2.3. What are these dimensions? The concept of poverty—whether absolute, relative, or multidimensional—does not fully capture the different facets of social exclusion. One can be relatively wealthy (and not captured by a poverty index) and still experience social exclusion; for example, as a result of discrimination. A measure to more fully capture all dimensions of social exclusion involved including all potential dimensions in which social exclusion may appear, be it material resources, income, housing, education, access to employment, or opportunities to participate in the local economy. To this end, Atkinson and Marlier (2010) and Labonté et al. (2011) have reviewed the existing theoretical and empirical literature on social exclusion. Labonté et al. (2011) have defined eight core dimensions along which social exclusion is likely to happen: 1. Income and material resources 2. Employment 3. Education and skills 4. Affordable and adequate housing 5. Health 6. Social resources and networks 7. Community resources and civic participation 8. Personal safety2 Material resources are the bedrock of social exclusion (when they are lacking) or inclusion (when they are adequate for both basic needs and normative social activities). The employment domain is particularly relevant, since its lack has adverse effects on material resources as well as on social participation (social networks, support). Education and skills provide capacity to participate socially and to engage in more secure forms of employment. Housing adequacy, affordability, and security together emerge as an important subset of material resources that extend beyond the basic need for shelter. The health domain is also relevant, since adverse health conditions can affect productivity levels, while having a disability can lead to social exclusion. Regarding the social resources 2 While earlier versions of this list included discrimination, it was dropped in the most recent version. 14 domain, institutionalized persons or those separated from their families generally have reduced social resources and are at greater risk of exclusion. On the other hand, having social resources (social networks, or the density of one’s social life; social support, or the quality of what those networks offer; and opportunities for different forms of social participation) are considered processes of inclusion. Community resources are identified as a separate domain and encompass access to community services and opportunities for political and civic participation. Finally, the personal safety domain refers to exposure to crime or discrimination that may affect or prevent individuals from participating in economic, social, civic, or political activities. The conceptual framework proposed by Labonté et al. (2011) overlaps with the framework developed by the World Bank (2013) to identify paths toward social inclusion. Individuals and groups want to be included in three interrelated domains: markets, services, and spaces (see Figure 1). The three domains represent both barriers to and opportunities for inclusion. Just as different dimensions of an individual’s life intersect, so do the three domains. Intervening in one domain without considering the others is likely to hamper the success of inclusion policies and programs. Figure 1. Working toward social inclusion—a framework Source: World Bank (2013). These two frameworks define the core set of expectations and priorities that should be included in an indicator set aiming to measure the various dimensions of social exclusion. Therefore, the final set of indicators that will be presented in Section 3 relies on these two analytical frameworks and divides indicators that measure progress toward social inclusion 15 between various dimensions: monetary poverty and material resources, access to the labor market, education and health, housing and living conditions, and community setting.3 2.4. Can the “depth” of social exclusion be measured? In addition to social exclusion’s “lateral” aspects discussed in the previous section, intensity or “depth” of social exclusion is also important. Miliband (2006) recommends considering social exclusion in three “dimensions”—width, depth, and concentration. Wide exclusion refers to the large number of people excluded on a single or small number of indicator(s). Concentrated exclusion refers to the geographic concentration of problems and to area exclusion. Deep exclusion refers to those excluded on multiple and overlapping dimensions. For example, the Regional Roma Survey (UNDP, World Bank, and EC, 2011) finds that the labor market outcomes among Hungarian Roma women are significantly worse than those among Roma men: only 13 percent of Roma women have a job, compared to 34 percent of Roma males and 51 percent of non-Roma females living nearby. This highlights a significant gender gap in terms of labor market access by Roma jobseekers. 4 Any choice of indicator set should be able to address these three aspects of social exclusion; that is, the indicators should capture a broad swath of dimensions (social processes) that characterize social exclusion; be capable of disaggregation to geographic areas (ideally local areas or even neighborhoods); and be linkable to specific individuals or groups. 3 Due to data limitations, the final set of indicators does not include social, political, and cultural resources, or civic and political participation. 4 The accompanying handbook, The People Behind the Numbers: Developing a Framework for the Effective Implementation of Local Equal Opportunity Plans, elaborates how complex social problems emanating from overlapping disadvantages can be broken down into individual elements, and which of these elements are best addressed at the local level. 16 3. Measuring social exclusion: Global and local lessons for Hungary Any method of measuring social inclusion needs to take into consideration the experience of earlier attempts to do so, both in Hungary and elsewhere in the world. These experiences include attempts to target (identifying a country’s lagging areas that should be emphasized when developing interventions) and monitor (determining what data should be regularly collected to identify whether the gap is closing between the most disadvantaged and the general population) social inclusion measures. 3.1. What has been done in Hungary so far? Over the past few years, various entities in Hungary have developed various sets of indicators to target those areas with the worst social inclusion outcomes. The HCSO used the 2001 and more recently the 2011 population census to produce a map of segregated census blocks using two indicators related to unemployment and education. In parallel, the Hungarian government defined the 33 most depressed microregions—based on economic, social, and infrastructure indicators—and used a similar approach, with minor changes, for the 2014–2020 period as well. 3.1.1. The HCSO used census-based indicators on education and unemployment to map marginalized communities Identifying marginalized areas based on data and evidence is the precondition for effectively targeting marginalized groups, including Roma. Similar to the World Bank–led exercise in Romania (see Box 2), socioeconomic indicators reflecting deprivation have also been shown to effectively grasp the most excluded social strata in Hungary. The HCSO was tasked with producing a map of segregated neighborhoods using census-block information from the 2001 population census, and then the 2011 census. Segregated neighborhoods were defined as parts of settlements where segregation is significantly higher than in other neighborhoods, and people face poor housing conditions and have limited access to basic services (water, sewage, social and healthcare services, and so on). The HCSO focused on two indicators that were closely linked to segregation: (i) a lack of work-able adults with regular income from employment; and (ii) work-able individuals with their highest school qualification below the 8th grade. For the first full EU financial period (2007–2013), 5 data from the 2001 population census were used, along with the following methodology: Urban areas were considered segregated if at least 50 percent of active-age residents (between ages 15 and 59) were unemployed and their highest school qualification did not exceed the 8th grade. A second tier consisted of urban areas threatened by segregation, in which the above-mentioned indicator took up a 5 Abridged Version of the Urban Development Manual (State Secretariat for Regional Development and Construction Ministry for National Development and Economy, March 2009). 17 value between 40 and 50 percent. In the case of the Budapest conurbation, urban areas were considered segregated if the value of the above-mentioned indicator attained or exceeded 35 percent; areas were considered to be threatened by segregation if the value of the indicator fell within 25–35 percent. In addition, data on social assistance uptake was used to more precisely define the scale of segregation. Areas where the proportion of regular social aid beneficiaries attained twice that of the city average (or the district average in Budapest) were considered to be segregated. Areas where the proportion of regular social aid beneficiaries attained 1.7 to 2 times the city’s (district’s) average were considered threatened by segregation (EC, 2011). For the current EU planning period (2014–2020), 6 data from the 2011 population census were used, along with the same methodology, only the thresholds were modified. Urban areas were considered segregated when at least 35 percent of the active-age residents (15– 59 years old) were unemployed and their highest school qualification did not exceed the 8th grade. Urban areas where the aforementioned indicator took up a value between 30 and 35 percent were considered threatened by segregation. In the case of the Budapest conurbation, urban areas were considered segregated if the indicator value attained or exceeded 20 or 25 percent for inner and outer districts, respectively. Areas were considered threatened by segregation if the indicator value fell within 15–20 or 20–25 percent for inner and outer districts, respectively. Settlements with 200 to 2,000 inhabitants kept the same threshold as before: 50 percent to be classified as a segregated neighborhood, and 40 to 50 percent to be considered as threatened by segregation. Overview of indicators used - One indicator related to low intensity of labor and one related to low level of education - Household-level data from the 2011 population census - Indicators available at the census-block level, but only every decade (10 years) 3.1.2. Targeting tool for complex social inclusion interventions designed by HCSO Increasing territorial disparities at the microregional level (LAU1, formerly NUTS4) was recognized as a major problem in the mid-2000s, partly as a result of social and ethnic tensions in some of the most depressed microregions, especially in northeast Hungary. The implementation of EU funds in the 2007–2013 financing period brought forth the need to target lagging areas where social inclusion programs should be focused. In these areas, “complex programs” were foreseen to be developed in groups of cca. 10–30 projects per microregion, covering human services and infrastructure development, economic development, transport and environmental infrastructure development, and urban development. For example, in the area of human development, a designated pool of funds 6 See http://net.jogtar.hu/jr/gen/hjegy_doc.cgi?docid=A1200314.KOR. 18 and thematic calls from the Human Resource Development Operational Program (OP) targeted these microregions in the 2007–2013 period. As a result, the HCSO was tasked in 2007 with defining a new set of indicators that would help target the most disadvantaged microregions. The HCSO had the opportunity to build on already elaborated-upon sets of indicators to identify lagging regions since the end of the 1980s. It initially used discriminant analysis to determine the best set of indicators for identifying depressed areas (1986–1993); subsequently switched to scoring models (1993– 2007); and finally moved to feature scaling (normalization) in 2014. While the set of indicators was limited to about 10 until 1996, more recent years have seen an expansion in the number and dimensions of indicators, reaching 20–30 indicators as of 2007 (Kezán and Szilágyi, 2015). During the 2007 exercise, a set of 31 economic, social, and infrastructure indicators were defined. Some were drawn from the previous census (6 years earlier, in 2001); others came from more recent data sources. The composite indicator was aggregated at the microregional level, for all 175 microregions, which were in turn ranked according to their final score and categorized into four groups of intervention (complex, medium, simple, and none). Indicators were chosen according to the following criteria: (i) indicators needed to be transparent, simple, reliable, and reproducible over time; and (ii) measurements needed to be consistent, so that the new methodology was comparable to earlier ones. There was no particular need to find indicators that would allow for tracking progress over time, as the goal of the composite indicator was to identify ex ante the most disadvantaged regions. In 2013, the 175 microregions were replaced by 198 districts (including the 23 districts of Budapest), and the HCSO undertook once more the task of identifying Hungary’s most depressed areas. The final set of indicators largely overlapped with those selected in 2007 (see Table 1) and identified 24 variables related to economic, social, and infrastructure dimensions. 7 Table 1. List of indicators targeting the most disadvantaged microregions/districts—2007 and 2014 ECONOMIC INDICATORS 2007 2014 Number of active business organizations, per 1,000 inhabitants X X Number of guest nights in tourism, per 1,000 inhabitants X 7 The legislative framework for these indicators is comprised of the following regulatory acts: (i) 290/2014 (XI. 26.) Government decree on the classification of the beneficiary districts; (ii) 105/2015 (IV. 23.) Government decree on the classification of the beneficiary communities and the classification system conditions; (iii) 1247/2015 (IV. 23.) Government resolution on measuring changes in regional differences in development of dynamic scorecard. An important aspect of the latter is the fact that it imposes the obligation to develop an indicator system aimed at tracking social changes at different territorial levels on an annual basis. 19 Number of retail shops, per 1,000 inhabitants X X Share of employment in agriculture, per total employment X Share of employment in services, per total employment X Growth rate of number of active business organizations X Amount of local tax revenues of municipalities, per inhabitant X Share of local tax revenues of municipalities, per total revenues X Number of researchers and developers, per 1,000 inhabitants X INFRASTRUCTURAL INDICATORS Share of flats supplied with the potable water pipes network X Share of flats supplied with the gas pipes network X Share of flats supplied with regular waste collection X X Length of sewer pipes, per potable water pipes X Share of flats supplied with the sewer pipes network X Road accessibility of closest county seat and microregional seat (minute) X Road accessibility of closest motorway (minute) X Share of paved roads, among all municipal roads X Number of phone subscribers, per 1,000 inhabitants X Number of cable TV subscribers, per 1,000 inhabitants X Number of broadband internet subscribers, per 1,000 inhabitants X X SOCIAL INDICATORS (1/2) Share of flats with at least 3 rooms and built in the past 5 years X X Share of non-comfort/substandard flats X Average price of used flats X Number of cars, per 1,000 inhabitants X Number of migrants, per 1,000 inhabitants X X Number of deaths, per 1,000 inhabitants X X Amount of personal income tax basis, per inhabitant X X X X Share of population in municipalities with density above 120 persons per sq 20 km Number of nursery and day care places, per 1,000 inhabitants below 2 years X Average life expectancy at birth, for men X Average life expectancy at birth, for women X SOCIAL INDICATORS (2/2) Rate of number of persons below 15 years, per number of persons above 60 X years Share of households without employment X Share of persons with secondary school graduation, among persons above 18 X X years Number of persons receiving regular social aid from the municipality, per X 1,000 inhabitants Share of persons receiving regular social aid from the municipality or X unemployment Share of persons receiving regular child protection aid, among persons below X X 24 years EMPLOYMENT INDICATORS Share of registered jobseekers, among working-age population X X Share of long-term registered jobseekers, among working-age population X X Share of low-educated (not more than primary school) registered jobseekers X Activity rate X Source: HCSO (2007; 2014). The final list of areas to be developed with complex programs changed significantly from 2007–2011 to 2014. Of the 33 microregions to be developed with complex programs in 2007, 9 were not identified as districts to be developed with a complex program. At the regional level, the number of areas to be developed with a complex program decreased most in South Transdanubia (from 8 to 4), and increased most in Northern Great Plain (from 8 to 12). At the county level, the number of areas to be developed with complex programs decreased most in Somogy (from 4 to 2), and increased most in Hajdú-Bihar (from 1 to 4). In addition, HCSO produced a map overlapping most disadvantaged municipalities and high unemployment rates (see Figure 2), and the latter proves to be extremely correlated with the former. Preferred districts and municipalities are defined according to very similar sets of indicators as those defined for the disadvantaged microregions, with minor differences (for example, life expectancy is used at the district level but not the municipality level, as data 21 would not be reliable). Therefore, the reason for having both is to be able to target disadvantaged municipalities in districts that would not be classified as disadvantaged. Figure 2. Most disadvantaged municipalities and unemployment rates Legend other high unemployment rate underdeveloped Source: HCSO (2015); Kezán and Szilágyi (2015). Overview of indicators used - Set of 24–31 indicators related to four dimensions of economic and social depression: economic, social, infrastructure, and education - Different sources of data, including population census (2001 and 2011), local administrative data, registers (for example, for tax revenues) - Indicators available at the microregional (or district) level, with varying regularity (census every 10 years, tax revenues every year, and so on) BOX 2. Identifying pockets of poverty at the subregional level: Mapping Romania’s marginalized communities In Romania, official municipal maps used by local government decision makers often do not show all of the marginalized (Roma) settlements, as these are typically of an informal nature. As a result, these communities’ needs are frequently overlooked in local development plans, including those funded with EU structural funds. However, a new approach for spending EU funds—community-led local development (CLLD)—allows EU-funded activities to be explicitly targeted to pockets of 22 deprived communities. To help the Romanian authorities design their CLLD program, the World Bank developed a methodology that highlighted the location of severely marginalized communities for each town and city in Romania—the Atlas of Urban Marginalized Areas in Romania, which presents maps that are based on data from the 2011 population and housing census. This tool uses a typology and corresponding indicators that are based on qualitative research and a review of earlier analysis and indices of urban marginalization. The maps use indicators at individual, household, and dwelling levels (such as education, employment, access to electricity, and so on) from the 2011 census. For each of these indicators, the values at the urban census sector level (areas of typically about 200 people) are determined for all urban census sectors and an urban threshold is then defined as the 80th percentile. For each urban census sector, it is subsequently determined whether its value is above the threshold for that indicator. If a census sector has a particular combination of indicators that are above their threshold, it is regarded as disadvantaged or marginalized. For a number of cities, maps at the census sector level were produced, displaying the typology of urban marginalized areas as determined by applying this methodology to the census data. An additional series of maps were produced that reflect information collected directly from urban authorities in Romania on whether marginalized communities existed in their municipality, and if so, where. For a subset of cities, maps were available from both the census-based method and the information gathered directly from urban authorities. Using census data to identify urban marginalized communities is a promising approach. However, further work is needed to assess its validity, including beyond urban areas. While maps of marginalized communities in urban areas can provide more finely tuned information about subnational or within-city variations in poverty and marginalization, and improve resource allocation, they cannot solve all development problems. To make the most of the information, it must be complemented with local, context-specific knowledge that draws on local expertise and community demand. In other words, after identifying the areas or populations in greatest need, it is necessary to understand why these places are poor. The reasons are likely to vary from place to place, and may include inadequate infrastructure, lack of economic activity, an insufficiently skilled workforce, or other factors. While the right combination of approaches will vary by country, the maps provide important information to help improve policies and programs to combat poverty and social exclusion. Source: World Bank (2015). 3.1.3. Monitoring indicators from NSIS The Government of Hungary is the first EU member state to develop and submit a national Roma inclusion strategy, which is titled National Social Inclusion Strategy: Extreme Poverty, Child Poverty, Roma. Subsequently, the government developed a monitoring system intended to track progress of Hungary’s National Social Inclusion Strategy (NSIS) in each of the areas identified as priority development dimensions: (i) poverty and social exclusion, with particular emphasis on the Roma population; (ii) reproduction of social exclusion; and (iii) equal access to economic opportunities. Indicators are further classified into second- and third-tier indicators, with each tier offering more in-depth understanding of each dimension of social exclusion. 23 The core social inclusion indicators—which largely overlap with the EU 2020 social indicators—are as follows: 1. Share of households living in poverty and social exclusion, or AROPE (EU 2020) 2. Share of households living in financial deprivation 3. Share of households with low work intensity 4. Employment rate 5. Share of children living in deep poverty 6. Share of 3–5 year olds enrolled in kindergarten 7. Share of children in the 6th grade that have parents with a maximum of 8 years of completed primary school who have obtained a score of 1 or below on standardized performance tests, compared to all 6th graders 8. Number of school dropouts by 10th grade, compared to the full cohort Beyond social inclusion indicators, a program monitoring system has also been applied and is being run by the state administration. Overview of indicators used: - Set of 8 core indicators related to social exclusion (poor economic conditions, employment, education) - Different sources of data, but many missing, and the monitoring system is under revision; data collection by the HCSO is ongoing - Few indicators available BOX 3. Europe-wide efforts to monitor national Roma inclusion strategies In addition to the progress the Government of Hungary has made in monitoring national social inclusion outcomes on the basis of the NSIS, considerable efforts are underway to improve results- based monitoring and evaluation (M&E) on Roma inclusion in other EU member states as well. Led by the EU Fundamental Rights Agency (FRA), a working group of member states is developing a model aimed at adopting a set of common rights-based indicators that can comprehensively assess Roma inclusion efforts at the EU level. It applies a so-called structure-process-outcome (S-P-O) indicator model that assesses (i) the legal and policy framework (structural indicators); (ii) the concrete measures to implement it (process indicators); and (iii) the achievements as observed for the target group(s), for example, Roma (outcome indicators). A recent transnational workshop on M&E organized by the European Social Fund (ESF)—Roma Inclusion Network in Madrid (November 13–14, 2014)—has also provided an opportunity for 24 member state participants to share past Roma inclusion investments, M&E experiences, and plans for the next programming period. The workshop has highlighted the fact that all countries are aware of the past programming period’s weak M&E systems, and that some progress is being made with regards to M&E in the 2014–2020 programming period, particularly in the areas of (i) better targeting of resources (such as using information on disadvantaged localities—the Czech Republic, Slovakia, Hungary); and (ii) monitoring whether projects are reaching Roma (in Hungary, Bulgaria) using self-identification. Participants discussed concrete ways to improve results targeting and monitoring using modern IT technology, and by making M&E systems much more inclusive toward the implementing organizations and final beneficiaries. Source: World Bank (2015). 3.1.4. Best predictors of AROP Finally, in an effort to provide geographically disaggregated estimates of poverty, the World Bank and the HCSO constructed a map of AROP estimates—one of the key indicators of poverty and social exclusion in the EU—at the microregional (“kistérség”, local administrative unit or LAU1) level in 2014 (World Bank, 2014). While this exercise only looked at one dimension of social exclusion—namely, monetary poverty based on income— it identified the best predictors of poverty from a subset of variables included both in the 2005 microcensus and the simultaneously collected 2005 EU Survey for Income and Living Conditions (EU-SILC). 8 Table 2. Best predictors of AROP at the microregion level—2005 Variables predicting log per capita disposable income Coefficient Employment No active members in the household -0.56 Two active members in the household -0.10 One or two employees, if household head is employer 0.08 More than two employees, if household head is employer 0.24 Number of employees in the household -0.28 8 Formally launched in 2004 in 15 countries and expanded in 2005 (to all 25 member states, Norway, Iceland), 2006 (to Bulgaria), and 2007 (to Romania, Switzerland, Turkey), the EU-SILC provides two types of annual data: (i) cross-sectional data pertaining to a given time or time period with variables on income, poverty, social exclusion, and other living conditions; (ii) longitudinal data pertaining to individual-level changes over time, observed periodically over a four-year period. EU-SILC is a multipurpose instrument that focuses mainly on income. Detailed data are collected on income components, mostly on personal income. Information on social exclusion, housing conditions, labor, education, and health is also collected. 25 Household head working in chemical manufacturing 0.23 Household head working in retail, trade, or repair of personal and household goods -0.10 Household head working in post or telecommunications 0.23 Household head without a contract at work -0.21 Household head with a contract at work for several months -0.16 Household head occupation (education & administration) 0.06 Education Maximum education (continuous) 0.02 Proportion of people with no education/adults in household -0.29 Proportion of people with primary education/adults in household -0.28 Proportion of people with secondary education, no final exam/adults in household -0.35 Proportion of people with secondary education/adults in household -0.23 Household composition Household head is woman -0.12 Household head divorced 0.07 Proportion of children between 0–5 years old -0.18 Number of individuals between 13–18 years old 0.11 Proportion of elderly members above 60 0.32 Household size of one or two members 0.06 Housing conditions No hot water in the dwelling -0.12 26 Paneled walls9 0.05 Income/monetary indicator Proportion of total amount of personal income tax, inhabitants 18+ 0.09 Geography County=9 -0.09 County=15 -0.07 County=18 -0.10 Region_1*Proportion of unemployed adults in the household/total no. of adults -0.23 Region_2*Proportion of unemployed adults in the household/total no. of adults -0.22 Region_3*Maximum education -0.01 Region_3*Household size 0.04 Region_4*Proportion of unemployed adults in the household/total no. of adults -0.35 Region_5*Proportion of unemployed adults in the household/total no. of adults -0.33 Region_7*Maximum education -0.01 Source: World Bank and HCSO staff calculations. The poverty mapping exercise relied on the model based on methodology from Elbers et al. (2003), using household-level data from the 2005 microcensus—a representative survey covering 2 percent of the Hungarian dwelling stock, conducted by the HCSO in April 2005— and the 2005 EU-SILC. The microcensus data covered a number of key household and individual characteristics, including (i) demography (age/sex profiles, marital status, household composition); (ii) employment (employment status, occupation, and industry); (iii) educational attainment; and (iv) information on dwellings (type of ownership, amenities, number and surface of rooms, type of sewage, type of walls). The poverty mapping model relied on data regarding total disposable household income (after transfers) per equivalent adult, which was available for 6,927 households. The disposable household income per equivalent adult formed the dependent variable of the models. This variable was chosen because it is also used to calculate the official measurement of poverty reported to the European Commission (AROP indicator). The 9 As used in prefabricated housing modules. 27 microcensus was only representative at the county level: for the purposes of constructing a unit-level model, the exercise relied on EU-SILC data on a number of household and personal characteristics (household composition, age, gender, level of education, employment status, and type of employment), as well as dwelling characteristics. Variables were selected for the modeling stage using comparable variables between the microcensus and EU-SILC, as well as auxiliary data at the settlement level (such as average schooling at the settlement and district level, average household size, average age of household head, among others). Several individual variables collected in the microcensus were used for creating means of geographical partitions; for instance, average schooling at the settlement and district level, average household size, and average household age, among other variables. The final list of variables is presented in Table 2 and the final map is shown in Figure 3. Figure 3. AROP rates at the microregion level, 2005 Source: World Bank (2014). Overview of indicators used - Set of 35 indicators related to 6 dimensions of poverty—employment, education, family composition, housing conditions, income, and geography - Household-level data from the 2005 microcensus and 2005 EU-SILC - Indicator available at the census-block level, but only every decade (10 years); or at the regional level every year from EU-SILC 3.2. International experience with measuring social inclusion outcomes at the national level There is a broad and diverging range of views on how to select indicators to examine social exclusion. Most methodologies include various dimensions under the terminology social 28 exclusion, and select different indicators for each field. The following sections are mostly based on a review of Labonté et al. (2011), and Levitas et al. (2007). Both reports present a meta-analysis of the different frameworks designed to analyze social inclusion/exclusion in Australia, Canada, France, the United Kingdom, and the EU. 3.2.1. Choice of dimensions A review of existing social exclusion frameworks, indicators, and measures led to the identification of eight main principle domains that capture processes of social exclusion/inclusion, as conceptualized by Labonté et al. (2011), and 10 topic areas in Levitas et al. (2007), most of which overlap: Table 3. Dimensions of social exclusion Labonté et al. (2011) Levitas et al. (2007)10 Income and material resources Material/economic resources Employment Economic participation Education and skills Culture, education, and skills Affordable and adequate housing Living environment Access to public and private services Health Health and well-being Social resources and networks Social participation Social resources Community resources and civic participation Political and civic participation Personal safety Crime, harm, and criminalization Source: Labonté et al. (2011); Levitas et al. (2007). The review by Labonté et al. (2011) of international evidence on choosing indicators to measure social exclusion shows that, in most cases, the set of indicators chosen falls within the aforementioned categories. Furthermore, in most cases, due to data availability and reliability of indicators, it is restricted to the following five categories: (i) monetary poverty; (ii) access to the labor market; (iii) education and health; (iv) housing and living conditions; and (v) macroeconomic conditions of place of residence. 3.2.2. Number of indicators Most countries finalize a set of 20 to 30 indicators organized in layers, or tiers, of indicators. The first tier is usually a restricted number of lead indicators, which cover the broad fields 10 This matrix is called the Bristol Social Exclusion Matrix, or B-SEM. 29 that have been considered the most important elements that lead to social exclusion. The second—and sometimes third—tiers of indicators support Tier I indicators and describe other dimensions of the problem in more depth. Atkinson, Marlier, and Nolan (2004) designed an indicator set for the EU that incorporates EU-specific social exclusion dimensions, and keeps in mind availability of data from EU-SILC and other datasets. The final set of indicators is presented in Table 4. It consists of two tiers of indicators, which were chosen for the EU as a whole. An additional third tier is left blank to be country specific, and depends on data availability as well as the specific focus researchers want to take regarding social inclusion/exclusion. Table 4. Atkinson et al. indicators for the EU Level 1/Tier I 1 The risk of financial poverty as measured by 50% and 60% of national median income 2 Income inequality as measured by the quintile share ratio; that is, the ratio of the share of national income received by the top 20% of households relative to the bottom 20% of households 3 The proportion of those aged 18–24 with only lower secondary education (and not in education or training) 4 Overall and long-term unemployment rates measured on an International Labor Organization basis 5 Proportion of population living in jobless households 6 Proportion of population dying before the age of 65, or the ratio of those in bottom and top quintile groups who classify their health as bad or very bad according to the World Health Organization definition 7 Proportion of people living in households lacking specified amenities or with specified housing faults Level 2/Tier II 8 Proportion of people in households below 40% and below 70% of median income, and proportion below 60% of the median fixed in real terms 9 Value of 60% of median threshold in purchasing power for two- and four-person households 10 Proportion of the population living in households permanently at risk of financial poverty 11 Mean and median equivalized poverty gap for a poverty line set at 60% median income (this measures depth of poverty by calculating the extent to which those in poverty fall 30 below the poverty line) 12 Income inequality as measured by the decile ration and the Gini coefficient 13 Proportion of the population aged 18–59 with only lower secondary education or less 14 Proportion of discouraged workers, proportion non-employed and proportion in involuntary part-time work, as a percentage of the total 18–64 population, excluding those in full-time education 15 Proportion of people living in jobless households with current income below 60% median 16 Proportion of employees living in households at risk of poverty (60% median) 17 Proportion of people who are low paid 18 Proportion of people unable to obtain medical treatment for financial reasons or because of waiting lists 19 Proportion of the population living in overcrowded housing 20 Proportion of people who have been in arrears on rent or mortgage payments 21 Proportion of people living in households unable to raise a specified sum in an emergency Indicators to be developed 22 Non-monetary indicators of deprivation 23 Differential access to education 24 Housing of poor environmental quality 25 Housing cost 26 Homelessness and precarious housing 27 Literacy and numeracy 28 Access to public and essential private services 29 Social participation and access to Internet Source: Atkinson and Marlier (2010). The indicator set developed by Atkinson et al. for the EU is not to be confused with the EU 2020 headline indicators, discussed in Section 1.2. The EU 2020 indicators express the five key objectives of the EU 2020 strategy (increasing the employment rate; increasing investments in R&D; attaining certain climate change and energy targets; reducing school dropout rates and increasing the share of people completing tertiary education; and lifting Europeans out of poverty and social exclusion) that have been determined at the EU level 31 and translated into national targets (and incorporated in national reform programs). Table 5 summarizes the headline indicators. Table 5. EU 2020 headline indicators Topic Headline indicator Employment Employment rate—age group 20–64 - females - males R&D Gross domestic expenditure on R&D Climate change and energy Greenhouse gas emission Share of renewable energy in gross final energy Primary energy consumption Final energy consumption Education Early leavers from education and training - females - males Tertiary educational attainment - females - males Poverty and social exclusion People at risk of poverty or social exclusion (share) People at risk of poverty or social exclusion (total) People living in household with very low work intensity People at risk of poverty after social transfers Severely materially deprived people Source: EUROSTAT (http://ec.europa.eu/eurostat/web/europe-2020-indicators). Most importantly, and closely related to the findings of HCSO presented in Figure 2, access to the labor market is shown to be a very good predictor of social exclusion. Furthermore, 32 most indicators are shown to be rather time invariant. This has been demonstrated by a longitudinal analysis of panel data in the United Kingdom. Burchardt et al. (1999) use the British Household Panel Survey, which has information on the same households spanning from 1991 to 1995, to investigate social exclusion. Using five dimensions of social exclusion—low income, low wealth, low production activity, political disengagement, and social isolation—Burchardt et al. show that there was little change in the value of the indicators in the five-year period studied. 11 3.2.3. Composite versus non-unified indicators Many researchers pool the indicators into a single social exclusion index (the BIP-40 indicator for France, for instance), 12 while others choose not to unify the various facets of exclusion and keep all indicators separately (Burchardt et al., 1999 for the United Kingdom, for instance). The advantage of composite indicators is that they allow for a single reading of the results: each unit of observation—county, year, and so on—is associated with just one number that is tracked across geographical units or over time, while single indicators need to be presented, compared, and tracked one by one. It is important to note that composite indicators come with two major caveats: (i) interpretation; and (ii) choice of aggregation methodology. When using composite indicators it is, in most cases, quite difficult to interpret the value associated with the composite indicator. For example, while a poverty rate of 60 percent means that 6 in 10 households live below the poverty line, there is not clear and simple explanation for an HDI of 60 percent. A composite indicator allows for ranking (60 percent is less than 70 and more than 50 percent) but not for interpretation. In addition, it may be difficult to justify the choice of a particular methodology to aggregate indicators into a single composite indicator, as there are no scientific guidelines regarding how to aggregate indicators into a single unified measure (thus different aggregation measures using the same set of indicators may yield different composite values, and different rankings across geographic areas, or over time). Because we want to retain the multidimensional picture of the phenomenon of social exclusion, the final set of indicators presented in Section 3 does not offer a composite—or unified—measure. 3.2.4. Disaggregation level and frequency at which indicators are collected A review of the different international examples mentioned above (Australia, Canada, France, the United Kingdom) shows that in most cases data is seldom collected on a regular basis, or at a highly disaggregated level. As part of the South Australian government’s social inclusion initiative, the Australian Workplace Innovation and Social Research Centre developed a model that provided both qualitative and quantitative indicators. The Northern 11 Belgium used the French BIP-40 framework. It recorded little change in the indicator values in the short term (Labonté et al., 2011). 12 The BIP-40 indicator was developed in 1999 and used until 2005 in France. It used a battery of 58 indicators, each of them normalized and then aggregated into a bundled “barometer.” It was, among others, criticized for being a “black box” (Hétfa and Alapítvány, 2013). 33 Adelaide Survey of Social Inclusion was implemented in 2005 but remained a one-off exercise, with measures that were unlikely to be integrated into routine data gathering or be administered in subsequent studies. In France, the central statistical office (INSEE) collected the BIP-40 barometer between 1999 and 2005 and produced statistics at the national level on an annual basis. Levitas et al. (2007) discuss the matrix of indicators to follow progress toward social inclusion, but rely on the UK’s Poverty and Social Exclusion survey, which was last fielded in 1999. Finally, the EU indicators developed by Laeken (Marlier, Atkinson, Cantillon, and Nolan, 2007) and refined by Atkinson and Marlier (2010) were not fully used to assess performance toward social inclusion across EU member states until 2004. Atkinson et al. (2005) report that the indicators have been used at the national level to (i) explain differences among EU member states; (ii) assist individual states in their policy development; (iii) promote “joined up government” by identifying where intersectoral work is required; and (iv) target setting. It is unclear the extent to which these four intents have been operationalized. 4. What indicators should be used in Hungary? The overview of international literature finds that there is no consensus about how to measure social exclusion, apart from the need to capture its multifaceted nature. What further complicates the matter is the tendency to use available quantitative measures, with social exclusion being defined retrospectively (by the choice of available indicators) rather than prospectively (ex ante through social theory). The construction of social indicators necessarily entails a compromise between the theoretical definition (demand-driven indicators) and what is empirically possible (supply-driven indicators). Data may simply not be available, may not be of adequate quality, or may not be sufficiently comparable across geographic areas or time. 4.1. Indicators that are suited to track progress toward social inclusion How often data is collected, at which level of disaggregation, and whether indicators are quantitative or qualitative depends on the reason why policy makers want to measure social exclusion. For targeting new policy programs, researchers may use data available up to a few years before a proposed program’s implementation, and may include indicators that are quite static. For outcome monitoring, or how “socially inclusive” Hungary is becoming, one would need a set of dynamic indicators, with data aggregated at a small geographical level. On the other hand, for policy monitoring, or identifying which policies are associated with increasing or decreasing social exclusion, one may rely on a mixture of qualitative and quantitative data, with detailed information to capture the breadth and depth of social exclusion. For policy and program planning, or for the evaluation of policies and programs, one would need highly disaggregated data, but not necessarily with high collection frequency. 34 The different sets of indicators defined for the various exercises undertaken by HCSO— mapping, targeting, and monitoring, as presented in Section 2—mostly overlap when it comes to themes/dimensions of social exclusion. Most include monetary poverty, access to employment, education, and sometimes housing and access to services. At the same time, the geographical level at which data is available, and the year(s) for which that information is available and meaningful, varies quite substantially. Mapping segregated census blocks relies on information available every decade, but at the census-block level. Targeting most disadvantaged areas requires more frequent data that is available annually and at the regional level, such as from information from EU-SILC or the latest census (every decade) at the microregional level, or administrative data collected with various frequencies at various levels of disaggregation. 4.2. Issues and dilemmas regarding collecting and tracking indicators to monitor progress on social inclusion in Hungary During the team’s discussions with HCSO in the course of 2015, various dilemmas and questions were raised regarding the selection of dynamic social inclusion monitoring indicators. In this section we reflect on these dilemmas, the lessons of which will be incorporated into a recommendation for a measurement tool that could serve as a “follow- up mechanism” for tracking development policy interventions in a social inclusion context. How can the final set of indicators developed by different government agencies be harmonized with indicators of general social development? Most of the core indicators presented in the final set are collected by local municipalities, the national household survey, the population census, or come from administrative data collected by different line ministries. The harmonization of the diversity of data requires strategic decisions on the one hand and considerable statistical and methodological efforts on the other. One must acknowledge that trade-offs between a statistically less robust public policy–related registry data (for example, number of visits to doctors or people receiving selected local subsidies) to hard social indicators (such as unemployment figures) are not always necessary, as both can inform policy making—though at other levels of robustness and relevance. An example of this dilemma is the case of urban segregation indicators. Altogether only two robust data components (education and employment) are enough to define the most problematic areas, but if the local level wants to have a deeper understanding of the local processes, several local registry-based indicators can be useful. Therefore, while the mainstream solution is to opt for sets of indicators (to ensure a robust analytical basis and a sensitive policy tracking tool), at the same time, qualitative information may better explain the field experience in cases of extremely poor and fragmented communities. For example, in a community where social development starts by helping people understand why it is important to engage with the social worker or spend a full day at school or work, sometimes very soft changes need to be captured, and these can only be achieved through qualitative information. 35 Which territorial level is the most relevant when analyzing different social phenomena and progress? Balancing the depth of information and the level of disaggregation is a complex exercise. Accurate information may only be available at the regional level, while microregions or districts and municipalities may have low capacity to collect information that would be relevant for tracking progress toward social inclusion. On the other hand, local-level information is sometimes aggregated to a larger territorial level, where its primary purpose is to enable comparisons between different geographical areas. In the Hungarian context, the core of the dilemma is related to the fact that the Hungarian public administration system is one of the most fragmented in Europe. This explains why the local level must often deal with very few inhabitants and operate at a very low capacity, covering a limited set of tasks in terms of service delivery, with a small delegated budget and few revenues. The size of the municipalities is also highly heterogeneous (from less than a hundred to as many as 1.7 million inhabitants). As a result, the concentration of problems and complexity of issues at the local level also diverge considerably. A more balanced analysis is possible if the microregions or districts are taken as the units of observation. At this level, the average population is 50,000, still with relatively large variations: Bélapátfalva has less than 9,000 and Miskolc has more than 250,000 inhabitants.13 In terms of policy analysis, considering the trade-off between reverting to exploring the social dynamics at the local level versus choosing larger (and hence more comparable) population size observation units depends on whether we consider the smallest local administrative unit as a competent stakeholder in the formation of inclusion/exclusion/development processes. So far, in Hungary, it is “local competence” to tackle social exclusion issues (as per the obligation to develop Local Equal Opportunity Plans (LEPs) and intervention strategies in every municipality), for example by designing an effective local social benefit system. More recently, however, the allocation mechanism for a considerable amount of social transfers and many social services has been shifted to the district level. In the realm of the benefits, those under district supervision are categorized as so-called “income-compensation benefits,” whereas it is the local level’s responsibility to design and raise funds for “expenditure-compensation benefits.” 14 While it can be meaningfully analyzed at the local level how the face and depth of social exclusion is shaping up, the phenomena leading to changing the levels of social exclusion—including changes deriving from the dynamics of the social transfer system—cannot be limited to the local level. Another important public policy realm limited in terms of meaningful analysis at the very local level is related to the labor market: despite the fact that indicators regarding unemployment have proved to be the most precise dynamic measures to track social change at the local level, the effects of “local” labor market policy interventions will necessarily go 13 See http://www.jaras.info.hu/jarasok-nepessege. 14 See http://www.kormanyhivatal.hu/download/1/39/d1000/szoc_tam_tajekoztato.docx. 36 beyond the boundaries of the local administrative units, and often also beyond the district level. 15 The education system also has “gaps.” For example, in 2010, close to 1,000 (every third) municipalities had no primary school.16 Thus, it is less meaningful to see whether school service is available at the local level as an indicator for access to services (and hence social inclusion) than it is to lift the geographical level of analysis to a (school) district level. This allows one to explore early school leaving, school performance survey results, absenteeism, and so on. Moreover, there are areas that do not have secondary education opportunities at all. 17 Thus, improvements to education completion will necessarily depend on other factors such as transportation options and scholarship availability. The effects of location and displacement can be also critical when defining the territorial level of analysis. For example, successful training and education programs—especially in substantially lagging regions—can either contribute to a decline in unemployment or can have the opposite effect. They can result in increased unemployment, as a trained labor force would be more mobile and able to move to regions with a higher demand for labor. The same can happen when production investments attract a labor force from another region, but the newly created jobs do not affect the local labor supply (for example, if people are not trained accordingly, lack necessary skills, and so on). Hence, local unemployment levels would not improve as a result of the investment; active labor would be moved from elsewhere and relatively lower activity rates would characterize their “sending” municipalities after the investment.18 In summary, while income levels can be measured (proxied) at the local level, broader social inclusion processes can only be tracked in larger geographic units. However, the analysis of the processes (and potential causalities) is hardly meaningful at a single level. For example, people may commute for jobs, but if there are issues regarding accessibility, they will be unable to take advantage of job opportunities. Moreover, social inclusion programs can be implemented in territorial arrangements that are disconnected from the territorial units in which the public administration system operates, especially if they are run by the nongovernmental organization (NGO) sector. Do all indicators require the same time frame, taking into account the potential lag in the impact of specific interventions on outcomes of interest, such as early childhood development 15 For more concrete examples of geographic relevance, see http://www.ksh.hu/interaktiv/terkepek/mo/ingc.html, “Helyben lakó és dolgozó a helyben foglalkoztatottak százalékában, 2011”—ratio of local inhabitants who are locally employed as a percentage of all locally employed, 2011, or “Naponta bejáró a helyben foglalkoztatottak százalékában, 2011”—daily commuters to the municipality as a percentage of the locally employed, 2011. 16 See http://www.ksh.hu/docs/hun/xftp/idoszaki/pdf/tarsatlasz.pdf. 17 For example, in northeast Hungary, in the formerly Bodrogközi microregion, which had 17,000 inhabitants, there were no secondary schools. 18 For a recent analysis on this and a critical review of impact measurement methods, see Dusek et al. (2014). 37 (ECD) or education and training interventions that have long-term impacts on the labor market? As demonstrated in the UK example using panel data (Burchardt et al., 1999), few indicators that track progress toward social inclusion improve within a five-year time frame. Depending on the types of indicators, some may be affected quickly, while others may take time to record improvement. Thus, tracking the latter and not seeing any changes would not necessarily point to an absence of progress toward social inclusion, but would just indicate the fact that one should wait longer to start measuring impact on those indicators. For example, in the case of ECD, kindergarten attendance among young children is an indicator that should be affected quickly, and could thus be measured annually, while cognitive improvements and long-term labor market inclusion would not be affected immediately. According to OECD data, investment in early childhood development will only be rewarding after several years, and at the latest by age 15 (OECD, n.d.). Trainings for adults who have been out of the labor market for a long time will have a very diverging impact regarding the time and geographic location where changes emerge; whereas, for example, job search assistance and job brokerage may produce more immediate results (OECD, 2005). In the Hungarian context, several evaluations commissioned by the former Hungarian Development Agency on measures financed in the 2007–2013 EU funding period have piloted various methodologies to adjust the assessments to time-related impact measurement constraints. For example, the evaluation of the developments in higher education, 19 the (quantitative) evaluation of health investments, 20 and the evaluation of measures targeting Roma integration 21 all point to the fact that in the case of impact assessments, the time frame of the measurement of results is critical. Sustainability issues would also very much depend on whether appropriate conditions would still exist in the long term to ensure that the results can unfold, and hence be measured at all. What level of heterogeneity is there when discussing different policies, such as education, labor, or social issues? As a result of the above, in the case of different sectoral policies, different time frames and territorial levels may be relevant to track progress toward social inclusion in Hungary. At the same time, the development in these subsectors is interrelated; thus, caveats at the local level may already highlight meaningful information for policy design at higher levels. Stakeholders should design a meaningful combination of geographical units to observe (including taking into account the center–periphery issue) and a timespan in which to track 19 See http://palyazat.gov.hu/download/48112/Fels%C5%91oktat%C3%A1si_%20%C3%A9rt%C3%A9kel%C3%A9s_z% C3%A1r%C3%B3jelent%C3%A9s_I.pdf. 20 See http://palyazat.gov.hu/download/48436/Eg%C3%A9szs%C3%A9g%C3%BCgyi_kvantitat%C3%ADv_%C3%A9rt% C3%A9kel%C3%A9s_Budapest_Int%C3%A9zet.pdf. 21 See http://palyazat.gov.hu/download/39813/Roma_ertekelesi_zarojelentes_V.pdf. 38 the processes. These should be designed according to the priorities set by the given inclusion policy agenda; for example, whether the focus should be on children whose life trajectories would unfold later; on the long-term unemployed; or on development of lagging regions with a multitude of challenges. What difficulties stem from establishing a composite/synthetic indicator of social exclusion, and what difficulties are associated with presenting a large set of single indicators, each of which reflects a different dimension of social exclusion? It may be difficult to justify the choice of a particular methodology to aggregate indicators into a single composite indicator, as there are no scientific guidelines for how to do this. Thus, different aggregation measures (including weighting) using the same set of indicators may give different composite values, and different rankings across geographic areas, or over time. The development of the Canadian Index of Well-being (CIW), for example, has taken almost eight years of work, involving scores of researchers and countless meetings before attaining a level of quality, quantity, and consensus regarded sufficient for its release— including selection of subindicators, choice of model, weighting indicators, and how to treat missing values (Levitas et al., 2007). On the other hand, presenting a large set of indicators may be confusing for policy makers, as many indicators are bound to worsen in the very same localities where other indicators drastically improve. To what extent may the data collected by the different line ministries, research institutes, various registers and the HCSO be combined or complement each other; or, for example, to what extent can they be appropriately used for disaggregation (to get local-level data)? As mentioned earlier, a sound framework for analyzing progress toward social inclusion should include systematic data collection; information from tailor-made surveys should also be included to more deeply understand some phenomena, on an ad hoc basis. Hence, data collected systematically by the HCSO or the line ministries should be preferred, and additional information collected by research institutes, tailor-made surveys, and qualitative assessments should be used as secondary data sources. Beyond the costs associated with data validation (it is expensive and time consuming to validate registry and other survey- based data), the variations of alternative data sources in territorial design and representation can further challenge the combination of data.22 At the same time, as highlighted above, there are some useful techniques for combining data resources to disaggregate data to lower territorial levels; the poverty mapping exercise discussed in Section 2.1.2 has applied such an approach. 22 For example, in education, the data collected in KIR-STAT (the national information system on education) is organized according to schools (operators and school buildings); hence, commuters are accounted for in their respective schools. 39 What tools are needed to explore whether policy interventions applied locally in the most disadvantaged settlements are the most adequate, taking into account that the list of settlements at the lowest end of the spectrum is roughly constant? Evaluating the relative impact of a certain social inclusion policy versus another set of decisions—for example, investing in access to labor markets through subsidized employment for the youth versus soft-skills development for prime-age adults, or constructing kindergarten facilities versus improving the transition from primary to secondary school— would require using robust impact evaluation methods, carefully comparing one set of disadvantaged settlements that benefit from the first policy option with another set of settlements benefiting from the second. Such experiments are few and far between, especially when it comes to comparing two options: most impact evaluations compare a situation where nothing is done to one where a program is put in place.23 Therefore, there is also considerable room in Hungary to design and implement targeted impact assessment exercises to pilot the adequacy of selected methods. 4.3. Policy objectives and considerations for developing an indicator set In order to allow the proposed indicator set to respond to the aforementioned requirements, issues, and dilemmas, not only is readily available geographically disaggregated and high quality data required; in addition, specific policy objectives should be established, and the indicator set should be aligned with these. The first policy objective is to identify whether certain groups (geographic, ethnic, demographic, and so on) are systematically excluded along the core dimensions of social inclusion (outcome monitoring). Viewed from this policy objective, the government should have in place a data collection system that captures most of the core dimensions, broken down by background characteristics such as geography, ethnicity, and demography. This type of outcome monitoring does not necessarily require a full census, as a representative sample of core groups (geographic, ethnic, demographic, and so on) may be used. Here, the EU-SILC goes a long way, as it captures most (if not all) of the core dimensions, and allows for some disaggregation (by demography, gender, and region). High-quality administrative data will also support the objectives of outcome monitoring, and this data source (at least for some data) can be accessible at highly disaggregated geographical levels. The second policy objective is to have an M&E system in place that enables decision makers to target their programs to socially excluded groups. Here, representative samples are clearly not sufficient, as data should be available at a highly disaggregated geographical level. However, while outcome monitoring should include at least some indicators that can be affected relatively quickly by changes in social inclusion/exclusion, some indicators used for targeting may not need to be affected relatively quickly or need to be available on a 23 For a summary of methods and numerous case studies, see for example Baker (2000). 40 regular basis. For example, the map of most disadvantaged regions or list of most disadvantaged groups is likely fairly stable over time. 4.4. The proposed indicator set for Hungary From the large set of indicators that are available within the Government of Hungary’s Territorial Development and Spatial Planning Information System (TEIR), the team selected indicators that (i) respond to the issues, challenges, and considerations identified in Sections 4.1, 4.2, and 4.3; and (ii) incorporate lessons from the international examples mentioned in Section 2. Moreover, the team specified (iii) what primary policy objective the indicator may serve (targeting versus outcome monitoring)—while keeping in mind that some indicators may be suited for both objectives. For the analysis, the team collected the existing data from TEIR and selected a subset of indicators directly from what was already collected. This set includes the majority of so- called objective indicators of non-monetary exclusion, such as the possession of material goods and facilities and physical conditions of life. At the same time, what may be called subjective indicators—such as self-assessment of general health conditions, economic hardship, and social isolation, or the expressed degree of satisfaction with various aspects of work and life—have not been included in the indicator set due to lack of available data, even though such data would also provide relevant information on social inclusion. Against this background, the team proposes using the set of social inclusion indicators summarized in Table 6. Considering the size of the Hungarian Roma population and the persistent gaps between Roma and non-Roma in Hungary and elsewhere in the region, it would be important for data to be collected in an ethnically disaggregated manner. While it is currently not possible to do so, the introduction of an ethnic identifier in upcoming surveys (the EU-SILC in particular) will make this feature available in the coming years. Table 6. Set of proposed indicators for Hungary Indicator Primary Source Level Frequency Coefficient policy of objective (after 2000) 24 variation 25 1. Monetary poverty and material resources 1.1 AROP rates (using Targeting EU-SILC Regiona Yearly N/A EUROSTAT’s methodology 24 There might be some variation in methodologies from year to year—such as incentives and sanctions regarding the registration of unemployed—which might impact the figures. 25 The average coefficient of variation is computed like so: we first take the standard deviation of each indicator over time at the settlement level. Then we normalize it by the average value of the indicator for each settlement over time. Finally, we take the mean over all settlements. 41 of 60% of median income) l 26 1.2 Income inequality Targeting EU-SILC Regiona l (or share of high-income TEIR Yearly 36% taxpayers on low-income Local taxpayers) (2005–2013) 1.3 Income growth of Outcome EU-SILC Regiona Yearly N/A bottom 40% monitoring l 1.4 Persistent risk of Outcome EU-SILC Regiona Yearly N/A poverty monitoring l 1.5 Intensity of poverty Outcome EU-SILC Regiona Yearly N/A (poverty gap) monitoring l 1.6 Share of households Outcome TEIR Yearly 23% receiving regular child monitoring benefit (2006–2013) 1.7 Financial inclusion/debt Outcome N/A monitoring 1.8 Dependency ratio Outcome TEIR Local Yearly 7% monitoring (2005–2013) 2. Employment/labor 2.1 Long-term Outcome TEIR Local Yearly 42% unemployment rate (180 monitoring days) (2003–2013) 2.2 Very long-term Outcome TEIR Local Yearly N/A unemployment rate monitoring 2.3 Share of population Targeting EU-SILC Regiona Yearly N/A living in jobless households l with current income below 60% of median income 2.4 Working poor (share of Targeting EU-SILC Regiona Yearly N/A working adults living in poor l households) 2.5 Share of households Targeting EU-SILC Regiona Yearly N/A with very low work intensity l 26 Or microregional if small area estimation (as discussed in Section 3.1) can be updated annually. 42 3. Education and health 3.1 Early school Outcome Admin Local Yearly N/A leavers/drop-out rates monitoring 3.2 Adults with low Outcome TEIR Local Census educational attainment monitoring N/A - 15–24 year-olds N/A - 15–59 year-olds 3.3 Low reading/literacy Outcome Admin Local Yearly N/A skills of pupils monitoring 3.4 Low birth weight Targeting TEIR Local Yearly N/A 4. Housing and living conditions 4.1 Water/sewage/gas Targeting TEIR Local Yearly 5%-11%- connection 18% (2000–2013) 4.2 Overcrowded flats Outcome TEIR Local Yearly 6% monitoring (2000–2013) 4.3 Households receiving Outcome TEIR Local Yearly 91% housing allowance monitoring (2003–2010) 4.4 Crime/violence Targeting TEIR Local Yearly 54% (2002–2013) 4.5. Confirming the validity of indicators As described by the literature on social exclusion indicators, many measures are not very sensitive to changes in the short term. The last column of Table 6 displays the coefficient of variation of all available indicators. The most “dynamic” indicators—that is, indicators whose average coefficient of variation at the settlement level over time is high—are the number of households receiving a housing allowance (91 percent); crime rate (54 percent); and long-term unemployment (42 percent). Income inequality (measured by the share of high-income taxpayers on low-income taxpayers) and share of households receiving children benefits show moderate dynamism, with respective coefficients of variation of 36 and 23 percent. 43 Finally, and unsurprisingly, indicators related to water, sewage, and gas, flat overcrowding, and dependency ratio show very little variability over time.27 The figures in Annex 2 show the difference between a steady indicator such as access to sewage, and a highly dynamic indicator such as long-term unemployment at the settlement level, through indicators of a random sample of 9 Hungarian settlements (Figures A2.1 and A2.2) as well the settlements with the highest and lowest relative variability (Figures A2.3 and A2.4, respectively). 27 Unfortunately, some of the more sensitive poverty-related dynamic indicators will become unavailable in the future, due to changes in administrative regulations and data collection. For example, the housing allowance scheme was redesigned in March 2015, allowing municipalities more freedom to decide whether to distribute this transfer. As a result, any data relating to this local-level subsidy that had been targeting income poor families will necessarily be biased, because some municipalities will report about it as a housing allowance, others as a local allowance, and still others will not grant any subsidies of this kind. Similarly, the regular child protection benefit that had been one of the best targeted social allowances for extremely poor families was changed as of 2013; thus, the administrative data on number of beneficiaries and costs relating to this benefit cannot be used for later years, either. 44 5. How to design a tracking tool that combines social inclusion indicators with project data? In previous sections we have identified the key components of an approach that enables policy makers to track the status of social inclusion at the subregional level. The following section explores ways to turn this approach into a tool that could combine locally available social exclusion data with project information so as to allow projects to be continuously tracked, with a view toward developing a feedback mechanism regarding whether funds are spent in areas of highest need. 5.1. Proposal for a tool to track development policy interventions in a social inclusion context In Section 4.4 we introduced a brief set of social indicators that would be appropriate to (i) accurately identify the different dimensions of social exclusion; (ii) be available at a geographically disaggregated level; (iii) be collected regularly, preferably on an annual basis; and (iv) be “dynamic”—that is, respond to local development dynamics as demonstrated by relative variability. The proposed indicators were listed in Table 6 in Section 4.4. Furthermore, as discussed in Section 4.1, some of these indicators would be less dynamic than others, but still relevant for tracking policy measures, and can be illustrative for looking at public policy outcomes. Moreover, some of the indicators would only be available at a level higher than the local one, which is often a more relevant territorial level for observation than the local level. In this section we explore the possibility of linking some of the above social indicators to development policy measures’ potential indicators, with a view toward delivering a meaningful tool for describing, targeting, and tracking the social processes in places where social inclusion interventions have taken place. We revert to the findings of earlier evaluation exercises that analyzed selected EU–co-funded measures’ impact mechanisms. Specifically, we draw on lessons from the recently completed evaluation of social inclusion measures implemented within the TÁMOP 5 between 2007 and 2012 28 (the Evaluation Report). In the 2007–2013 period, TÁMOP 5 included most of the measures for social inclusion activities and developments targeting the poorest and most disadvantaged communities in Hungary, among them children and the long-term unemployed. 29 Our analysis includes 28 The report is available at http://www.nfu.hu/download/48414/T%C3%A1rsadalmi_%20befogad%C3%A1s_%C3%A9rt%C3%A9kel%C3%A 9si_jelent%C3%A9s.pdf and its annexes are available at http://www.nfu.hu/download/48413/T%C3%A1rsadalmi%20befogad%C3%A1s_esettanulmanyok.pdf. 29 The evaluation was based on document review, qualitative analyses (including case studies), and quantitative analyses. One quantitative analysis was based on a tailor-made data collection where a representative sample from all beneficiaries was compiled. An online survey was organized, with a 30 percent to 35 percent response rate. The number of project-level responses was close to 1,000. The survey and analysis of results was carried 45 measures that are relevant to the current exercise, and excludes many that are not.30 Therefore, our findings do not paint a full picture and are meant to be illustrative inputs for designing similar future exercises, with a view toward encouraging the design of a more comprehensive tracking and monitoring system for the new (2014–2020) EU funding period. The measures discussed in the Evaluation Report cover several thematic areas, and the analysis relies on a hypothesized impact mechanism: the thematic areas and impact mechanisms are discussed in Annex I of this report. Projects supported between 2007 and 2012 show considerable heterogeneity; moreover, the target organizations are also diverse—national-level institutions, microregions, municipalities, and NGOs, as well as consortia of various compositions were eligible for funding under the themes. The mechanisms behind the desired project outcomes are also diverse, and the differences in project design (for example, launch period, size of the beneficiary group, required intensity of work with beneficiaries, multiplication of effects) display a variety of ways to address selected social problems with TÁMOP 5 funding. 5.2. Topics and hypothesized impact mechanism of selected social inclusion measures Table 7 demonstrates a way to link the EU–co-funded project with tracking (local) social processes, based on the content analysis delivered by the Evaluation Report. Based on the results of the content analysis, we develop a general list of social challenges addressed. Since the social problems to be addressed can be defined based on a simple analysis, if necessary, their coherence with various strategic documents (such as relevant OPs, NSIS, or sectoral strategies) can be easily checked and updated. Second, we link these with the list of calls that directly or indirectly address the social challenge in question. (Some calls designed to approach complex social situations would match more than one social challenge.) Third, we select one or more proxy indicators that would best serve to track the development of the given social challenge. Finally, we list the indicators that are available and have proven to be dynamic and measureable at the lowest possible geographic level (as discussed in Section 3.1) while keeping in mind the limitations of this approach (as discussed in Section 4.2). out by REVITA Foundation; the evaluation was led by Hétfa Research Institute and Metropolitan Research Institute, Budapest. 30 For example, TÁMOP 5 had financed, among other initiatives, drug prevention programs, rehabilitation of ex- convicts, victim protection, methodological developments in a background institution of the Ministry of Human Capacities (MHC), programs for people living with disabilities, and so on. 46 Table 7. Matching social challenges with measures and tracking indicators Social problem Proxy social Examples for Closest Missing addressed indicator to measures in matching indicator/notes track relevance TÁMOP 5 tracking of measure indicator children with share of TÁMOP 5.1.1.- Number of 3.1 Early school constrained population with 09/1-2, TÁMOP people without leavers/drop- school careers low education 5.2.2-08/1, 2, having out rates (L) a (i.e. 8th TÁMOP 5.2.2- completed the Available in the grade)/Early 10/1, TÁMOP first class of framework of school leavers / 5.2.3-A-11 primary school the EU2020 dropouts TÁMOP 5.2.3- among those data collection A-12/1 over 10 years by the (L, Census) background institute to the Ministry of Human Capacities child poverty share of children TÁMOP 5.2.2- 1.6 Share of living under the 08/1, 2, households poverty line TÁMOP 5.2.2- receiving 10/1, TÁMOP regular child 5.2.3-A-11, benefit from TÁMOP 5.2.3- among those A-12/1 under 18 years (L) 2.5 Share of very low work intensity households (L, meaning households with no employed person) among all households deviances crime rates TÁMOP 5.1.1.- 5.1 Number of 09/1-2, TÁMOP registered 5.2.5-08/1/C crimes/violence per 100 persons (L) low share of young TÁMOP 5.1.1.- Share of participation of unemployed 09/1-2 unemployed young in the among the 47 Social problem Proxy social Examples for Closest Missing addressed indicator to measures in matching indicator/notes track relevance TÁMOP 5 tracking of measure indicator labour market labour market entrants (L) digital illiteracy digital illiteracy TÁMOP 5.1.1.- X 09/3 measurable e.g. by nr of households connected to broad band internet (L) low activity activity rate, TÁMOP 5.1.1.- 2.1 Long-term rates/large unemployment 09/1-2, TÁMOP unemployment unemployment rate, long-term 5.1.1.-09/3, rate (180 days) unemployment TÁMOP 5.1.1- (L), 2.6 Share of rate 09/6-7, households TÁMOP- with very low 5.3.1/08/1, 2, work intensity TÁMOP-5.3.1- among all C-09/2, households (L) TÁMOP- 5.3.1/08/2, TÁMOP-5.3.9- 11/1 gaps in / lack of share of local TÁMOP 5.1.1- Segregation additionally: (selected) residents 09/4-5, TÁMOP index produced measurable quality receiving social 5.1.1-09/8-9, by the HCSO e.g. by social/human benefits, share TÁMOP 5.1.3- based on 2011 various sub- service delivery of local 09/1,2, TÁMOP Census data sectoral data for various residents on 5.2.2-08/1, 2, on service gaps target groups selected TÁMOP 5.2.2- via SZOCIR (L) (e.g. at local transfers, nr of 10/1, TÁMOP level, home clients per staff 5.2.3-A-11, care, social, employed in the TÁMOP 5.2.3- child social A-12/1, TÁMOP protection, assistance/home 5.2.5-08/1/A, youth welfare care/welfare TÁMOP service, services service sector 5.2.5.A-10/1, for people with TÁMOP disabilities) 5.2.5.A-10/2, TÁMOP- 5.3.8.A2-12/1, TÁMOP- 48 Social problem Proxy social Examples for Closest Missing addressed indicator to measures in matching indicator/notes track relevance TÁMOP 5 tracking of measure indicator 5.3.8.A2-12/2, TÁMOP- 5.3.8.A3-12/1, TÁMOP- 5.3.8.A3-12/2, TÁMOP - 5.4.1/08/, TÁMOP - 5.4.2/08/1, TÁMOP 5.4.3- 09/1, TÁMOP 5.4.3-09/2, TÁMOP-5.4.3- 10/1, TÁMOP- 5.4.3-10/2, TÁMOP -5.4.4- 09/1/A, TÁMOP -5.4.4- 09/1/B, TÁMOP -5.4.4- 09/1/C, TÁMOP -5.4.4-09/2/A, TÁMOP -5.4.4- 09/2/B, TÁMOP -5.4.4- 09/2/C, TÁMOP- 5.4.5/07/1 és 5.4.5-09/1, TÁMOP- 5.4.6.A-12/2, TÁMOP-5.4.9- 11/1, TÁMOP - 5.5.2 /09/2, TÁMOP - 5.5.2 /09/3, TÁMOP- 5.5.2/10/4, TÁMOP- 5.5.3/08/01, TÁMOP- 5.5.3/08/02, TÁMOP-5.5.3- 49 Social problem Proxy social Examples for Closest Missing addressed indicator to measures in matching indicator/notes track relevance TÁMOP 5 tracking of measure indicator 09/1 low community TÁMOP 5.2.5- X cohesion 08/1/B homelessness nr of homeless TÁMOP- X 5.3.3/08/1, measurable TÁMOP- e.g. by 5.3.3/08/2, available beds TÁMOP-5.3.3- in homeless 10/1, TÁMOP- service 5.3.3-10/2 provision (L) indebtedness / level of TAMOP-5.3.5- 4.3 Share of housing cost household debt, 09/1 people overburden housing cost receiving overburden housing allowance per 100 residents (L) housing share of people TÁMOP-5.3.6- 4.1 Share of segregation of living in 11/1 dwellings Roma / people inadequate connected to living in poverty housing of Roma sewage (L) neighbourhoods origin Nr of households as gas consumers among all dwellings (L), Amount of electricity provided to households per residents (L) Share of dwellings connected to the water system (L) 4.2 Overcrowded flats: Nr of 50 Social problem Proxy social Examples for Closest Missing addressed indicator to measures in matching indicator/notes track relevance TÁMOP 5 tracking of measure indicator dwellings in / number of resident population at the end of the year based on the Census data (L) low labour activity rate of TÁMOP- 2.1 Long-term X market people with 5.3.8.A2-12/1, unemployment measurable participation of disabilities TÁMOP- rate (180 days) e.g. by nr of people with 5.3.8.A2-12/2, (L) people on disabilities TÁMOP- Number of disability 5.3.8.A3-12/1, people pension (L) TÁMOP- receiving 5.3.8.A3-12/2, disability TÁMOP- benefits (L) 5.4.5/07/1 és 5.4.5-09/1, TÁMOP- 5.4.6.A-12/2, TÁMOP- 5.4.7/08/1, TÁMOP- 5.4.7/08/2, TÁMOP- 5.4.7/09/1, TÁMOP- 5.4.8/08/1 discrimination statistics on TÁMOP– Share of Roma X of Roma and cases at the 5.5.4.A-09/1, at local level (L, can be people living Equal Treatment TÁMOP– Census data) obtained from with disabilities Body and in 5.5.4.B-09/1, the Equal and other court TAMOP – Treatment vulnerable 5.5.5/08/1, Authority groups TÁMOP 5.5.7/08/1 low consumer TÁMOP- X awareness 5.5.6/08/1, TÁMOP- 5.5.6/08/2 51 Social problem Proxy social Examples for Closest Missing addressed indicator to measures in matching indicator/notes track relevance TÁMOP 5 tracking of measure indicator exclusion of nr of people TAMOP – 5.1 Number of people with with criminal 5.6.1.A-11/1., registered criminal records records TAMOP – crimes/violence 5.6.1.A-11/3, per 100 TAMOP – persons (L) 5.6.1.A-11/ 4., TÁMOP 5.6.2- 10/1 high crime rates crime rates TAMOP – 5.1 Number of among young 5.6.1.A-11/1, registered TÁMOP- crimes/violence 5.6.1.B-12/1, per 100 TÁMOP- persons (L) 5.6.2/08/1, TÁMOP 5.6.2- 10/1 *collected at (L)=local; (R)=regional level; **=the benefit’s figures are not available after 2013, but since the analysis concerns the period until 2013, this seems to be an appropriate indicator; *** the benefit was cut and redesigned after 2015, but since the measures concern the period until 2013, this seems to be an appropriate indicator The analysis demonstrates that most of the challenges and associated activities could be tracked with an indicator from the proposed indicator set. It is therefore possible to develop a feedback mechanism that is based on whether thematic activities are undertaken in the geographic areas with the highest need in a certain theme. This feedback mechanism would be based on tracking the indicators at the lowest possible geographical level and comparing them with projects’ geographic locations. For example, an analysis of how TÁMOP 5 funding sources had reached localities or districts (or any higher geographical level) that experience child poverty could be undertaken using a map with the rate of beneficiaries of the regular child benefit, including the information of the location of projects funded under the relevant calls (in this case, TÁMOP 5.2.2-08/1 and 2, TÁMOP 5.2.2-10/1, TÁMOP 5.2.3-A-11, TÁMOP 5.2.3-A-12/1). 31 For another example on unemployment rates, displaying projects under TÁMOP 5.1.1-09/1-2, TÁMOP 5.1.1-09/3, TÁMOP 5.1.1-09/6-7, TÁMOP-5.3.1-08/1 and 2, TÁMOP 5.3.1-C-09/2, TÁMOP 5.3.1-08/2, TÁMOP 5.3.9-11/1 could serve to track where this 31 For the 2014–2020 period, an alternative indicator will have to be defined for this topic. 52 problem has been addressed via labor market inclusion projects funded under TÁMOP 5. These examples are elaborated in more detail in Annex 2. 5.3. Next steps toward an improved tracking tool As demonstrated by the example of selected TÁMOP 5 2007–2012 period calls, the inclusion of “social challenge” dimensions in the database of calls/measures can be a useful tool for: 1. displaying the dynamics of (selected) social challenges as proxied through low administrative-level social indicators; and subsequently 2. tracking whether any funding is addressing these social challenges in the given locality/district/and so on in a given time period. To this end, routinely combining maps of subregional social exclusion indicators with project data would require the regular collection of at least the following project information: • the social challenge the call/measure intends to address via an ex post content analysis of the calls or via an ex ante solution, as proposed below for the 2014–2020 period; • the time frame in which the call is launched and the projects are financed (from and until what year); • number of project beneficiaries; and • the location(s) of the implementation. Additionally, the Cohesion Policy Regulations 2014–2020 entail strengthened monitoring and evaluation arrangements, with the goal of ensuring that (i) robust and reliable data are available; (ii) these data can be aggregated at the EU level; and (iii) evaluation activities focus on assessing the effectiveness and impact of ESF support. To this end, member states are now required to ensure that data collection systems provide electronically structured data about the participants for each priority axis, broken down by investment category. Annex B and C of the Monitoring and Evaluation of Cohesion Policy guidance note (EC, 2015) offers a list and definition of common and youth employment indicators (output and results). The tracking tool can accommodate and make this information available as well. As the tool is not designed for robust impact analysis, but rather for simply visualizing information on social processes and linking it with data on funding and expenditure, no conclusions can be made regarding the causalities and the level of change. However, the tool does allow one to make observations regarding what changes were ongoing at the time selected projects in the given geographic area were funded, and whether the locally implemented projects responded to local challenges. Furthermore, the analysis underlying the proposed tool builds on the Evaluation Report’s content analysis of all related calls. The 53 analysis helped identify the social challenges, and many of these could be directly linked to the proposed set of dynamic indicators. Due to the limited data offered by the management authority’s database on EU–co-funded projects (EMIR), it is unrealistic to re-elaborate the content of all past calls related to social inclusion. 32 At the same time, for the 2014–2020 period, the relevant administrative body or bodies (the Prime Minister’s Office or the respective departments in the line ministries) should be encouraged to add additional dimensions to routinely collected data when they administer or design the call. These dimensions should correspond to the dynamic indicator set and could allow for a more sophisticated classification of EU–co-funded projects, thereby making it possible to track the local challenges addressed. These dimensions should also be linked to EU 2020 targets. The dimensions should, at a minimum, include: 1. constrained school careers/low education 2. child poverty 3. crime or deviance 4. low employment/activity levels 5. gaps in/lack of (selected) quality social/human service delivery for various target groups (at the local level, home care, social, child protection, youth welfare service, services for people with disabilities) 6. indebtedness/housing cost overburden 7. housing segregation of Roma/people living in impoverished neighborhoods 8. discrimination of vulnerable groups (such as Roma, people with disabilities, and so on) 33 The above list should be matched and complemented with dimensions included as relevant dimension/indicators in the forthcoming Operational Programs (OPs), which include a social inclusion dimension and other dimensions that are being defined as a basis for indicators of the forthcoming revision of the NSIS II in the course of 2015. It is important to ensure that indicator definitions are consistent across policy objectives. This means choosing, whenever possible, the same definitions for project monitoring indicators that are also part of the national-level outcome monitoring. Such consistency will help explain movements—or lack thereof—in outcome indicators. 32 Currently, EMIR only includes information about whether the project is implemented in a lagging region; whether it targets Roma, people with disabilities, or any vulnerable group; and whether it has contributed to an improvement in their quality of life. Information appears to be incomplete, and hence its use is limited. 33 The proposed dimensions are not necessarily complementary. 54 It is also important that national key projects co-funded by EU resources also report project data in a territorially disaggregated manner. To this end, it is important that government agencies that are beneficiaries of EU-funded interventions track resource allocation and project indicators in a territorially disaggregated manner, and that the Prime Minister’s Office enables the EU project database to accommodate this reporting mechanism. 5.4. Recommendations for targeting and monitoring EU–co-funded social inclusion investments in the 2014–2020 period 1. Establish an institutional/operational framework for the regular collection of social inclusion indicators in a territorially and ethnically disaggregated manner—a proposed set of indicators can be found in Table 6. Responsible agency: Ministry of Human Capacities and Hungarian Central Statistical Office Time frame: 2016 and onwards Expected costs: low (most of the indicators are based on existing data) 2. Establish a collection framework of project data (including for national key projects) for the 2014–2020 project period, corresponding to and consistent with the dynamic indicator set. It should include (but not necessarily be limited to) the following dimensions: o constrained school careers/low education o child poverty o crime or deviance o low employment/activity levels o gaps in/lack of (selected) quality social/human service delivery for various target groups (at the local level, home care, social, child protection, youth welfare service, services for people with disabilities) o indebtedness/housing cost overburden o housing segregation of Roma/people living in impoverished neighborhoods o discrimination of vulnerable groups (such as Roma, people with disabilities, and so on) 34 Responsible agencies: Ministry of Human Capacities and Prime Minister’s Office 34 The proposed dimensions are not necessarily complementary. 55 Time frame: January–June 2016 Expected costs: low (requires the addition of a few variables to the EU project database) 3. Complement the project data collection framework with o further relevant dimensions based on the forthcoming OPs, which have a social inclusion dimension; o dimensions that are being defined as a basis for indicators of the forthcoming revision of the NSIS II; and o indicators aimed at monitoring the performance of each relevant OP priority axis, broken down by investment category. Responsible agencies: Ministry of Human Capacities and Prime Minister’s Office Time frame: January–June 2016 Expected costs: medium (requires closer coordination between government agencies) 4. Map and publish the social inclusion indicators by using a GIS software application or the national mapping platform for administrative data, TEIR TETA Responsible agencies: Ministry of Human Capacities and Lechner Lajos Knowledge Center Time frame: 2016 and onwards Expected costs: low (requires the use of the existing GIS platform) 5. Continuously map and publish project data by using a GIS software application or the national mapping platform for administrative data, TEIR TETA Responsible agencies: Ministry of Human Capacities, Prime Minister’s Office, and Lechner Lajos Knowledge Center Time frame: 2016 and onwards Expected costs: low (requires the continued use of the existing GIS platform) 56 6. References Atkinson, A., B. Cantillon, E. Marlier, and B. Nolan. 2005. “Taking Forward the EU Social Inclusion Process.” Independent Report commissioned by the Luxembourg Presidency of the Council of the European Union, Government of Luxembourg, Luxembourg. Atkinson, A. B., and E. Marlier. 2010. “Analyzing and Measuring Social Inclusion in a Global Context.” United Nations, Department of Economic and Social Affairs (DESA), New York, ST/ESA/325. Atkinson, A. B., E. Marlier, and B. Nolan. 2004. “Indicators and Targets for Social Inclusion in the European Union.” Journal of Common Market Studies 42 (1): 47–75. Baker, J. L. 2000. Evaluating the Impact of Development Projects on Poverty. Washington, DC: World Bank. Burchardt, T., J. Le Grand, and D. Piachaud. 1999. “Social Exclusion in Britain 1991–1995.” Social Policy and Administration 33 (3): 227–44. Deaton, A. 1997. The Analysis of Household Surveys: A Microeconometric Approach to Development Policy. Washington, DC: World Bank. Dusek, T., and M. Lukovics. 2014. “Az ELI és az ELI Science Park gazdasági hatásvizsgálata.” Területi Statisztika 54 (3): 202–19. Elbers C., J. O. Lanjouw, and P. Lanjouw. 2003. “Micro-Level Estimation of Poverty and Inequality.” Econometrica 71 (1): 355–64. EUROSTAT. 2015. “Europe 2020 Indicators—Poverty and Social Exclusion.” http://ec.europa.eu/eurostat/statistics- explained/index.php/Europe_2020_indicators_-_poverty_and_social_exclusion. EC (European Commission). 2011. “On the Territorial Aspects of Extreme Poverty: Drawing Up a European Extreme Poverty Map.” European Platform for Roma Inclusion. http://ec.europa.eu/justice/discrimination/files/romaplatform_discussion _paper_poverty_2011_en.pdf. EC. 2015. “Programming Period 2014-2020. Monitoring and Evaluation European Cohesion Policy.” Guidance Document, European Social Fund. Hétfa, and R. Alapítvány. 2013. “A társadalmi befogadást szolgáló fejlesztések (TÁMOP 5. prioritás) értékelése.” Budapest: Hétfa. Ivanov, A., and J. Kagin. 2014. “Roma Poverty from a Human Development Perspective.” Roma Inclusion Working Papers, United Nations Development Programme, Istanbul. 57 Kezán, A., and D. Szilágyi. 2015. “Measuring the Disparities in the State of Development of LAU1 and LAU2 Units.” Presentation at the Hungarian Central Statistical Office. Labonté, R., A. Hadi, and X. E. Kauffmann. 2011. “Indicators of Social Exclusion and Inclusion: A Critical and Comparative Analysis of the Literature.” É/Exchange Working Papers, vol. 2, no. 8, University of Ottawa. Levitas, R., C. Pantazis, E. Fahmy, D. Gordon, E. Lloyd, and D. Patsios, 2007. The Multi- dimensional Analysis of Social Exclusion. London: Department for Communities and Local Government (DCLG). Marlier, E., A. B. Atkinson, B. Cantillon, and B. Nolan. 2007. The EU and Social Inclusion: Facing the Challenges. Bristol: Policy Press. Miliband, D. 2006. Social Exclusion: The Next Steps Forward. London: Centre for Analysis of Social Exclusion. London: LSE and ESRC Research Centre. OECD (Organisation for Economic Co-operation and Development). n.d. “Investing in High-Quality Early Childhood Education and Care (ECEC).” http://www.oecd.org/education/school/48980282.pdf. OECD. 2005. “Labour Market Programmes and Activation Strategies: Evaluating the Impacts.” In OECD Employment Outlook, 173– 208. http://www.oecd.org/els/emp/36780874.pdf. Ravallion, M., and B. Bidani. 1994. “How Robust Is a Poverty Profile?” World Bank Economic Review 8 (1): 75–102. Ravallion, M. 1992. “Poverty Comparisons: A Guide to Concepts and Methods.” Living Standards Measurement Study (LSMS) Working Paper, No. LSM 88, World Bank, Washington, DC. Silver, H. 1994. “Social Exclusion and Social Solidarity.” International Labour Review 133 (5– 6): 531–78. UNDP (United Nations Development Programme), World Bank, and EC. 2011. Regional Roma Survey. World Bank. 2013. Inclusion Matters: The Foundation for Shared Prosperity (Advance Edition). Washington, DC: World Bank. World Bank. 2014. “Small Area Estimates of Poverty in Hungary: A Methodological Report.” Mimeo, World Bank, Washington, DC. World Bank. 2015. Equality of Opportunity for Marginalized Roma: A Pathway to Shared Prosperity in Europe. Washington, DC: World Bank. 58 Annex I. Summary of TÁMOP 5 measures and hypothesized impact mechanisms Code of the measure Topic Goal Hypothesized impact mechanism Child and youth protection Through preventative and Reducing and preventing deviances, children TÁMOP 5.1.1-09/1-2 (funding available only for the intervention programs, to help will improve their school careers and most disadvantaged microregions) youth enter the labor market facilitate their later labor market integration. Development of digital Developing digital literacy competence will TÁMOP 5.1.1-09/3 competences (funding available To develop digital literacy help people use the Internet to, among only for the most disadvantaged other things, search for jobs. microregions) Local community development Capacity of organizations that work with the TÁMOP 5.1.1-09/4-5 programs (funding available only To develop community programs poor will strengthen, and new services will for the most disadvantaged and services emerge that will help reduce social microregions) exclusion. Training and employment for To help disadvantaged groups The chances of participants in the labor TÁMOP 5.1.1-09/6-7 vulnerable groups (funding integrate into the labor market market will improve and they will obtain available only for the most and access services, training, and new skills and on-the-job training. disadvantaged microregions) programs Training and employment for Developing service delivery and the NGO TÁMOP 5.1.1-09/8-9 professionals (funding available To strengthen local human sector will improve access to services and only for the most disadvantaged capacities and civil society reduce social exclusion. microregions) Community-based development To integrate those living in deep TÁMOP 5.1.3-09/1 activities for the integration of poverty via the tools of social Access to social services for the poor will and 2 people living in deep poverty— and community work, and to improve, and hence the poorest will increase professional coordination design selected public services their community activity. that match local demand 59 Code of the measure Topic Goal Hypothesized impact mechanism Establishment of the methodology To elaborate program elements for scaling up of the Chance for that target vulnerable children TÁMOP 5.2.1-07/1 Children program (ECD) and aged 0–7 with special attention Ensuring good quality project calls will TÁMOP 5.2.1-09/1 follow-up of local projects paid to children 0–5 who are not improve the implementation’s effectiveness. in preschool, to enhance their chances in society Scaling up the ECD-focused To develop skills and complex TÁMOP 5.2.2-08/1 Chance for Children program to programs for vulnerable children ECD facilitates later school careers of and 2 the country level, focusing on the aged 0–7, with special attention children, and hence their chances in the TÁMOP 5.2.2-10/1 most disadvantaged microregions paid to children 0–5 who are not labor market will improve. in preschool TÁMOP 5.2.3-A-11 Scaling up the ECD-focused To reduce child poverty and Improved ECD, community development, TÁMOP 5.2.3-A-12/1 Chance for Children to the country prevent the reproduction of and local human service delivery will level poverty improve access to quality services. TÁMOP 5.2.5-08/1/A Integration program of children To compensate for the Facilitating access to child protection TÁMOP 5.2.5-A-10/1 and youth—child protection disadvantages experienced by services will help make families less TÁMOP 5.2.5-A-10/2 component affected school-aged children vulnerable. and youth Integration program of children To improve the quality of life of TÁMOP 5.2.5-08/1/B Youth activities in the community will and youth—youth-focused disadvantaged youth and increase with the help of the service. component develop their community skills Integration program of children To compensate for the TÁMOP 5.2.5-08/1/C and youth—drug consumption– disadvantages experienced by Through preventative programs, deviant related component affected school-age children and behavior will decrease. youth, preventative programs TÁMOP 5.3.1-08/1 “First Step”—enabling and Programs to enable and prepare Improving the employability of the target and 2 preparatory programs for for independent living, and group and helping them acquire new TÁMOP 5.3.1-C-09/2 independent living for people with programs for preparing the steps competencies and knowledge will facilitate 60 Code of the measure Topic Goal Hypothesized impact mechanism TÁMOP 5.3.1-08/2 low reintegration chances into the and motivating to enter the labor their labor market integration. labor market market, supporting services Targeted (single beneficiary) To develop homeless project for the professional and TÁMOP 5.3.3-08/1 organizations and related Educating organizations that work with the methodological elaboration of the TÁMOP 5.3.3-08/2 services that would serve the homeless and offering complex labor market program for social and labor TÁMOP 5.3.3-10/1 social and labor market and housing programs will improve the market integration of homeless TÁMOP 5.3.3-10/2 integration of homeless people, chances for social reintegration of homeless people and programs for social enhancing their professional, people. and labor market integration of housing, and social situations homeless people Pilot program for prevention of Scaling up debt management programs and To launch debt management TÁMOP 5.3.5-09/1 arrears traps introducing alternative services will help programs and related stabilize indebted beneficiaries’ financial preventative programs situation. Complex poverty/Roma The labor market and social integration settlement program (ensuring chances of people living in segregated To help the integration of people TÁMOP 5.3.6-11/1 access to complex human neighborhoods will improve through living in segregated services) complex programs like social and community environments development, education, health, training, and employment actions. Supporting motivation trainings Through needs assessments and tailor-made TÁMOP 5.3.8-A2-12/1 and related events at employers, To help the labor market development, and by creating the TÁMOP 5.3.8-A2-12/2 targeting the most vulnerable integration of people with nationwide supporting network, the target TÁMOP 5.3.8-A3-12/1 groups in order to facilitate their reduced workability and access group will have a better chance to find labor rehabilitation services chances in the labor market employment on the open labor market. 61 Code of the measure Topic Goal Hypothesized impact mechanism Supporting motivation trainings To help the labor market Through needs assessments and tailor-made and related events at labor integration of people with development, and by creating the TÁMOP 5.3.8-A3-12/2 market services, targeting the reduced workability and help nationwide supporting network, the target most vulnerable groups in order them receive labor rehabilitation group will have a better chance to find to facilitate their chances in the services employment on the open labor market. labor market Learning partnerships for To develop competencies for Tailor-made trainings will improve the TÁMOP 5.3.9-11/1 enhancing employability vulnerable people and trainings potential labor market participation of the according to their special target group. education needs Modernization of social services, fostering the capacity of central To develop social, child welfare, TÁMOP 5.4.1-08/1 Based on methodological development, and regional strategic planning, child protection, and drug services will become more effective. elaborating decisions related to prevention services social policy Central developments of social To modernize social services via TÁMOP 5.4.2-08/1 Based on methodological development, informatics the development of a centralized services will become more effective. electronic service Development of home care To develop services related to TÁMOP 5.4.3-09/1 The quality of life of beneficiaries who home care TÁMOP 5.4.3-09/2 receive home care will improve, and TÁMOP 5.4.3-10/1 recipients of home care transfer will have TÁMOP 5.4.3-10/2 more chance to find employment. 62 Code of the measure Topic Goal Hypothesized impact mechanism Development of social trainings, professional trainings (including skills and higher level trainings), TÁMOP 5.4.4-09/1/A fostering local training capacities TÁMOP 5.4.4-09/1/B To train social, child welfare, and Developing social protection, child welfare, TÁMOP 5.4.4-09/1/C child protection staff, and to and child protection services will improve TÁMOP 5.4.4-09/2/A develop trainings (and training beneficiaries’ access to quality services. TÁMOP 5.4.4-09/2/B materials TÁMOP 5.4.4-09/2/C TÁMOP 5.4.5-07/1 Creating a barrier-free To develop methodological and Raising professional standards will improve TÁMOP 5.4.5-09/1 environment via physical and info- training materials the quality of service delivery. communication tools Scaling up knowledge and service development relating to the To disseminate the methods and TÁMOP 5.4.6-A-12/2 Trainings will prepare providers for creation of barrier-free the training materials elaborated investments in equal treatment. environments via physical and in TÁMOP 5.3.4 info-communication tools TÁMOP 5.4.7-08/1 Development of basic The rehabilitation program will help visually TÁMOP 5.4.7-08/2 rehabilitation services for the To provide basic rehabilitation impaired people more easily integrate into TÁMOP 5.4.7-09/1 visually impaired services for the visually impaired the labor market. Enhancing the professional To modernize the services Modernizing the system and creating a well- TÁMOP 5.4.8-08/1 background for complex relating to people living with educated institution will make active labor rehabilitation disabilities market policies more effective. 63 Code of the measure Topic Goal Hypothesized impact mechanism Pilot program for the functional To develop the most essential TÁMOP 5.4.9-11/1 Basic service delivery will be more effective combining of basic services social and child welfare services based on the combination of services. and coordinate service delivery Supporting local programs and Implementing the horizontal goals will TÁMOP 5.5.1-A-10/1 To implement programs that initiatives to implement horizontal improve the local conditions for the social facilitate TÁMOP horizontal goals goals* integration of vulnerable groups. TÁMOP 5.5.2-09/1 Scaling up voluntary work— To establish the professional Standardizing voluntary work will improve central coordination foundation of voluntary work service quality. Scaling up voluntary work—local To scale up voluntary work by Scaling up voluntary work will improve the TÁMOP 5.5.2-09/2 projects implementing voluntary capacities of CSOs and state- and TÁMOP 5.5.2-09/3 programs and fostering civil municipality-run institutions; TÁMOP 5.5.2-10/4 society organizations that deal standardization will also help develop with voluntary work programs’ professional quality. TÁMOP 5.5.3-08/01 Supporting organizations to CSOs will more easily participate in service TÁMOP 5.5.3-08/02 develop and service civil society To develop civil society delivery; thus, capacities of such services will TÁMOP 5.5.3-09/1 organizations organizations improve, and in lagging regions they might fill gaps in service delivery. Supporting antidiscrimination programs in the media— Discriminated and vulnerable people will TÁMOP 5.5.4-A-09/1 component A: media-related To train and employ people at become employed, hence their training and employment of risk of discrimination representation will improve, which will people with Roma background foster their impact on opinion building. and people living with disabilities Supporting antidiscrimination To produce media broadcasts programs in the media— Media representation of discriminated TÁMOP 5.5.4-B-09/1 that establish a positive component B: enhancing the people will improve and reduce negative perception of discriminated reduction of discrimination stereotypes about them. groups through the media 64 Code of the measure Topic Goal Hypothesized impact mechanism Fighting discrimination—forming Uncovering discriminatory actions will To foster actions against public perception and fostering improve the advocacy capacities of affected TÁMOP 5.5.5-08/1 discrimination and strengthen the authorities’ work (single groups; awareness-raising programs will the advocacy capacity of beneficiary: Equal Treatment reduce the prevalence of discriminatory discriminated groups Authority) actions. Dissemination of the importance Fostering the capacities of organizations that TÁMOP 5.5.6-08/1 of consumer protection by To promote the importance of deal with consumer protection and TÁMOP 5.5.6-08/2 forming awareness-based consumer protection disseminating knowledge will make consumer behavior advocacy activities more effective. Development of the network of To develop advocacy work advocacy organizations from relating to making legal TÁMOP 5.5.7-08/1 among the NGO/legal The effectiveness of legal protection will representatives available for sick, representatives for sick or improve. institutional clients and in the institutionalized clients and in the realm of child protection realm of child protection Enhancing the chances of social The programs will improve the life chances integration for incarcerated To reintegrate people with TÁMOP 5.6.1-A-11/1 of those who have criminal records, and people and those under criminal records, prevent crime preventative programs will reduce crime patronage based on training and and victimization levels, especially among young offenders. employment programs Special integration and The chance to attend trainings and obtain TÁMOP 5.6.1-A-11/3 reintegration activities for To promote the social and labor marketable professional skills, along with TÁMOP 5.6.1-A-11/4 incarcerated persons and those market integration of people incentives to the business sector, will under patronage and living in with criminal records improve the target group’s chances of corrective institutions integrating into the labor market. 65 Code of the measure Topic Goal Hypothesized impact mechanism Assisting groups of minors and To foster the cooperation of the youth via crime-prevention police and the receiving Curricular and extracurricular activities will TÁMOP 5.6.1-B-12/1 programs who are especially institutions to address the needs improve youth’s knowledge and self-esteem exposed to committing crimes and of minors and youth especially and professionals’ methodological toolkit, becoming victims prone to committing crimes and which will in turn decrease crime levels. becoming victims Assisting the methodological To reduce crimes among minors Needs assessment and professional trainings TÁMOP 5.6.2-08/1 development crime-prevention and youth, prevent victimization, will enhance more effective implementation and reintegration programs and assist the reintegration of of crime-prevention and reintegration fostering social cohesion offenders programs. Assisting the methodological Training professionals and offering development crime-prevention information and support services for victims, To reduce crimes among minors and reintegration programs and elaborating a stepwise reintegration TÁMOP 5.6.2-10/1 and youth, prevent victimization, fostering social cohesion—Phase II program for people with criminal records, and assist the reintegration of will enhance their reintegration to society offenders and into the labor market, which will also reduce victimization. Source: Hétfa and Alapítvány, 2013, pp. 46–49. Hypothesized impact mechanisms were elaborated based on content analysis of the calls as indicated in the report. *The horizontal goals are: facilitating equal chances, sustainability, facilitating territorial cohesion, international and interregional cooperation, scaling up of social innovation, and transfer of experiences. 66 Annex II. Relative variability of selected indicators Figure A2.1. Access to sewage—Nine random settlements (%, 2000–2013) 100 90 80 Milota 70 Gyöngyöstarján Kishartyán 60 Zalaszentgrót 50 Domaszék 40 Pócspetri 30 Felsőzsolca Sásd 20 Tab 10 0 20002001200220032004200520062007200820092010201120122013 Source: World Bank staff calculations. Figure A2.2. Long-term unemployment—Nine random settlements (%, 2000–2013) 500, 450, 400, Milota 350, Gyöngyöstarján Kishartyán 300, Zalaszentgrót 250, Domaszék 200, Pócspetri 150, Felsőzsolca Sásd 100, Tab 50, , 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Source: World Bank staff calculations. 67 Figure A2.3. Access to sewage—Settlements with the lowest coefficient of variation 110, 100, Nemesvámos Almásfüzitő 90, Balatonszárszó Szekszárd 80, Tatabánya Komló Keszthely 70, Siklós Sárhida 60, Kincsesbánya 50, 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Source: World Bank staff calculations. Figure A2.4. Long-term unemployment—Settlements with the highest coefficient of variation 250, 200, Vasegerszeg Iván Árpás 150, Salköveskút Pusztavacs 100, Kup Örkény Kőszegdoroszló 50, Kéleshalom Vereb , Source: World Bank staff calculations. 68 Annex III. Mapping for results in Hungary Leveraging the approach introduced in Section 5.2 of this report, this Annex demonstrates an example of how to use subregional information on human development with a project data overlay. The example uses long-term unemployment data available at the microregional level, combined with project data from EU–co-funded interventions aimed at increasing employability and implemented in the 2007–2013 period. 35 As the starting point of the analysis, our first map (Figure 1) demonstrates the levels of long- term unemployment in 2007, as well as the locations and relative size of relevant employability interventions funded from TÁMOP 5 between 2007 and 2013. Figure A3.1. Microregional long-term unemployment rate (2007) and employability projects (2007–2013) Long-term unemployment rates have been calculated as the number of jobseekers registered for longer than 180 days, for every 1,000 working-age (18–59) individuals in each microregion. Employability projects are relevant TÁMOP 5 projects discussed in Table 7 of Section 5.2 (TÁMOP 5.1.1-09/1-2, TÁMOP 5.1.1-09/3, TÁMOP 5.1.1-09/6-7, TÁMOP 5.3.1-08/1 and 2, TÁMOP 5.3.1-C-09/2, TÁMOP 5.3.1-08/2, TÁMOP 5.3.9-11/1). Projects were geocoded and subsequently mapped. Bubble sizes correspond to the amount of project funds disbursed during the project cycle. Color scale gradients start at 16 (lightest) and extend to 97 (darkest). 35 The GIS tool used for this exercise is an ArcGIS Online platform (http://www.arcgis.com/online). This platform is designed to showcase multiple map layers for data visualization and analytical purposes (also used for mapping World Bank projects, available at http://maps.worldbank.org). Unemployment and population data for map layers has been obtained from TEIR, and joined to microregional shapefiles using ArcMap 10.3 software. Project information has been provided by the Prime Minister’s Office: the project data has subsequently been cleaned, geocoded, and uploaded to ArcGIS Online platform. 69 The map indicates a mixed territorial targeting accuracy of TÁMOP 5 projects aiming to promote employability between 2007 and 2013. Findings include: • employability interventions were focused on many microregions with high levels of long-term unemployment, particularly those in northeastern and southwestern Hungary—partly as a result of territorially targeting funds to the most disadvantaged microregions (TÁMOP 5.1.1) that are also the microregions with high levels of long- term unemployment; • some microregions with levels of unemployment in 2007 comparable to those receiving large volumes of projects have not benefited from EU funding at all, including Pétervásár (northern Hungary), Püspökladány (Northern Great Plain), and Nagyatád (South Transdanubia). It is important to note that none of these were among the most disadvantaged microregions—that is, areas targeted through TÁMOP 5.1.1; • many microregions with the country’s lowest levels of long-term unemployment (northwest and central Hungary) have also benefited from employability interventions; 36 • many microregions where long-term unemployment levels are not the highest, but are still high (in the northeast, east or southwest) have not received any EU funding at all. In the second step of the analysis, we look at the “final picture”—that is, long-term unemployment data in 2013, keeping the same project overlay. 36 This demonstration ignores the possibility that interventions may have also been driven by higher levels of unemployment among certain groups, such as youth, without affecting long-term unemployment rate overall. These issues could of course be revealed by further analysis of unemployment data disaggregated by geography, ethnicity, gender, and age. 70 Figure A3.2. Microregional long-term unemployment rate (2013) and employability projects (2007–2013) Long-term unemployment rates have been calculated as the number of jobseekers registered for longer than 180 days, for every 1,000 working-age (18–59) individuals for each microregion. Employability projects are relevant TÁMOP 5 projects discussed in Table 7 of Section 5.2 (TÁMOP 5.1.1-09/1-2, TÁMOP 5.1.1-09/3, TÁMOP 5.1.1-09/6-7, TÁMOP-5.3.1-08/1 and 2, TÁMOP 5.3.1-C-09/2, TÁMOP 5.3.1-08/2, TÁMOP 5.3.9-11/1). Projects were geocoded and mapped. Bubble sizes correspond to the amount of project funds disbursed during the project cycle. Color gradients start at 16 (lightest) and extend to 97 (darkest), equivalent to 2007 levels. The first noticeable change is how much “lighter” the colors have become all around the country: the fact that the color scale gradients of the 2007 and 2013 maps are normalized suggests a significant improvement in terms of long-term unemployment since 2007. Indeed, the average number of long-term unemployed per 1,000 working-age individuals in each microregion has considerably decreased overall, from 54 in 2007 to 41 in 2013. The map also demonstrates that territorial disparities have decreased somewhat—that is, the difference between leading and lagging microregions is considerably smaller overall. Moreover, there have been cases of “isolated” developments—that is, there are a few freestanding microregions in overall lagging areas where long-term unemployment has improved more than in neighboring microregions (for example, the Sátoraljaújhely microregion in the northeast or the Szigetvár microregion in the southwest). The GIS tool also provides an opportunity to showcase detailed project information about employability projects. By clicking on a project location bubble, a pop-up window can share key project information (location, title, target groups, amount, date, and so on) to external stakeholders. This feature enhances transparency, accountability, and also provides a 71 platform on which to disseminate and exchange knowledge about EU-funded employability projects. Figure A3.3. The platform offers detailed project information While the tool offers an approach that is easy to comprehend for policy audiences and clients alike, it is limited in terms of interpretability. Most importantly, the tool is unable to identify the extent of change that is in fact attributable to EU-funded employability interventions between 2007 and 2013. The gradually increasing size of the public works program since 2009 (in 2014, public works contributed approximately 1.5 percentage points toward the decrease in the Hungarian unemployment rate) has likely had a considerably higher impact on employment levels in many microregions. Also, private-sector investments in some lagging microregions may also have contributed to improving employment figures at the local level, which the tool is unable to capture. These and other dilemmas of the proposed approach are elaborated in Section 5 of this report. Nevertheless, the data visualization feature offers inputs to targeting and monitoring EU funds. It also offers a solid basis upon which to introduce a results-based approach to OP implementation in the 2014– 2020 period, as well as accompanying features that enhance transparency, accountability, and citizen feedback. 72