Pre-Primary Education in Mongolia Access, Quality of Service Delivery, & Child Development Outcomes March 2017 i Table of Contents Acknowledgements....................................................................................................................................... 4 Abbreviations and Acronyms ........................................................................................................................ 5 Glossary ......................................................................................................................................................... 6 Executive Summary....................................................................................................................................... 7 Chapter 1 Early Childhood Education in Mongolia: Operation & Financing ............................................... 11 1.1 Why invest in preschool? .................................................................................................................. 11 1.2 The basic structure of preschool education in Mongolia and operational characteristics .............. 14 1.3 Coverage of preschool services in Mongolia .................................................................................... 16 1.4 Public Expenditure on Pre-School Education .................................................................................... 20 1.5 The Pre-School Education Financing System .................................................................................... 23 1.6 Provision of meals and school supplies: A closer examination......................................................... 24 1.7 Conclusions ....................................................................................................................................... 25 Chapter 2 Quality of kindergartens in Mongolia ........................................................................................ 28 2.1 Assuring quality of kindergarten services in Mongolia ..................................................................... 29 2.2 Measuring quality in Mongolian kindergartens: The World Bank Kindergarten Survey 2015 ......... 30 2.2.1 Structural characteristics of sampled kindergartens ................................................................. 33 2.2.2 Quality by kindergarten type and location ................................................................................ 37 2.2.3 Details of performance on the ECEMI quality subscales ........................................................... 39 2.3 Conclusions ....................................................................................................................................... 43 Chapter 3 School Readiness Among 5-year Old Children in Mongolian Preschools .................................. 47 3.1 The 2015 World Bank Kindergarten Survey: Data collection, sample characteristics, and ECE outcomes ................................................................................................................................................ 48 3.1.1 Survey sample and data collection ............................................................................................ 48 3.1.2 Household characteristics of 5-year old children enrolled in kindergartens............................. 51 3.1.3 The household environment and parental engagement at home ............................................ 55 3.1.4 ECE outcomes among five-year old kindergarten-enrolled children ......................................... 58 3.2 ECE outcomes in a home-based school preparation intervention in Mongolia ............................... 64 3.3 ECE outcomes among children in the MICS ...................................................................................... 68 3.4 Conclusions ....................................................................................................................................... 71 Chapter 4 Key Action Areas Moving Forward ............................................................................................. 75 Key action area 1: Improve access to ECE in rural areas through a tested community- or home-based ECE modality ........................................................................................................................................... 76 1 Key action area 2: Improve access to formal ECE services in urban areas, with a potentially expanded role for the private sector ....................................................................................................................... 78 Key action area 3: Improve quality of fixed public kindergartens through investments targeted to rural areas ........................................................................................................................................................ 79 Key action area 4: Undertake further analytical work to examine ECE financing norms, and address gender and other disadvantages in early development outcomes ........................................................ 81 References .................................................................................................................................................. 82 Annexes ....................................................................................................................................................... 86 Annex 1 History of preschool education and alternative programs in Mongolia................................... 86 Annex 2 Determinants of preschool enrollment in Mongolia: Regression output table ....................... 88 Annex 3 Herders’ children enrolled in preschools in Mongolia.............................................................. 89 Annex 4 List of school and sanitation supplies contributed by parents of kindergarten children ......... 90 Annex 5 Kindergarten exposure among 5-year old children enrolled in Mongolian kindergartens, 2015 ................................................................................................................................................................ 92 Annex 6 Construction of the household wealth index in the WB kindergarten survey ......................... 93 Annex 7 Comparison of the 2015 WB quality survey sample with the 2010 MICS ................................ 95 Sample of children enrolled in public and private preschools............................................................ 96 Sample of children enrolled in ger kindergartens............................................................................... 96 Annex 8 Distribution of normalized scores in child development domains tested under the MELQO instrument among 5-year children in Mongolian kindergartens, 2015.................................................. 99 Annex 9 Relationship of household-, individual, and school-level characteristics with normalized scores on the MELQO direct assessment instrument: Regression output tables ................................ 101 Cognitive development and language .............................................................................................. 101 Social/emotional development......................................................................................................... 103 Executive function/self-regulation.................................................................................................... 104 Fine motor development .................................................................................................................. 105 Annex 10 Influence of maternal education and other socioeconomic characteristics on MELQO child assessment scores................................................................................................................................. 107 Annex 11 ECE outcomes in the Save the Children Japan (SCJ) and ger-kindergarten samples: Notes and regression output tables ....................................................................................................................... 110 Distribution of ages ........................................................................................................................... 110 Family engagement at home ............................................................................................................ 110 ECE outcomes.................................................................................................................................... 111 Annex 12 Determinants of ECE outcomes in Mongolia: Regression output table ............................... 112 2 3 Acknowledgements This report was written by a team comprised of Rabia Ali (Task Team Leader, GED02), Moukim Temourov (GED05), and Takiko Igarashi (GED02). The Mongolia Marketing Consulting Group (MMCG) carried out the 2015 World Bank Kindergarten Quality Survey. Pagma Genden (GHN02) provided research support and extensive coordination with the Ministries of Education and Finance, and international development partners. Save the Children Japan (SCJ) conducted ECE assessments among children enrolled in SCJ’s home-based school preparation program. Ms. Genden, Tungalag Chuluun (GSP02), and Tsendsuren Tumee (formerly ECD officer with UNICEF Mongolia) provided extensive editing and quality assurance checks on the Mongolian and English versions of the report. Peer reviewers of the concept note and the draft report included Amanda Devercelli, Nazmul Chaudhary, and Sophie Naudeau. The work was carried out under the guidance of Harry Patrinos (Practice Manager, GED02), Luis Benveniste (former Practice Manger, East Asia Education), and James Anderson (Country Manager, Mongolia). The team thanks the following individuals both within and outside the Bank for valuable advice through the various stages of this report, from project concept through drafting of the report: the MELQO team and especially Amanda Devercelli (WB), Abbie Raikes (UNICEF), and Ivelina Borisova (Save the Children); Linda Platas and Tsendsuren Tumee (UNICEF) in planning for the report, and the tireless efforts of Mongolkhatan Gunsen (Mongolian Institute of Educational Research) in developing research questions and report themes suitable to the context of Mongolia. The draft report benefited extensively from feedback and comments provided by Elena Glinskaya, Luis Benveniste, Amer Hasan, and Xiaoyan Liang. A special thanks to Munkh-Orgil.A and Densmaa.B at MECS and Sanjaadorj.M at MOF for assistance in gathering data on ECE expenditures. The team is especially grateful to Ivelina Borisova (Save the Children) for leading the instrument adaptation workshop, with participation from Gereltuya Tsegmid and Suvddelger (SCJ), Myagmar.B (Mongolian State University of Education), Munkhzaya (Ulaanbaatar Education Department), Amirdai.G and Norjkhorloo.N (MECS), Sumiya O. and Dulamsuren.S (MMCG) for participating in the adaptation workshop and advising the data collection and framing of research questions, and continued support through survey preparation and administration. The team thanks the MMCG staff, under the leadership of Bum-Erdene and Davaasuren.Ch., for their management of the data collection process, timely responses to challenges faced in the field, and flexibility in accommodating the special needs of the study. The team also thanks the Bayan-Ulgii education department, including Serkhgul, for their efforts to educate the team on context of service provision in remote areas of rural Mongolia. On data analysis specifically, valuable advice was provided by Abbie Raikes, Natalie Koziol (University of Nebraska-Lincoln), and Souhila Messaoud-Galusi (formerly WB). Ulziimaa Erdene, Otgonbayar Yadmaa, and Anna Coronado from the World Bank provided administrative assistance throughout the preparation of the report, from initiation of activities with the adaptation workshop to ongoing engagement with the government and others in Mongolia. 4 Abbreviations and Acronyms WB World Bank UB Ulaanbaatar MECS Ministry of Education, Culture and Science MICS Multiple Indicator Cluster Survey ECD Early Childhood Development ECE Early Childhood Education SCJ Save the Children Japan 5 Glossary Aimag: An administrative district equivalent to a province. There are 21 aimags in Mongolia. District: The report discusses municipal districts (duureg), which are a second-level administrative unit that is separate from rural districts (soum). There are 9 districts in UB, six of which are contiguous. Ger: A portable tent structure traditionally used by herders for shelter. A ger has a collapsible circular wooden frame that is covered with felt and heated by a small stove. Soum: A rural administrative subdivision of an aimag. There are 331 soums in Mongolia. 6 Executive Summary Investing in high-quality ECE is a cost-effective strategy, and can yield returns of up to 10 percent per annum. Mongolia’s Sustainable Development Vision 2030 places early childhood education (ECE) at the starting point of the journey towards a “knowledge-based society and a skillful Mongolian”. The Vision’s stated goal is to ensure that every child is enrolled at pre-school education facilities, meeting standard requirements and providing the basis for learning the Mongolian language and culture. These goals are motivated by evidence from around the world on the long-term economic benefits of ECE. Cognitive and social skills, the latter defined as attentiveness, perseverance, impulse control, and sociability, drive education and life success. No surprise then that inequality in early childhood experiences and learning – arising from limitations in any of genetic, parental, and environmental resources at, before, or immediately after birth – that constrain development of these skills also produces inequality in ability, achievement, health, and adult success. Further, adverse early environments create deficits in skills and abilities that drive down productivity and increase social costs—thereby adding to financial deficits borne by the public. But the adverse impacts of these limitations can be overturned through investments in high-quality ECE that provide children and their parents the resources they need to properly develop the cognitive and Figure 1 Returns to a unit dollar invested in selected personality skills that create productivity. development interventions, early years to adulthood Short-term costs of such interventions are more than offset by the immediate and long- term benefits through reduction in the need for special education and remediation, better health outcomes, reduced need for social services, lower criminal justice costs and increased self-sufficiency and productivity among families. And, the earlier the investment, the greater the return (figure 1). A critical time to shape productivity is from birth to age five years, when the brain develops Source: Heckman (2008) rapidly to build the foundation of cognitive and character skills necessary for success in later life. Despite impressive gains in coverage, Mongolian children in greatest need of ECE are still not covered. The size of government spending on ECE indicates that the subsector remains a priority for Mongolia and absorbs 20 percent of the country’s education expenditure, a relatively high share among peers and globally. ECE services grew rapidly in the last two decades, with formal preschool services now covering about 70 percent of children aged 3-5 years. Another 9 percent of children are covered under alternative preschool services. There are no gender gaps in access. But challenges persist. • Geographic and ethnic gaps remain large (figure 2). Enrollment rates in rural areas lag behind those elsewhere in the country, standing at a mere 46 percent among children between the ages of 36- 59 months (UNICEF, 2015). UB lags behind aimag and soum centers by a sizable 10 percentage points. The 7 enrollment rate among Kazakh households is a full 24 Figure 2 Percentage of children 36-59 percentage points lower than among those in the majority months enrolled in a formal ECE program, Khalkh ethnicity. Mongolia, 2013 • Household wealth is a key determinant of preschool 100 80 Percent of chilren (%) enrollment, with children from households in the poorest 60 quintile of wealth almost 40 percentage points less likely to 40 enroll in preschool than those from the richer quintiles. 20 0 • Herders’ children remain underrepresented in the Aimag center Khalkh Kazakh Other Capital city Soum center Rural center system, relative to their share in the country’s population. • Given that ECE enrollment rates are lowest among the most socioeconomically disadvantaged in the country and those with the lowest wealth, the largest share of the public Location Ethnicity of subsidy to ECE is accruing to the wealthiest. household head Assessment of early development outcomes among Source: UNICEF (2013) kindergarten-enrolled children reveals large socioeconomic gaps in school readiness. Children in rural public kindergartens lag behind those in urban areas in a number of cognitive and non- cognitive abilities (figure 3). The socioeconomic status of a child’s household exerts a significant influence on development outcomes after accounting for kindergarten-level factors including quality and exposure. Other than kindergarten quality, attainment of a college degree by a child’s mother was the only factor significantly associated with higher outcomes in all five domains of early development tested, after controlling for the full set of household-, individual- and kindergarten-level factors. Being male, of Kazakh ethnicity, or needing special assistance in the classroom was associated with poorer outcomes. Figure 3 Mean normalized child assessment scores Further, existing alternative ECE services are not among 5-year old children enrolled in fixed public sufficient to close the school readiness gap in hard- kindergartens, by location, Mongolia, 2015 to-reach populations. Among these, mobile or ger 0.50 UB kindergartens are most commonly used to target the 0.40 Aimag center 0.30 Soum nomadic population. Cognitive and non-cognitive 0.20 skills among children enrolled in ger kindergartens, 0.10 0.00 more than halfway through the three- to four-week -0.10 operation of these kindergartens, lag significantly -0.20 behind those of children in fixed kindergartens. Thus, -0.30 while ger kindergartens are received enthusiastically by parents and help increase access to ECE in the herder population, the short duration of exposure offered drastically reduces the chances that enrolled children will catch up with others. Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) Key action areas moving forward Action area 1: Expand access to preschool services in rural areas, prioritizing community- or home-based modalities for hard-to-reach populations such as nomadic herders 8 This report shows that increasing intensity of family engagement with children on activities at home, such as reading, singing, and playing together, is associated with significantly higher outcomes in numeracy and language, underlining the potential for a home-based ECE modality to improve school readiness among hard-to-reach populations. One such intervention, implemented in Mongolia by Save the Children Japan (SCJ) between 2012 and 2016, aimed to enhance school preparation through use of a home-based curriculum, capacity building and technical support for parents and teachers, and community mobilization. Participating children exhibited ECE outcomes considerably superior to those of comparable children in ger kindergartens, after accounting for social, economic, and individual characteristics. The intervention was far more effective in improving school readiness among vulnerable children than alternative preschool services currently employed. Action area 2: Improve access to formal ECE services in urban areas and relieve classroom congestion, with a potentially expanded role for the private sector In urban areas, further expansion of fixed kindergarten services is needed, through construction of new public kindergartens. Given the tremendous growth in the private sector, public financing of privately- owned kindergartens is also an option, especially in Ulaanbaatar. This expansion of services is critical, as Space constraints are common: One in three kindergartens doesn’t have sufficient space for indoor play activities, while only half have adequate gross motor play equipment. Kindergarten group sizes are very large and unwieldy across the country, with a mean of 42 children per group, rising to 50 in urban areas. Notably, in center-based ECE programs reporting high long-term returns in the US, the teacher-student ratio is merely 1:6. Second, when considering the potential for an expanded role for the private sector in provision of kindergarten services, incentives need to be designed to encourage quality rather than just increased enrollments. This report shows that private kindergartens serving a richer population in non-ger areas of UB are of higher quality and produce better outcomes than those serving the poorer, ger areas. Thus care should be taken to provide the right incentives, oversight, and/or technical assistance to ensure service provision of adequate quality, and minimized quality differences. Action area 3: Improve quality of fixed public kindergartens through investments targeted to rural areas A quality assurance mechanism is needed to systematically monitor compliance with quality standards and assess quality in the public and private sectors. Without regular monitoring for compliance with existing standards, or assessments of quality, it is not possible to identify areas for improvement for targeting of future investments. One option is to develop a quality scorecard for use at the kindergarten level. The data collected through the scorecard could be collated by provincial-level or other local education authorities, and then transmitted to MECS following the existing procedures governing flow of data underlying the annual production of the education and science statistical yearbook. In order to improve the equity and effectiveness of ECE investments, quality improvement efforts should target rural areas. This report shows that kindergarten quality is the single most important factor determining level of school preparedness among children in kindergartens, and that simply enrolling children in kindergartens for longer durations or at younger ages, at existing levels of quality, is not enough. Rural kindergartens, in particular, lag behind in all aspects of quality. Reductions in the urban- rural gap in child development cannot be achieved without increased financing for learning materials, improving their variety and the resulting range of classroom activities offered. Overall, there is scope for 9 improvement in about a third of kindergartens. Further, quality improvement efforts should focus on the math and numbers environment, since kindergartens across the country perform lowest on this subscale – taking care, of course, to do this without drilling children on numbers, rote learning, or creating a high- pressure environment in the classroom. Action area 4: Undertake further analytical work on financing norms and demand-side financing ECE financing norms and expectations about out-of-pocket contributions need further examination in the context of overall financing of the education sector, with an eye towards improving equity. Existing financing norms based on location or type of kindergarten have not effectively addressed quality differentials. Out-of-pocket payments for ECE also lead to higher financial burden on families from remote areas, herders, and others of low socioeconomic status. Further analytical work could explore mitigation options through subsidies to providers (public or private) or parents to access services, with varying degrees of targeting. Demand-side solutions such as conditional cash transfers or vouchers should be considered. Linkages may be possible through existing social welfare programs, for example the Child Money Program. Programs can be targeted to, weighted in favor of, or means-tested for disadvantaged families. Measures can include prescribing income eligibility rules for public ECE services, and settling the amounts of co-payments. Education providers may also receive supplementary per capita funding to accelerate educational outcomes in target population groups. Table 1 Summary of key action areas to improve the reach and quality of ECE services in Mongolia Key action area Actions Outputs or impacts desired 1. Improve access to ECE in rural Deploy a community- or Improve school preparedness in hard-to-reach areas home-based modality populations such as nomadic herders targeting the household environment Evaluate pros and cons of ger Improve efficiency of public spending on ECE; kindergartens relative to those improve access to ECE services in herder population of other ECE modalities 2. Improve access to formal ECE Invest to expand fixed Improved access to ECE services; reduce congestion services in urban areas, with a kindergarten services in the in classrooms. potentially expanded role for the public or private sectors private sector Prepare for expanded role of Reduce quality gaps in the private sector the private sector, and design incentives for quality 3. Improve quality of fixed public Develop a quality assurance Monitor compliance with quality standards and kindergartens through investments mechanism assess quality on a regular basis targeted to rural areas Target quality improvements Improve equity and effectiveness of current public to rural areas investments Improve the math and Improve cognitive outcomes numbers environment Improve exposure to numbers Reduce socioeconomic gaps in cognitive and and letters in classroom language outcomes practice, using existing child- centered approach 4. Undertake further analytical ECE financing norms and out- Generate information to enable design of targeted work in key areas of-pocket contributions interventions to improve equity of public Male disadvantage in early investments development Specific needs due to minority status and disabilities 10 Chapter 1 Early Childhood Education in Mongolia: Operation & Financing Sustaining spending on early childhood education (ECE) at 24 percent of its education budget, equivalent to about 1 percent of GDP, Mongolia is a relatively high spender on pre-primary education. Basic education has historically remained a priority investment, and within the sector, pre-primary education recognized as the stepping stone to school, critical to school readiness. The present chapter shows however that while Mongolia has demonstrated robust growth in access to ECE, challenges persist. Geographic and ethnic gaps remain large, and household wealth is a key determinant of preschool enrollment. Herders’ children remain underrepresented in the system, relative to their share in the country’s population. An examination of spending patterns and budget allocation reveals weaknesses. After covering the priority expenditure categories of salaries, meals, and utilities, budget constraints do not allow the state to cover most learning materials, including school supplies, teaching materials, extracurricular activities and other activities. Thus, kindergartens are left to their own devices to provide learning materials and supplies through contributions from parents, raising funds, or transfers from local governments. Differences in capacity to mobilize such local resources can lead to differences in quality of learning environment across kindergartens in different parts of the country. Also, given that ECE enrollment rates are lowest among the most socioeconomically disadvantaged in the country and those with the lowest wealth, the largest share of the public subsidy towards ECE is accruing to the wealthy. This chapter is structured as follows. Section 1 provides an overview of the rationale for public investment in ECE across the world. Section 2 describes key ECE services in Mongolia, while section 3 shows who has access to these services. Sections 4 and 5 review trends in public spending, and highlight the key characteristics of ECE financing in the country. Section 6 takes a closer look at provision of meals and learning materials and supplies, as these play a central role in determining quality of kindergartens. Section 7 concludes with key messages. 1.1 Why invest in preschool? Development in early childhood is an important predictor of success in adulthood, and investments in early development, including education, can yield high returns of up to 10 percent per year. The influences and factors affecting a child’s early years impact the development of skills that determine outcomes later in life (Cunha and Heckman, 2007; Heckman, 2006; Cunha et al, 2005; Carneiro & Heckman, 2003). Children who enter school ill-prepared are more likely to have poor academic performance, repeat grades, and drop out of school than children whose cognitive skills and overall school readiness are higher upon primary school entry (Heckman and Masterov, 2007; Reynolds et al, 2001; Feinstein, 2003; Pianta & McCoy, 1997; Currie & Thomas, 1999). Small-scale studies in Guatemala, South Africa, and Jamaica also show that children with low levels of cognitive development in early childhood do poorly in school (Stith et al., 2003; Liddell and Rae, 2001; Walker et al., 2005). Non-cognitive dimensions, particularly socio-emotional development, are also important (Heckman, 2007). Investments in ECE promote school readiness and better education outcomes (Lynch, 2005), and have been linked to life-long benefits for participants and society in general, including increased wage-earning potential, decreased incarceration, and lower reliance on social welfare (Schweinhart et al, 2005; Heckman, 2012). Investing in high-quality ECE is a cost-effective strategy, and can yield returns of up to 10 percent per annum (Heckman, 2008). And the earlier the investment, the higher the return (figure 1.1). Research from 11 multiple disciplines makes clear that outcomes in early childhood are malleable, although the window of opportunity may be short, especially for cognitive outcomes and nutritional status 1. ECE investments can also address early Figure 1.1 Returns to a unit dollar invested in selected inequalities in opportunity. While genetic development interventions, early years to adulthood influences account for half of the observed variance in cognitive abilities between children, the quality of a child’s environment, early stimulation and learning opportunities account for the other half (Fernald et al, 2009). Research from the US shows that gaps in cognitive and non-cognitive ability appear very early. At age 3, the difference in cognitive scores between children of college graduates and high school dropouts is almost 1.5 standard deviations, and the difference is Source: Heckman (2008) stable until (at least) 18 years of age (Heckman, 2008) 2. Average achievement gaps in math and reading between children in the top and bottom income quintiles are more than a full standard deviation at the beginning of kindergarten (Duncan and Magnuson, 2013). Comparable evidence for low- and middle- income countries is limited, but findings are similar 3. By the time children enter primary school, gaps in cognitive development are already apparent, with children from lower socioeconomic backgrounds lagging noticeably behind children from wealthier backgrounds. For many children, these gaps apparent at school entry continue to widen with age (Paxson & Schady, 2007; Grantham-McGregor et al, 2007). Through efforts to improve a child’s early environment, policymakers can impact children’s development and reduce gaps (Heckman, 2006). Preschool investments that focus on improving access without due consideration to quality of services will not improve early development outcomes, and may even cause harm. Monitoring and quality assurance is essential to protect children’s wellbeing and to promote their development and learning. A focus on access alone, or at the expense of quality, can jeopardize the very benefits that policymakers hope children will gain through preschool. Evaluations of preschools from settings as diverse as Guinea, Cape Verde, and Bangladesh record associations between measures of quality and cognitive outcomes 1 Almond and Currie (2011), and Cunha et al. (2006) are reviews for the United States; Engle et al. (2007; 2011) and Behrman et al. (2013) are reviews for developing countries that focus primarily on the medical literature. 2 See also Brooks-Gunn at al. (2006) and Cunha and Heckman (2007). 3 A comparative analysis of five Latin American countries (Schady et al, 2014) found large differences in language development among children across quartiles within countries, bigger than one standard deviation in urban Colombia and rural Ecuador, and larger than 0.75 standard deviations in the urban and rural areas of all five countries. Among poor children in rural Ecuador, differences in cognitive development between children of higher and lower socioeconomic status were substantial at young ages, including in vocabulary, memory and visual integration. The socioeconomic gradients in vocabulary increased between 3 and 5 years of age (Schady, 2011, which builds on Paxson and Schady, 2007). Studies on Indonesia, Madagascar, and Cambodia and Mozambique all find substantial socioeconomic differences in cognitive development at young ages, with gaps generally increasing with age for some, but not all, indicators of cognitive development (Indonesia: Jung and Hasan, 2014; Madagascar: Fernald et al., 2011; Cambodia and Mozambique: Naudeau et al., 2011. See also Halpern et al., 1996; Ghuman et al., 2005; Grantham-McGregor et al., 2007; Bouguen et al., 2013). 12 (Jaramillo & Tietjen, 2002; Aboud, 2006 and 2007). In contrast, if programs are not of high quality, the impact on children can be negligible, or even detrimental (Bernal, 2008 and 2010; Naudeau et al, 2011). In Mongolia, the commitment to ECE in financial terms is large, but very little is systematically known about the quality of services provided. Later chapters of the present report seek to fill this gap in knowledge. In sum, Mongolia’s investments in preschool education, described in this chapter, are motivated by evidence from around the world on the long-term economic benefits of investing in ECE. Basic education has historically remained a priority investment, and the within the sector, pre-primary education recognized as the stepping stone to school, critical to school readiness. Preschool education legislation and policies governing provision of ECE services in Mongolia set the stage for public investments (box 1.1). Two pieces of legislation are of note. One is the Law of Primary and Secondary Education, and the second is the Mongolian Pre-school Education Law. Box 1.1 Provision of preschool services in Mongolia: The regulatory environment Preschool education legislation and policies governing provision of ECE services in Mongolia set the stage for public investments. Two pieces of legislation are of note. The Law of Primary and Secondary Education laid out the objective of preschool education as helping children develop their minds, bodies and personalities by providing an educational environment conducive to development of talents, abilities and life skills. The Mongolian Pre-school Education Law stipulates that the provision of books, manuals and appropriate toys for children attending state-owned kindergartens as well as the norm-based variable costs, will be financed through the state budget. With recent amendments to the Law, parents are now obligated to cover part of meal costs. At the same time, the mandatory age for entry into primary school was lowered from 7 to 6 years, which heightened interest in adequacy of ECE services in the country to ensure school readiness at a younger age. Finally, the National Policy on Integrated Early Childhood Development (2005) has been adopted jointly by the Ministry of Education and Science, the Ministry of Health, and the Ministry of Social Welfare and Labor. The policy coordinates the efforts of state central organizations in charge of mother and child health care, nutrition, social welfare and protection, and education. In addition to encouraging a coordinated and comprehensive approach to provide quality ECD services, the Policy also emphasizes development of alternative forms of service delivery with greater participation of the private sector and other stakeholders, with a focus on the disadvantaged groups of remote rural children and children with disabilities (UNESCO, 2008) 4. 4 In addition to building kindergartens for children aged 2–5, the policy provides in-service training for all ECE teachers in Mongolia, consultation services for families, mothers, and women through pediatric and psychological home services, non-formal education services, telephone and internet counseling services, training and information campaigns on radio, television, in newspapers, and other print media. These outreach activities include counseling information on nutrition, hygiene, immunization, health care, and child-rearing practices that can stimulate social and cognitive development. 13 1.2 The basic structure of preschool education in Mongolia and operational characteristics Delivery of education services in Mongolia, and ultimately education outcomes, needs to be contextualized within the host of features that make the country unique. First, delivery of services in the public sector is challenging – and expensive – given the country’s sparse population and wide geographical spread. With just 1.7 persons per square kilometer of land, Mongolia has the lowest population density in the world. Of the total population of just above 3 million people, roughly a quarter are nomadic herders living in portable traditional tents known as gers, generally set up at least 5–15 km from each other. Climate conditions are extreme, with temperatures ranging from 40 degrees to minus 50 degrees Celsius during the year. These factors constrain not just the operational and logistical aspects of service delivery, but can also contribute to higher costs of providing services. The primary ECE service type in Mongolia constitutes conventional “fixed” kindergartens. Fixed kindergartens, which make up roughly 90 percent of preschool institutions in the country, generally operate eight hours a day, five days a week from September to June. They provide training for speech development, basic mathematical abstractions, music and singing, physical development and fine arts. In addition, they conduct excursions and health enhancement activities. A small number of kindergartens provide boarding facilities for children. Alternative preschool services are also provided, among which ger kindergartens are most common. Mongolia’s geographical spread has led to the creation of alternative preschool programs, in addition to the conventional fixed kindergartens. Another factor is the traditional nomadic lifestyle practiced by roughly a quarter of the Mongolian population, which makes it nearly impossible for ECE services to reach herders’ children. To overcome this barrier, the law on preschool education provides a legal framework for alternative forms of preschool services, which most public kindergartens offer today (box 1.2), and are implemented via three modalities: mobile teachers (or traveling teachers), mobile or ger-kindergartens, and shift classes (table 1.1). The most common among these are the ger-kindergartens, which operate as satellite kindergartens affiliated with fixed kindergartens. Table 1.1 Alternative preschool programs in Mongolia Alternative Pre-school Mobile teachers (Traveling Individual home visits, mapping, and Programs in Mongolia teachers) outreach Mobile ger-kindergartens Clustering families and deliver ECE using gers Shift classes Shortened day class for children living nearer to ECE facilities Source: Ministry of Education, Culture, and Science, UNICEF Box 1.2 Ger kindergartens: A Mongolian innovation for provision of alternative ECE services in the nomadic population The ger kindergarten is an innovative adaptation of the conventional kindergarten to the unique circumstances of Mongolia, and as such merits a more detailed description (annex 1 provides further information on how the concept of the ger kindergarten evolved). Ger kindergartens are designed to suit the socioeconomic and cultural setting of the nomadic population in Mongolia. Each ger- 14 kindergarten comprises two gers – traditional forms of dwelling in Mongolia made of hard wooden structures and thick felt – one for teaching and learning, and the other for cooking and sleeping. The life of a ger can be 10-20 years depending on the quality of materials. Inside the classroom, there are low tables and chairs, learning materials, and bed mats for napping. Each ger-kindergarten is intended to accommodate around 20-25 children aged between 1-5 years. Mobile ger-kindergartens operate in the summer only, and reach out to herders in locations where they tend to move to during the warmer summer months. The gers are loaded in a pick-up truck and driven to the countryside location where about 10-15 herding families can be found close to each other. It takes Mongolians a couple of hours to set up a typical ger. The ger kindergartens, along with teachers, are transported to rural and remote areas that are more easily accessible for herders and where their children are able to attend the kindergarten daily. The kindergartens start operations after the summer recess begins, and the teachers stay in the ger for the duration of its operation. As herders’ families are usually resident in a very isolated environment, the ger kindergartens provide not only early childhood education, but also the opportunity for children to socialize with other children. Ger kindergartens operate for a full 8-hour day but only for a limited time frame during the summer (June to August) when herder families’ move on the steppe in search of fresh grazing ground. Parents or relatives bring children in the morning and fetch them in the late afternoon, but may sometimes leave children with teachers overnight. The kindergarten will stay in the same location for three to six weeks, and sometimes will move with the herders to follow them. Ger kindergarten services are not typically offered outside the summer months. Ger-kindergartens are managed by the regular kindergartens they are attached to. MECS covers the cost of teacher salaries and other recurrent expenses, typically providing an operating budget of 21 days. Very limited supplementary resources from soum governments may be available to some kindergartens. In almost all cases, the gers themselves, along with the furniture and learning materials in them, were procured through donor-financed projects. The number of preschool establishments is increasing, largely due to a surge in the number of private kindergartens. Provision of ECE services has steadily grown. With 687 kindergartens operating in the country during the 2004-5 school year, the number rose by 87 percent to 1,288 in 2014-15 (figure 1.2). There were 675 mobile ger-kindergarten Figure 1.2 Trend in number of public and private classes in 2014, 200 shift classes, and 8 mobile kindergartens, Mongolia, 2004-14 teacher groups. A significant share of the growth in services came from the private 1200 30% sector, whose share in kindergartens rose 1100 Private Number of kindergartens Public 25% from 7 to 35 percent during this period. 78 1000 900 Share of private percent of private kindergartens in the 20% Percentage 800 country are located in Ulaanbaatar. 700 15% 600 Alternative preschool services, on the other 10% 500 hand, are provided by the state only. Despite 400 5% the increase in the number of private 300 200 0% kindergartens, the state still remains the major provider of preschool services, with public kindergartens registering about 90 Source: MES (2014) percent of enrolled children. 15 Kindergarten staff consist of principals, teachers, assistant teachers, methodologists, and others. The preschool education establishments, public and private, currently employ 24,179 people, including 6,833 teachers, 6,133 assistant teachers, 1,204 principals, 510 methodologists, and 9,499 support staff. While according to norms each group of children or classroom should be managed by a teacher, supported by an assistant teacher, in practice the ratio of assistant teachers to teachers is slightly under 1. The 2010- 2015 data show that the number of teaching staff (teachers, teaching assistants, and teachers of music) working in kindergartens has been increasing, particularly those working in private kindergartens. On the other hand, the number of teachers in delivering alternative preschool services has decreased as more children entered regular kindergartens. Table 1.2 Number of preschool education staff, Mongolia, 2015 Principals Methodologists Teachers Assistant Other Total teachers Staff Public 798 449 5,542 4,971 8,244 20,004 Private 406 61 1,291 1,162 1,255 4,175 Total 1,204 510 6,833 6,133 9,499 24,179 Source: MECS (2016) Table 1.3: Composition of preschool teaching staff, Mongolia, 2015 Academic year Teachers Of which: Assistant Group Music Shift Mobile teachers 2011-2012 4,907 4,241 479 94 93 4,326 2012-2013 5,332 4,632 516 88 96 4,769 2013-2014 5,895 5,177 530 96 92 5,249 2014-2015 6,158 5,446 568 72 72 5,589 2015-2016 6,833 6,052 639 65 77 6,133 Source: MECS (2016) 1.3 Coverage of preschool services in Mongolia The number of children enrolled in preschools has steadily increased in the past decade. Between 2005 and 2016, the total number of children enrolled in PSIs increased by an astounding 84 percent from 122,536 to 225,388, with 201,834 children attending kindergartens and 23,554 children receiving alternative preschool services (table 1.4). While preschool enrolment numbers have risen among children of all ages, growth has been particularly strong among children of 2 years old and younger. On the other hand, during the same period the number of children 6 years old enrolled in preschool has decreased as more children of this age entered primary school. Table 1.4 Number of children enrolled in preschool education institutions Academic year Number Age groups Up to 2 years 2 years 3-5 years 6 years 2011-2012 164,263 5,383 30,937 124,442 3,501 2012-2013 180,969 5,920 30,673 142,125 2,251 2013-2014 193,672 6,560 35,587 149,932 1,593 2014-2015 206,636 8,085 41,453 155,478 1,620 2015-2016 225,388 8,213 46,799 168,723 1,653 Source: MECS (2016) 16 The number of children receiving alternative ECE services has been largely stagnant, with a slight decline in recent years. Figure 1.3a presents the change in number of students enrolled in regular and alternative preschool programs. While the overall pre-primary enrollment is increasing in the country, the number of children enrolled in alternative preschool services has slightly declined since 2010. Various factors could be contributing to this decline, including increased availability of regular kindergartens, out-migration to urban centers, fiscal difficulty in operation of programs at the level of subnational governments, and quality of alternative preschool program. Figure 1.3a Trend in number of children enrolled in Figure 1.3b Percentage of children aged 36-59 preschools institutions, Mongolia, 2005-15 months enrolled in a formal early childhood education program, Mongolia, 2013 Number of children in fixed 225,000 100 kindergartens 80 Percent of chilren (%) 200,000 60 175,000 Number of children in 40 150,000 alternative pre-school 20 0 125,000 programs Aimag center Kazakh Other Khalkh Capital city Soum center Rural center 100,000 75,000 50,000 25,000 - Location Ethnicity of household head Source: MECS (2015) Source: UNICEF (2013) Mongolia’s preschool enrollment rate grew rapidly in the last two decades, and is now closer to that in significantly more productive economies than its middle-income peers. Within the context of expanding services described earlier, contact with ECE services has steadily risen among Mongolian children, with the gross enrollment ratio 5 in pre-primary education rising from under 40 percent in 1991 to over 80 percent during the 2014-15 school year. These official estimates include, however, contact with both formal and alternative preschool services. In the 2014-15 school year, the GER for formal or fixed services stood at 71.3 percent, with the remaining 9.2 percent in alternative preschool services. UNICEF’s Multiple Cluster Indicator Survey (MICS), which likely captures the formal or fixed share of preschool enrollments, echoes the official estimates. In 2013, the survey reported a preschool enrollment rate of 68 percent among children between the ages of 36-59 months, up from 58 percent in 2010, a sizable increase over a three-year period. This increase has placed Mongolia’s pre-primary enrollment rate closer to that of significantly more productive economics, in per capita terms, than to its middle-income peers (figure 1.4). In spite of growth in coverage, geographic and ethnic gaps remain large. Enrollment rates in rural locations lag behind those elsewhere in the country, standing at a mere 46 percent among children between the ages of 36-59 months (figure 1.3b). Notably, the enrollment rate in UB, the capital city, also lags behind that in aimag and soum centers by a sizable 10 percentage points. Clearly, the rural disadvantage is largest, but even in UB, access is a problem for some. The ethnicity gap is also very large: 5 The gross enrollment ratio (GER) is the number of children enrolled as a fraction of children aged 2-5 years. 17 the enrollment rate is a full 24 percentage points lower among Kazakh households than among those in the majority Khalkh ethnicity. Figure 1.4 Pre-primary gross enrollment ratios in Mongolia and selected Eastern European, Asian and Latin American economies 140 Pre-primary gross enrolment ratio (%) Chile 120 Ukraine Belarus Czech Rep. 100 Malaysia Latvia Moldova Romania Lithuania Slovakia 80 Bulgaria Hungary Slovenia Russia Poland Greece Vietnam China Argentina Venezuela Serbia Uruguay 60 Bolivia Turkmenistan Croatia Indonesia Kazakhstan Armenia Costa Rica 40 Paraguay Lao Macedonia Turkey Kyrgyz Rep. Azerbaijan 20 India Tajikistan 0 0 5,000 10,000 15,000 20,000 25,000 GDP per capita in 2015 (constant 2010 US$) Source: World Bank Development Inducators (2016) Figure 1.5 Children enrolled in preschools as a fraction of children aged 2-5 years, by aimag, Mongolia, 2014-2015 UVS in alternative pre-school TOV programs ARKHANGAI in KHENTII kindergartens SUKHBAATAR GOVISUMBER DUNDGOVI OMNOGOVI DORNOGOVI ORKHON ULAANBAATAR 0 10 20 30 40 50 60 70 80 90 100 110 120 Source: MES (2015) Enrollment rates in fixed kindergartens are highest in UB, Eastern, and Central regions of Mongolia, while rates in alternative preschool services are highest in the Western and Khangai regions. The GER 18 in fixed kindergartens exhibits considerable variation across aimags 6 (figure 1.5), ranging from 55 percent in Bayan-Ulgii and Ovorkhangai aimags to 93 percent in Govisumber aimag. In Ulaanbaatar, it stood at 74 percent. Alternative preschool services, including ger kindergartens, are concentrated in areas with large shares of the nomadic population and hence low enrollments in fixed kindergartens – these are broadly aimags in the Western Region and Khangai Region (North), where the GER of alternative programs is around 20 percent 7. Roughly one third to quarter of the children attending preschool are in the alternative system in these regions. Overall, coverage ranges from 1.5 percent in Darkhan-uul aimag to 27 percent in Bayankhongor aimag. Household wealth is a key determinant of preschool enrollment, and the Kazakh disadvantage remains sizable after controlling for household characteristics. Among household-level background factors, household wealth is the key determinant of whether a child is enrolled in preschool (annex 2). After controlling for the full set of household level factors available in the 2010 round of the MICS, the coefficients on the wealth index variables remain large and significant (figure 1.6). Children from households in the poorest quintile of wealth are almost 40 percentage points less likely to enroll in preschool than those from the richer quintiles. After controlling for the full set of economic variables, the Kazakh disadvantage is no longer statistically significant, but the magnitude of the coefficient remains sizable and large at 14 percentage points. There are no gender gaps. Despite increases in coverage among herders’ Figure 1.6 Relationship between probability of preschool enrollment and household wealth among children, they remain underrepresented in the children aged 36-59 months, Mongolia, 20108 preschool education system. In the last decade, there was a five-fold increase in children 0.6 Percent (%) enrolled from herders’ families. Coverage of 0.4 herders’ children in preschool education rapidly 0.2 increased in early 2000s but it has stagnated in recent years (annex 3). In 2014-2015, 14 0.0 percent of children receiving preschool services No predictors With predictors came from herder families. The share is low Quintile 2 Quintile 3 considering that the herder population is about Quintile 4 Quntile 5 (richest) a quarter of the country’s population. Source: Authors' calculations using MICS (2010) 6 Mongolia is divided administratively into Ulaanbaatar (UB) and 21 aimags (provinces). The aimags are divided into districts known as soums, and soums are divided into baghs (villages). UB is divided into districts and khoroos (city wards). 7 These two regions showed the lowest preschool attendance rate in the MICS data. 8 The heights of the bars equal the difference between the share of children attending preschool in households in a particular wealth quintile, and those in the poorest quintile (omitted). All differences are significant at the 1 percent level. Predictors include maternal education, gender of child, location, region, ethnicity and religion of household head, and type of dwelling, among others. 19 1.4 Public Expenditure on Pre-School Education Education’s share in government spending in the past decade has fluctuated 9. Historically, Mongolia gives education a high priority and spending on education constitutes the largest share of government resources. As share of GDP, government expenditures on education reached the peak of 8.5% in 2001, but they have been fluctuating since, more downwards in recent years, dropping to an estimated 4.1% in 2014. The wide fluctuations are due to changes in GDP associated with Mongolia’s highly volatile macroeconomic environment. Education expenditures, in real terms, have held steady. While public spending on education has consumed a signification portion of domestic revenues, reaching above 30% in 2010, this indicator has been also trending downwards decreasing to about 16% in 2014. Figure 1.7 Public expenditure on education, Figure 1.8 Public expenditure on education by Mongolia, 2000-14 education level, Mongolia, 2008-12 Education Expenditures, 2000-2014 Education Expenditures, by level (%) 35.0% 100 30.0% 25.0% 80 20.0% 60 15.0% 10.0% 40 5.0% 20 0.0% 2000 2005 2010 0 % of GDP 2008 2009 2010 2011 2012 % of Domestic Revenue (Excluding Grants) Pre-school General Higher Source: MECS (2015) Traditionally, expenditures in education have been weighted towards primary and secondary education, but recent data show increasing trends for spending on preschool education. General education accounts for about 70 percent of all students in the Mongolian education systems and spending on general education, a priority of the Government, makes up more than a half of overall education expenditures. The share of spending for higher education has been falling in recent years, reflecting a reduction of state subsidy to the sub-sector that the Government believes can rely most on private financing (through tuition fees). Government spending on preschool education, on the other hand, has been relatively consistent, 22-24% of total public education expenditures, as the government makes this more of a priority to encourage on-time enrollment of six year-olds to Grade 1. More recently, donors have been also allocating significant funds for this subsector. Preschool education in Mongolia is free and the state spends more on preschool education than most countries. In terms of both expenditure as a percent of GDP (1%) and as a percentage of total public expenditures on education (24%), Mongolia spends more on preschool education compared to other middle-income countries, including advanced OECD economies. 9 This section draws heavily upon the World Bank’s Mongolia Public Expenditure Review (2008). Updated information will be made available through the ongoing PER exercise expected to be completed in September, 2017; preliminary analyses suggest that the expenditure patterns have held consistently since 2008. 20 Figure 1.9a Nominal public expenditure on Figure 1.9b Real public expenditure on preschool preschool education, Mongolia education, Mongolia Public expenditures on education Public expenditures on education (MNT, million, nominal values) 956,095 (MNT, million, 2008 prices) 575,569 562,946 418,832 462,799 363,462 353,404 387,826 363,462 360,249 221,751 89,907 82,240 83,559 105,157 133,938 89,907 83,833 99,712 141,340 2008 2009 2010 2011 2012 2008 2009 2010 2011 2012 Preschool Education Preschool Education Source: MECS (2015) Figure 1.10 Preprimary spending as percent of government education expenditures, Mongolia and other middle-income countries Ukraine Tajikistan Russian Federation Republic of Moldova Peru Mongolia Kyrgyzstan Kazakhstan Guatemala Chile Armenia 0.0 5.0 10.0 15.0 20.0 25.0 Source: UNESCO (2014 or latest year available) Employee wages constitute the lion share of recurrent public expenditure on PSIs, followed by meals, utilities, and transfers to the private sector. Historically, capital expenditures made up a significant portion of government spending on education, including for preschool, but, as discussed later in this chapter, the 2012 Integrated Budget Law (IBL) delegated responsibilities and flow of funds for capital expenditures to local government authorities, so currently public spending on education covers only recurrent expenditures. For preschool education, the largest spending categories for recurrent costs are employee wages and associated social insurance benefits (65%); meals (18%); utilities (10%), and subsidies to the private sector (7%) 10. Given the country’s vast geography and harsh winters, the cost of heating, as part of utility costs, is a large recurrent spending item that all Mongolian educational institutions face. In addition, the government also spends a large share of its budget for providing meals to children enrolled in preschool. Note that learning materials and school supplies are included in the very small “other” category; as discussed later, these expenses are largely financed by parents at the kindergarten level. 10 According to the current system, the state transfers to private kindergartens cover only the variable costs incurred by the private kindergartens, estimated in the same manner as for public kindergartens, as explained later in this section. 21 Regular kindergartens absorb the lion share of public expenditure on ECE, followed by ger kindergartens. 88 percent of recurrent public expenditure on preschools goes towards operating regular fixed kindergartens. Ger kindergartens, the most common form of alternative preschool service used, absorb another 6 percent, followed by 24 hour kindergartens (2 percent), shift groups (2 percent), and mobile teacher services (< 1 percent). The per capita cost of providing preschool services varies across different types of establishments, highest for 24-hour kindergartens and lowest for shift groups. According to the 2015 expenditure data, the per-student average annual cost of providing preschool services is the highest for 24-hour kindergartens, followed by regular kindergartens, mobile preschools, shift groups, and mobile teacher services. The high cost of 24 hour kindergartens is associated with the fact that they provide a full range of boarding services (room, meals, utilities, etc.), while alternative preschools are less costly because they do not provide meals, like in the case of shift groups and mobile teacher services, or do not involve heating, like in the case of mobile preschools that function only during the summer. Figure 1.11 Composition of preschool expenditures in Mongolia Preschool Expenditures, 2000-15 (%) private Preschool Expenditures, 2015 sector subsidies/ wages 100 7% 90 80 70 60 meals 50 18% 40 30 wages and 20 bonuses 10 58% 0 2000 2005 2010 2015 heating & Salaries Social insurance electricity Office building expenses Office supplies & water 10% Meals Private sector subsidies benefits Other 7% Source: MECS (2016) Table 1.5 Expenditures by type of preschool establishments, Mongolia, 2015 Number of Total expenditures, As % of total Per-student average students, annual expenditures cost, annual annual average (MT, million ) MT, million) 24 hour kindergartens 3,094 3,944,581 2.19 1,275 Shift groups 6,563 3,716,323 2.06 566 Mobile preschool 17,696 11,649,373 6.46 658 Mobile teacher services 2,554 1,500,216 0.83 587 Regular kindergartens 154,788 159,414,866 88.45 1,030 22 Total 184,695 180,225,359 100.00 976 Source: MECS (2015) 1.5 The Pre-School Education Financing System 11 Under the IBL, provision of preschool education and general education in Mongolia is delegated to subnational government and financed by state and local budget. As prescribed in the law, the MECS develops short- and long-term Education Development Plans with the participation of subnational government and other related organizations. The Ministry’s plan, together with the norms and normative amounts set by the central government, serve as a base for developing short- and long-term education development plans by the aimags, soums, and districts. Following the approval of local education development plans by the regional Assemblies, the MECS estimates the amount of earmarked transfers within the budget constraint approved by the central government and sends it to the Education Department of the aimags. Earmarked transfers allocated by the central government are the main revenue source for educational institutions at the subnational level, including for preschool establishments. Under the 2011 BL, the aimag authorities are responsible for maintenance of the buildings of educational institutions since these buildings are local property, so educational institutions, including preschools, are also eligible for receiving funds from the Local Development Fund (LDF). Budget Allocation Budget allocations for preschool institutions are based on the per-capita principle of money following students. The education budget distinguishes between variable and fixed costs. Fixed costs cover items such as heating, water, electricity and sewage, which are allocated based on past expenditures. Variable costs cover teacher salaries and other recurrent expenditures (teacher training, books, postage, communication costs, etc.). The salary part of the per-student variable cost includes only the cost of teaching staff, while all support staff, an essential proportion staff for preschools, are included as part of the fixed cost. Salary levels for teachers in preschool education are comparable with those in the rest of the education sector, but their overall remunerations are lower because they do not have additional co- payments that school teachers receive for such tasks as reviewing homework, teaching extra hours, etc. Variable costs are allocated based on the location and type of school. According to the recent changes introduced by the government, the average per-student variable costs for kindergartens, both public and private, are set based on the three administrative locations in the country (Ulaanbaatar, aimag center, and other than aimag center). Traditionally, in order to provide incentives for the education staff working in rural areas, the variable costs for aimag and lower level districts are set at higher rates than in Ulaanbaatar. Once approved by the government, the normative means for kindergartens serve as a base for calculating variable costs for other types of preschools (24 hour kindergartens and alternative PSIs), when the adjustment coefficients are used to take into account specific features of alternative preschool services. The current average per-student variable costs and adjustment coefficients approved by the government in 2012 are provided in the tables below. Table 1.6: Average Per-Student Variable Costs in Kindergartens (‘000 MNT) 11 This section draws heavily on “Mongolia: Service Delivery in a Decentralized Environment” (World Bank, 2015) and “Mongolia Public Financial Management Performance Report” (World Bank, 2015). 23 Location Salary Costs Social Insurance Other Variable Costs Total Variable costs Benefits Other than aimag (province) centers 760.1 83.6 27.9 871.6 Aimag centers 616.7 67.8 25.1 709.6 Capital 615.0 67.7 22.8 705.5 Source: MECS (2015) Table 1.7: Adjustment Coefficients of Average Per-Student Variable Costs Type of students Salary variable cost coefficient Other variable costs coefficient Children of 24hr/boarding kindergartens 1.5 1.04 Disabled children 1.3 1.07 Shift group children 0.6 0.5 Mobile group children 0.7 0.5 Source: MECS (2015) The per-capita funding formula used in the education sector of Mongolia is relatively simple and transparent and is not overburdened with too many adjusters. International experience with per-capita funding formulas suggests that the simpler the formula the greater its transparency. A transparent formula has the advantage of being easily understood by the policy-making community and school stakeholders, minimizing the costs of data collection and analysis, and allowing for greater accountability at the local level because the amounts to be transferred are known. Although simplicity in a formula can come into conflict with equity or effectiveness considerations, which may require more adjustment factors, it is important to consider these trade-offs. 1.6 Provision of meals and school supplies: A closer examination Although provision of meals absorbs a significant share of resources spent on preschool, meals provided are still not of adequate quality. Currently the cost of meals in public kindergartens is fully covered by the state, and, as discussed earlier, provision of meals is one of the largest spending category for preschool establishments. The current norms for financing of meals for students in preschool establishments, along with the norms for students in primary education and in dormitories of general education and vocational training, were prescribed by the Government’s resolution from October 27, 2012. According to the resolution, the daily per-child cost of providing meals is set at 1,650 MT ($0.82) for regular kindergartens and at 2,400 MT ($1.20) for 24 hour kindergartens. While the government intended to adjust the norms annually for inflation, due to the financial constraints faced by the state, the norms have remained unchanged till now. To improve resources available for meals, recent amendments made to the preschool education law require parents to contribute to meal costs. Since the per-child amount allocated for meals by the state remains rather modest, it does not provide sufficient space for provision of nutritionally reach and diverse meals for children enrolled in kindergartens. The daily amount allocated for meals is not sufficient to provide children with three hot meals and two teas, as prescribed by the regulations, so parents were asked to make informal contributions to cover the deficit. To address this, the government has introduced reforms to formalize parents’ contributions for provision of meals. According to the draft resolution, parents will be asked to provide a daily amount equal to that currently provided by the state (1,650 MT), so that the total meal allocation per child is doubled. 24 School supplies and learning materials are largely financed by out-of-pocket expenditure by parents. With the government covering the priority expenditure categories of salaries, meals, and utilities, little is allocated to financing of school supplies, teaching and learning materials, extracurricular activities and other educational activities. In the absence of government funds, parents contribute a set of school and sanitation supplies per the list prescribed by the ministry of education (annex 4). This list of school supplies includes a variety of pencils, notebooks, markers, scissors, glue and other supplies, while the list of sanitation supplies include soap, toilet paper, toothpaste and brush, and other essentials. The amount of supplies are differentiated among different age groups and types of kindergartens, regular vs boarding. Thus, parental contributions make up the largest share of funds allocated for purchasing school supplies, organizing extracurricular activities, and even minor repair works in preschool establishments. With the new law requiring additional parental contributions to meals, out-of-pocket expenditure on preschool education will rise in the near future. 1.7 Conclusions The equity of public spending 12 on ECE should be improved through better targeting of ECE services to the rural and most disadvantaged populations. The finding that ECE enrollment rates are lowest among the most socioeconomically disadvantaged in the country and those with the lowest wealth, all concentrated in rural areas, indicates that the largest share of the public subsidy towards ECE is accruing to the wealthy. Going forward, improving access to ECE among the difficult-to-reach rural populations will be key to improving the overall equity of Mongolia’s public investment on ECE. Given the higher costs of reaching these populations through formal preschool services, existing alternative, non-formal ECE services should be carefully evaluated to explore potential for scale-up. Other sections of this report shed light on potential options. The ECE financing mechanism and allocation rules also need closer examination to explore potential for equity improvements. Existing norms for financing variable costs in kindergartens take into account location of a kindergarten to adjust salaries, social benefits, and other variable costs, and teacher salaries are also adjusted for disabilities among children. As chapter 2 will show, however, the norms are not leveling out regional inequities in quality. Further, no provisions are made to offset the financial burden on families, which is higher for those from remote areas, herders, and others of low socio-economic status. Instead, both rich and poor families with children enrolled in kindergartens contribute equally to learning materials needed in kindergartens, and in the near future, will do so for meals as well. To improve equity of the system, both the norms and expectations about out-of-pocket contributions should be examined, with due consideration to the welfare implications of each in the context of overall financing of the education sector. Budget constraints leading to low provision of learning materials and supplies may contribute to variation in kindergarten quality. Recent trends in government spending on preschool education confirm that the subsector remains a priority for Mongolia. Among its peers and globally, Mongolia spends a relatively high share of its education expenditure on ECE. Budget constraints, however, are evident when 12 A related question is one of efficiency of public spending on ECE, which cannot be answered solely on the basis of information presented in this report. It is generally accepted that the factors described earlier in this chapter – namely, geography and population density – make social sector service delivery expensive in Mongolia. But that doesn’t answer the question of whether there is a need to re-allocate resources within the subsector. This would require a comprehensive approach beyond the scope of this report, including an examination of expenditures in the education sector as a whole, an exercise planned as part of analytical work in the future. 25 examining the composition of expenditure. After meeting basic salary needs and other recurrent costs that are absolutely necessary for operation of PSIs, relatively little is left for learning materials and school supplies. As such, kindergartens must largely access learning materials and supplies through their own initiative, whether through contributions from parents, raising funds, or transfers from local governments. Given that this capacity to mobilize local or community resources likely varies greatly in different parts of the country, it may be that the material learning environment inside kindergartens – e.g. availability and quality of learning materials, their variety, the range of classrooms activities offered, etc., will be significantly different from place to place. Rural areas may be especially disadvantaged. Chapter 2 explores whether this is indeed the case. Revision of ECE financing norms should be considered together with feasibility of demand-side options in the context of overall financing of the education sector, with an eye towards improving equity. Private contributions to the system currently cover learning materials and supplies, and meals in the future. The welfare implications of the out-of-pocket expenditure should be monitored. Those currently out of the system are likely to be the most economically disadvantaged, so monitoring is needed to ensure that private expenditures are not prohibitively high or pose a barrier to entry into preschool. Clear state guidance is needed to better regulate private contributions in the sector. At the same time, demand-side solutions such as conditional cash transfers or vouchers could have a role to play in improving access and/or relieving the disproportionate burden of out-of-pocket costs among the poorest. Linkages may be possible through existing social welfare programs, for example the Child Money Program. Programs can be targeted to, weighted in favor of, or means-tested for disadvantaged families. Further analytical work should be undertaken to examine the feasibility of the demand-side financing options. Information on local capital investments in ECE should be centrally available for monitoring by MECS, especially given that physical infrastructure and maintenance needs at PSIs are often not met. Another area where budget constraints come into play is maintenance and upkeep of physical infrastructure in kindergartens. The expenditure analysis in this chapter shows that there is very little expenditure on this. In interviews conducted for this report, a number of kindergartens in UB and other locations reported that the resources they request for maintenance are not granted to them – as a result, many kindergarten buildings constructed in the 1960s and 1970s are in a state of disrepair. It is unclear how this situation will change in the medium run under decentralization of authority over budget decisions pertaining to capital expenditures. Monitoring local investments in preschool (and indeed other education subsectors) should be a priority for central government authorities. However, that MECS does not have access to these capital expenditure data indicates that such monitoring is not currently happening. The potential for an expanded role of the private sector in provision of ECE services should be explored further. In urban areas, and especially in UB, the private sector has played an important role in the expansion of preschool services, and, in light of fiscal constraints faced by the government, it would be useful to explore whether an appropriate mix of incentives could help bring in more private service providers to the sector. The growth of the private sector comes with opportunities and challenges. A starting point is to identify which segments of the population the sector is currently serving, e.g. just the rich, or a more diverse population, and how quality differs from that in public kindergartens. Chapters 2 and 3 will shed light on these questions. Alternative preschool services hold promise for reaching children of herders and others in remote areas, but further efforts are needed to explore cost effective modalities. Provision of alternative preschool 26 services, of which ger kindergartens form the largest share, is significantly cheaper in per capita terms than are regular services. However, it should be noted that ger kindergartens operate for very short periods of time, four weeks at most, in the summer only. The short period of exposure likely limits the impact on ECE outcomes. At the same time, increasing exposure time will make the intervention more expensive, and may also not be logistically possible. Chapters 2 and 3 elaborate on these issues, and encourage consideration of community- and home-based interventions that target the home environment as a potential alternative to formal kindergarten exposure. 27 Chapter 2 Quality of kindergartens in Mongolia Chapter 1 showed that recent trends in government spending on preschool education confirm that the subsector remains a priority for Mongolia, but budget constraints leave much of the responsibility for provision of learning materials and supplies to kindergartens themselves, and to parents. Different kindergartens in turn have varying capacities to mobilize resources locally, whether through contributions from parents, raising funds, or receiving transfers from local governments. Given this, it is likely that quality, both in terms of structure (kindergarten infrastructure and the material learning environment) and in terms of the process of service delivery (classroom interactions and opportunities for cognitive and non-cognitive development), will vary across kindergartens. These differences could then lead to differences in ECE outcomes across children. The differences in availability and quality of learning materials, their variety, range of classroom activities offered, etc., may disproportionally disadvantage rural areas or other locations where poverty is concentrated, e.g. ger-areas of UB. Fragmented evidence suggests that the ECE subsector in Mongolia faces problems stemming from overcrowding, poor infrastructure and lack of learning materials, and inadequate budget allocation at the facility level. Still, to date, there has been no systematic, facility-level assessment based on a standard set of criteria against which individual kindergartens can be assessed and compared. Further, there is very little information that delineates physical infrastructure and structural inputs from process quality in terms of teacher-student interactions and learning activities. Finally, no information is available on ECE in the private sector. The present chapter seeks to bridge these gaps in knowledge, and relies on the findings of a World Bank- financed survey that measured quality of public and private kindergartens in 2015. The survey carried out two rounds of data collection. One round implemented the survey in conventional or “fixed” kindergartens in urban and rural areas of Mongolia. Another round of was carried out in mobile or ger kindergartens, an alternative preschool service offered in Mongolia for a short period of time each year, targeted to the specific needs of children from the nomadic herder population largely outside the reach of conventional preschool services. Overall, kindergartens across the country performed highest on classroom interactions, and lowest on the numeracy environment. Fixed public kindergartens stood out for their large kindergarten group sizes, and a very large rural disadvantage in almost all aspects of quality assessed. While most kindergartens had a basic set of learning and play materials to support numeracy, literacy, and other activities, they often did not fully meet the standards assessed due to lack of variety and adequacy of these materials. Ger kindergartens outperformed fixed kindergartens on Interactions, but performed lower on all other domains of quality assessed. This is not surprising given the significant resource constraints and challenging service delivery environment that these kindergartens operate under. Private kindergartens performed similar to public ones, slightly ahead in Interactions, and lagging behind very slightly in other domains. Private kindergartens in ger-areas of UB lagged behind those in non-ger areas, but the differences were small in comparison with urban-rural quality differences in the public sector. This chapter is structured as follows. Section 1 describes the quality assurance system in place in Mongolia. Section 2 defines the structural and process elements of kindergarten quality in the context of the present report, and describes the World Bank Kindergarten Quality Survey, the data collection process, and results, disaggregated by type of kindergarten (public “fixed” kindergartens, private 28 kindergartens, and ger kindergartens). Details of performance on each of the quality subscales are also discussed. Section 3 concludes with some discussion on the main issues that emerge from the analysis. 2.1 Assuring quality of kindergarten services in Mongolia Monitoring and assuring quality of services is essential to protect children’s well-being and to promote their development and learning. Often, particularly in light of political and budget pressures, policymakers focus efforts on expanding access to ECD services. While expanding access is critical, it must be done with a commensurate focus on ensuring quality. A focus on access alone, or at the expense of quality, jeopardizes the very benefits that policymakers hope children will gain through preschool and other ECD interventions. Evaluations of preschools from settings as diverse as Guinea, Cape Verde and Bangladesh record associations between measures of quality and cognitive outcomes (Jaramillo & Tietjen, 2002; Aboud, 2007). In contrast, if programs are not of high quality, the impact on children can be negligible, or even detrimental. The analytical framework underlying the World Bank’s Systems Approach for Better Education Results (SABER) ECD diagnostic tool is useful for examining Mongolia’s ECE quality assurance functions 13. Within the framework, quality assurance is one of the three key policy goals of effective ECD systems, with three sets of actions or policy levers associated with it: (i) data availability; (ii) quality standards; and (iii) compliance with standards. This section provides an overview on each of these in the context of Mongolia. Survey and administrative data on access to ECE services and certain ECD outcomes are routinely collected and available in Mongolia. Administrative data are collected on access to ECE services, including on number of children enrolled in preprimary school (disaggregated by province and rural and urban areas), average number of students per teacher, and level of financial commitment to ECE. Mongolia participates in UNICEF’s Multiple Indicator Cluster Survey (MICS), and data are available on all early childhood development outcomes collected routinely under the survey rounds conducted in 2010 and 2013 14. Quality standards for ECE services are fairly detailed. National learning standards are laid out in the Kindergarten Curriculum, and areas of learning include motor, social emotional, language, and math skills, as well as interaction with the natural environment, art, and music. All kindergartens are required to follow the Curriculum. Teachers monitor children’s development, and record progress regularly through the school year in each area of learning. Quantitative targets for performance are not set. Guidelines exist governing interactions, including on promotion of positive behaviors, behavior management and discipline, and prohibition of corporal punishment. Physical environment requirements include standards for buildings, facilities, equipment, outdoor play, and separate sleeping rooms. Health/safety and nutrition/food service requirements also exist 15. There are no specific standards for community 13 Implementation of the full SABER-ECD tool was beyond the scope of this report. The Framework and its elements are used here only to anchor the description of Mongolia’s quality assurance functions for kindergartens. 14 At the time of writing, the final report for the 2013 MICS was available, but not the survey data itself. 15 In recent years, UNICEF has provided extensive support to schools and kindergartens in target areas across Mongolia to help draft emergency preparedness plans in conjunction with local emergency agencies (See UNICEF, 2014 and 2015). Also, most kindergartens in the country lack in-house sanitation facilities, and the cold weather prevents children from using outdoor facilities with adverse impact on health. To address this, UNICEF supported development of the national policy document, ‘Norms and Requirements for Water, Sanitation and Hygiene (WASH) in Schools, Dormitories and Kindergartens’, approved in 2015 with an official decree issued by the ministries of education, health, and finance. 29 engagement, but the national curriculum broadly emphasizes family and community engagement. Events are regularly held at kindergartens, showcasing, for example, children’s talents, and are attended by children’s families and local education authorities. Employment as a kindergarten teacher is possible after acquisition of a four-year Bachelor’s degree from a preschool teacher training institute. Kindergarten teachers are graduates of one of the twelve public or private teacher training institutes in Mongolia. Students aspiring to become kindergarten teachers enter these institutes upon completion of twelve years of education, and after passing an entry examination. Graduates of all teaching institutes complete four years of training to acquire a Bachelor’s degree. MSUE also runs a two-year kindergarten teacher program for students who already have received a higher education in another discipline (e.g. engineering) and wish to become kindergarten teachers. There is however no formal mechanism for centrally monitoring adherence to standards, especially with regard to inputs that directly impact skill development in the classroom. The Ministry of Education, Culture and Science (MECS) oversees all preschool institutions. Public and private kindergartens are inspected annually to check for compliance with physical infrastructure standards and safety requirements, but this is no more than an audit to ensure that kindergartens spend financing allocated in line with approved expenditure plans. There is no regular monitoring to ensure that learning environment needs, for example, school supplies such as print, art and play materials, are indeed met, or that teaching practice and methods are of adequate quality 16. The latter is largely conducted within kindergartens by principals or methodologists to advise teachers on improvements. Kindergarten administrators are not required to conduct formal self-assessments or prepare quality service plans. Overall, thus, there is no mechanism available to the Ministry for determining differences in learning environment needs or process quality across kindergartens. In particular, the standard for group size is not met, reportedly due to capacity constraints in kindergartens. On paper, staffing ratios are set at 25 children per kindergarten group. In practice, as discussed later in this report, group sizes are twice as large. In background interviews conducted prior to this study, school directors reported that demand for kindergartens is so high that parents line up for hours or sometimes days outside kindergartens prior to the enrollment day. In all kindergartens visited, principals showed waiting lists of parents with information to contact them in case a spot opens up. In such cases, kindergartens decide to enroll more children than required based on Ministry norms. 2.2 Measuring quality in Mongolian kindergartens: The World Bank Kindergarten Survey 2015 It is important to distinguish between the structural and process dimensions of quality. Structural dimensions of quality refer to the presence (or absence) of resources that can facilitate the interactions that should take place in a learning environment (Berlinski and Schady, 2015). They can include aspects related to infrastructure (space, lighting, furniture, and equipment); elements related to health, sanitation, and safety (health protocols, emergency procedures); the characteristics of educators and caregivers (their pre-service and in-service training, experience, salaries); and the characteristics of the group of children under their responsibility (size, age range, caregiver-to-child ratios). Process dimensions 16 In principle, provincial and UB city education authorities are required to carry out such monitoring regularly. There is however no mechanism for formally reporting findings and recommendations resulting from this monitoring, or collating them for policy planning purposes at the central level. 30 of quality refer to the elements of daycare that directly impact a child’s day-to-day experience, learning, and development. They focus on the implementation of the curriculum (if one is available) and, in particular, on the frequency, types, and quality of interactions between children and their caregivers, between children and their peers, and between caregivers and parents. In the WB survey, kindergarten quality was assessed using Save the Children’s Early Childhood Environment Monitoring Instrument (ECEMI). The modified version of the Early Childhood Environment Rating Scale (ECERS) used in this study was developed by Save the Children, and is henceforth referred to as the Early Childhood Environment Monitoring Instrument (ECEMI) 17. The ECEMI, like the ECERS, is an observation-based assessment of the learning environment of early childhood centers, and uses the quality domains or “subscales” of ECERS-Revised Space and Furnishings (arrangement of space both indoors and outdoors), Activities (materials and activities offered to children), Program Structure (the schedule of the day, including routines and activities) and Interactions (supervision and interactions including language that occur in the classroom) (ECERS-R, Harms, Clifford, & Cryer, 1998) 18, and ECERS– Extension Literacy and Math subscales (ECERS-E Sylva, Siraj-Blatchford, Taggart, 2003). These six subscales were singled out for attention because they form the basis of a preschool curriculum and prepare children for primary school. All together across the six subscales of the ECEMI, there are 28 items (table 2.1). Each item has a description of what is to be observed for each rating ranging from 1 (inadequate), 3 (minimal), or 5 (good). A final rating for each item is then determined based on a set of rules 19. The average score for each quality subscale is constructed by averaging the score across the items in that subscale. Then, scores on the seven subscales are averaged to provide a total (mean) ECEMI score. The ECEMI measures both structural and process dimensions of quality, but not all subscales allow a clear separation between the two. The Space and Furnishings subscale assesses aspects related to infrastructure like space, lighting, furniture, and equipment, and as such is a measure of structural quality. The Interactions and Program Structure subscales measure process quality. In contrast, the remaining subscales of Activities, and the Language and Literacy and Math environments measure and aggregate across both aspects of the quality. For example, the Activities subscale assesses not just whether a certain number of musical instruments are available in the classroom, but also whether these are used as part of daily classroom practice. The Math environment subscale assesses availability of appropriate print materials, and also measures whether the teacher incorporates expression of numbers or other math learning into daily activities. Structural aspects of quality relating to children, including age range, teacher- to-child ratios, kindergarten size, the age composition of classes, and length of teacher-child relationship, were not measured as part of the ECEMI, but separately in the survey. 17 The tool has been used and validated in more than six countries by Save the Children, but it was new to Mongolia before the assessment in 2015. The adaptation workshop for the ECEMI was led by Save the Children, with participation from MECS, UNICEF-Mongolia, and other local ECD experts. Prior to observation in the field, all raters were provided with training on the use of the ECEMI. This training included practice observations in kindergarten settings similar to those that would be rated as part of the study, and subsequent review of these practice ratings. 18 The full ECERS-R includes the subscales of Personal Care Routines, Language-Reasoning, and Parents and Staff. 19 Raters began by assessing whether the requirements for a score of 1 were met; if they were, the rater assessed whether the indicators for a score of 3 were met, and so on through the scale. The ‘stop-scoring’ method was used. Stop-scoring means that when all indicators for a certain score were not met, scoring was stopped and that score was awarded regardless of whether later indicators for higher scores were met. 31 Table 2.1 Early Childhood Environment Monitoring Instrument (ECEMI): Quality subscales and items Structural quality Process quality Structure and process quality combined Subscale Item Subscale Item Subscale Item 1. Space & Indoor space 2. Program Time 4. Activities Use of fine motor skill materials furnishings Structure table/schedule Furnishings Free play Art materials and use Arrangement Group time Music materials and use for play Gross motor 3. Disciplinary Block/construction games play Interactions practices Teacher-child Drama/imaginative play materials interaction Relationship Nature/science/health materials between children 5. Language Printed materials & literacy Books and writing environment Teacher reading with kids Poems Emergent writing Talking/listening 6. Math & Numbers and counting numbers Reading & expressing numbers environment Application of counting Distinguishing/comparing/collating Science/experiment Data on quality of services provided in kindergartens was collected in a sample of 171 public kindergarten groups (in 101 kindergartens) and 55 private kindergarten groups (in 50 kindergartens). Data collection took place in October, 2015, and kindergartens were sampled from across the five regions of Mongolia (table 2.2). Fixed public kindergartens sampled included 35 kindergartens in UB, and 66 from across eight provinces of Mongolia. In UB, locations were included to cover ger and non-ger areas. Urban growth in UB in recent years has largely taken the form of ger areas, where most of the poorest live, areas which lack adequate connection to network services and are heavily prone to natural hazards. Low- income residents that populate these areas build their own gers, but lack basic services and infrastructure provision, such as paved streets, parks, streetlights, water connection and sanitation 20. Kindergartens were randomly selected in each location from a census of public and private kindergartens. Outside UB, in each aimag, kindergartens were randomly selected from aimag centers (urban center of the aimag) and soums (villages) 21. Data on quality of services provided in kindergartens was also collected on a sample of 101 ger kindergartens. Ger kindergartens are the by far the most common type of alternative preschool service offered in Mongolia. Since these operate only during the summer months, when fixed kindergartens are closed, the data collection in these kindergartens took place between June and July, 2015. Also, these kindergartens are concentrated in regions of Mongolia where coverage of fixed kindergartens is 20 World Bank (2015) 21 Private kindergartens exist in urban areas only, and are largely concentrated in UB. Therefore, the sample has no private kindergartens in soums/villages. 32 considerably lower. Because of this, the locations selected for sampling were somewhat different for ger kindergartens than for fixed kindergartens. Table 2.2 Number of kindergarten groups observed in the WB Kindergarten Survey sample Region Aimag Ger kindergarten Public (fixed) kindergarten Private kindergarten Eastern Khentii 20 20 Central Tuv 10 4 Selenge 3 Dornogovi 9 14 1 Darkhan-uul 2 16 2 Khangai Ovorkhangai 18 11 1 Khuvsgul 24 11 Western Bayan-ulgii 15 18 Ulaanbaatar 74 51 Total 101 171 55 The findings in this section reflect conditions under which 5-year olds typically go to kindergartens in Mongolia, and are not representative of all enrolled children. There are two key points to note when interpreting the findings in this section. The first is that the ECEMI is implemented at the group level, not at the level of the kindergarten. In some kindergartens thus, more than one group of children was observed, leading to more than one set of ECEMI quality observations for that kindergarten. Second, the objective of the study as a whole was to measure outcomes among 5-year old children attending kindergartens, and in addition, measure quality in the kindergartens these children are enrolled in. So the number of groups observed in a particular kindergarten depended on the number of groups that had 5- year old children in that kindergarten. For example, if a kindergarten had two groups with 5-year old children, then both groups were observed and included in the survey. See chapter 3 for more details on sampling of children. Overall, care should be taken to interpret the findings in this section as representative of the conditions under which 5-year olds typically go to kindergartens in Mongolia, and not as representative of all enrolled children (i.e. those of other ages). 2.2.1 Structural characteristics of sampled kindergartens Children’s group sizes in fixed public kindergartens are very large, particularly in urban areas, while those in private kindergartens are significantly smaller. Kindergarten group sizes vary greatly across the country, and across types of kindergartens (table 2.2). In public kindergartens, the average number of students enrolled per group stood at 42 22. The number further varies significantly across regions, hovering just above 50 in UB ger and non-ger areas, to 37 in aimag centers and 34 in soums. Overcrowding of classrooms is thus a particularly large concern in UB, but even outside UB, group sizes are very large. Group sizes in ger kindergartens, as expected given the kindergartens’ size and purpose, are smaller, with an average of 22 children enrolled per group in soums and 27 in aimag centers. Even ger kindergartens, thus, are generally enrolled at or above their maximum capacity of 20. Only one of the 100 ger kindergartens sampled had less than 10 children enrolled. 22 In public fixed and ger kindergartens, each group has one teacher and one assistant teacher. 33 Group sizes in private kindergartens are significantly lower, varying from a low of about 20 children enrolled per group in non-ger areas of UB and Aimag Centers, to somewhat larger at 25 children per group in UB ger areas. Table 2.3 General characteristics of kindergartens in Mongolia, WB Kindergarten Survey, 2015 Public-ger Public – fixed Private kindergarten kindergarten kindergarten Location of kindergarten UB non-ger areas 0 31.6 56.4 UB ger areas 0 11.7 36.4 Aimag center 7.1 28.7 7.3 Soum 92.9 28.1 0 Total 100 100 100 No. of staff employed in kindergarten Mean Median Mean Median Mean Median UB non-ger areas 32.5 28 10.8 9 UB ger areas 31.5 32 10.6 9 Aimag center 25.7 27 28 27.5 10.7 7 Soum 2.6 2 18.8 19 Total 4.2 2 26.3 25 10.7 9 Kindergarten group size Mean SD Mean SD Mean SD UB non-ger areas 51.481 9.31 19.839 6.357 UB ger areas 52 9.947 25.15 8.261 Aimag center 27.714 4.889 36.51 7.433 20.5 3.317 Soum 22.308 5.385 34.021 8.091 Total 22.694 5.508 42.351 11.775 21.818 7.318 Percent children present today Mean SD Mean SD Mean SD UB non-ger areas 81.912 9.212 79.697 15.195 UB ger areas 78.37 13.604 75.747 17.401 Aimag center 76.268 16.712 86.79 8.096 76.368 10.656 Soum 69.897 15.814 79.477 9.841 Total 70.352 15.875 82.212 10.117 78.019 15.648 Source: Author’s calculations using data from the World Bank Kindergarten Quality Survey (2015) Size of public kindergartens declines from urban to rural areas, while private kindergartens are significantly smaller. Overall the size of kindergartens, proxied by the number of staff employed, also depends on the region of the country. Outside UB, school sizes are significantly smaller. The mean number of staff employed per public kindergarten was 26, ranging from 19 in soum areas to just above 30 in UB. Private kindergartens were roughly half as big, employing an average of 11 employees per kindergarten and no significant differences between ger and non-ger areas of UB. Ger kindergartens in soums employed between 2-3 staff members per each, which is consistent with the operation of ger-kindergartens (see chapter 1). Ger kindergartens in aimag centers reported employing a larger number of staff (average of 27) because they operated in tandem with a fixed kindergarten, with fixed kindergarten staff also working in the ger part of the kindergarten. In public kindergarten groups, 5-year olds are rarely mixed with children of other ages, unlike in private and ger kindergarten groups, where mixing is substantial. 34 In fixed public kindergartens, there is relatively little mixing of age groups in the kindergarten groups observed (table 2.4). In the groups sampled for the survey, a group on average consisted of 83 percent 5- year old children, 16 percent 3-4 year olds, and only 1 percent of 6 year olds. The averages do mask considerable variation across location. Compared to other areas, soums were the only area where kindergarten groups had a significantly higher share of younger children – an average of 23 percent of the group, compared, for example, to 15 percent in non-ger areas of UB. Table 2.4 Composition of kindergarten groups by age in Mongolian kindergartens, 2015 Public-ger kindergartens Public - fixed kindergartens Private kindergartens 3-4 year 5-year 6-year 3-4 years 5-year 6-year 3-4 years 5-year 6-year olds olds olds olds olds olds olds olds olds UB non-ger areas 15 84.5 0.6 44.1 54.3 1.7 UB ger 14.7 85.3 0 51.8 47.8 0.4 areas Aimag 83.2 14 2.9 11.5 87.7 0.8 30.5 67.5 2 center Soum 61.1 24.5 14.3 23.4 74.5 2.1 Total 62.7 23.8 13.5 16.3 82.7 1 45.9 52.9 1.2 Source: Author’s calculations using data from the World Bank Kindergarten Quality Survey (2015) Private kindergarten groups had significantly more mixing of age groups. On average, 53 percent of a group consisted of 5-year old children, 46 percent of lower ages, and 1.2 percent of 6 year-olds. The average share of 5-year olds varied from 48 percent of the group in UB ger areas to 54 percent in non-ger areas. Preschool teachers in Mongolia are trained to work with students of all age groups but not on how to work with mixed age groups all together in one kindergarten group. This suggests that teachers in private kindergartens may not be able to meet the specific needs of each age group. Ger kindergartens in stark contrast to both private and public fixed kindergartens had very low share of 5-year olds, and significantly more younger and older children. An average group consisted of 24 percent 5-year olds, 63 percent 3-4 year olds, and 14 percent 6-year olds. Such a high level of mixing of ages, in a context where teachers are not formally trained on working with mixed age groups, suggests that teachers are unlikely to meet the development needs of children. Children in fixed public kindergartens had been in the preschool of current enrollment longer than those in private kindergartens. Children in ger kindergartens had significantly lower prior preschool exposure. Data was collected on number of years each child tested had been enrolled in preschool. The 5-year old children surveyed in fixed public kindergartens had been in preschool an average of 27.2 month (median of 25 months) at the time of the survey (see annex 5). The mean number of years were higher in non-ger areas of UB and in aimag centers than in soums, with a difference in average of 3 months. Children in private kindergartens had been in preschool for a significantly lower average of 22.6 months (median of 25 months), with the average ranging from 20 months in ger areas of UB to 24 in non-ger areas. Interestingly, children in private kindergartens had been enrolled in their current kindergarten for a significantly lower duration of time than those in public kindergartens (15.3 months compared to 25.2 months). Further, within private kindergartens, duration of enrollment at current kindergarten varied significantly from 12.8 in ger areas of UB to 17.4 months in non-ger areas. In other words, not only have the children in private kindergartens had significantly lower exposure to preschool on average, they are 35 also likelier to switch kindergartens. This suggests that once children are enrolled in a public kindergarten, they tend to stay enrolled in one place, but not so in private kindergartens. Children in ger kindergartens had had little prior exposure to preschool. For 41 percent of children, the kindergarten they were currently enrolled in was their first-ever exposure to preschool. However, a fairly high share (59 percent) of children had had prior exposure to preschool of some sort. Among these, 50 percent had been enrolled previously in another kindergarten for 5 months or less, but on average, a child with previous preschool experience had attended preschool for 10 months. Teacher-student relationships in private kindergartens lasted a significantly shorter period of time than in public kindergartens, indicating lower stability in children’s preschool environment. On average, a 5-year old child in a fixed public kindergarten had been with his present teacher for 1.3 years, at the time of the survey (annex 5). The average varied from 1.1 year in soums to the highest observed of 1.5 years in aimag centers. The median was very close to the average. In private kindergartens, teacher-student relationships seemed to be much less stable. Not only had the average child been taught by his present teacher for an average of 0.6 years, almost half the duration in public kindergartens, 50 percent of these children had been taught by their current teacher for at most 0.1 year. This, combined with the finding that most children in private kindergartens had been enrolled in the current kindergarten for at least 15 months, suggests that teacher turnover in private kindergartens is significantly higher than that in public ones. Attendance was similar in public and private kindergartens, and time spent in school very rarely fell below 4 days a week. Attendance was significantly lower in ger kindergartens. The teacher of each kindergarten group reported the number of enrolled children present in class on the day of the survey visit. On average, about 80 percent of enrolled children were present in public kindergartens, with no significant differences between UB and non-UB areas. Attendance at private kindergartens was similar (78 percent of enrolled children), but slightly lower in ger areas of UB (75 percent). Ger kindergartens had the lowest share of enrolled children present, at an average of 70 percent. This is not unexpected, given the remoteness and special constraints of access in the population targeted by these kindergartens. For all children included in the survey, attendance in the last week was also recorded. On average, a child present in a public kindergarten on the day of the survey had attended kindergarten for 41 hours the week before, with little variation in the mean across rural and urban areas. In UB and aimag centers, 95 percent of children surveyed had attended kindergarten for at least 32 hours in the past week, and at least 24 hours in soums. In ger kindergartens, the mean number of hours attended last week stood at 58. This is very high, but many ger kindergartens report operating 7 days a week. Also, the variation in attendance in the ger- kindergarten population is higher than that in fixed kindergartens. 10 percent of the children attended kindergarten for 16 hours or less in the previous week, and 25 percent for at most 32 hours. Again, not surprising, given the special constraints of the target population. Attendance in last 7 days at private kindergartens was the same as that in public fixed kindergartens, with no differences in means across urban and rural areas. Children in private kindergartens in the UB ger area 36 had somewhat lower attendance, with the 5th percentile 8 hours lower than that in public fixed kindergartens. 2.2.2 Quality by kindergarten type and location Public fixed kindergartens performed best on interactions, and lowest on the math and number environment. Overall, public kindergartens perform well on the quality index as measured by the ECEMI, with a total mean score of 4.1 out of 5 (table 2.5). This implies an overall performance rating between “minimal” (3) and “good” (5). Performance on individual subscales sheds light on aspects that could be improved. The highest score (4.5) was achieved on the Interactions subscale. Space and furnishings (4.3) and program structure (4.2) followed closely behind. Activities stood at 4.17, comparable to the language and literacy environment score of 4.16. The lowest score was achieved on the math environment subscale (3.63). Table 2.5 Mean scores on ECEMI quality subscales in Mongolian kindergartens, 2015 Ger Public – fixed Private Mean SD Mean SD Mean SD Structural quality Space and furnishings 3.75 1.199 4.259 0.871 4.127 0.938 Process quality Interactions 4.74 0.63 4.548 0.778 4.642 0.669 Program structure 3.71 1.201 4.201 0.959 4.188 0.931 Structure & process Activities 2.798 0.988 4.171 0.871 3.918 0.929 quality Literacy 3.815 1.057 4.164 0.755 4.052 0.823 Math 3.118 1.126 3.633 0.99 3.506 1.008 Total mean ECEMI quality score 3.655 0.805 4.162 0.731 4.072 0.707 Source: Author’s calculations using data from the World Bank Kindergarten Quality Survey (2015) Except in Interactions, private kindergartens performed similar to public ones on all measures of structure and process quality, lagging behind very slightly. Private kindergartens performed very similar to public kindergartens, with an overall mean quality score of 4.07, lower than public schools by a mere 0.09 points. Private kindergartens scored lower than public ones in four of the five subscales, but the differences were generally small (ranging from 0.013 points in program structure to 0.25 in activities). The exception was in interactions, where private kindergartens scored 0.11 points higher. Ger kindergartens out-performed fixed kindergartens in Interactions, but scored significantly lower on all other measures of structural and process quality. Given the difficult situation under which ger kindergartens operate, it was expected that these would perform worse on the quality ratings than fixed kindergartens. Overall this was true: ger kindergartens were a full half-point lower on the total mean ECEMI score. Compared to fixed public kindergartens, ger kindergartens scored half-point lower on space and furnishings. In literacy, the difference was smaller, at a third of a point, and larger in math, at 0.50 points. The biggest difference was in the activities score, with ger kindergartens lagging behind a full 1.35 points. Surprisingly, the interactions score was 0.23 points higher than fixed kindergartens. While ger kindergartens clearly lag behind others, it is commendable that in spite of the tremendous constraints of delivering services in a difficult environment and unfavorable living conditions for kindergarten teachers, the kindergartens still achieve an overall rating of 3.6, significantly above the “minimal” rating of 3. Each subscale score (except Activities) ranges between the “minimal” and “good” rating. It appears that teachers make good use of the limited space, learning materials, and other resources available to them to create a supportive learning environment for children. Improvements may be possible, but the efforts made so far are impressive. 37 Across all subscales, among public kindergartens, rural kindergartens lagged significantly behind. The average scores on each subscale mask considerable variation across the type of area the kindergarten is situated in (figure 2.1). In general, fixed kindergartens in UB ger and non-ger areas perform similarly. Aimag center kindergartens lag behind significantly, while soum kindergartens perform the lowest. The soum kindergarten disadvantage is large, ranging from roughly a gap of half a point on the ratings for space, program structure, and math, to a third of a point on literacy. On activities, soum kindergartens are almost a full point behind UB kindergartens. Ger kindergartens in aimag centers perform better than those in soums. Ger kindergartens in aimag centers perform better than those in soums in almost all subscales, with the largest differences (0.9 points) in the space and activities category. Gaps on all other subscales range from one- to two-thirds of a point, except on interactions, where there is no significant difference between the two. This may very well be due to the fact that the “parent” kindergartens, under which aimag center ger kindergartens operate as “satellites”, have access to more financial resources than do parent kindergartens in soums. Figure 2.1 ECEMI quality scores in fixed public Figure 2.2 ECEMI quality scores in private kindergartens, by location, Mongolia, 2015 kindergartens, by location, Mongolia, 2015 5 UB 5 UB non-ger Aimag center 4.5 Soum 4.5 UB ger 4 4 3.5 3.5 3 3 Literacy environment Literacy environment Space/furnishings Space/furnishings Interactions Interactions Program structure Program structure Activities Math environment Activities Structure Process Structure & process Total Structure Process Structure & process Math environment Total Source: Author’s calculations using data from the World Bank Kindergarten Quality Survey (2015) Private kindergartens in ger-areas of UB lag behind those in non-ger areas, but differences are small in comparison with urban-rural quality differences in the public sector. When covering private fixed kindergartens in ger and non-ger areas of UB, one key expectation had been that we would find lower quality kindergartens in the lower-income neighborhoods. Indeed, the ger-area kindergartens in our sample do lag behind non-ger area kindergartens (figure 2.2), but the differences are far lower than differences between rural and urban kindergartens in the public sector (figure 2.1). Thus, overall, quality in private kindergartens is comparable to that in public ones, but public kindergartens exhibit far higher variation in quality depending on where they are located. 38 2.2.3 Details of performance on the ECEMI quality subscales ECEMI subscales assessing structural quality only The Space and Furnishings score ranged mostly between “minimal” and “good”, with improvements needed in about a third of public kindergartens. Scores indicate a connection with high student teacher ratios. Space and furnishings was the second highest-rated category among fixed public kindergartens. This is the category where issues of congestion and over-crowdedness in kindergartens tend to play out. Kindergartens perform generally well on measures of appropriateness of furniture, ventilation and light of indoor space, and existence of learning centers in the classroom (83-88 percent of kindergartens). But a significantly lower share has sufficient space for indoor movement and activities, and for appropriate positioning of learning centers in the classroom (73-78 percent). Only 66 percent had sufficient space for simultaneous indoor play activities, 54 percent had enough gross motor equipment, and 60 percent made any significant use of this equipment. Overall, 37 percent of the fixed kindergartens scored a perfect 5 or “good” rating on this subscale, and another 37 percent scored greater than or equal to 4 but less than 5. Private fixed kindergartens performed only marginally worse than public ones, but the notable differences were observed in existence of learning centers in classrooms (19 percentage points less likely) and appropriateness of positioning of these centers (14 percentage points lower). The only dimension where private kindergartens performed significantly better was on adequacy of gross motor equipment (8 percentage points more likely). Overall, 35 percent of private kindergartens scored a 5 or “good” rating on this subscale. Given the infrastructural limitations ger kindergartens operate in, it is surprising that they do almost as well as the fixed kindergartens on some dimensions of space. Roughly the same share of ger kindergarten as fixed ones had adequate indoor room for movement and space for simultaneous activities. But adequacy of space depends on the number of children enrolled. We saw earlier that some fixed kindergartens are extremely crowded. Thus in some instances the space available in a fixed kindergarten may be highly inappropriate for a group of 60 children, while a ger-kindergarten with more limited space may still serve very well for a group of 15 children. Not surprisingly, though, ger-kindergartens do lag behind considerably on a number of other dimensions. 11 percent fewer kindergartens have good ventilation and light, or child-sized furnishings. About 60 percent had appropriate learning centers, and roughly 40 percent had or made use of adequate gross motor equipment. Overall, only 25 percent of ger kindergartens scored a perfect 5 or “good” rating on this subscale, and another 31 percent scored greater than or equal to 4 but less than 5. ECEMI subscales assessing process quality only With the majority of kindergartens scoring a “good” on the Interactions subscale, this was the highest scoring category across kindergartens. Ger kindergartens out-performed fixed ones. Interactions was the highest scoring category across public, private and ger kindergartens. Teachers in between 80-87 percent of fixed public kindergartens used non-punitive discipline methods rather than physical punishment, showed warmth and respect towards children, expressed sympathy towards those in discomfort or distress, modeled good social skills, and supported children in developing appropriate social behavior with peers. A somewhat lower fraction of kindergartens (70 percent) were observed to be set up to avoid conflict, for example, duplicate toys were often not accessible or there was no protected 39 space for a child to play with a favorite toy. Overall, 63 percent of the fixed kindergartens scored a 5 or “good” rating on this subscale. While private kindergartens performed marginally better than fixed public kindergartens, striking differences emerged in comparison with ger kindergartens, which were significantly more likely (10-14 percentage point difference) to demonstrate these characteristics than public kindergartens. Private kindergartens performed only marginally better than public fixed ones. The only significant difference was the higher share of kindergartens set up to avoid conflict (80 percent). Overall, 66 percent of the private kindergartens scored a 5 or “good” rating on this subscale, and 67 percent of ger kindergartens. The Program Structure score ranged mostly between “minimal” and “good”, with improvements needed in about a third of public kindergartens. Scores indicate a connection with high student teacher ratios. The third highest scoring category was program structure. On free play, public kindergartens perform well: in roughly 80-90 percent, free play occurs for a substantial portion of the day, and supervision and access to toys is adequate. On scheduling, most kindergartens (64-72 percent depending on the specific indicator) strike a balance between structure and flexibility, variety in activities, and use of time, but there is room for improvement. On group time, ratings also tend to be low. Whole-group gatherings are limited to a short period in only 63 percent of kindergartens, and activities done in small groups in 68 percent. Routine activities (hygiene, etc.) were much more likely to be done in small groups (81 percent of kindergartens). Overall, 46 percent of the fixed kindergartens scored a 5 or “good” rating on this subscale, and another 21 percent scored greater than or equal to 4 but less than 5. Private kindergartens were generally similar to public ones in scheduling. In free play, the notable difference was that a significantly lower share of kindergartens (10 percentage points lower) had adequate toys and materials. In group time, 8 percent fewer kindergartens conducted activities in small groups. Overall, 36 percent of the fixed kindergartens scored a 5 or “good” rating on this subscale. Compared to public fixed kindergartens, ger kindergartens were at a significant disadvantage along almost all dimensions of program structure. 8-12 percent fewer kindergartens could demonstrate adequate balance between flexibility and structure and periods of waiting during transition in activities were long. 11-21 percent fewer had adequate free play time, supervision during free play, or adequate number of toys. Activities in small groups were similarly limited. For example, routine tasks were done in small groups in only half of ger kindergartens, 27 percentage points lower than in fixed public kindergartens. Overall, 31 percent of ger kindergartens scored a perfect 5 or “good” rating on this subscale, and another 18 percent scored greater than or equal to 4 but less than 5. ECEMI subscales assessing structural and process quality together The Activities score ranged mostly between “minimal” and “good”, with improvements needed in about a third of public kindergartens. Lack of variety in materials was the main constraint. Activities was the fourth highest scoring category. Most fixed public kindergartens perform well, with 60- 70 percent of kindergartens reporting adequate availability and regular use of materials for fine motor development, art, music, blocks or construction games, dramatic play, and nature and science materials. Ratings on the lower side were due to lack of variety in art materials used daily (62 percent had four or more types), absence of a corner reserved for blocks/construction games (57 percent had such a corner), 40 lack of variety in drama or imaginative play materials (44 percent could involve play on numerous themes) and in nature/science materials (57 percent had three or more types), and lack of organization of nature/science materials (64 percent had well-organized materials in good condition). Overall, thus, there is a significant share of kindergartens (30-40 percent, depending on the specific activity type) that do not have adequate materials, and many more still lack the variety in materials desired by international standards. Overall, 16 percent of the fixed kindergartens scored a perfect 5 or “good” rating on this subscale, and another 56 percent scored greater than or equal to 4 but less than 5. Private kindergartens lag behind slightly, and the disadvantages are concentrated in a few key dimensions. 18 percent fewer kindergartens have adequate variety of art materials, and 20 percent fewer have weekly music activities. Between 8-13 percent fewer have adequate variety or use of drama materials, and between 16-21 percent fewer have adequate availability or use of nature and science materials. Overall, 16 percent of the private kindergartens scored a perfect 5 or “good” rating on this subscale. Ger kindergartens are the furthest behind. Activities is the lowest scoring category for ger kindergartens; they perform significantly better on the language and literacy and math environment subscales. The differences on use in fine motor skills equipment/materials and blocks and construction games are sizeable, with ger kindergartens anywhere between 13 to 21 percentage points less likely to perform well on these dimensions than fixed public kindergartens, depending on the specific item and indicator in question. In art, the differences are huge, ranging from 24 to 32 percentage points less likely to either have or use the type of material in question. But the largest differences are in activities for drama, and nature/science activities. Ger kindergartens have far fewer art materials (41 percent have 2-3 simple musical instruments compared to 97 percent of fixed kindergartens), and 47 percent have at least one music and dancing lesson per week (97 percent of fixed kindergartens). Only 4 percent of kindergartens had singing activities; and a mere 10 percent had 2-3 music materials (compared to 81 percent of fixed kindergartens with 4 or more types of music materials). Between 20-27 percent of kindergartens had provision (space, adequate materials, time allocated) for drama or imaginative play comparable to that in fixed kindergartens, and between 18-24 percent for nature and science-related activities. Between 31-45 percent only have a minimal level of provision for imaginative/dramatic play compared to 85-93 percent of fixed kindergartens; 44-54 percent a minimal level for nature and science compared to 88-94 percent of fixed kindergartens. Overall, none of the ger kindergartens scored a perfect 5 or “good” rating on this subscale, and only 18 percent scored greater than or equal to 4 but less than 5. Clearly, given the space constraints and limited resources available for ger kindergartens, it is not reasonable to expect that they would have nearly the same number of materials or opportunities for activities available at fixed kindergartens. Indeed, the quality assessment reveals that many ger kindergartens provide a minimal level of activities, with less variety than fixed kindergartens. The Language Environment score ranged mostly between “minimal” and “good”, with improvements needed in about a third of public kindergartens. Lack of variety in materials was sometimes a constraint, and the main area for improvement was classroom set-up and engagement of children. Note: The ECEMI items assessed in this subscale do not intend to measure whether children are being pushed to read or recognize letters, or whether they are able to do so successfully. Instead, the intent is to observe whether the kindergarten provided an opportunity for exposure to books and other print materials, and that the kindergarten set-up allowed children the option to look at or read books by themselves. 41 Language and literacy was the second-to-lowest score achieved. Poems was the most highly rated category, with teachers in more than 86 percent of fixed kindergartens regularly reading poems, singing, and keeping children engaged. On other items, kindergartens perform at a mixed of “minimal” and “good” levels. 47 percent of kindergartens had a good variety of labeled pictures materials (5 or more examples), and 57 percent had a set up to assist children in recognizing words. Over 60 percent of groups had a good variety of books, and a corner where children can read independently. At least 70 percent were observed to have active engagement of children in reading activities, and opportunities for independent reading. Dedicated spaces for writing exist in all kindergartens (more than 80 percent), but engagement of children is somewhat limited (60 percent). Plans to facilitate discussion (talking and listening) were developed and used by teachers in less than half the kindergartens, and 60 percent or more displayed children engaged in discussions with the teacher facilitating. Overall, 16 percent of the fixed kindergartens scored a perfect 5 or “good” rating on this subscale, and another 55 percent scored greater than or equal to 4 but less than 5. Private kindergartens perform somewhat lower than public ones. Availability of a comparable variety of labeled pictures is higher (9-10 percent kindergartens more), but in most dimensions, private kindergartens perform somewhat lower. Between 10-16 percent fewer kindergartens perform at a comparable level of performance on teachers reading books with children and poems. Overall, 16 percent of the private kindergartens scored a perfect 5 or “good” rating on this subscale. Ger kindergartens perform significantly lower than both private and fixed public kindergartens, but the differences are not as pronounced as those in the Activities subscale. Between 5-15 percent fewer kindergartens perform as well as fixed ones on almost all dimensions of literacy. The three exceptions were variety of labeled pictures, child engagement in writing activities, and planning of discussions to facilitate talking and listening, on each of which a somewhat higher fraction of ger kindergartens were rated “good”. Overall, 24 percent of the ger kindergartens scored a perfect 5 or “good” rating on this subscale, and another 26 percent scored greater than or equal to 4 but less than 5. The Math and Numbers Environment score was limited mostly to “minimal”, and this was the lowest performing subscale. Most kindergartens lack of availability of basic learning materials, and offer limited opportunities for exposure to numbers. Note: The ECEMI items assessed in this subscale do not intend to measure whether children are being pushed to read or recognize numbers, or whether they are able to do so successfully. Instead, the intent is to observe whether the kindergarten provided an opportunity for exposure to books and other print materials, and that the kindergarten set-up allowed children the option to look at or read books by themselves. For fixed public kindergartens, math was the lowest rated subscale. On the math/number environment, kindergartens generally performed well – roughly 80 percent or more had a variety of developmentally appropriate materials of various types available and being used regularly. While counting practice and incorporation of counting into the daily interaction was happening at 73 percent or more of kindergartens, relatively few (40 percent) had teachers actively using numbers when communication with children. Fewer kindergartens (61-71 percent) had either the materials or engagement of children in reading or recognizing numbers, or in distinguishing, collating or comparing. Understanding or working with shapes was less common (49-53 percent), and engagement of children in observing or performing simple science experiments was very rare (13 percent). Overall, a mere 6 percent of the fixed kindergartens scored a 42 perfect 5 or “good” rating on this subscale, and another 43 percent scored greater than or equal to 4 but less than 5. Private kindergartens a little lower than public ones on all dimensions of this subscale. Gaps were observed in practice of counting activities, and distinguishing, comparing and collating materials and activities (9-10 percent fewer kindergartens). Overall, 6 percent of private kindergartens scored a perfect 5 or “good” rating on this subscale, and another 38 percent scored greater than or equal to 4 but less than 5. Ger kindergartens performed significantly worse than fixed ones on all dimensions of the math subscale. Particularly large gaps (more than 15 percent fewer kindergartens) exist in counting materials or their use, or reading and recognizing numbers. Verbal counting activities were very rare, with 33 percent fewer kindergartens conducting verbal multiple counting practices on a daily basis. Overall, only 7 percent of ger kindergartens scored a perfect 5 or “good” rating on this subscale, and another 24 percent scored greater than or equal to 4 but less than 5. 2.3 Conclusions Quality standards are in place, but there is no mechanism to systematically monitor compliance or assess quality in the public and private sectors. Mongolia has fairly well-developed standards governing both structural and process aspects of quality. Standards exist for kindergarten infrastructure and learning materials, and a kindergarten curriculum lays out expectations regarding child development in key learning areas. However, without regular monitoring for compliance or assessments of quality, it is not possible to identify areas for improvement for targeting of future investments. This chapter documented large differences in quality between urban and rural public kindergartens, and among private kindergartens in UB. To ensure that the observed quality differences do not become larger over time, and to target resources to address the most urgent needs, it will be important for MECS to develop a mechanism to monitor quality on a regular basis, in both public and private kindergarten across different locations. Among public fixed kindergartens, the rural disadvantage in structural and process quality is large and likely related to resource constraints highlighted in chapter 1. Chapter 1 showed that despite being a relatively high spender on preschool education, the state is unable to provide learning materials and supplies due to budget constraints. Given that public kindergartens across urban and rural areas are likely to have varying access to supplementary resources through local government or private sources, this may contribute to variation in kindergarten quality – especially in structural quality. The present chapter documents that such variation in quality does indeed exist, except in the process quality subscales of Interactions, in which most kindergartens perform very well. Moreover, the areas where kindergartens are less likely to meet ECEMI standards, i.e. achieve a rating of 5 (“good”), are precisely the areas where financing is limited, i.e. in availability of adequate learning materials, their variety, the resulting range of classroom activities offered, etc. With most fixed public kindergartens scoring between a “minimal” and “good” on each subscale except Interactions, there is scope for improvement in both structure and process quality. The analysis in this section has shown that a variable share of fixed public kindergartens fully meets the standards assessed in the ECEMI (between 6 – 63 percent of kindergartens scored a full 5 or “good”, depending on the 43 subscale) 23. Interactions (discipline, teacher-child and child-child interactions) are a particular strength of fixed kindergartens. While the dimensions of activities, space, and program structure are also strong, with between 67-74 percent of kindergartens acquiring a score of at least 4 on these subscales, there is room for improvement in the remaining third of kindergartens. Some of the dimensions of poor performance on the space and program structure subscales indicate a connection with high student teacher ratios described earlier. Table 2.6 summarizes the specific interventions that may be needed. Table 2.6 Specific areas for improvement in quality in fixed public kindergartens in Mongolia Subscale ECEMI Share of Specific interventions needed Score kindergartens (Ordered from most urgent to least) needing improvements Improve… Math & 3.633 20-30% Frequency of counting practices and incorporation of counting into daily numbers interaction environment Materials and engagement of children in reading or recognizing numbers 40-50% Materials and engagement in distinguishing, collating or comparing Understanding or working with shapes 60% Teachers' use of numbers when communicating with children 80-90% Engagement of children in observing or performing simple science experiments Language & 4.164 30-40% Child engagement in reading activities literacy Classroom set-up for reading activities (a corner where children can read environment independently, assistance in recognizing words) 40-50% Variety of labeled picture materials (5 or more examples) and books Teacher planning to facilitate discussion (talking and listening) Activities 4.171 30-40% Availability of materials for fine motor skills (art, music, blocks/construction games, dramatic play, and nature and science materials) Variety in art materials used daily (four or more types) and nature/science materials (three or more types); Create space for play reserved for blocks/construction games Organization of nature/science materials and maintain working condition 50-60% Variety in dramatic or imaginative play materials Program 4.201 30-40% Group size to increase small-group activity time and its frequency structure In scheduling, balance between structure and flexibility, variety in activities, and efficiency in use of time Space and 4.259 20-30% Space for indoor movement and activities, and appropriate positioning of learning furnishings centers in classroom Group size to ensure sufficient space for simultaneous indoor play activities 40-50% Availability and use of gross motor equipment Interactions 4.548 30% Classroom set up to avoid conflict between children (provide duplicate toys or protected space for favorite toy) Most fixed public kindergartens have a basic set of learning and play materials to support activities, but cannot meet the ECEMI standard of “good” due to lack of variety and adequacy of these materials. A constraint which kept many kindergartens from scoring a 5 (“good”) was lack of variety in learning or play materials. Basic materials most critically needed are provided; it is only in the adequacy and/or variety of these materials that kindergartens are found wanting. On the literacy environment subscale, while 23 Note that had quality been measured using the full ECERS-R instead of the ECEMI, the scores would most like have been lower, given that a “5” or “good” on the ECERS-R on some items has a somewhat higher number of criteria that need to be met, in order to achieve a rating of 5. 44 materials are not that much of a problem, classroom set-up and methods to expose children to reading need to be improved. Performance on both the structural and process aspects of the math and numbers environment is very low. Across all kindergartens, the poorest performance is on the math environment, where not even half the kindergartens achieve a rating of 4 (i.e. halfway between “minimal” and “good”). Constraints faced were both structural, for example, lack of availability of basic learning materials, and limited opportunities to read or recognize numbers and work with shapes, as well as process-related, for example, lack of exposure to numbers in general classroom interactions. This is needed in addition to verbal counting exercises, which were found to be standard in most (but not all) kindergartens. In recent years, Mongolia has made impressive strides in improving access to reading books and other language materials at the kindergarten- and primary school levels, in part thanks to external donor funding 24, but numeracy has not been emphasized to the same degree. The recently developed kindergarten curriculum reform also has relatively less focus on numbers and math skills when compared to social emotional development and motor skills. The difference in quality between private kindergartens in ger- and non-ger areas of UB highlights the need for design of incentives to encourage quality rather than just increased enrollments. Overall, the present chapter shows that, on average, private kindergartens lag behind public ones only slightly in both structure and process quality. the Interactions subscale is an exception, where private kindergartens perform slightly better. But there are concerns not captured by the ECEMI. First, substantial mixing of age groups in private kindergarten groups, in a context where formal teacher training does not include training on working with mixed age group classrooms, significantly limits the teacher’s ability to meet the children’s needs. Second, children in private kindergartens tend to switch kindergartens significantly more than those in public ones and teacher-student relationships last a shorter period of time. The data indicate that teacher turnover is significantly higher than that in the public sector, leading to lower stability in children’s preschool environment. This suggests that the consideration of an expanded role of the private sector recommended in Chapter 1, while perfectly reasonable, should be treated with caution, and any incentives offered should be designed to encourage quality rather than just improved coverage of preschool services. Further work is needed to understand the constraints faced by the private sector in improving quality of services in disadvantaged areas of UB. Private kindergartens in ger areas of UB visited for this study were small businesses owned by capital-constrained owners, who reported difficulty in paying rents and utility bills. The lag in quality relative to non-ger area private kindergartens may be linked to lack of resources and difficulties in making the operation of these kindergartens profitable, especially given that most private kindergartens are very small with small enrollments and group sizes. Another factor may be the population served by low-quality private kindergartens. If the catchment area is a low-income one, these private kindergartens may be enrolling children only from low-income households, whose constrained ability to pay fees will not permit enough revenue generation to invest in quality improvements. The quality differences observed between kindergartens in ger and non-ger areas of UB lend some support to this hypothesis. Chapter 3 sheds further light on this question. 24 The Rural Education and Development (READ) project implemented under financing from the World Bank is an example. 45 Ger kindergartens perform lower than fixed ones on all ECEMI domains of process and structural quality except Interactions, where they outperform fixed kindergartens. Given the significant resource constraints and challenging service delivery environment that ger-kindergartens operate under, it is not surprising that on most dimensions of quality, ger kindergartens lag significantly behind. Except for Interactions, in which they outperform fixed public and private kindergartens, performance on Space and Furnishings, Program Structure, Literacy, and Activities is low. In Space, this is due to limitations placed by ger structures on ventilation and light. In the Activities, Literacy, and Math environment subscales, the differences are due to lack of variety in materials available for activities. The fact that gers, along with all equipment and furnishings, must be moved from place to place to follow herders imposes practical limits on the number and variety of materials that can be transported. Finally, lower performance on Program Structure indicates that teachers’ plans for use of the time available in class could be improved. The brief duration of operation of ger kindergartens, and to a lesser extent, the limitations in their learning environments, caution against high expectations regarding their impact on ECE outcomes. Overall, these factors suggest that the impact of ger kindergartens in terms of improving ECE outcomes in disadvantaged populations is expected to be limited. Given the brief duration of exposure of four weeks or less, the quality of services provided, and the limitations on improvement in quality, it is unlikely that the impact of a ger kindergarten would come close to the potential effect of an average of a couple of years of attendance in a regular public kindergarten, especially given the superior quality exhibited by the latter. At present, there is no evidence available from anywhere in the world on positive impacts of preschool with less than two months of exposure, even in the best possible quality settings. The fact that ger kindergartens target the most disadvantaged populations in the country only makes the task all the harder. Thus, while ger-kindergartens have played an impressive role in expanding access to ECE, and have been enthusiastically received across the country, expectations regarding their potential impact on outcomes should be tempered and aligned with their duration and quality of the learning environments they offer. Chapter 3 sheds further light on the issue. The next chapter presents findings on outcomes among 5-year old children attending public, private, and ger kindergartens described in the present chapter. Of particular interest are gaps in outcomes between children from high- and low-income households, and whether preschool attendance has an effect on reducing any gaps observed. A key question to be explored is thus whether the variation in quality documented so far contributes in any way to any variation observed in ECE outcomes. Chapter 3 attempts to shed light on these questions by presenting findings on ECE outcomes among 5-year old children attending the public, private and ger kindergartens discussed in this chapter. 46 Chapter 3 School Readiness Among 5-year Old Children in Mongolian Preschools Chapter 2 showed that across and within regular public, private and ger kindergarten, differences in quality are sizable. There remains considerable scope for improvement, and the extent depends on the quality subscale considered. For example, while most fixed public kindergartens have a basic set of learning materials and supplies, they often do not meet standards assessed due to lack of variety and adequacy of these materials. The program focus on math and numbers is very low. Private kindergartens meet many of the same standards as public ones, and like public kindergartens, exhibit significant location-specific variation in quality. Finally, given the short duration of operation of ger kindergartens, they are unlikely to enable enrolled children to catch up to others. The present chapter takes the discussion on quality of preschools in Mongolia forward by adding to it what is now known about ECE outcomes among children enrolled in kindergartens. To do this, it relies on the findings of the same World Bank-financed survey described in Chapter 2, the WB Kindergarten Survey, which measured process quality of public and private kindergartens in 2015. Among 5-year old children attending the kindergartens covered, the survey also measured ECE outcomes in the domains of numeracy, language, social emotional skills, executive functions, and fine motor skills. The survey was carried out in two rounds of data collection. One round implemented the survey in conventional or “fixed” kindergartens in urban and rural areas of Mongolia. Another round of data collection was carried out in ger kindergartens. The survey’s findings on determinants of child outcomes resonate with those from settings across the world. The mean kindergarten quality score was the only factor among kindergarten characteristics that was significantly associated with higher outcomes in all domains of early development assessed. Clearly, quality matters. Longer exposure to preschool was associated with better outcomes, and was statistically significant in the numeracy, literacy, and social emotional domains. The socioeconomic status of a child’s household continued to exert a significant influence on outcomes in all domains after controlling for kindergarten-level factors including quality and exposure. Increasing intensity of family engagement with children on activities at home was associated with significantly higher outcomes in numeracy and language, and marginally so in social emotional skills. While Kazakh children lagged behind others in language and social emotional skills, the gap disappeared after accounting for kindergarten quality. Finally, needing special assistance in the classroom lowered outcomes. The chapter also presents findings from an analysis of assessment data collected for children who participated in a home-based school preparation program implemented by Save the Children Japan (SCJ) in 30 soums of four aimags of Mongolia. The intervention targeted the most socially and economically disadvantaged communities in the country, and focused on herders’ children with no access to formal preschool services. Since the target group is very similar to that of ger kindergarten services, the ger- kindergarten sample of the WB Kindergarten survey offers a plausible comparison group to which the children in the SCJ intervention can be compared. While concerns with data quality and representativeness of the SCJ sample should not be ignored, and hard evidence on impacts is not available, the data do show that when compared to attendance at ger kindergartens, participation in the SCJ intervention was associated with a far more supportive at-home ECE environment at end-line, improved family engagement with children in activities that nurture early development, and significantly superior development outcomes in all five domains of skills assessed. Thus evidence from both the WB 47 and SCJ surveys underscores the potential for home-based ECE modalities to improve outcomes among Mongolian children outside the reach of kindergarten services. This chapter is structured as follows. Section 1 focuses on the WB Kindergarten Survey, and describes the data collection process undertaken (World Bank Kindergarten Quality Survey) to measure ECE outcomes among children enrolled in Mongolian preschool institutions. This is followed by presentation of survey findings on background characteristics of 5-year old children enrolled in different types of kindergartens. This sheds light on “targeting” of preschool services, and highlights which segments of the Mongolian population each type of institution serves. The next subsection describes the household ECE environment and parental engagement with children at home, focusing on factors shown to influence early development outcomes in other settings. The final subsection presents early development outcomes, disaggregated as in Chapter 2 by type of kindergarten (public “fixed” kindergartens, private kindergartens, and ger kindergartens). Of key interest is the extent to which the variation in quality documented in chapter 2, combined with household- and student-level factors, helps explain the variation observed in outcomes. Section 2 presents findings on child assessments conducted by SCJ for children enrolled in the SCJ home-based school preparation program, and compares outcomes to those among children enrolled in ger kindergartens in the WB survey. Section 3 exploits data on ECE outcomes in UNICEF’s most recent available round of the MICS survey for Mongolia, which allows a limited comparison with the WB survey. Section 4 then concludes with discussion of the main issues highlighted and potential avenues forward. 3.1 The 2015 World Bank Kindergarten Survey: Data collection, sample characteristics, and ECE outcomes 3.1.1 Survey sample and data collection Data on ECE outcomes were collected on 1,996 five-year old children in a sample of 101 public (regular) kindergartens and 430 children in a sample of 50 private kindergartens 25. Data collection was limited to 5-year olds only. This was because the age of entry to primary school is 6 years in Mongolia, and MECS was interested in assessing school readiness and learning gaps in this age group. Data collection took place in October, 2015, and locations in which kindergartens were sampled were selected across the five regions of Mongolia (table 3.1). Fixed public kindergartens sampled included 35 kindergartens in UB, and 66 from across eight provinces of Mongolia. In UB, locations were included to cover the low-income “ger” and relatively well-off “non-ger” areas 26. Outside UB, in each aimag, kindergartens were selected from aimag centers (the urban center of the aimag) and soums (villages). Private kindergartens exist in urban areas only, and are largely concentrated in UB. Therefore, selected locations for these kindergartens were limited to UB and aimag centers. Table 3.1 Number of children assessed in the WB Kindergarten Survey sample Region Aimag Ger Public fixed Private 25 A maximum of 20 children were selected in each kindergarten. At each kindergarten, selection of children took as place as follows. First, the number of groups with 5-year old children was determined. An equal number of children was then randomly selected from each group, with the number of selected children adding up across groups to 20 in each kindergarten. 26 Urban growth in UB in recent years has largely taken the form of ger areas, where most of the poorest live, areas which lack adequate connection to network services and are heavily prone to natural hazards. Low-income residents that populate these areas build their own gers, but lack basic services and infrastructure provision, such as paved streets, parks, streetlights, water connection and sanitation (World Bank, 2015). 48 Eastern Khentii 90 220 Central Tuv 85 40 Selenge 40 Dornogovi 38 205 15 Darkhan-uul 7 141 19 Khangai Ovorkhangai 126 212 8 Khuvsgul 101 140 Western Bayan-ulgii 75 260 Ulaanbaatar 738 388 Total 534 1996 430 Data on outcomes was also collected on 534 five-year old children in a sample of 101 ger kindergartens, as these are by far the most common type of alternative preschool service offered in Mongolia. Since ger kindergartens operate only during the summer months, when fixed kindergartens are closed, the data collection in these kindergartens took place between June and July, 2015. Also, these kindergartens are concentrated in regions of Mongolia where coverage of fixed kindergartens is considerably lower (section 3.1). Because of this, the locations selected for sampling were restricted to aimag centers and soums of selected aimags. ECE outcomes were assessed using the Measuring Early Learning Quality and Outcomes (MELQO) instrument. This is a new instrument recently developed in collaboration between the World Bank, UNICEF, UNESCO, the Center for Universal Education at Brookings Institution, and experts from around the world (box 3.1). The domains of child development assessed by the instrument included cognitive skills, language skills, motor development, social/emotional development, and executive function/self- regulation27. In addition to implementing the MELQO instrument, background information on household characteristics was also collected, including socioeconomic status (such as education), and the ECE environment within the household. Box 3.1 MELQO: A collaborative effort to make early learning assessment more accessible globally The Measuring Early Learning Quality and Outcomes project aims to improve outcomes for young children by making early learning assessment more accessible worldwide. Building on the strengths of measurement initiatives that have already been developed, the project has developed two tools: one to measure child development and the other to measure the quality of early learning environments. The project also provides governments with guidance to responsibly and effectively take early learning quality and outcome measures to scale. 27 Cognitive skills include pre-literacy, problem solving, measurement and comparison, analytical thinking, memory and early mathematical and number sense. Language skills include a child’s knowledge and use of words, both in print and oral form. Motor development includes fine and gross motor skills, and measures a child’s capacity to control their body movements to perform everyday tasks, such as walking, running, or jumping, as well as drawing, writing, holding utensils and picking up objects. Social/emotional development includes a child’s awareness of their feelings and those of others. It also measures children’s social interactions and how they manage their behaviors. Executive function/self-regulation includes self-control (inhibition and delaying gratification), persistence, and the ability to initiate action and sustain attention (MELQO, 2015). 49 Key objectives and features of MELQO include: • To produce open-source, free assessments with good technical guidance • To provide tools that can be used to link child development assessment with quality of early learning environment assessment • To develop a common core that can be used in existing surveys and instruments to produce regionally and globally-comparable data • To work with countries to ensure the tools can be used within national monitoring and evaluation systems The project is led by the World Bank, UNICEF, UNESCO, and the Center for Universal Education at Brookings Institution and includes experts from around the world. The prototype for measuring early learning and child development has been developed and includes parent/teacher report and a direct assessment instrument for children ages 3 to 7 years old. The prototype includes a set of items that measure cognitive, language, and executive function skills, as well as socio-emotional development. Pre-field testing of the prototype has taken place in Bangladesh, Kenya, Madagascar, Sudan, and Tanzania, with more countries (e.g. Nicaragua) in the planning stage or joining soon. The content of the assessment is drawn from existing assessments and will have similarities with items from many existing tools, including: ASQ, EDI, IDELA, PRIDI, SDQ, SWAN, and WCARO, among others. In addition, the child development assessment is aligned to the Early Grade Reading Assessment (EGRA) and the Early Grade Math Assessment (EGMA). MELQO child development and learning tools would ideally produce a nationally-representative distribution of child development, which could then inform the allocation of funds, inform curriculum design and teacher training programs, and monitoring the percentage of children developmentally on track. Source: MELQO (2015) Age, gender, and ethnicity of children in survey sample: The mean age of children in the public and private fixed kindergarten sample was 63 months (5.3 years). The mean age was significantly higher at 65 months in the ger kindergarten sample, with a considerably wider spread in ages 28. Just under half the sample in both fixed and ger kindergartens consisted of girls. 78 and 80 percent of the children in ger and fixed kindergartens respectively were from households whose head was of the Khlakh ethnicity 29; 14 and 13 percent were Kazakh, and 7 percent respectively were of some other ethnicity 30. 5 percent of five-year old children in public kindergartens had a teacher-reported disability that required special assistance. In private kindergartens, the share was higher. While measures of physical wellbeing (e.g. anthropometric measurements) were not included in the study, teachers were asked to 28 Due to the difference in ages between the ger kindergarten and fixed kindergarten sample, any relevant comparisons in outcomes are always presented after restricting the ger kindergarten sample to 5 year olds only. All relevant analysis presented later in this report controls for age. 29 95 percent of Mongolia’s population is of the Khalkh ethnicity. 30 “Other” ethnicity includes Durvud, Bayd, Uriankhai, Buriad, and others. 50 report on general wellbeing observed in the classroom. Slightly over 10 percent of children in public fixed kindergartens were reported to fall sick or get tired often, while 5 percent had a disability that required special assistance. The private and ger kindergarten samples had lower shares of children observed to fall sick often (5 percent), but tiredness was just as commonly reported as in the public fixed kindergarten sample. Prevalence of disabilities in ger kindergartens was comparable to other public kindergartens, but was twice as high in the private kindergarten sample (10 percent). 3.1.2 Household characteristics of 5-year old children enrolled in kindergartens One of the primary concerns of this report is to determine who benefits from public investments in preschool education, and how the quality of services accessed varies by socioeconomic status. To that end, the survey collected data on a handful of key social and economic indicators for each child included in the assessment. Interviews were conducted with parents or primary caregivers to collect information on parental education, employment status, and assets owned by the household, among other areas. Through these data, we are better able to understand the social and economic characteristics of beneficiaries of preschool services, and can begin to examine how well services reach those who need them the most, and the extent to which public investments in preschool education are helping level the playing field for children born into disadvantaged households vis-à-vis those from the richest. The survey also collected data on a handful of key aspects of the child’s home environment that can impact ECE outcomes, independently of what happens in preschools 31. Findings are summarized in this section. Public sector fixed kindergartens serve children from households from a wide range of socioeconomic backgrounds. Educational attainment among mothers varied significantly across fixed public kindergartens, depending on location (figure 3.1). Not surprisingly, non-ger areas of UB had the highest attainments, with 71 percent of mothers in the sample having completed an undergraduate degree or higher. In ger areas of UB and aimag centers, this declined to 46-48 percent, and in soums, the share stood at 34 percent. The share of mothers with secondary education or less rose from 5 percent in UB non ger areas to 26 percent in soums. While educational attainment among fathers was generally lower than that among mothers (figure 3.2), the pattern across locations was similar. The share completing undergraduate degrees or higher declined from 58 to 21 percent between UB non-ger areas to soums, while primary or lower increased from 0 to 11 percent of children. In terms of employment status of the household head, the picture is similar: increasing disadvantage from UB non-ger areas to soums (figure 3.3). The share of children from households where the household head was either unemployed or reported no occupation rose from 10 percent in non-ger areas of UB to 21 percent in soums. Aimag centers and soum kindergartens also had significantly higher share of children from households whose heads practiced animal husbandry as the primary occupation, at 6 and 25 percent of children respectively compared to 1 percent in UB. Another dimension of socioeconomic status of children enrolled in kindergartens is a measure of the wealth of each child’s household. In the absence of data on household consumption and income, household wealth can be proxied by an “asset index”, a linear index constructed using asset ownership 31 This included measuring the extent to which parents were involved in each of the following: reading books to children or spending time with them looking at picture books, telling stories, singing songs, playing, spending time together outdoors, or naming, counting or drawing things together. 51 indicators, using principal components analysis to derive weights (see annex 6 for details on construction of the index). Figure 3.4 shows the mean score on the wealth index for households of children in public kindergartens in each location. The scores mirror the findings on educational attainments and employment status: wealth scores decline from an average of 0.70 in UB non-ger areas, to 0.33 in ger areas, to -0.24 in soums. Figure 3.1 Maternal education among 5-year old children enrolled in kindergartens, Mongolia, 2015 Public-ger Public - fixed 80 71 60 46 4548 46 37 39 40 40 33 34 29 24 17 17 17 18 20 8 7 7 8 1 4 1 1 4 1 0 0 Percent Aimag center Soum UB non-ger UB gerAimag center Soum Private 88 79 80 60 46 42 Primary or lower 40 21 Secondary 20 9 7 High 4 3 2 Undergrad or above 0 UB non-ger UB ger Aimag center DKN Graphs by sample2 Figure 3.2 Paternal education among 5-year old children enrolled in kindergartens, Mongolia, 2015 Public-ger Public - fixed 80 58 60 50 40 41 40 36 40 35 29 28 29 31 26 24 23 21 20 15 10 8 10 10 11 3 6 5 5 4 0 1 0 Percent Aimag center Soum UB non-ger UB ger Aimag center Soum Private 77 80 60 48 42 36 40 31 Primary or lower Secondary 20 15 12 13 10 7 High 4 4 2 Undergrad or above 0 UB non-ger UB ger Aimag center DKN Graphs by sample2 52 Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) Figure 3.3 Occupations of heads of households of 5-year old children enrolled in kindergartens, Mongolia, 2015 80 Public-ger Public - fixed 74 59 60 50 53 47 37 40 29 31 29 25 23 25 18 21 17 20 13 12 10 10 8 6 4 1 1 0 Percent Aimag center Soum UB non-ger UB ger Aimag center Soum Private 80 60 48 48 45 43 44 40 34 18 Paid 20 12 Self-employed 7 1 Animal husbandry 0 UB non-ger UB ger Aimag center None Graphs by sample2 Figure 3.4 Wealth asset index scores of households of 5-year old children enrolled in kindergartens, Mongolia, 2015 Public-ger Public - fixed Private 1.08 1 0.70 0.66 Mean asset index score .5 0.33 0.21 0.05 0 -0.24 -.5 -0.50 -1 -1.26 -1.5 UB non-ger UB ger Aimag center Soum Graphs by sample2 Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) 53 Annex 7 compares the WB quality survey sample with that of the 2010 Multiple Indicator Cluster Survey (MICS), which is a nationally representative survey. Children from the poorest households in the WB subsample of public kindergartens look similar to those in the poorest households in the MICS in terms of household asset ownership and parental education, while children from the richest households tend to be better off. Private kindergartens tend to enroll more children from either the highest or lowest ends of the SES spectrum than do public kindergartens. In non-ger areas of UB, private kindergartens serve a significantly different population from that served by public kindergartens (figures 3.1-3.4). In employment, the differences are muted, the main difference being that household heads of enrolled children were significantly more likely to be self-employed than to engage in paid employment, and were slightly less likely to report being unemployed or to have no occupation. The striking differences emerge when looking at educational attainments among parents of enrolled children. The share of mothers with an undergraduate degree or higher is 88 percent, a full 17 percentage points higher than that in public kindergartens that serve the same area. The difference is even higher for fathers – they are 19 percentage points more likely to have an undergraduate degree or higher. Further, the mean wealth index score of children’s households stands at 1.08, significantly higher than that of households of children in public kindergartens in the same location (0.70). In UB ger areas, in contrast, the differences between student populations served by public and private kindergartens were very small. Maternal educational attainment is not significantly different, and attainment among fathers also looks similar. Among the latter, undergraduate or higher completion was very slightly higher, and also a somewhat higher share of secondary or less as well as unknowns. The mean wealth index score of children’s households stands at 0.21, slightly lower than that of households of children in public kindergartens in the same location (0.33). These findings imply that private kindergartens Figure 3.5 Wealth index scores of households of 5-year in non-ger areas of UB serve a population that is old children enrolled in UB area kindergartens, significantly better off on multiple Mongolia, 2015 socioeconomic dimensions than that in non-ger 20 areas. Furthermore, the public and private sectors in UB seem to be catering to the needs 15 of different populations, with the private sector serving a higher relative share of the richest Percent 10 (and the poorest, to a lesser degree), and the public sector serving a broader spectrum of 5 households (figure 3.5). Annex 7 compares the WB quality survey 0 sample with that of the 2010 Multiple Indicator -2 -1 0 Predicted index - national 1 2 Cluster Survey (MICS), which is a nationally Public UB Private UB representative survey. Children from the Source: Author’s calculations using data from World Bank poorest households in the WB subsample of Kindergarten Quality Survey (2015) private kindergartens look similar to those in the poorest households in the MICS in terms of household asset ownership and parental education, while children from the richest households tend to be better off. 54 In soums, ger kindergartens are successful in targeting the most vulnerable children, but less so in aimag centers. Compared to fixed kindergartens, ger kindergartens on paper have a completely different target population: children of the hard-to-reach nomadic herders, who, due to their nomadic lifestyles, are unable to enroll children in fixed kindergartens. Overall, in the ger kindergartens operating in soums, this objective is being met to quite an extent (figures 3.1-3.4): 74 percent of the children in the sample were from households practicing animal husbandry, and another 10 percent were either not working or had no occupation. Educational attainments among parents tell the same story: maternal educational attainment is far lower than in any other location covered under fixed public or private kindergartens; the same holds true among fathers. Finally, children enrolled in ger kindergartens in soums come from households with an average wealth index of -1.26, the lowest in the entire sample for any location surveyed, and more than 2 standard deviations lower than households of children enrolled in private kindergartens in non-ger areas of UB. In terms of targeting the disadvantaged, this is quite impressive. However, there still remains a notable 16 percent of children from households engaged in paid employment or were self-employed. Moreover, in ger kindergartens operating in aimag centers, only 8 percent of children came from households practicing animal husbandry, and the largest share were from households either not working or had no occupation (50 percent). This may be justifiable if this share represents a segment of the vulnerable population that is not employed and doesn’t have access to fixed kindergartens. But it is much harder to justify the remaining 42 percent of children that come from household heads engaged in paid employment or are self-employed. Overall, households of children in aimag center ger-kindergartens score an average of -0.5 which is significantly lower than that in other kindergartens, but again significantly better off than those in soums (-1.26). Targeting of ger kindergartens in aimag centers also seems off if we look at educational attainment among parents. 46 and 10 percent of mothers and fathers respectively have completed tertiary education. This is very high, and certainly does not indicate that these kindergartens in aimag centers are targeting the worst off in the location. Annex 7 compares the WB quality survey ger-kindergarten subsample with a comparable subsample of out-of-preschool 5-year old children in the 2010 Multiple Indicator Cluster Survey (MICS). As the latter is a nationally representative survey, such a comparison allows us to say something about the socioeconomic characteristics of children enrolled in ger kindergartens relative to the population of out- of-school 5-year olds nationally. Indeed, we find that while the WB survey sample includes children from some of the most disadvantaged communities in Mongolia, children from well-off households are also enrolled in ger-kindergartens. Part of the puzzle may be the high unmet demand for preschool services described earlier. In interviews conducted for this report with kindergarten authorities, many directors reported that they allowed children who had been turned away or waitlisted in fixed kindergartens due to lack of availability of spaces to enroll in ger kindergartens. This may be one factor underlying the relatively high share of children who do not come from herders’ families. Another factor may simply be that there are fewer herders’ families present in aimag centers, compared to soums. 3.1.3 The household environment and parental engagement at home Evidence from other settings shows that children’s household environment influences their development outcomes. When or not children are enrolled in an ECE program, most children spend a 55 substantial part of their lives at home with their mother or primary caregiver. Therefore, it is important to assess whether the children in our sample are growing up in environments that are likely to promote positive child development. The WB survey collected data on the type of toys children had access to when at home, whether children were left at home without adequate supervision, and whether household members engage with children in selected activities at home. Of particular interest is whether parents look at or read books with children, or tell stories. Children whose parents engage in these activities with them usually know more words, have better cognitive abilities, are more interested in books, and become better readers in the future (Duursma et al., 2008). The type of toy most commonly used by children when at home varied by type of kindergarten, but most children had access to at least one type. Wealth gaps were sizable for private kindergartens. Toys: In fixed kindergartens, almost all children were reported to play at home with manufactured toys or those bought in a shop (98 percent); the share was somewhat lower among children from households in the poorest quintile (92 percent). Play with homemade toys was much lower (28 percent) or other objects in the household or outside (65 percent), and the wealth gap was large (20-23 percentage points). In private kindergartens, almost all children were reported to play at home with manufactured toys, but the wealth gap was larger (16 percentage points). Otherwise, children in the private kindergarten sample were similar to those in public fixed ones. Play with homemade toys was much lower (35 percent) or other objects in the household or outside (69 percent), and the wealth gap was large (19-23 percentage points). Most of the children enrolled in ger kindergartens also tended to play at home with manufactured toys (92 percent), and use of homemade (20 percent) or household objects (60 percent) was comparable to that in fixed kindergartens. Children were rarely left at home alone or with inadequate supervision. Leaving children alone at home was reported to be very low. Children were left alone at home for an hour or more for an average of no more than 0.09 days in the past week for any kindergarten type. Leaving children under the supervision of older children (10 years or younger) was also very rare - an average of no more than 0.16 days in the past week for any kindergarten type. Mothers of children in fixed public kindergartens were much more likely than fathers to be involved with at-home activities with children, and wealth gaps in involvement were large. Depending on the activity, mothers of between 30 percent (played with children) and 42 percent of children (spent time outdoors) had engaged with the child in any of the listed activities in the past three days. Children from richer households were significantly more likely to be engaged in the listed activities (figure 3.6) 32. Fathers were much less likely to engage with their children (10-22 percent); Like mothers, for fathers, time spent outdoors was the most common activity with a child. Difference between the richest and poorest quintiles was again very large (10 - 21 percentage points, highest for outdoor activity and playing). Siblings were generally even less likely to engage in activities with children surveyed (9-14 percent), perhaps because there was a relatively small number of households with more than one child 32 It is likely that the composition of activities that children are engaged in at home changes from season to season. Fixed public and private kindergartens were surveyed in October, 2015, which was a relatively mild winter month, with opportunities to spend time outdoors. 56 in our sample 33. Engagement in past three days with other individuals (aunts, grandparents, etc.) was similarly low (6-11 percent of children). On average, both parents of children in private kindergartens were likelier to engage with children in activities at home compared to public kindergartens, but wealth gaps were significantly larger. Depending on the activity, mothers of between 36 percent (time spent outdoors) and 48 percent of children (singing with children) had engaged with the child in any of the listed activities in the past three days. Overall thus, mothers’ engagement with children is somewhat higher than that in the public kindergartens sample (about 6 percentage points). Importantly, differences between the richest and poorest households also tended to be much larger, with the largest wealth gap in telling stories to children (34 percentage points), followed by reading books and singing songs (25-26 percentage points). Compared to the public kindergarten sample, Figure 3.6 At-home engagement of mothers with fathers were much more likely to engage with their children in past three days, by wealth quintile, children ranging from 17 percent in reading or among 5-year old children enrolled in public looking at books to 32-33 percent in outdoors time kindergartens, Mongolia, 2015 and playing. Difference between the richest and 0% 20% 40% 60% poorest quintiles was again larger than that in the 22% Read or looked at books public kindergartens group (14-40 percentage 44% points, highest for outdoor activity). Siblings were Told stories 24% 40% generally less likely to engage in these activities with 21% these children (10-18 percent), consistent with the Sang songs 46% fact that there was a relatively small number of 32% Spent time outdoors households with more than one child in our sample. 51% However, sibling engagement was overall higher Played 17% 34% than that in the public kindergarten sample. 23% Engagement in past three days with other Named/counted/drew 47% individuals (aunts, grandparents, etc.) was also slightly higher (8-13 percent of children), with Poorest Middle Richest wealth gaps going in either direction (e.g. poorer Source: Author’s calculations using data from World Bank households were significantly more likely to have Kindergarten Quality Survey (2015) children engaged with books or singing with an “other” person, but the reverse was true for telling stories and playing). Engagement of children attending ger kindergartens in at-home activities was very low compared to fixed kindergartens. Depending on the activity, mothers in between 13 percent (played with children) and 28 percent of children (singing) had engaged with the child in any of the listed activities in the past three days. This is significantly lower than in the public kindergarten sample. Children from richer households were significantly more likely to be engaged in the listed activities (19-41 percentage point difference, highest in telling stories; lowest difference in engaging with books). Fathers were much less likely to engage with 33 An interesting related point is that for all activities, the children from the lowest quintile households were very slightly more likely to be engaged in activities with siblings. In our sample, poorer households tend to have a larger number of young children in the household than do richer ones. 57 their children (3-19 percent); time spent outdoors was the most common activity with a child. Siblings were generally more likely to engage in these activities with these children (16-33 percent) than either fathers or mothers – indeed, sibling engagement was significantly higher than that in both the public fixed and private kindergarten samples. The poorer households targeted by ger kindergartens have a higher number of young children in the household compared to others. Engagement in past three days with other individuals (aunts, grandparents, etc.) was the lowest among all samples (4-8 percent of children) 3.1.4 ECE outcomes among five-year old kindergarten-enrolled children This section explores the relationship between performance on each domain of child development and household-, child- and school-level factors. We know from experience around the world that household- and child-level background factors impact ECE outcomes. Household wealth and parental education in particular has been associated with superior outcomes, and parenting practice, including level of engagement with children at home, also matters. Thus, even if the quality of children’s experience and service deliver within kindergartens is held constant, it is likely that children from different socioeconomic backgrounds will develop skills differently and at different paces, even if they attended exactly the same kindergartens with no difference in quality of environment within the kindergarten (which we know is never the case). From a policy perspective, thus, it becomes important to understand how much of the variation we see in outcomes among children can be attributed to differences in background, as opposed to differences in quality of kindergartens they attend. In both cognitive development and language, children in ger kindergartens lag significantly behind children in public fixed kindergartens, while those in private kindergartens score the highest. Annex 8 shows the distribution of cognitive scores, expressed in standard deviation units (see box 3.2), for public fixed kindergartens, ger kindergartens, and private kindergartens. The distribution of scores in ger kindergartens tends to be skewed toward the lower end of the distribution of scores in public fixed kindergartens, while the opposite is true for private kindergartens, in which scores are higher. Indeed, the mean score in ger kindergartens is a full 0.30 SD units lower than that in public kindergartens (figure 3.7), while that in private kindergartens is 0.30 SD units higher. Both differences are significant at the 1 percent level (see box 3.2). Similarly, the mean language score in ger kindergartens is 0.43 SD units lower than that in public kindergartens, while that in private kindergartens is 0.19 SD units higher. Box 3.2 Analyzing MELQO child assessment results: Normalization of scores and statistical significance Creating normalized assessment scores The first step in analysis of the MELQO child assessment results is creation of a normalized assessment score for each domain of child development, for each child assessed. Domains tested using the MELQO instrument were cognitive skills, language, social/emotional development, executive function and fine motor skills. In each domain, normalized scores 34 were calculated for a pooled cross-section consisting of children from all three samples, i.e. public, ger and private kindergartens. With the scores thus expressed in standard deviation (SD) units, it is possible to compare differences in performance in each domain across the three samples, as well 34 Normalized scores are Z scores, calculated by subtracting from each child’s score the average score obtained on each domain and dividing by the standard deviation of scores on that domain. The scores were thus expressed in standard deviation (SD) units. 58 as to explore household-, child- and school-level determinants of outcomes observed. When scores are expressed in SD units, they can be positive or negative. A normalized score of “0” implies that the child scored the mean score. The larger the normalized score, the higher the child scored above the mean. Negative values of the normalized score indicate that the child scored below the average score. Thus, a normalized score of “-0.5 SD” is higher than a normalized score of “-1 SD”; a normalized score of “0” is higher than both of these; and a normalized score of “1.2 SD” would be the best of the four scores. Understanding statistical significance When looking at the differences in average scores between different groups of children, this section often reports whether the differences are “statistically significant” or not, at the 1, 5, or 10 percent levels. Also, in regression analysis, in which we are interested in examining whether background and kindergarten-level factors influence outcomes or scores, the section similarly reports on whether the coefficient on a particular factor is “statistically significant” or not. What does this mean? All estimates reported here are based on surveys, and thus are subject to error. In this report, a test of statistical significance always refers to the test of whether an estimate (either a difference in means, or a regression coefficient) is statistically different from (i.e. larger or smaller than) zero. Intuitively, the test of statistical significance reports on the level of precision with which the estimate can be stated to be “different” from zero, and gives the reader a sense of how large or small the error is. The smaller the significance level, the more precisely estimated the number, and the more confident we can be that the estimate in question is different from zero and is “statistically significant”. Stated differently, the probability that a regression coefficient significant at the 1 percent level is truly different from zero in the population surveyed is higher than that for one reported at the 5 or 10 percent significance levels. Figure 3.7 Mean normalized MELQO child assessment scores among 5-year old children, by kindergarten type, Mongolia, 2015 0.3 0.2 0.1 Mean score in SD units 0.0 -0.1 -0.2 -0.3 -0.4 -0.5 Ger -0.6 Public - fixed -0.7 Private -0.8 Cognitive Language Social emotional Executive function Fine motor Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) Notes: Children enrolled in ger kindergartens were assessed after the kindergartens had been operatingf for about 2-3 weeks, which is more than half-way through the total duration of operation for a typical ger kindergarten. Across public, private, and ger kindergartens, scores display a generally similar pattern in other domains as well. In other domains, performance followed a similar pattern across the public, ger, and private kindergartens (see annex 8. In social emotional development, children in ger kindergartens lagged 59 significantly behind those in public kindergartens (an average of 0.84 SD units – figure 3.7), while those in private kindergartens scored on average 0.19 SD lower. Socioemotional development was the only domain on which private kindergartens lagged behind public kindergartens. In executive function, children enrolled in ger kindergartens lagged significantly behind public kindergartens by just over a third of a standard deviation, while private kindergartens were a quarter of a standard deviation ahead. In fine motor skills, children enrolled in ger kindergartens lagged significantly behind public kindergartens by about half a standard deviation, but there was no significant difference between private and public fixed kindergartens. In cognitive and language skills, and executive function, children in public kindergartens in rural areas were at a significant disadvantage compared to those in urban areas. In fixed public kindergartens, child development outcomes for the cognitive, language and executive function domains are highest in UB, followed by aimag centers, with children in soum kindergartens lagging significantly behind (figure 3.8). The difference in average scores between soums and UB ranged from 0.22 to 0.28 SD units depending on the domain. In social emotional skills, children in aimag center kindergartens scored on average highest among the three locations, with no significant difference between children in UB and soum kindergartens. The same was true for the fine motor skills domain. Figure 3.8 Mean normalized MELQO child Figure 3.9 Mean normalized MELQO child assessment scores among 5-year old children assessment scores among 5-year old children enrolled in fixed public kindergartens, by location, enrolled in private kindergartens, by location, Mongolia, 2015 Mongolia, 2015 0.50 UB 0.50 UB non-ger 0.40 Aimag center 0.40 UB ger 0.30 Soum 0.30 0.20 0.20 0.10 0.10 0.00 0.00 -0.10 -0.10 -0.20 -0.20 -0.30 -0.30 Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) In UB, children enrolled in private kindergartens in ger areas lag behind those in non-ger areas. In private kindergartens, children in ger areas of UB lagged significantly behind those in non-ger areas, in all five domains of child development assessed (figure 3.9). The ger-area disadvantage was particularly large in the domains of cognitive development and executive function (0.6 SD units), followed by language (0.4 SD) and fine motor skills. Importantly, children in ger-area kindergartens performed, on average, worse than children in public kindergartens located in soums (figure 3.8). Among children in fixed public kindergartens, measures of household socioeconomic status were significant predictors of all outcomes. Among household level factors analyzed (figure 3.10), the coefficients on maternal educational attainment were large, other factors held constant, especially those 60 for language and fine motor skills, ranging from a difference in scores of between 0.49 to 0.63 SD between children of mothers with primary education or less compared to those with a college degree or higher. The effects of father’s education attainment were also significant, but coefficients were generally smaller than those on maternal education 35. Other things held equal, household wealth was associated with higher outcomes, particularly in math skills and executive function, where the gaps between the poorest and richest households came up to 0.31 SD. Finally, effect of the full set of household socioeconomic variables was jointly significant in each domain of outcomes assessed, over and above the effect of kindergarten-level variables (annex 10). Figure 3.10 Relationship of household characteristics and preschool exposure with normalized scores on the MELQO direct assessment instrument, 5-year old children enrolled in Mongolian public kindergartens, 2015 Standard deviation units -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.0706*** 0.0692*** At home engagement 0.0493* 0.343*** Household chracteristics 0.487*** Maternal education 0.421** 0.318*** 0.631*** 0.269*** 0.201** Paternal education 0.311*** 0.186* Household wealth 0.317*** Numeracy Literacy 0.140*** 0.244*** Quality score Preschool exposure 0.429*** 0.100*** Social emotional 0.106** Executive 0.0737*** function 0.0645** Time in kindergarten 0.0549** Draw-a-man Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) 35 While the effect of different levels of parental education was not always statistically significant, the magnitude of coefficients was comparable to those of other statistically significant predictors. Importantly, the coefficients on indicator variables for parental education were jointly significant for each assessment domain (annex 10). 61 Notes: See annex 9 for the complete regression output tables, and discussion of results in each domain of child development. Regression results reported here control for the full set of hosuehold-, indivudal- and school-level characteristics, also described in the annex. Significance levels: *** = 1 percent, ** = 5 percent, * = 10 percent. Bar heights represent the change in normalized test scores associated with: (i) a 1-SD increase in the home engagement index (see box 3.3); (ii) children of mothers with a college degree or higher relative to those with primary education or less; (iii) children of fathers with a college degree or higher relative to those with primary education or less; (iv) children from households in the richest quntile of wealth relative to those in the poorest quintile; (v) a 1-point increase in the total ECEMI quality score; (vi) an additional year completed in kindergarten. Increasing intensity of family engagement with children on activities at home was associated with significantly higher outcomes in numeracy and language, and marginally so in social emotional skills. A 1-SD increase in the at-home engagement index (box 3.3) was associated with an increase in normalized scores of about 0.07 SD on the cognitive and language domains. The finding points to the potential significance of home-based ECE activities in improving school readiness, among children with limited or no access to formal preschool services. This may be especially relevant for the poorest households, where parental engagement with the child in activities relevant for ECE is considerably lower than in richer households (section 3.3). Box 3.3 : Constructing the at-home engagement index To measure the effect of the household environment and specifically parental engagement at home, the data collected through select questions in the WB Kindergarten Survey was used to create the “at- home engagement index” referred to in figure 3.10. As discussed in section 3.3, for each child assessed, the Survey collected data on whether the child’s mother, father, sibling, or any other individual at home had engaged with the child in a specific set of activities in the last three days. These activities were reading or looking at books, telling stories, singing songs, spending time together outdoors, playing, and naming, counting or drawing objects. For each child, an engagement intensity score was first created by adding up all the instances of activities reported, across the different household members reported to have engaged with the child in the three-day period preceding the survey. For example, consider a child with whom the mother engaged in five activities, the father engaged in one activity, while a sibling engaged in two. The engagement intensity score for this child was 8. Such a construct leads to a score expressed in units of “person-activity”. Note that, by construction, this score does not measure how long the activity lasted, or how frequently it may have occurred during the three- day period (these data were not collected in the survey). The only aspect captured was whether the activity occurred or not. The engagement intensity score was then normalized and expressed in units of standard deviations using the same method employed for standardization of child assessment scores (box 3.2). A standard deviation (SD) of the distribution of scores was equal to 2.6 person-activities. In other words, the at- home engagement effect displayed in figure 3.10, i.e. the difference of 0.07 SD in the normalized cognitive skills score, was associated with an increase in the engagement intensity score of 2.6 person- activities – or, stated differently, the equivalent of a household member engaging with the child in between 2 to 3 activities in the three-day period prior to the survey. 62 Longer exposure to preschool was associated with better outcomes in the numeracy, language, and social emotional skill domains. An additional year of kindergarten exposure was associated with between 0.05 to 0.07 SD higher scores on the cognitive, literacy, and social emotional skills domains (figure 3.10). Stated differently, among 5-years olds assessed, children who had just started kindergarten at the time of the survey scored 0.11-0.15 SD lower than children who had been in kindergarten for a full two years, other factors held constant. These effects are encouraging, given that evidence from other studies examining the relationship between duration of preschool and child development outcomes is mixed with analyses finding both negative and positive impacts of preschool exposure (Vandell, 2004; Brinkman et al, 2016). Figure 3.11 Relationship of selected background characteristics with normalized scores on the MELQO direct assessment instrument, 5-year old children enrolled in Mongolian public kindergartens, 2015 Standard deviation units -0.7 -0.5 -0.3 -0.1 0.1 0.3 0.177*** No kindergarten predictors 0.0984* 0.158*** 0.231*** Girls 0.155*** With kindergarten predictors 0.160*** 0.227*** -0.309** No kindergarten predictors -0.461*** Kazakh household With kindergarten predictors Numeracy Literacy -0.384*** -0.368*** No kindergarten predictors -0.219* Social emotional -0.172* Disability Executive -0.414*** function -0.442*** With kindergarten predictors -0.317*** -0.197* Draw-a-man Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) Notes: See annex 9 for the complete regression output tables, and discussion of results in each domain of child development. Regression results reported here control for the full set of hosuehold- and individual-level characteristics, and also show results upon addition of the full set of kinderdarten-level controls. See annex for complete list of controls. Significance levels: *** = 1 percent, ** = 5 percent, * = 10 percent. Bar heights present the difference in normalized test scores associated with: (i) being female; (ii) belonging to a Kazakh household; (iii) having some kind of teacher-reported disability. 63 The association of outcomes with quality in kindergartens varied across domains, but effects were largest in the social emotional domain, followed by language. The estimated effect sizes of observed classroom quality are large, even after controlling for all available household-, individual, and kindergarten level controls (figure 3.10). A 1-point increase in classroom quality was associated with an increase of between 0.10 SD (executive development) and 0.43 SD (social emotional skills), depending on the development domain considered. This range of effect sizes is higher than that reported in other studies that include extensive control variables, or use fixed effects to estimate the effect of classroom quality, both in the United States and in developing countries. For example, large scale studies from the United States report effect sizes ranging from 0.04 (Duncan, 2003) to 0.18 (Peisner-Feinberg et al., 2001) on children’s language and cognitive development. For socioemotional skills, prior studies report effect sizes as small as 0.02 in the U.S. (Peisner-Feinberg et al., 2001) to as large as 0.13 in the U.K. (Sylva et al., 2006). Meanwhile, an East African study reports an effect size of 0.15 on children’s cognitive skills (Malmberg et al., 2011). A study from Ecuador, which uses CLASS instead of ECERS-R, shows similar relationships between classroom quality and children’s learning outcomes with effect sizes ranging from 0.06 to 0.13. Brinkman et al (2016) report an effect of between 0.071 to 0.082 SD for a 1-SD increase in classroom quality in rural Indonesia. Being male, Kazakh, or needing special assistance in the classroom was associated with lower scores. Girls score higher than boys on almost all domains of child development assessed in this study (figure 3.11). The gender gap in outcomes favored girls in each of the five domains tested, but was sizable and statistically significant in the language, executive function, and fine motor skill domains (between 0.15- 0.22 SD units higher), after accounting for all other household-, individual, and school-level characteristics. Children from Kazakh households scored significantly lower than others on language (0.31 SD lower) and social emotional skills (0.46 SD lower), after controlling for all household- and individual-level controls. An encouraging finding, however, was that after adding controls for school level variables such as length of exposure to preschool and quality of services, the Kazakh disadvantage was significantly reduced and became statistically insignificant. Disabilities among children were associated with large gaps in outcomes across all but the fine motor skills domain (the gap ranged from 0.2 to 0.44 SD units). 3.2 ECE outcomes in a home-based school preparation intervention in Mongolia The MELQO assessment instruments were used by Save the Children Japan (SCJ) to assess children who participated in the NGO’s home-based school preparation intervention in Mongolia. The limited potential of formal preschool services to reach remote and isolated subpopulations in Mongolia, including the herder population, has necessitated exploration of alternative modalities of ECE. Mobile or ger kindergartens offer one approach, but their reach is hampered by logistical and financial constraints. The analysis in the preceding section in this chapter underlined the significance of family engagement at home in activities with children, and as such, interventions that boost at-home engagement with children offer a promising approach to improve early development outcomes among those with no access to preschool. In this spirit, between 2012 and 2017, Save the Children Japan (SCJ) implemented a project titled “Improving Primary Education Outcomes for the Most Vulnerable Children in Rural Mongolia”. One of the three project components was a home-based school preparation program targeting herders’ children between 5-6 years of age. The component aimed to enhance school preparation through use of a home- based curriculum, capacity building and technical support for parents and teachers, and community mobilization. See box 4.1 for more details. In 2015, using the same MELQO instruments employed in the WB Kindergarten Survey, SCJ assessed all children that had completed this home-based ECE intervention. 64 The SCJ data provide an opportunity to compare early development outcomes among children in the SCJ intervention to those in the ger-kindergarten sample of the WB Kindergarten Survey. Similar to ger kindergartens, the SCJ intervention targeted children with no access to formal preschool services. One question that then arises is whether it is possible, using the assessment data on children in the SCJ intervention, and the data on outcomes among children enrolled in ger kindergartens presented earlier in this chapter, to explore how effectively participation in the home-based ECE modality improved child development outcomes relative to attendance at a ger kindergarten. A comparison of impacts is not directly possible, as neither intervention has a true control group available to which differences in outcomes can be compared. Children in both interventions were assessed either at the very end or close to the end of the intervention, so that the “end-line” outcomes collected cannot be compared to outcomes at the start of the intervention. These limitations notwithstanding, if it were possible to establish that children in the SCJ intervention were, on average, “identical” or very similar to those in ger kindergartens, in particular with regard to social and economic characteristics and critical individual-level characteristics such as age, then, under the assumption that the only other factor influencing outcomes in the two groups was the nature of the intervention they underwent, a comparison of outcomes across the two samples could be useful. The results from such an exercise should however be treated with extreme caution and as suggestive only, and certainly never as implying impacts attributable to either intervention, as the influence of unobserved confounding factors will bias results. Data collection in the two samples was conducted by two different sets of enumerators, under different field conditions and processes, which limits comparability of the two samples. The findings from the analysis of data on children in the SCJ sample must also be interpreted in the light of several other important points. The first is that the enumerators who collected the data for SCJ were not the same ones as those in the WB Kindergarten survey, although they did participate in the same enumeration trainings. In particular, while SCJ used its own employees to assess children in its home-based ECE intervention, the WB survey employed a third-party survey firm independent of the education ministry. Data on a large number of children had to be excluded from the analysis due to inadequate quality, raising concerns about representativeness of the SCJ sample. Assessment data for a large number of children in the SCJ intervention could not be used due to poor data quality and unresolved data collection issues. SCJ originally assessed 490 children in its intervention, across the project’s 30 soums in 4 aimags. Of these, only 315 questionnaires were deemed to be of adequate quality and completeness for electronic data entry. A key issue was that the parent questionnaire could not be implemented in many cases, and in others, the student and parent data could not be matched due to coding and enumeration errors. As such, the findings presented in this analysis should be treated with caution, as there are currently no data available to assess the extent to which the children included in the sample represent the target population covered under the SCJ intervention. In particular, if low data quality or completeness was in some way connected to poor performance on the MELQO tasks during the assessment, then the results on development outcomes presented in this analysis may be biased upwards. Children in the SCJ sample come from households that are somewhat more socially and economically disadvantaged than those in the ger-kindergarten sample. The first question to answer is whether the SCJ and ger kindergarten samples are indeed comparable. If, for example, the SCJ sample has children from households with higher socioeconomic status, then we know a priori that these children are expected to perform better on the MELQO assessment due to a more supportive home environment at baseline, regardless of the impact of the intervention. Similarly, older children will exhibit superior skills, 65 even in the absence of a targeted intervention. Table 3.2 shows key sample characteristics for children in the two samples. The average age is slightly higher in the SCJ sample, as is the share of boys. Unlike the ger-kindergarten sample which covered the province of Bayan-Ulgii and hence included children from Kazakh households, the SCJ sample does not include children of Kazakh ethnicity, but 10 percent of the sample still consists of children from non-Kazakh minority backgrounds. On other key dimensions of social and economic status, children in the SCJ sample are somewhat worse off than those in the ger- kindergarten sample. 32 percent of mothers and 20 percent of fathers in the SCJ sample had completed high school or higher, compared to 41 percent of mothers and 27 percent of fathers in the ger- kindergarten sample. 16 percent of SCJ children came from a household whose head was not employed at the time of the survey, compared to 10 percent in ger-kindergarten sample. The share reporting animal husbandry as primary occupation was slightly lower in the SCJ sample. The average wealth index score was also somewhat lower. Table 3.2 Sample characteristics for children enrolled in ger kindergartens and the SCJ intervention, Mongolia, 2015 Age (months) Share of girls (percent) Children 36 Mean SD Median Ger kindergarten 72.9 2.1 73 46.5 155 SCJ program 73.6 2.2 74 43.1 248 Ethnicity of HH head Khalkh Kazakh Other minority Ger kindergarten 76.1 19.4 4.5 SCJ program 89.9 0 10.1 Maternal education (percent) Primary or lower Secondary High Undergrad or above Ger kindergarten 17.5 41.6 33.1 7.8 SCJ program 19.4 48.8 28.2 3.6 Paternal education (percent) Primary or lower Secondary High Undergrad or above Unknown Ger kindergarten 27.9 44.8 25.3 1.9 0 SCJ program 32.1 42.7 18.3 2.4 4.5 Job type of HH head (percent) Paid Self-employed Animal None husbandry Ger kindergarten 9.7 5.2 74.8 10.3 SCJ program 9.3 3.6 71 16.1 Mean wealth index score Mean SD Median Ger kindergarten -0.98 0.78 -1.2 SCJ program -1.15 0.69 -1.3 Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) and SCJ (2015) Despite lower socioeconomic status, homes of children in the SCJ intervention exhibited an environment far more supportive of early development than those of children in ger kindergartens. Another important input to production of early development skills is the home environment and family 36 The target age group for the ger-kindergarten sample in the WB Kindergarten Survey was children between the ages of 5-6 years. Most of the children in the SCJ sample were over 6 years of age. In order to make the two samples comparable, it was thus necessary to truncate the samples to limit to an age group where the two samples overlap (annex 11). This was determined to be the ages of 70 – 77 months, and reduced the number of usable observations in the ger-kindergarten sample from 534 to 155 children, and from 315 to 248 in the SCJ sample. 66 engagement at home in ECE-relevant activities. This is also critical because parental engagement was the key mechanism through which the SCJ intervention intended to improve outcomes. Figure 3.12a shows that access to manufactured or store-bought toys was high in both samples (91-93 percent of children), but children in the SCJ sample were notably more likely to play with homemade toys and household objects. Similarly, mothers of children in the SCJ sample were much more likely to engage with children in supportive activities at home (figure 3.12b). Overall, the at-home engagement index score (see box 3.3 for construction of this index) is significantly higher in the SCJ sample by 1.7 SD (significant at the 99 percent level), and the difference remains large and statistically significant even after controlling for the full set of individual- and household-level controls (annex 11). Given that these data were collected at the end of the intervention, there is no way to ascertain that these measures improved over the duration of the intervention and that the improvement was solely caused by the intervention, but the fact that at end-line the SCJ households exhibited a more supportive ECE environment for children than those in the ger-kindergarten group is encouraging. Figure 3.12a Types of toys children play with at Figure 3.12b At-home engagement of mothers with home, among children enrolled in ger children in past three days, among children enrolled in kindergartens and the SCJ intervention, ger kindergartens and the SCJ intervention, Mongolia, Mongolia, 2015 2015 Named/counted/drew 59% 83 17% Household objects SCJ 66 Played 44% Ger 10% Manufactured/store- 91 Spent time outdoors 60% 21% bought 93 Sang songs 66% 28% 65 Homemade Told stories 62% 19 21% SCJ Ger Read or looked at books 72% 0 20 40 60 80 100 24% Percent of children 0% 20% 40% 60% 80% Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) and SCJ (2015) At the end of the SCJ intervention, outcomes in each of the five domains of early development assessed were far superior in the SCJ sample than in ger kindergartens. To the extent that socioeconomic status contributes to child development outcomes, the fact that children in the SCJ sample come from households at a greater social and economic disadvantage than those in the ger-kindergarten sample suggests that improving child outcomes in the SCJ sample would be a more challenging task than in the ger-kindergarten sample. However, as shown in table 3.3, SCJ children, at the end of the intervention, performed significantly better than children in ger-kindergartens in all dimensions of early development skills. The advantage varied from 1.06 SD in executive function to 0.19 SD in fine motor skills. Strikingly, this advantage persists and remains very high even after controlling for all observable individual and household-level characteristics (figure 3.13). 67 Table 3.3 Mean normalized MELQO child assessment scores among children enrolled in ger kindergartens and the SCJ intervention, Mongolia, 2015 (standard deviation units) Cognitive Language Social emotional Executive function Fine motor Ger kindergarten 0.257 0.087 -0.418 -0.022 0.081 SCJ program 1.031 1.01 0.318 1.042 0.267 SCJ program advantage 0.774 0.923 0.736 1.064 0.186 Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) and SCJ (2015) The “SCJ intervention” effect remains Figure 3.13 The SCJ program advantage in mean normalized very large even after controlling for the MELQO assessment scores among children enrolled in ger home environment and at-home family kindergartens and the SCJ intervention, Mongolia, 2015 engagement, in addition to other social 1.2 1.1 and economic characteristics. After Difference in mean scores (SD) 1.0 0.9 controlling for all background factors, 0.8 0.7 0.8 surprisingly, the SCJ indicator coefficient 0.8 0.7 remains statistically significant and large – 0.6 0.5 0.4 larger in fact than any other coefficient 0.4 0.3 (figure 3.13 and annex 11). This is true 0.2 0.2 even after controlling for the influence of 0.0 family engagement at home, which is Cognitive Language Social Executive Fine purportedly the key mechanism through emotional function motor which the impact of the SCJ intervention is No predictors expected to be mediated. Stated With all household and individual predictors differently, if it were true that the primary mechanism through which the effect of Source: Author’s calculations using data from World Bank Kindergarten the intervention was to be mediated was Quality Survey (2015) and SCJ (2015) Note: Bar heights represent the size of the difference in mean MELQO indeed captured by the at-home scores between children in the SCJ and ger-kindergarten samples. All engagement index included in the differences were significant at the 1 percent level. Regression tables are regressions, we would expect the presented in annex 11. coefficient on the SCJ indicator variable to be significantly reduced once the index is included in the regression. While the coefficient does reduce in size by about a third of an SD in each domain of development, the fact that it remains so large and highly significant suggests that there is an ”SCJ intervention effect” that is not explained completely by the engagement index variable used. One possibility is that as the index captured only whether specific ECE-relevant activities occurred in the home, and as it does not capture how long the activity lasted or how frequently it occurred during the period of measurement, the index captures only part of the information needed to describe the true effect of the SCJ intervention on parental engagement at home. 3.3 ECE outcomes among children in the MICS The 2010 round of UNICEF’s Multiple Indicator Cluster Survey (MICS) sheds light on a limited set of ECE outcomes among children under-5 years of age. The 2010 and 2013 rounds of UNICEF’s MICS collected data on a limited set of ECE outcomes in the domains of literary and numeracy, physical ability, 68 socioemotional skills, and learning, among children aged 36-59 months 37. A key strength of these data is that since the MICS is a household survey with national coverage, it opens up the possibility of examining gaps in outcomes between children enrolled in preschool and those not enrolled. The MICS collects ECE outcome data through a total of fourteen binary (yes/no) indicators. As figure 3.14a shows, however, the indicators used pose measurement challenges from the outset. On nine of the fourteen indicators, either close to a 100 percent of children, or very few, are able to perform the task in question. Thus, while the data shed light on a handful of dimensions on which the children either perform very poorly or very well, the binary nature of the measures and their limited number does not permit establishment of a plausible distribution of outcomes across children. Parental/caregiver reports rated lowest performance in the literacy and numeracy domains. On the MICS literacy indicators, parental reports place children very low (figure 3.14a). Alphabet recognition is not included in the preschool curriculum, but the fact that 15 percent of Mongolian children between the ages of 36-59 months could name at least 10 letters of the alphabet shows the emergence of a learning gap that is likely to grow at older ages. The ability to read at least four simple words was even rarer. The MICS numeracy indicators are less stringent than the literacy ones, and we start to see some variation in reported ability. 66-70 percent of children can identify some colors and shapes, respectively; 86 percent can count; and 61 percent were reported to recognize numbers up to 10. In the social emotional domain, while almost all children get along well with other children, 26 percent were reported to kick, bite, or hit other children, easily distracted (29 percent). On physical ability, almost all children could perform the two fine motor tasks examined; and 14 percent were reported to be sometimes too weak to play. In the learning domain, almost all children could follow simple directions on how to do something correctly, or conducted simple tasks independently. Children in rural areas and those not enrolled in preschool lagged significantly behind others. Figure 3.14b presents the differences in MICS ECE outcomes among children across different locations in Mongolia, excluding certain indicators in which either almost all or too few children meet the specified criteria (see figure note). The only domain with a significant level of variation in outcomes is literacy/numeracy, in which children in rural areas lag significantly behind urban ones. Further, children not attending preschool tended to lag behind those attending preschool, and the difference was particularly large in the literacy/numeracy domain. In numeracy, social and economic household factors were important predictors of success, as was enrollment in preschool. With children performing either very well or very poorly on most of the ECE indicators in the MICS, the resulting lack of variation made it difficult to explore differences across children with different demographic and socioeconomic characteristics 38. The only domains where it was possible to conduct such analysis were numeracy and social emotional skills. Compared to the literacy domain, the numeracy indicators are less stringent, and we start to see some variation in reported ability. Similar to the findings using the MELQO instrument, among household level factors, maternal education was a key 37 At the time of writing, data from the 2013 round of the MICS were not made publicly available, but the final report was available. All analysis using the MICS data in this report is thus based on the 2010 round. Where possible, survey results described in the final report for the 2013 MICS are presented. 38 Specifically, the effects of background or individual level factors cannot be precisely estimated. For example, regressing the binary dependent variable “Child knows up to 10 letters of the alphabet?” on even a limited set of household- and individual-level characteristics yields coefficients on which the standard errors are typically far larger than the coefficient itself. 69 determinant, holding other factors constant, and in terms of the ECE environment at home, the number of children books at home (annex 12). For example, children of mothers with a college degree were 16 percentage points likelier to recognize numbers up to 10 relative to those with primary education or less, and availability of between 1-5 children books (or higher) in the home was associated with a 10 percentage point higher probability of meeting the same indicator. Girls were likelier (7 percentage point difference) to meet the criteria than boys, while Kazakh children lagged behind others (16 percentage points), other factors held equal. Notably, the effect of being enrolled in a pre-primary institution was significant and remained large after controlling for all household level characteristics (20 percentage points). Figure 3.14a Percentage of children aged 36-59 Figure 3.14b Percentage of children aged 36-59 months reported able to perform MICS ECE outcome months developmentally on track (country- tasks, Mongolia, 2010 specific criteria), Mongolia, 2010 Independently execute Learning tasks Learning Follow simple directions Social-Emotional Easily distracted Social-Emotional Kick/bite/hurt others Get along w/ other children Sometimes too weak to Physical** play Physical Hold spoon, etc. Pick up small object Literacy/numeracy* Recognize numbers up to 10 0 20 40 60 80 100 Numeracy Count Percent (%) Idenify shapes Pre-school: Not attending Identify colors Pre-school: Attending Location: Rural center Read >=4 words Literacy Location: Rural center Recognize >=10 letters Location: Soum center 0 20 40 60 80 100 Location: Aimag center Percent (%) Location: Capital city Source: Authors' calculations using MICS (2010) Source: UNICEF (2013) Notes: * Literacy-numeracy: Developmentally on track if at least two of the following are true: the child can identify some colours, or identify simple shapes, or count. Indicators excluded were recognition of at least ten letters of the alphabet, read at least four simple words, and recognize numbers up to 10. ** Physical: Developmentally on track if at least two of the following is true: A child can pick up a small objects, or is not sometimes too sick to play 70 Boys lagged behind girls in social emotional skills, while preschool enrollment and other background factors had no significant influence. Household socioeconomic factors had no influence on performance on negative social behaviors, but girls continued to outperform boys (6 percentage points less likely to exhibit negative social behaviors). Unlike in the WB survey sample, there was no effect of preschool exposure. While direct comparisons are not possible, the results from the MICS survey on numeracy outcomes echo the findings of the WB survey. A direct comparison of the WB survey with the MICS is not possible or valid due to a number of reasons. One, the number of items assessed in the MELQO instrument is far higher than that in the MICS in each domain. Second, the children in the MICS are younger. The MICS covers children under 5 years of age, younger than the WB study sample of children aged between and including 5 and 6 years 39. Even so, the fact that differences in numeracy outcomes are sizable even in the younger sample in the MICS is a warning that gaps in learning start at the earliest ages. These differences notwithstanding, the numeracy results from the MICS survey echo the findings of the WB survey. The gender gap is prominent, as is the role of maternal education. Preschool enrollment matters even after controlling for household- and individual-level characteristics. In the social emotional skills domain, the only other domain in which a qualitative comparison is possible, only the gender gap is significant in the MICS data. The WB survey, on the other hand, showed effects of a number of social and economic characteristics, and a sizable effect of exposure to preschool. These results may differ due to a number of reasons, but may largely be due to the fact that the items assessed in the two surveys were completely different 40. 3.4 Conclusions For children in public kindergartens, quality was the only factor among kindergarten characteristics that was significantly associated with higher outcomes in all five domains of early development. The magnitude of the association of outcomes with quality in kindergartens varied, but was consistently significant at the 1 percent level across four of the five domains of early development assessed; for fine motor skills, it was significant at the 5 percent level. Effects were largest in the social emotional skills domain, followed by language. These findings echo similar findings from settings across the world, and highlight the key takeaway message that access to preschool matters, but will improve early development outcomes only if preschool services are of adequate quality. Longer exposure to preschool was also associated with better outcomes, and was statistically significant in the numeracy, literacy, and social emotional domains. Duration of kindergarten exposure was 39 A final difference is that outcomes in the MICS were reported by parents or primary caregivers, while children were directly assessed in the WB study. A parent may or may not be aware of specific skills a child has, which introduces the potential for error in reporting, especially in literacy and numeracy skills where household factors may limit opportunities for parental observation of the child’s writing and reading skills. Both approaches – parent reports vs. trained assessors – have pros and cons, but, net of these, it appears that the MELQO instrument allows a more effective mapping out of the distribution of outcomes across the children surveyed, which in turn allows for more effective measurement of the gaps between children from different backgrounds. 40 In the MELQO assessment, children were asked to name friends, and to verbally react to hypothetical situations (sharing, empathy towards others’ emotions, understanding own feelings) in order to gauge emotional responses. This had no overlap whatsoever with the MICS indicators (figure 3.12a). In numeracy, the MICS items were also present in the MELQO instrument, even if in a somewhat modified fashion. The only exception was ability to identify colors, which was not assessed in the MELQO instrument. 71 positively associated with early development outcomes, over and above the effect of kindergarten quality, and after controlling for all other factors. The effect was statistically significant for three of the five domains assessed: numeracy (1 percent level), literacy, and social emotional skills (5 percent level). The socioeconomic status of a child’s household continued to exert a significant influence on outcomes in all domains, after controlling for kindergarten-level factors. Other than kindergarten quality, attainment of a college degree by a child’s mother was the only factor significantly associated with higher outcomes in all five domains of early development, after controlling for the full set of household-, individual- and kindergarten-level factors. As the coefficient on the college degree indicator variable represents the difference in outcomes relative to mothers with primary education or lower, it essentially captures the inequality in the broader Mongolian population between those with the lowest levels of educational attainment and those with the highest 41. Regardless of how the parental education variables were entered into the regression models, they were always jointly significant in all five domains. Household wealth was also associated with higher outcomes, but the coefficients were significant for numeracy and executive function domains only, and marginally significant for literacy. Overall, the effects of parental education are an encouraging finding, in the sense that general education sector investments that effectively improve educational attainment in the Mongolian population will also tend to have an intergenerational positive impact on early childhood outcomes, in the long term. Higher at-home family engagement was associated with better performance on cognitive and language skills. Engagement with children on activities at home was positively associated with higher outcomes in all domains, but the effect was statistically significant only for cognitive and language skills. In social emotional skills, the effect was marginally significant. This suggests that a home-based ECE intervention that increases intensity of family engagement with children on ECE-relevant activities at home would have a beneficial effect on early development outcomes, independently of whether a child is enrolled in kindergarten or not. But such interventions may be especially relevant for the poorest and most disadvantaged households, where parental engagement in activities important for ECE outcomes is considerably lower than in richer households. While concerns about sample representativeness cannot be ignored, the findings on superior outcomes exhibited by children completing SCJ’s home-based school preparation program are very encouraging. Analysis of data from SCJ’s home-based school preparation intervention provides limited but very encouraging evidence on the potential for home-based school ECE interventions to improve child development outcomes in places with no or little access to formal preschool services. While concerns with data quality and representativeness of the SCJ sample should not be ignored, and hard evidence on impacts is not available, the data do show that when compared to attendance at ger kindergartens, participation in the SCJ intervention was associated with a more supportive at-home ECE environment at end-line, improved family engagement with children in activities that nurture early development, and significantly superior development outcomes in all five domains of skills assessed. Evidence from both the WB and SCJ surveys underscores the potential for home-based ECE modalities to improve outcomes among Mongolian children outside the reach of kindergarten services. Given the developmental benefits associated with at home-family engagement and the SCJ model of home-based 41 In the 2013 MICS, 8.2 percent of adult women had primary or lower education only, while 34 percent had completed college (UNICEF, 2016). 72 school preparation, a household-based approach may be applicable more broadly for improving school readiness among Mongolian children with limited or no access to formal preschool services. Chapter 4 discusses some options to this end. It is important to note, however, that a home-based intervention may not be able to provide all the benefits of formal preschool. Preschool provides an opportunity for children to interact with one another and to develop socio-emotional skills, including exposure to team work, conflict-resolution, different types of individuals, etc., in a way that home environments – even in the most educated/wealthy households – may not provide. In the WB survey data, social emotional skills was the domain with the highest coefficient on kindergarten quality, indicating that this domain was particularly sensitive to preschool environment. Notably, children in the SCJ sample did exhibit social emotional skills superior to children in the ger-kindergarten sample. Future assessments of the SCJ model should identify design features that may have facilitated development of these skills. Children in ger kindergartens lagged significantly behind others more than mid-way through operation, and are thus unlikely to catch up given the typical duration of ger kindergarten operation. Chapter 2 found that ger kindergartens perform lower than fixed ones on all assessed domains of process and structural quality except Interactions, and that logistical challenges of operation limit scope for quality improvement. Further, given the brief duration of operation of ger kindergartens, and to a lesser extent, the limitations in their learning environments, chapter 2 cautioned against high expectations regarding their impact on outcomes. The present chapter adds to this by showing that children assessed in ger kindergartens do lag significantly behind others. Ger kindergartens included in the survey had been operating for 2-3 weeks when ECE outcomes were assessed. While the survey by design cannot measure the impact of services, the finding that mid-way through operation42, outcomes stood significantly below those in fixed kindergartens indicates that impact is likely to be limited, and certainly will not bring enrolled children even marginally close to the outcomes observed among children after 1-2 years of kindergarten exposure. Further, at present, there is no evidence available from anywhere in the world on positive impacts of preschool with less than two months of exposure, even in the best possible quality settings. The limitations notwithstanding, ger kindergartens are valued highly by parents, and plans for future provision should evaluate their costs and benefits in relation to other ECE modalities. Mongolia has shown a real commitment to serving difficult-to-reach populations through impressive efforts in recent years to expand the coverage of ger kindergartens. These have been received enthusiastically by parents, with high satisfaction rates (World Bank, 2015). For the vast majority of children from herders’ families, these kindergartens are the only possible opportunity for exposure to preschool and development of social emotional skills in particular, which are markedly less sensitive to home-based interventions. At the same time, given where ger kindergartens stand now in terms of duration of exposure and quality of learning environment, prospects of enabling enrolled children to catch up with others remain low. A key challenge is that even if the duration of exposure were increased, cost implications would be significant 43. Further, the space constraints and challenging logistics of operating these kindergartens make it practically impossible to close the quality gap. Future provision of these kindergartens and decisions about 42 The vast majority of ger kindergartens do not operate for more than four weeks in the summer. 43 Doubling the exposure time would roughly double recurrent expenditure from about 658 million Tugrik per capita per year (chapter 1) to over 1,030 million Tugriks, i.e. it would make ger kindergartens more expensive than regular ones, in per capita terms. 73 ECE financing should take all costs and benefits into account, including those associated with alternative ECE modalities such as community- or home-based interventions. In private kindergartens, the ger-area disadvantage in outcomes is consistent with quality differences between kindergartens in ger and non-ger areas of UB. Chapter 2 showed that while on average private kindergartens perform similar to public ones in terms of quality, private kindergartens serving a richer population in non-ger areas of UB are of higher quality than those serving the poorer, ger areas. The present chapter shows that early development outcomes among children in ger-area private kindergartens lag far behind those in non-ger areas. This suggests that factors influencing quality and hence outcomes will operate differently than those in the public sector, and will need careful consideration when planning for an expanded role of the private sector in provision of ECE services. Early development outcomes exhibit disadvantages related to gender, ethnicity, and disabilities among children. The male disadvantage in domains of language, executive function and fine motor skills warrants further investigation, given that a male disadvantage is also observed at higher levels in terms of educational attainment of the adult population. Qualitative research work would also shed light on any social or cultural factors within the home or school environment that might help explain the gender gap. Also, Kazakh children lag significantly behind others in the language and social emotional domains, but after controlling for supply side factors, this gap disappears. This is encouraging, and indicates that the minority disadvantage can be reduced through targeted quality improvements. In addition, the special needs of children with disabilities will also require targeted actions. 74 Chapter 4 Key Action Areas Moving Forward Earlier chapters of this report examined kindergarten quality and early development outcomes among Mongolian children to assess the effectiveness and equity of public investments in early childhood education (ECE), which are high in Mongolia when compared to peer countries. The analysis shows that while progress has been made in improving access to preschool overall in the country, the most disadvantaged and vulnerable are still excluded from the system. Further, an assessment of outcomes shows that even after significant exposure to formal preschool services in the public sector, socioeconomic gaps in outcomes remain large. At the same time, potential areas where relatively low- cost investments could reap significant gains remain unexploited. Key among these is use of home- or community-based ECE interventions that could improve school preparedness among children in rural areas not enrolled in preschools, for example the home-based school preparation intervention implemented by Save the Children Japan (SCJ). Another is the potential for an expanded role for the private sector in urban areas. Table 4.1 summarizes the key actions areas proposed, and the present chapter discusses these and other key recommendations moving forward. Table 4.1 Summary of key action areas to improve the reach and quality of ECE services in Mongolia Key action area Actions Outputs or impacts desired 1. Improve access to ECE in rural Deploy a community- or Improve school preparedness in hard-to-reach areas home-based modality populations such as nomadic herders targeting the household environment Evaluate pros and cons of ger Improve efficiency of public spending on ECE; kindergartens relative to those improve access to ECE services in herder population of other ECE modalities 2. Improve access to formal ECE Invest to expand fixed Improved access to ECE services; reduce congestion services in urban areas, with a kindergarten services in the in classrooms. potentially expanded role for the public or private sectors private sector Prepare for expanded role of Reduce quality gaps in the private sector the private sector, and design incentives for quality 3. Improve quality of fixed public Develop a quality assurance Monitor compliance with quality standards and kindergartens through investments mechanism assess quality on a regular basis targeted to rural areas Target quality improvements Improve equity and effectiveness of current public to rural areas investments Improve the math and Improve cognitive outcomes numbers environment Improve exposure to numbers Reduce socioeconomic gaps in cognitive and and letters in classroom language outcomes practice, using existing child- centered approach 4. Undertake further analytical ECE financing norms and Generate information to enable design of targeted work in key areas demand-side financing options interventions to improve equity of public Male disadvantage in early investments development Specific needs due to ethnicity and disabilities 75 Key action area 1: Improve access to ECE in rural areas through a tested community- or home-based ECE modality Addressing inequities in access to ECE services through an approach targeted to rural areas is urgently needed. Using data from the MICS surveys, this report confirms that access to formal preschool services in rural areas of Mongolia remains very low, and that children not enrolled in preschool come from some of the most socioeconomically disadvantaged communities in the country. This report also showed that even among those enrolled in kindergartens, early development outcomes are lowest among those from low socioeconomic status households. Thus outcomes among children with no or little exposure to preschool are likely to be even lower than those described in this report. Finally, children enrolled longer in formal preschool exhibit superior early development outcomes compared to those with less exposure, an effect that remains after controlling for most household-, individual-, and kindergarten-level characteristics. The gaps in development outcomes implied by these findings, if left unchecked, will only grow over time, as shown by existing micro-economic literature that confirms the life-long impacts of cognitive and non-cognitive development in early childhood. Improving access to ECE services in rural areas, the most disadvantaged in the country, should thus be a priority. In rural areas, including in hard-to-reach populations such as nomadic herders, community- or home- based modalities targeting the household environment should be explored. The analysis in this report shows that engaging children in activities such as reading, singing, and playing, outside of kindergartens and within the child’s home, can boost early cognitive and language skills. It is possible that these effects would extend to community-level interventions of adequate quality that target improvements in the home environment. Indeed, in the home-based school preparation program currently being implemented by Save the Children Japan (box 4.1), parents anecdotally report increased attentiveness to and engagement with reading materials, and improved communication skills and confidence. While the program has not been rigorously evaluated through a randomized control trial approach, or measurement of baseline and end-line ECE outcomes, assessment data indicate that when compared to attendance at ger kindergartens, participation in the SCJ intervention was associated with a far more supportive at-home ECE environment at end-line, improved family engagement with children in activities that nurture early development, and significantly superior development outcomes in all five domains of skills assessed – even after controlling for all social, economic, and individual characteristics. This is consistent with informal feedback from grade-1 primary school teachers, who report markedly higher school preparedness among children covered under the intervention, compared to children from similar backgrounds with no or little prior exposure to preschool. Indeed, teachers with experience with children exposed to ger kindergartens and to those in the SCJ intervention reported increased school preparedness in the latter group 44. Box 4.1: Two examples of home-based school preparation programs A home-based school preparation program for primary school entrants in rural Mongolia 44 Teacher reports assert that mixing of different age groups in ger kindergarten, combined with the short duration of exposure, reduces the effect of the intervention relative to that of SCJ’s home-based one, in which children receive one-on-one attention from a parent, sustained over a period of 5-6 months. 76 Between 2012 and 2016, Save the Children Japan (SCJ) implemented a project titled “Improving Primary Education Outcomes for the Most Vulnerable Children in Rural Mongolia” with a grant from the Japan Social Development Fund (JSDF). One of the three key project components is “Strengthening School Preparation Programs for New School Entrants (5-6 years of age) in the Community”. This component aims to enhance home-based school preparation through use of a home-based curriculum, capacity building and technical support, and community mobilization. The project was implemented in 30 soums across four aimags. Parents play the role of preschool teachers. A guidance book for parents and a workbook for children are provided with methodological assistance and guidance for parents. Additional digital tools (recoded instructions) are also provided to support both parents and children. Parents are recommended to spend 20-30 minutes a day and total of 3-4 productive hours with their children in a week. A “mobile toy and book library” was developed to enhance school preparation through use of toy and book kits at home. Ten sets of boxes containing 10 children’s books and 3 toys each were distributed to one kindergarten per soum center. Parents borrow the kits and work at home with their children for one to two weeks. Child development is recorded in a special notebook by parents. In the notebook, parents answer questions, and children draw pictures about storybooks read by parents. Librarians then check the notebooks when parents come to the library for changing the kits. At the end of the program every child will have had a chance to be exposed to 30 new educational toys and 100 books which facilitates development of cognitive and language skills. Children are assessed on cognitive, behavioral, physical and socioemotional development at the start and end of the program through use of a simple questionnaire. Mentor teachers at a soum kindergarten carry out this assessment along with a child development notebook after completion of the program, which lasts 96 days, and usage of all kits. In Uyanga soum of Uvurkhangai aimag (reported in May 2015), there was an increase of 26 percentage points in the overall development among children who participated in the program (the assessment report and data are not yet available) 45. National scale-up of Turkey’s Mother-Child Education Program In Turkey, the recently scaled-up highly successful Mother Child Education Program (MOCEP) is a 25 week-long training program for mothers and their 5-6 year-old children. The Program was found to have positive effects on children’s overall school readiness and on mothers’ interest in school, self- esteem, and communication with husbands. The Mother Child Education Foundation (ACEV) provided training of trainers, local coordinators, and supervisors, and overall program management and quality control services. The Turkish Ministry of National Education (MoNE) provided for its adult trainer staff to be trained by ACEV as group leaders, and space for courses to be held at the 500+ adult education centers all around Turkey. The World Bank provided initial funding for expansion. Between 1993 and 2010, ACEV trained over 1,100 group leaders and 150 local coordinators, and MOCEP was scaled up to 71 of 81 Turkish provinces, reaching over 310,000 mothers and children. During this period, $17 million 45 The intervention has introduced another innovation in delivery of ECE services in Mongolia. The project created a community education council (EEC) which linked schools, families, and kindergartens in each soum. Mentor teachers were trained for the program and play important roles including instructing parents and assessing children. Before the project these three parties were disconnected, but children’s transition from preschool to primary schools was supported with the EEC’s role. 77 was spent on all management and implementation costs, averaging $56 per beneficiary. In 2010, MOCEP was nationalized within MoNE’s larger National Family Program developed with ACEV’s support, providing an opportunity to go to further scale. Per beneficiary costs had dropped below $40 before this transfer to MoNE, as economies of scale were fully in place, reaping benefits of system investments made in previous years. Outside of Turkey, through local partnerships, MOCEP was translated into English, Arabic, and Spanish and reached over 12,000 mothers and children in Europe (Belgium, France, Switzerland), the Middle East (Bahrain, Jordan, Lebanon, Saudi Arabia), and Latin America (Mexico). MOCEP’s content and methodology also inspired ACEV to develop additional parenting (including fathers), adult training, and ECE programs, including TV/internet/mobile-based ones. Through face to face programs only, ACEV has reached over 1 million beneficiaries and trained nearly 10,000 group leaders to deliver these courses in Turkey and 13 in other countries (Germany, Netherlands, Turkish Republic of Northern Cyprus, UK and Laos in addition to the eight mentioned above). Future planning for financing and providing ger kindergartens in hard-to-reach populations should be based on an evaluation of costs and benefits relative to those of other ECE modalities. Mongolia has shown a real commitment to serving difficult-to-reach populations through impressive efforts in recent years to expand the coverage of ger kindergartens. These have been received enthusiastically by parents, with high satisfaction rates. For the vast majority of children from herders’ families, these kindergartens are the only possible opportunity for exposure to preschool and development of social emotional skills in particular, which are markedly less sensitive to home-based interventions. At the same time, as this report has argued, the short duration of exposure and lower quality of learning environment drastically reduces the chances that enrolled children will catch up with children in fixed kindergartens. A key challenge is that increasing exposure time will make the intervention prohibitively expensive. Further, the space constraints and challenging logistics of operating these kindergartens make it practically impossible to close the quality gap. Future provision of these kindergartens and decisions about ECE financing should take these costs and benefits into account, including those associated with alternative ECE modalities such as community- or home-based interventions. Key action area 2: Improve access to formal ECE services in urban areas, with a potentially expanded role for the private sector In urban areas, especially Ulaanbaatar, investments to expand fixed kindergarten services in the public or private sectors could improve access, but cost implications need to be explored. In urban areas, further expansion of fixed kindergarten services could be considered, possibly through construction of new public kindergartens. This is especially true of Ulaanbaatar, where 28 percent of children between the ages of 3 and 5 years are out of preschool. While higher population densities in urban areas should allow provision of services at lower cost than in rural areas, the implications for the already high expenditure on ECE services should be further explored 46. Also, as discussed in this report, urban Mongolia has experienced tremendous growth in provision of kindergarten services in the private sector. Thus 46 At time of writing, expenditure data disaggregated at the province and capital city levels were not available to explore the cost implications. 78 public financing of privately-owned kindergartens is also an option that could be used to improve access. Further work is needed to understand the constraints faced in expansion of private kindergartens. The difference in quality between private kindergartens in ger- and non-ger areas of UB highlights the need for design of incentives to encourage quality rather than just increased enrollments. This report showed that while on average private kindergartens perform similar to public ones in terms of quality, private kindergartens serving a richer population in non-ger areas of UB are of higher quality than those serving the poorer, ger areas. The analysis also shows that children in ger-area private kindergartens lag far behind those in non-ger areas. Thus while the potential for an expanded role of the private sector should be considered, care should be taken to provide the right incentives, oversight, and/or technical assistance to ensure service provision of adequate quality, and minimized quality differences. Further work is needed to understand factors constraining quality in the private sector. Key action area 3: Improve quality of fixed public kindergartens through investments targeted to rural areas A quality assurance mechanism is needed to systematically monitor compliance with quality standards and assess quality in the public and private sectors. Mongolia has fairly well-developed standards governing both structural and process aspects of quality. However, without regular monitoring for compliance or assessments of quality, it is not possible to identify areas for improvement for targeting of future investments. To ensure that the observed quality differences documented in this report do not become larger over time, and to target resources to address the most urgent needs, it will be important for MECS to develop a mechanism to monitor quality on a regular basis, in both public and private kindergarten across different locations. For example, a quality scorecard could be developed for use at the kindergarten level. The data collected through the scorecard could be collated by provincial-level or other local education authorities, and then transmitted to MECS following the existing procedures governing flow of data underlying the annual production of the education and science statistical yearbook. In order to improve the equity and effectiveness of current public investments, efforts to improve quality should target rural areas. This report showed that kindergarten quality matters for child outcomes in Mongolia, but that quality is low in at least a third or more of kindergartens (table 2.6). These are concentrated in rural areas. While the Ministry’s recent early childhood curriculum reform program is a step in the right direction, a national reform affecting all kindergartens dilutes the focus on rural areas, which need the most attention. Interventions targeted to rural kindergartens should help reduce the urban-rural gap observed in child development outcomes. Quality improvement efforts could also focus on the math and numbers environment. Kindergartens across the country perform lowest on the math and numbers environment. The Ministry’s recent reform effort has focused on training teachers on social emotional development of children, and language skills. For example, kindergartens were recently provided with books and language materials. Cognitive skills take a back seat, and executive function even more so. These are precisely the domains in which socioeconomic status and other background factors of children in our sample influence outcomes significantly more than preschool exposure and differences in kindergarten quality. Targeted interventions to rural kindergartens, with a strengthened focus on the math environment, will help further reduce the urban-rural gap in child development outcomes. 79 Current exposure to numbers and letters in classroom practice should be re-considered, given the large socioeconomic gaps in cognitive and language skills, which, left unchecked, will widen with age. The large socioeconomic gaps especially on math and language skills indicate that children in public kindergartens perform at very different levels. Evidence from around the world suggests that these gaps persist and remain largely unchanged upon entry into school. Even though number or letter recognition, for example, is not part of the official curriculum in Mongolia, children from better-off households demonstrate good working knowledge of these. The report confirms that learning gaps start early and exist along dimensions of socioeconomic status, and that they remain sizable even after significant exposure to preschool 47. Children from poorer backgrounds are unlikely to catch up at later ages, and the current preschool environment is not helping bridge these gaps. To help reduce these disparities, it may be necessary to re-evaluate classrooms methods, especially those underlying math skills, executive function, and language. It is critical however to note that this shouldn’t translate into drilling children on numbers, rote learning, or create a high-pressure environment in the classroom. Mongolian kindergartens already incorporate child-centered approaches into the classroom experience, and this emphasis should be maintained. Reducing group size in public kindergartens may help improve the impact of preschool exposure, although this will also require expansion of fixed kindergarten services. In our public kindergarten sample, group size did not turn up as significant predictor of assessment scores. This may be due to the fact that most kindergarten groups were very large; the minimum group size was 21. Thus there really is no variation in our data to test whether children develop skills better at a reasonably low group size of 20 or less. Further, as seen earlier, the activities and space and interactions subscales of the ECEMI reflected constraints related to overcrowding in the classroom, so the quality score entered into the regressions also captures some of the expected effect. Finally, it is possible that kindergartens in Mongolia are simply operating with sizes that are beyond some a threshold after which class sizes don’t matter. Overall though, it is clear that big group sizes make it difficult if not impossible for the teacher to give adequate one-on- attention to children, so relieving space constraints and accommodating enrolled children in a larger number of smaller groups may be important to improving the quality and impact of services. Financing options for quality improvement efforts should explore the potential for parental contributions, with due consideration to welfare impacts of increased out-of-pocket payments. Private contributions to the system currently cover learning materials and supplies, and will cover meals in the future. Further potential for parental contributions should be explored, to supplement public investments made to improve quality. As demand for preschool services outstrips supply, given the congestion in classrooms and reports of unmet demand from kindergarten authorities across the country, it is likely that parents would be willing to make increased contributions, provided these are allocated transparently and effectively towards the most urgent quality needs. At the same time, the welfare implications of out-of- pocket expenditure should be monitored. Those currently out of the system are the most economically disadvantaged, so monitoring is needed to ensure that private expenditures are not prohibitively high or pose a barrier to entry into preschool. Contributions could also be linked to socioeconomic status, with lower contributions from the more economically disadvantaged communities (see key action area 4). Clear state guidance is needed to regulate private contributions in the sector. 47 In the public fixed kindergarten sample, most children had been in preschool for at least a couple of years (chapter 2). 80 Key action area 4: Undertake further analytical work to examine ECE financing norms, and address gender and other disadvantages in early development outcomes ECE financing norms and options for demand-side financing need further examination in the context of overall financing of the education sector, with an eye towards improving equity. Chapters 1 and 2 showed that large differences in kindergarten quality and outcomes persist across the country, indicating that existing norms based on location or type of kindergarten are not effective in addressing inequities. They also do not address the higher financial burden on families from remote areas, herders, and others of low socio-economic status. This suggests a need to examine the equity implications of the current financing mechanism and its allocation rules. Mongolia can also look to the experience of countries across the world that have adopted a variety of strategies to address inequities. Countries as diverse as Australia, Hungary, United States, and United Kingdom all provide subsidies to providers (public or private) or parents to access services, with varying degrees of targeting. Demand-side solutions such as conditional cash transfers or vouchers should be considered. Linkages may be possible through existing social welfare programs, for example the Child Money Program. Programs can be targeted to, weighted in favor of, or means-tested for disadvantaged families. Measures can also include prescribing income eligibility rules for public ECE services, and settling the amounts of co-payments (either in absolute terms or as a sliding scale proportion of the costs of the program). In Australia, education providers receive supplementary per capita funding on an as-needed basis to accelerate educational outcomes for Indigenous Australians. In Brazil, additional resources are provided to localities that are unable to guarantee a minimum level of expenditure per child per year. Further analytical work is needed to understand why boys lag behind girls in certain dimensions of early development. The male disadvantage in domains of language, executive function and fine motor skills warrants further investigation, given that a male disadvantage is also observed at higher levels in terms of educational attainment of the adult population: 34 percent of women aged 15-49 years in the 2013 MICS had a attained a college or university degree, compared to 21 percent of men (UNICEF, 2013). Early grade assessments to be conducted during the 2016-17 school year as part of the Ministry’s Education Quality Reform Project will shed further light on whether girls tend to outperform boys in the early grades as well. Qualitative research work would also shed light on any social or cultural factors within the home or school environment that might help explain the gender gap. Targeted interventions are needed to address disadvantage associated with ethnic status, and special needs related to disabilities among children. Kazakh children lag significantly behind others in the language and social emotional domains, but after controlling for supply side factors, this gap disappears. This is encouraging, and indicates that the minority disadvantage can be reduced through targeted quality improvements. In addition, the special needs of children with disabilities will also require targeted actions. 81 References Aboud, F. E. 2006. “Evaluation of an Early Childhood Preschool Program in Rural Bangladesh.” Early Childhood Research Quarterly 21 (1): 46-60. Aboud, F.E. 2007. “Evaluation of an Early Childhood Parenting Programme in Rural Bangladesh.” Journal of Health Population and Nutrition 25: 3-13. Almond, Douglas, and Janet Currie. 2011. "Human Capital Development before Age Five." In Orley Ashenfelter and David Card, eds., Handbook of Labor Economics. North Holland: Amsterdam, pp. 1315- 486. Behrman, Jere, Lia Fernald, and Patrice Engle. 2013. "Preschool Programs in Developing Countries." P. Glewwe, Education Policy in Developing Countries. Chicago: University of Chicago Press. Berlinski, Samuel, and Norbert Schady, eds. The Early Years: Child Well-being and the Role of Public Policy. Palgrave Macmillan, 2015. Bernal, Raquel. "The effect of maternal employment and child care on children's cognitive development." International Economic Review 49.4 (2008): 1173-1209. Bernal, Raquel. "The impact of a technical training program for childcare providers on children’s wellbeing." Documentos CEDE. Bogotá: Universidad de los Andes (2010). Bouguen, A., D. Filmer, K. Macours, S. Naudeau. 2013. Impact Evaluation of Three Types of Early Childhood Development Interventions in Cambodia. Washington, DC: World Bank Policy Research Working Paper Series 6540. Brooks-Gunn, Jeann, Flavio Cunha, Greg Duncan, James Heckman, and Aaron Sojourner. 2006. “A Reanalysis of the IHDP Program.” Unpublished manuscript, Infant Health and Development Program, Northwestern University, 2006. Carneiro, P.M., and J.J. Heckman. 2003. “Human Capital Policy” in J. Heckman and A. Krueger (eds) Inequality in America: What Role for Human Capital Policy? MIT Press, Cambridge. Case, Anne, and Christina Paxson. 2008. “Stature and Status: Height, Ability, and Labor Market Outcomes.” Journal of Political Economy 116(3): 499-532. Cunha, Flavio, James Heckman and Lance Lochner. 2006. “Interpreting the Evidence on Life Cycle Skill Formation.” In Eric Hanushek and Finis Welch, eds., Handbook of the Economics of Education. North Holland: Amsterdam, pp. 697–812. Cunha, Flavio, and James Heckman. 2007. “The Technology of Skill Formation.” American Economic Review 97(2): 31-47. Currie, J. and D. Thomas. 1999. “Early Test Scores, Socioeconomic Status and Future Outcomes.” NBER Working Paper 6943. National Bureau of Economic Research, Cambridge, MA. Duncan, Greg J., and Katherine Magnuson. 2013. “Investing in Preschool Programs.” Journal of Economic Perspectives 27(2): 109-32. 82 Engle, Patrice, Maureen Black, Jere Behrman, Meena Cabral de Mello, Paul Gertler, Lydia Kapiriri, Reynaldo Martorell, and Mary Eming Young. 2007. “Strategies to Avoid the Loss of Developmental Potential in More Than 200 Million Children in the Developing World.” The Lancet 369(9557): 229-42. Engle, Patrice, Lia Fernald, Harold Alderman, Jere Behrman, Chloe O'Gara, Aisha Yousafzai, Meena Cabral de Mello, Melissa Hidrobo, Nurper Ulkuer, Ilgi Ertem, S, Iltus, and the Global Child Development Steering Group. 2011. “Strategies for Reducing Inequalities and Improving Developmental Outcomes for Young Children in Low and Middle Income Countries.” The Lancet 378(9799): 1339-53. Feinstein, L. 2003. “Inequality in the Early Cognitive Development of British Children in the 1970 Cohort.” Economica 70 (1): 73-97 Fernald, L., P. Kariger, P. Engle, and A. Raikes. 2009. “Examining Early Child Development in Low-Income Countries: A Toolkit for the Assessment of Children in the First Five Years of Life.” Washington, DC: World Bank. Fernald, Lia, Ann Weber, Emmanuela Galasso, and Lisy Ratsifandrihamanana. 2011. “Socioeconomic Gradients and Child Development in a Very Low Income Population: Evidence from Madagascar.” Developmental Science 14(4): 832-47. Ghuman, S., Behrman, J. R., Borja, J. B., Gultiano, S., & King, E. M. (2005). Family Background, Service Providers, and Early Childhood Development in the Philippines: Proxies and Interactions. Economic Development and Cultural Change , 54 (1), 129-164. Grantham-McGregor, Cheung, B., Cueto, Glewwe, Richer, & Trupp. (2007). Developmental potential in the first five years for children in developing countries. The Lancet , 369 (9555), 60-70. Halpern, R., Barros, F. C., Horta, B. L., & Victoria, C. (1996). “Desenvolvimento neuropsicomotor aos 12 meses de idade em uma coorte de base populacional no Sul do Brasil: diferenciais conforme peso ao nascer e renda familiar. Saúde Pública , 12 (1), S73-S78. Heckman, J. 2006. “Skill Formation and the Economics of Investing in Disadvantaged Children.” Science 312 (5782): 1900–02. Heckman, James J. 2007. “The Technology, and Neuroscience of Human Capital Formation.” Proceedings of the National Academy of Sciences, 104(33): 13250-13255. Heckman, James J. "Schools, skills, and synapses." Economic inquiry 46.3 (2008): 289-324. Heckman, J. J., and D. V. Masterov. 2007. “The Productivity Argument for Investing in Young Children.” Applied Economic Perspectives and Policy 29(3): 446–493. Heckman, James J. "Invest in early childhood development: Reduce deficits, strengthen the economy." The Heckman Equation 7 (2012). Jaramillo, A., and K. Tietjen. 2001. Early Childhood Development in Africa: Can We Do More and Better for Less? A Look at the Impact and Implications of Preschools in Cape Verde and Guinea. World Bank Africa Region Human Development Series. Washington, DC: World Bank. Jung, Haeil; Hasan, Amer. 2014. The Impact of Early Childhood Education on Early Achievement Gaps : Evidence from the Indonesia Early Childhood Education and Development Project. World Bank, 83 Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/18340 License: CC BY 3.0 IGO. Liddell C, and Rae G. (2001). Predicting early grade retention: a longitudinal investigation of primary school progress in a sample of rural South African children. British Journal of Educational Psychology, 71 (3), 413- 28. Lynch, R.G. 2005. “Early Childhood Investments Yield Big Payoff.” Policy Perspectives. San Franciso, CA: WestEd. Ministry of Education and Science (MES). 2014. Statistical Yearbook Education and Science: 2013-2014 Academic Year, 2013 Fiscal Year. Ulaanbaatar, Mongolia. Naudeau, Sophie, Sebastian Martínez, Patrick Premand, and Deon Filmer. 2011. “Cognitive Development among Young Children in Low-Income Countries. In No Small Matter: The Impact of Poverty, Shocks, and Human Capital Investments in Early Childhood Development, Harold Alderman, ed., The World Bank, Washington, DC. Paxson, C., and N. Schady. 2007. “Cognitive Development among young Children in Ecuador: The Role of Wealth, Health, and Parenting.” Journal of Human Resources 42 (1): 49-84. Pianta, R.C. and S.J. Mcoy. 1997. “The First Day of School: The Predictive Validity of Early School Screening.” Journal of Applied Developmental Psychology 18 (1): 1-22. Reynolds, A.J., J. A. Temple, D. L. Robertson, and E. A. Mann. 2001. “Long Term Effects of an Early Childhood Intervention on Educational Achievement and Juvenile Arrest- a 15- Year Follow-Up of Low- Income Children in Public Schools,” Journal of the American Medical Association 285 (18): 2339-46. Save the Children (2006) Education of Kazakh children: A situation analysis. United Kingdom. Schady, Norbert. 2011. “Parental Education, Vocabulary, and Cognitive Development in Early Childhood: Longitudinal Evidence from Ecuador.” American Journal of Public Health 101(12): 2299-307. Schady, N., Behrman, J. R., Araujo, M. C., Azuero, R., Bernal, R., Bravo, D., ... & Vakis, R. (2014). Wealth gradients in early childhood cognitive development in five Latin American countries. Schochet, P. Z. (2005). Statistical power for random assignment evaluations of education programs. Princeton, NJ: Mathematica Policy Research. Schweinhart, L. J., J. Montie, Z. Xiang, W. S. Barnett, C. R. Belfield, and M. Nores. 2005. “Lifetime Effects: The High/Scope Perry Pre-school Study through Age 40.” High/Scope Educational Research Foundation, Ypsilanti, Michigan. Stith, A., Gorman, K., & Choudhury, N. (2003). The Effects of Psychosocial Risk and Gender on school attainment in Guatemala. Applied Psychology, 52 (4), 614−629. Walker, Chang, Powell, & Grantham-McGregor. (2005). “Effects of Early Childhood Psychosocial Stimulation and Nutritional Supplementation on Cognition and Education in Growth-Stunted Jamaican Children: Prospective Cohort Study. The Lancet, 369 (9556). 84 World Bank. 2013. What matters most for early childhood development? A framework paper. Systems Approach for Better Education Results (SABER) Working Paper Series. Washington DC. World Development Indicators. 2014. The World Bank. Washington DC. World Bank. 2015. “IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-11836) ON A GLOBAL PARTNERSHIP FOR EDUCATION FUND GRANT To MONGOLIA FOR AN EARLY CHILDHOOD EDUCATION PROJECT”. Washington, DC. UNESCO Beijing. UNESCO National Education Support Strategy (UNESS): Mongolia. Beijing, January 2008. UNESCO/IBE. 2011. World Data on Education. VII Ed. 2010/11. Mongolia. UNICEF. 2016. UNICEF Annual Report 2015: Mongolia. Downloaded from http://www.unicef.org/about/annualreport/index_90469.html. 85 Annexes Annex 1 History of preschool education and alternative programs in Mongolia Early Childhood Education in Mongolia faced a serious challenge during the period of transition to democracy and to an open market economy in the early 1990s. Under the socialist regime, access to preschool education was rather unconstrained. Each soum (village) used to have at least a state-financed nursery for children aged 0-2 years old and a kindergarten for those aged 3-6 years old. However, with reduced economic capacity in the country immediately after the transition, subnational government budgets strained provision of preschool education especially in rural and remote areas. Many nurseries and kindergartens closed down because of fiscal deficits among local governments, and about 2,000 of these teachers lost their jobs (UNESCO, xx). The number of pre-primary enrollees dropped sharply (Figure A1). Herders and poor children in rural and remote areas were hit hardest. Figure A1 Pre-primary education, enrollment in 1989 to 2012 100 150 Thousands people Enrolment in pre-primary education, both sexes (number) 80 Gross enrolment ratio, pre-primary, both sexes (%) 100 60 40 50 20 0 0 (Source: UNESCO Institute for Statistics) Remoteness and severe weather conditions are a substantial challenge in Mongolia’s ECE development in remote areas. Mongolia is the least densely populated in the world. Even with the population growth in recent years, the population density has remained under two per square kilometers (WDI data). 54.6% of total children under the age six years old live in rural areas (UNICEF, xx). With the financial difficulty, these natural conditions had made it more difficult for the state to ensure the preschool education access for rural children. The concept of mobile ger-kindergartens emerged against a backdrop of these challenging conditions. It was international agencies and NGO, such as United Nations Children’s Fund (UNICEF) and Save the Children who initially piloted this concept in 1994 to serve difficult-to-reach children of nomadic herders’ children, who are often from poorest group in the country with unstable earnings and distributed particularly in the west and east side of the country. Children were able to receive subsidized school meals and learn in the school preparation classes before entering schools for the first time. Seeing the positive outcomes, the initiative of mobile ger-kindergartens was later replicated at other locations and soon became a wide-spread practice across the country, known as a key intervention of “alternative forms” to provide preschool education (Save the Children, 2009). 86 The policy environment to enhance preschool education, particularly alternative preschool programs, has been evolved since the mid-1990s. In 1995, the government established the National Program on Preschool Strengthening. One of the key goals of the program was to create a favorable environment for preschool education for both nomadic and sedentary populations. It was however an ambitious plan for 1995 to 2000 under the fiscal budget difficulty. In 2005, National Policy of Integrated Early Childhood Development Policy was endorsed by a joint order of the ministry of health, education, labor and social welfare to promote policy coordination among various stakeholders. The major breakthrough was the adoption of the Law on Pre-school Education in 2008, which stipulated that the provision of food, books, manuals and appropriate toys for children attending state-owned kindergartens as well as the norm-based variable costs, should be financed from the state budget. Prior to the law’s implementation, parents were obligated to pay 50 percent of food costs and other expenses, which were major obstacles for parents to enroll their children to preschool. In addition, the government decided to provide its financial and technical support in developing non-formal early childhood development programs, which had been supported earlier by external stakeholders only. (UNESCO, xx; UNICEF, xx) 87 Annex 2 Determinants of preschool enrollment in Mongolia: Regression output table Relationship of household- and individual-level characteristics with preschool enrollment among children aged 36-59 months, Mongolia, MICS, 2010 (1) (2) (3) VARIABLES Enrollment Enrollment Enrollment Mother's Primary 0.0169 0.0713 Education (0.0633) (0.0873) Basic (lower secondary) 0.191*** 0.148* (0.0550) (0.0754) Upper secondary 0.287*** 0.123 (0.0541) (0.0757) Vocational 0.342*** 0.130 (0.0700) (0.0889) College, university 0.491*** 0.175** (0.0535) (0.0800) Wealth Index Second 0.314*** 0.200*** (0.0337) (0.0511) Middle 0.490*** 0.406*** (0.0360) (0.0673) Fourth 0.500*** 0.434*** (0.0372) (0.0811) Richest 0.564*** 0.413*** (0.0391) (0.137) Gender Female 0.0255 (0.0280) Region Khangai -0.0718 (0.0504) Central -0.0254 (0.0489) Eastern -0.0311 (0.0484) Ulaanbaatar -0.242*** (0.0517) Ethnicity Kazak -0.137 (0.0857) Constant 0.309*** 0.280*** 0.497** (0.0479) (0.0215) (0.216) Full set of household and individual controls No No Yes Observations 1,320 1,320 1,008 R-squared 0.112 0.208 0.144 Standard errors in parentheses. Household- and individual-level controls not shown include indicators for age group, location, “other” ethnicity, religion, material of roof, dwelling type, ownership of dwelling and agricultural land, livestock, bank account, sanitation facilities at home, and number of children in the household. *** p<0.01, ** p<0.05, * p<0.1 88 Annex 3 Herders’ children enrolled in preschools in Mongolia Figure A3 Share of herders’ children of the total pre-school children (%) 20.0% 15.0% 10.0% 5.0% 0.0% 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Source: MECS (2015) 89 Annex 4 List of school and sanitation supplies contributed by parents of kindergarten children Quantity No. Name of Quality standard Regular Kindergartens 24hrs Boarding Kindergartens supplies Small Mid Senior & Small Mid Group Senior & Group Group Preschool Group Preschool School supplies 1 Drawing Semi hard HB, B3, 1pc 3 pcs 2pcs 3pcs pencil H3, M2 HB B3, H3, M2 B3, H3, B3, H3, M2 M2 2 Color Bright colors, soft 1 box 1box 1 box 1box pencils as easy to color 12 colors 12 colors 12 colors 12 colors To be thick, length Length Length 15 Length 15 Length 15 8-15 cm 8cm cm cm cm 3 Pencil Must be easy to use 1pc 1pc 1pc 1pc sharpener 4 Colored No odors, without - - 1box - - 1box Markers toxic effects 12 colors 12 colors 5 Crayons With plastic 2pcs 2pcs 1pc 2pcs 2pcs 1pc case/covering, 6 colors 6 colors 12 colors 6 colors 6 colors 12 colors length 8-15 cm, not Length Length Length 15 Length Length Length 15 sharpened but 8cm 8cm cm 8cm 8cm cm rolled 6 Drawing Thick water 3pcs 4pcs 5pcs 4pcs 5pcs 6pcs Notebook absorbing paper, 12 pages 20 pages 24 pages 12 pages 20 pages 24 pages easy to rip 7 Map Paper Thick, and smooth 1pc 1pc 1pc 1pc 1pc 1pc surfaced 8 Water Each color of 2pcs 2pcs 3pcs 2pcs 2pcs 3pcs Colours, watercolour to be 6 colors 6 colors 12 colors 6 colors 6 colors 12 colors Gouache separate gouache gouache watercolor gouache gouache watercolor containers, non- 2pcs 2pcs toxic Gouache Gouache Gouache in 1pc 1pc containers, 12ml 9 Brushes Easy to hold, round 1pc 2pcs 2pcs 1pc 2pcs 3pcs and square shaped. Square Round Round No. Square Round No. Round No. Brush hair soft, No.8-10 No. 3-4, 3-4, No.8-10 3-4, 3-4, thin, length 10- Square Square Square Square 15cm No.8-10 No.8-10 No.8-10 No.8-10 10 Colour A4 or 20*28cm, 1 set 2sets 3sets 2 sets 3 sets 3sets papers 14*20cm sized 6 colors 8-10 10-12 6colors 8-10 10-12 14*20cm colors colors 14*20cm colors colors 14*20cm 20*28cm 14*20cm 20*28cm 11 Dough Odorless, without 1 box 2 boxes 2 boxes 1box 2 boxes 2 boxes color imprints, non- 4-6 colors 4-6 colors 8-10 colors 4-6 colors 4-6 colors 8-10 colors sticking, in containers 12 Scissors Without sharp 1pc 1pc 1pc 1pc 1pc 1pc edges, with lid Length 5- Length 5- Length 11- Length 5-7 Length 11- Length 11- cover, easy to use. 7 cm 7 cm 13cm cm 13cm 13cm Length of cutting 7- 13 cm 13 Glue Odorless, well 2pcs 3pcs 3pcs 3pcs 4pcs 4pcs adhesive, 8-15gr 8gr 8gr 15gr 8gr 8gr 15gr 90 14 Ruler No sharp edges, safe, flexible - - 1pc - - 1pc plastic. Length 15- 20 cm. Different shapes 15 Notebooks 12 pages. In 4pcs of 4pcs of (math & accordance with - - each - - each letter) state standards Sanitation supplies 16 Hand soap For child use, 250 4pcs 4pcs 4pcs 6pcs 6pcs 6pcs (liquid) ml container 17 Toothbrush Suitable for age of 2pcs 2pcs 2pcs 2pcs 3pcs 3pcs child, soft 18 Toothpaste For child use, no 2pcs 2pcs 3 pcs 2pcs 3 pcs 3 pcs less than 75ml, 99gr 19 Toilet paper MNS 4273:1995 5pcs 4pcs 4pcs 6pcs 6pcs 6pcs 20 Paper Each with 250 5pcs 4pcs 4pcs 6pcs 5pcs 5pcs Napkins napkins in a box 21 Hand soap For child use, 4pcs 3pcs 5pcs 4pcs (hard) doesn’t cause - - allergic reactions 91 Annex 5 Kindergarten exposure among 5-year old children enrolled in Mongolian kindergartens, 2015 Ger kindergartens Public - fixed kindergartens Private kindergartens Months enrolled in preschool Mean Median Mean Median Mean Median UB non-ger areas 28.4 28.5 24.3 25 UB ger areas 27.2 25 19.8 18.5 Aimag center 9.2 2 28.7 34 22.5 21 Soum 6.2 2 25.4 25 Total 6.4 2 27.4 25 22.7 25 No. of years teacher has known child Mean Median Mean Median Mean Median UB non-ger areas 1.3 1.1 0.6 0.1 UB ger areas 1.3 1.1 0.5 0.1 Aimag center 0 0 1.5 1.1 0.3 0.1 Soum 0.3 0 1.1 1 Total 0.3 0 1.3 1.1 0.5 0.1 Months enrolled in current kindergarten q1.8.2 area Mean Median Mean Median Mean Median UB non-ger areas 24.4 25 17.4 13 UB ger areas 25.3 25 12.8 13 Aimag center 1 26.5 25 9.9 13 Soum 1 24.5 25 Total 1 25.2 25 15.3 13 Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) 92 Annex 6 Construction of the household wealth index in the WB kindergarten survey In the absence of survey questions on income or expenditures, household wealth is typically proxied by an “asset index”, a linear index constructed using asset ownership indicators. The index is understood to capture long-term household wealth as reflected in ownership of assets, and represents a ranking of households by wealth, from poorest to richest. It is important to note that the index does not provide information on absolute poverty, current income or expenditure levels, and the wealth scores calculated are applicable for only the particular data set they are based on. For further information, see Filmer and Pritchett (2001), Rutstein and Johnson (2004) and Rutstein (2008). The WB 2015 kindergarten survey collected data on asset ownership in households of all children surveyed. For each child, parents or primary caregivers reported on whether the household or at least one person in the household owned a particular asset (see table A6 for the full list of assets). The statistical procedure of principal components was used to determine the weights for an index of the asset variables. Principal components is a technique for extracting from a set of variables those few orthogonal linear combinations of the variables that capture the common information most successfully. Intuitively the first principal component of a set of variables is the linear index of all the variables that captures the largest amount of information that is common to all of the variables. First, initial factor scores are calculated for the pooled subsamples in the WB survey (namely, the public, private, and ger kindergarten samples). Then, separate factor scores are calculated for households in urban and rural areas. Finally, the urban and rural factor scores are regressed on the initial factor scores to obtain the combined factor scores for all households in the three subsamples. Table A6 compares the average ownership of each asset across the poorest and richest households, and shows that the asset index is internally coherent, with ownership differing widely across the poorest and richest households for each asset. Table A6 Summary statistics for variables entering the computation of the first principal component: Households of children enrolled in public fixed kindergartens 48, Mongolia, 2015 Wealth index quintile 1 2 3 4 5 Electricity 93.4 98.9 99.8 99.8 100 Renewable energy generator 19.2 13.5 8.8 7.2 4.8 Computer 1.4 15.5 56.9 90.2 98.2 Internet connection 0 2.9 19.2 69.3 97.3 Television 93.4 98 99 99.4 100 Radio 22.5 22.8 19.8 16.2 19.5 Non-mobile telephone 8 8.9 7.8 11.9 31.2 refrigerator 50.7 93.8 99.2 99.4 100 Washing machine 38 93.8 99.6 99.6 99.7 Vacuum cleaner 2.3 41.7 82.4 94.7 99.4 Library 19.7 40.6 64.3 81.1 96.4 Watch 25.8 42.6 62.7 82.8 95.8 Mobile telephone 96.2 99.8 99.8 99.8 100 Camera 0.5 4.9 14.3 33.8 67.9 Bicycle 20.2 39.2 53.5 65.4 75.7 Motorcycle 44.1 35 24.7 15.6 9 Car or truck 22.1 49.2 71.8 80.7 93.7 Animal-drawn cart 3.8 1.8 2.4 1.2 0.3 Tractor 1.4 2 3.3 1.2 1.2 48 Tables for the private and ger kindergarten subsamples are available from the authors upon request. 93 Bank account 45.1 79.8 94.5 98 100 Dwelling type==Ger 68.1 51.9 22.9 4.1 0 Agricultural land (m2) 1071.6 915.9 654.1 807.7 3727.6 Animals 70.9 90.6 57.2 29.9 23.4 Mean wealth index score -1.4 -0.5 0.2 0.7 1.2 Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) 94 Annex 7 Comparison of the 2015 WB quality survey sample with the 2010 MICS The Multiple Indicator Cluster Survey (MICS) is a nationally representative household survey conducted every few years in Mongolia by UNICEF. The broad coverage of the survey provides a useful reference point against which the WB 2015 kindergarten survey sample can be compared. The MICS sample is representative at the regional level, and covers all 21 aimags and the capital city of Ulaanbaatar, while the WB sample was limited to a representative aimag in each of the five regions of the country, covering a total of eight aimags and Ulaanbaatar. A comparison of the two samples can thus shed light on how applicable the WB survey findings are more broadly for the country as a whole. Given the significance of socioeconomic status in determining human development outcomes, the economic characteristics of the WB sample are a key area of interest. Here, we exploit the overlap in data collected in the two surveys, focusing on ownership of selected assets, type of dwelling (see annex 6 for details), and parental education. Figure A7.1 Household asset ownership and household characteristics among preschool-enrolled 5-year old children in the 2010 MICS and the 2015 WB surveys MICS 0 28 96 dwelling Private 0 Ger 29 92 Public fixed 0 23 68 MICS 24 50 23 Car Private 78 98 0 Public fixed 72 94 22 50 Camera MICS 0 10 Private 20 74 0 Public fixed 14 68 1 MICS 50 83 Watch 34 Private 73 98 25 Public fixed 63 96 26 MICS 48 Library 5 23 Private 62 94 0 Public fixed 64 96 20 Vaccuum 87 cleaner MICS 1 31 Private 69 99 0 Public fixed 82 99 2 Computer MICS 11 65 0 Private 60 96 8 Public fixed 57 98 1 0 10 20 30 40 50 60 70 80 90 100 Richest wealth quintile Middle quntile Poorest wealth quintile Source: Author’s calculations using data from the World Bank Kindergarten Quality Survey (2015) and the 2010 MICS 95 Sample of children enrolled in public and private preschools Figure A7.1 presents the share of preschool-enrolled 5-year old children in the 2010 MICS whose households own a selected set of consumer goods, in the poorest, middle, and richest quintiles of the household wealth index. The figure also presents the same information for children in fixed public and private kindergartens in the 2015 WB kindergarten survey. For children in public kindergartens, the poorest quintile is very similar in the two surveys in terms of ownership of durable assets, but the middle and richest quintiles in the WB sample are better off than those in the MICS. Also, educational attainment among mothers is similar in both the poorest and richest quintiles (figure A7.2). In private kindergartens, the lowest quintile in the WB sample is poorer in terms of asset ownership, while the middle and richest quintiles are richer. Parental education attainment is significantly higher in the WB sample. Compared to the MICS 2010 sample, thus, Figure A7.2 Maternal education among preschool- overall, children from the poorest households enrolled 5-year old children in the 2010 MICS and the in the WB public and private school subsamples 2015 WB surveys come from similar households, while children 100 85 90 83 from the richest households tend to be better 80 off. This may in part be due to the fact that 60 50 47 35 asset ownership has increased over time 40 20 25 29 between the two survey years of 2010 and 20 10 8 00 50 3 4 2015. Indeed, the final report of the 2013 MICS 0 does suggest that across the two MICS cross- Public Private MICS Public Private MICS sections, asset ownership was higher in 2013 Primary or lower Undergrad or above than in 2010. Further analysis once the 2013 Poorest Middle Richest MICS data are available will shed light on this question. Source: Source: Author’s calculations using data from the World Bank Kindergarten Quality Survey (2015) Sample of children enrolled in ger and the 2010 MICS kindergartens Figure A7.3 Household asset ownership and household characteristics among ger-kindergarten children in the 2015 WB surveys, compared to out-of-preschool 5-year old children in the 2015 MICS 96 MICS 0 dwellin Ger 40 94 0 g Ger 18 77 MICS 38 44 Car 19 Ger 61 100 49 Camera MICS 13 12 4 Ger 21 80 3 Watch MICS 44 88 28 Ger 60 64 44 Library MICS 28 50 2 Ger 71 100 17 Vaccuum Computer cleaner MICS 20 63 0 Ger 80 100 0 MICS 12 63 0 Ger 64 100 2 0 10 20 30 40 50 60 70 80 90 100 Richest wealth quintile Middle quntile Poorest wealth quintile Source: Source: Author’s calculations using data from the World Bank Kindergarten Quality Survey (2015) and the 2010 MICS While the 2010 MICS collected data on whether children were enrolled in preschool at the time of the survey, the survey doesn’t distinguish between type of preschool institution. Since children enrolled in ger kindergartens have very low exposure to Figure A7.4 Maternal education among ger- preschool in any given year, it is likely that ger- kindergarten children in the 2015 WB survey, kindergarten enrollment is not picked up in the compared to out-of-preschool 5-year old children in survey, and that these children are captured as the 2015 MICS being out-of-preschool in the MICS. Thus, the 100 88 imperfect but closest group relevant for 80 60 37 3740 32 40 23 comparison in the MICS is the restricted sample of 0 0 0 0 3 1 20 5-year old children who are not enrolled in 0 preschool in the MICS. Ger MICS Ger MICS Primary or lower Undergrad or To that end, figure A7.3 presents the share of 5- above year old children in the 2010 MICS who are not Poorest Middle Richest enrolled in preschool and whose households own a selected set of consumer goods, in the poorest, Source: Source: Author’s calculations using data middle, and richest quintiles of the household from the World Bank Kindergarten Quality Survey wealth index. The figure also presents the same (2015) and the 2010 MICS 97 information for children in ger kindergartens in the 2015 WB kindergarten survey. In asset ownership, the poorest quintile in the WB sample looks similar to the MICS in some respects, but not in others. The richest quintile is significantly better off in the WB sample. In terms of educational attainment among parents, however, the children in the WB sample are significantly worse off than those in the MICS sample (figure A7.4). This suggests that the ger kindergarten sample in the WB survey includes children from some of the most disadvantaged communities in Mongolia, but some children from well-off households are also enrolled. 98 Annex 8 Distribution of normalized scores in child development domains tested under the MELQO instrument among 5-year children in Mongolian kindergartens, 2015 Figure A8.1 Normalized cognitive scores among 5- year old children enrolled in kindergartens, Mongolia, 2015 610 8 Percent 4 2 0 -3 -2 -1 0 1 2 z_num2_tot Ger Public - fixed Private Figure A8.2 Normalized language scores among 5- Figure A8.3 Normalized social/emotional scores year old children enrolled in kindergartens, among 5-year old children enrolled in Mongolia, 2015 kindergartens, Mongolia, 2015 15 15 10 10 Percent Percent 5 5 0 0 -4 -2 0 2 4 -3 -2 -1 0 1 2 z_lit4_tot z_soc3_tot Ger Public - fixed Private Ger Public - fixed Private Figure A8.4 Normalized executive function scores Figure A8.5 Normalized fine motor scores among 5- among 5-year old children enrolled in year old children enrolled in kindergartens, kindergartens, Mongolia, 2015 Mongolia, 2015 99 40 20 30 15 Percent Percent 20 10 10 5 0 0 -2 0 2 4 6 -3 -2 -1 0 1 z_mov5_tot z_drw7_tot Ger Public - fixed Private Ger Public - fixed Private Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) 100 Annex 9 Relationship of household-, individual, and school-level characteristics with normalized scores on the MELQO direct assessment instrument: Regression output tables (among 5-year old children enrolled in Mongolian public kindergartens) The analysis presented in section 3.4 was conducted in the following four stages: 1. The first step in analysis of the MELQO child assessment results is creation of a normalized assessment score for each domain of child development, for each child assessed. Domains tested using the MELQO instrument were cognitive skills, language, social/emotional development, executive function and fine motor skills. In each domain, normalized scores 49 were calculated for a pooled cross-section consisting of children from all three samples, i.e. public, ger and private kindergartens. With the scores thus expressed in standard deviation (SD) units, it is possible to compare differences in performance in each domain across the three samples, as well as to explore household-, child- and school-level determinants of outcomes observed. 2. Normalized scores are regressed on household- and child-level characteristics, separately for each domain 50. Use of the background data on the social and economic status of children’s households provides an opportunity to understand how outcomes vary by these factors among children enrolled in public kindergartens. All effects described in the text were statistically significant at the 1 percent level, unless otherwise noted. 3. The third stage investigates how performance on each domain varies by children’s exposure to the preschool system and kindergarten quality, in the absence of household- and child-level controls. Children who attend higher quality kindergartens or have been in the system longer are expected to have higher scores on domains of cognitive and non-cognitive development. Further, classroom characteristics can impact outcomes, such as class size, composition of class by age, and the size of the kindergarten. All control variables and related sample characteristics are discussed in chapter 2. 4. The fourth stage of the analysis introduces both supply and demand side factors into the regression specification at the same time. Addition of the supply side variables allows us to investigate whether household or individual level characteristics continue to matter once we control for the differences in quality of services and duration of exposure. Cognitive development and language Table A9.1 Relationship of household-, individual, and school-level characteristics with normalized cognitive skills scores (1) (2) (3) (4) (5) VARIABLES z_num2_tot z_num2_tot z_num2_tot z_num2_tot z_num2_tot Wealth index quintile = 2 0.263*** 0.155** 0.144** (0.0850) (0.0719) (0.0669) Wealth index quintile = 3 0.344*** 0.194** 0.176** (0.0807) (0.0867) (0.0774) Wealth index quintile = 4 0.445*** 0.224** 0.201** (0.0915) (0.0922) (0.0835) 49 Normalized scores are Z scores, calculated by subtracting from each child’s score the average score obtained on each domain and dividing by the standard deviation of scores on that domain. The scores were thus expressed in standard deviation (SD) units. 50 The rest of the analysis (stage 2-5) is restricted to analysis of outcomes among children in fixed public kindergartens only. 101 Wealth index quintile = 5 0.593*** 0.349*** 0.311*** (0.0970) (0.107) (0.0981) Age (months) 0.0799*** 0.0812*** 0.0819*** (0.00783) (0.00731) (0.00736) Child is female 0.0787 0.0776 0.0665 (0.0518) (0.0527) (0.0511) Maternal education completed = High 0.156 0.0617 0.0405 (0.135) (0.132) (0.131) Maternal education completed = Secondary 0.212* 0.121 0.133 (0.113) (0.106) (0.0987) Maternal education completed = College or higher 0.480*** 0.359*** 0.343*** (0.113) (0.100) (0.0993) Household member engagement index 0.0709*** 0.0652*** 0.0706*** (0.0226) (0.0210) (0.0210) Mean quality (ECEMI) score 0.162*** 0.128*** 0.140*** (0.0488) (0.0422) (0.0307) Group size 0.00208 -0.000286 (0.00374) (0.00302) Years completed in kg 0.156*** 0.0737*** (0.0298) (0.0272) No. of employees >20 & < 40 -0.0510 -0.119** (0.0672) (0.0549) No. of employees >=40 0.0302 -0.0554 (0.114) (0.0884) Time since teacher became child's teacher (mo) -0.0152 -0.0250 (0.0205) (0.0196) Constant -5.807*** -5.720*** -0.680*** -0.762* -6.458*** (0.525) (0.511) (0.198) (0.405) (0.584) Full set of household and individual controls No Yes No No Yes Full set of school controls No No No Yes Yes Observations 1,995 1,995 1,995 1,994 1,994 R-squared 0.185 0.219 0.015 0.069 0.239 Robust standard errors clustered at the kindergarten level in parentheses. Household- and individual-level controls not shown include indicators for paternal education, ethnicity, employment status, an indicator for disability and household location. School level controls not shown include share of enrolled students in group present in the kindergarten at time of survey, and indicators for age composition of the group. *** p<0.01, ** p<0.05, * p<0.1 Table A9.2 Relationship of household-, individual, and school-level characteristics with normalized language skills scores (1) (2) (3) (4) (5) VARIABLES z_lit4_tot z_lit4_tot z_lit4_tot z_lit4_tot z_lit4_tot Wealth index quintile = 2 0.322*** 0.175** 0.158** (0.108) (0.0768) (0.0749) Wealth index quintile = 3 0.330*** 0.159** 0.147** (0.0994) (0.0742) (0.0709) Wealth index quintile = 4 0.447*** 0.197** 0.171** (0.148) (0.0905) (0.0785) Wealth index quintile = 5 0.495*** 0.185 0.186* (0.150) (0.121) (0.107) Age (months) 0.0616*** 0.0641*** 0.0644*** (0.00940) (0.00829) (0.00825) Child is female 0.181*** 0.177*** 0.155*** 102 (0.0410) (0.0405) (0.0389) Maternal education completed = High 0.420*** 0.349*** 0.307** (0.128) (0.114) (0.121) Maternal education completed = Secondary 0.445*** 0.328*** 0.363*** (0.101) (0.106) (0.0971) Maternal education completed = College or higher 0.589*** 0.486*** 0.487*** (0.118) (0.106) (0.111) Household member engagement index 0.0674*** 0.0570** 0.0692*** (0.0254) (0.0233) (0.0219) Mean quality (ECEMI) score 0.231*** 0.221*** 0.244*** (0.0445) (0.0455) (0.0372) Group size 0.000625 -0.00169 (0.00439) (0.00445) Years completed in kg 0.129*** 0.0645** (0.0291) (0.0312) No. of employees >20 & < 40 -0.0723 -0.0650 (0.0875) (0.0854) No. of employees >=40 -0.207 -0.254* (0.137) (0.130) Time since teacher became child's teacher (mo) -0.0294 -0.0341 (0.0284) (0.0297) Constant -4.826*** -4.622*** -0.926*** -0.771 -5.627*** (0.596) (0.614) (0.183) (0.466) (0.750) Full set of household and individual controls No Yes No No Yes Full set of school controls No No No Yes Yes Observations 1,995 1,995 1,995 1,994 1,994 R-squared 0.130 0.191 0.030 0.113 0.241 Robust standard errors clustered at the kindergarten level in parentheses. Household- and individual-level controls not shown include indicators for paternal education, ethnicity, employment status, an indicator for disability and household location. School level controls not shown include share of enrolled students in group present in the kindergarten at time of survey, and indicators for age composition of the group. *** p<0.01, ** p<0.05, * p<0.1 Social/emotional development Table A9.3 Relationship of household-, individual, and school-level characteristics with normalized social emotional skills scores (1) (2) (3) (4) (5) VARIABLES z_soc3_tot z_soc3_tot z_soc3_tot z_soc3_tot z_soc3_tot Wealth index quintile = 2 0.0918 -0.0471 -0.0404 (0.145) (0.110) (0.0960) Wealth index quintile = 3 0.202 0.0718 0.0736 (0.180) (0.134) (0.122) Wealth index quintile = 4 0.225 0.0692 0.0623 (0.198) (0.128) (0.124) Wealth index quintile = 5 0.172 -0.00260 -0.0196 (0.187) (0.131) (0.113) Age (months) 0.00524 0.00733 0.00832 (0.00858) (0.00836) (0.00786) Child is female 0.101* 0.0984* 0.0704 (0.0542) (0.0519) (0.0478) Maternal education completed = High 0.423* 0.319 0.243 (0.220) (0.214) (0.186) Maternal education completed = Secondary 0.553** 0.381 0.366** (0.241) (0.232) (0.173) 103 Maternal education completed = College or higher 0.566** 0.465** 0.421** (0.240) (0.213) (0.170) Household member engagement index 0.0470 0.0378 0.0493* (0.0428) (0.0316) (0.0273) Mean quality (ECEMI) score 0.417*** 0.426*** 0.429*** (0.0553) (0.0508) (0.0490) Group size 0.00676 0.00281 (0.00448) (0.00480) Years completed in kg 0.0899*** 0.0549** (0.0211) (0.0229) No. of employees >20 & < 40 -0.265** -0.222* (0.117) (0.115) No. of employees >=40 -0.336** -0.351** (0.137) (0.136) Time since teacher became child's teacher (mo) 0.00388 0.00530 (0.0266) (0.0250) Constant -0.930* -0.906 -1.574*** -1.545*** -2.637*** (0.556) (0.572) (0.236) (0.450) (0.676) Full set of household and individual controls No Yes No No Yes Full set of school controls No No No Yes Yes Observations 1,995 1,995 1,995 1,994 1,994 R-squared 0.038 0.125 0.093 0.179 0.230 Robust standard errors clustered at the kindergarten level in parentheses. Household- and individual-level controls not shown include indicators for paternal education, ethnicity, employment status, an indicator for disability and household location. School level controls not shown include share of enrolled students in group present in the kindergarten at time of survey, and indicators for age composition of the group. *** p<0.01, ** p<0.05, * p<0.1 Executive function/self-regulation Table A9.4 Relationship of household-, individual, and school-level characteristics with normalized executive function scores (1) (2) (3) (4) (5) VARIABLES z_mov5_tot z_mov5_tot z_mov5_tot z_mov5_tot z_mov5_tot Wealth index quintile = 2 0.222** 0.162** 0.161* (0.0922) (0.0796) (0.0855) Wealth index quintile = 3 0.311*** 0.236*** 0.232*** (0.0734) (0.0699) (0.0737) Wealth index quintile = 4 0.370*** 0.233*** 0.232** (0.0893) (0.0884) (0.0952) Wealth index quintile = 5 0.497*** 0.337*** 0.317*** (0.101) (0.110) (0.117) Age (months) 0.0590*** 0.0615*** 0.0624*** (0.00850) (0.00786) (0.00775) Child is female 0.146*** 0.158*** 0.160*** (0.0493) (0.0487) (0.0480) Maternal education completed = High 0.119 0.0835 0.0677 (0.108) (0.113) (0.113) Maternal education completed = Secondary 0.229** 0.239** 0.231** (0.0983) (0.101) (0.101) Maternal education completed = College or higher 0.384*** 0.336*** 0.318*** (0.0970) (0.0998) (0.100) Household member engagement index 0.00573 0.0159 0.0257 104 (0.0227) (0.0224) (0.0223) Mean quality (ECEMI) score 0.139*** 0.113** 0.100*** (0.0453) (0.0462) (0.0378) Group size -0.000784 -0.00130 (0.00418) (0.00416) Years completed in kg 0.0718** 0.0145 (0.0318) (0.0291) No. of employees >20 & < 40 0.0250 -0.0292 (0.0715) (0.0636) No. of employees >=40 0.0723 0.0512 (0.126) (0.114) Time since teacher became child's teacher (mo) -0.00878 -0.0200 (0.0268) (0.0236) Constant -4.388*** -4.488*** -0.561*** -0.685 -5.233*** (0.570) (0.595) (0.176) (0.477) (0.757) Full set of household and individual controls No Yes No No Yes Full set of school controls No No No Yes Yes Observations 1,995 1,995 1,995 1,994 1,994 R-squared 0.101 0.135 0.010 0.038 0.145 Robust standard errors clustered at the kindergarten level in parentheses. Household- and individual-level controls not shown include indicators for paternal education, ethnicity, employment status, an indicator for disability and household location. School level controls not shown include share of enrolled students in group present in the kindergarten at time of survey, and indicators for age composition of the group. *** p<0.01, ** p<0.05, * p<0.1 Fine motor development Table A9.5 Relationship of household-, individual, and school-level characteristics with normalized fine motor scores (1) (2) (3) (4) (5) VARIABLES z_drw7_tot z_drw7_tot z_drw7_tot z_drw7_tot z_drw7_tot Wealth index quintile = 2 0.0951 0.0630 0.0679 (0.0998) (0.0945) (0.0972) Wealth index quintile = 3 0.191* 0.136 0.146 (0.114) (0.102) (0.100) Wealth index quintile = 4 0.0977 0.0257 0.0298 (0.103) (0.115) (0.116) Wealth index quintile = 5 0.0864 0.0432 0.0355 (0.127) (0.152) (0.144) Age (months) 0.0282*** 0.0280*** 0.0283*** (0.00949) (0.00891) (0.00927) Child is female 0.234*** 0.231*** 0.227*** (0.0426) (0.0415) (0.0413) Maternal education completed = High 0.643*** 0.612*** 0.584*** (0.193) (0.166) (0.175) Maternal education completed = Secondary 0.732*** 0.690*** 0.654*** (0.179) (0.161) (0.166) Maternal education completed = College or higher 0.730*** 0.669*** 0.631*** (0.152) (0.140) (0.148) Household member engagement index 0.0542 0.0560 0.0554 (0.0378) (0.0343) (0.0344) Mean quality (ECEMI) score 0.125*** 0.113** 0.106** (0.0415) (0.0469) (0.0458) Group size 0.00538 0.00167 105 (0.00402) (0.00415) Years completed in kg 0.0722*** 0.0283 (0.0259) (0.0270) No. of employees >20 & < 40 -0.0290 -0.0998 (0.0882) (0.0811) No. of employees >=40 0.122 -0.000823 (0.125) (0.109) Time since teacher became child's teacher (mo) -0.0147 -0.0222 (0.0246) (0.0250) Constant -2.634*** -2.789*** -0.432** -0.466 -2.781*** (0.686) (0.625) (0.168) (0.370) (0.786) Full set of household and individual controls No Yes No No Yes Full set of school controls No No No Yes Yes Observations 1,995 1,995 1,995 1,994 1,994 R-squared 0.063 0.084 0.008 0.021 0.099 Robust standard errors clustered at the kindergarten level in parentheses. Household- and individual-level controls not shown include indicators for paternal education, ethnicity, employment status, an indicator for disability and household location. School level controls not shown include share of enrolled students in group present in the kindergarten at time of survey, and indicators for age composition of the group. *** p<0.01, ** p<0.05, * p<0.1 106 Annex 10 Influence of maternal education and other socioeconomic characteristics on MELQO child assessment scores Figure A10.1 presents the regression results Figure A10.1 Relationship of maternal education with for the influence of maternal educational normalized scores on the MELQO direct assessment attainment on child assessment scores, instrument, 5-year old children enrolled in Mongolian controlling for the full set of individual, public kindergartens, 2015 household, and school-level characteristics 0.70 0.654*** 0.631*** (See annex 9 for full regression results). Secondary High College 0.584*** 0.60 Other factors held constant, children of 0.487*** 0.50 0.421** mothers with primary school completion or 0.40 0.363*** 0.366** 0.343*** lower perform worse than those with higher 0.307** 0.318*** levels of education with the largest 0.30 0.231** difference associated with attainment of a 0.20 college degree or higher, followed by a high 0.10 school degree. The exception was the 0.00 numeracy domain, in which only the coefficient on the college degree dummy was statistically significant. Overall, the signs and relative sizes on all education dummies were in the expected direction, increasing in magnitude with level of education. Source: Author’s calculations using data from the World Bank Kindergarten Quality Survey (2015) Pairwise comparisons of the coefficients on Notes: See annex 9 for the complete regression output tables, and discussion of results in each domain of child development. Regression the education dummies allow us to see results reported here control for the full set of hosuehold-, indivudal- whether the differences were relative only to and school-level characteristics, also described in the annex. children whose mothers had primary school Significance levels: *** = 1 percent, ** = 5 percent, * = 10 percent. Bar heights represent the change in normalized test scores associated or lower education (table A10.1). The with the child’s mother having completed lower secondary school, difference between those with a secondary high school, or college degree or higher, relative to those with and high school degree, holding other factors primary education or less. constant, was statistically significant only for the executive function domain; that between those with a secondary school and college or higher degree was significant for numeracy, literacy, and executive function, and marginally significant for social emotional skills; and finally that between high school and college for numeracy and literacy skills only. Statistical significance aside, it is notable that the magnitudes of the coefficients on secondary and high school education were robust to the inclusion of supply side variables (school characteristics, see annex 9). Table A10.1 Pairwise comparisons of coefficients on maternal education dummy variables (excluding primary education or lower) DIFFERENCE BETWEEN COEFFICIENTS ON… High and secondary Secondary and college High and college NUMERACY Estimate 0.093 0.302*** 0.210*** F (1, 100) 1.3 10.8 13.1 P-value 0.255 0.001 0.000 SOCIAL EMOTIONAL Estimate 0.123 0.178* 0.054 F (1, 100) 2.2 3.9 0.8 P-value 0.137 0.051 0.387 107 LITERACY Estimate 0.056 0.180** 0.124** F (1, 100) 0.7 6.1 4.9 P-value 0.399 0.015 0.028 EXECUTIVE FUNCTION Estimate 0.163** 0.250*** 0.087 F (1, 100) 4.3 10.3 2.0 P-value 0.041 0.002 0.160 FINE MOTOR Estimate 0.070 0.047 -0.023 F (1, 100) 0.4 0.2 0.1 P-value 0.508 0.666 0.752 Finally, table A10.2 presents F-tests for the joint significance of the maternal and paternal education indicator variables. For all five domains tested, the null hypothesis that the coefficients on parental education are jointly zero is soundly rejected at the 1 percent level. Table A10.2 Tests of joint significance of maternal and paternal education dummy variables BLOCK F BLOCK DF RESIDUAL DF PR > F R2 CHANGE IN R2 NUMERACY Full model without parental 26.1 35 100 0.000 0.220 education Full model WITH parental 6.0 8 100 0.000 0.239 0.019 education SOCIAL EMOTIONAL Full model without parental 33.1 35 100 0.000 0.227 education Full model WITH parental 4.0 8 100 0.000 0.241 0.014 education LITERACY Full model without parental 12.1 35 100 0.000 0.216 education Full model WITH parental 3.1 8 100 0.003 0.230 0.014 education EXECUTIVE FUNCTION Full model without parental 11.7 35 100 0.000 0.130 education Full model WITH parental 6.1 8 100 0.000 0.145 0.015 education FINE MOTOR Full model without parental 6.3 35 100 0.000 0.079 education Full model WITH parental 3.7 8 100 0.001 0.099 0.020 education Table A10.3 presents results of F-tests for the joint contribution of the full set of demand side variables (household characteristics) and supply side variables (school characteristics), sequentially added. For each domain, the models are built in two sequences. The “base” model in both includes age and gender of child. In the first sequence, school variables are added in the first step, followed by socioeconomic characteristics. In the second, social and economic characteristics are entered after the supply side ones. With the exception of executive function and fine motor skills, in all domains, the F-statistics for the joint additional contribution of school characteristics, over and above that of social and economic characteristics, are significant at the 1 percent level. The same is true of F-statistics for the additional contribution of household characteristics, over and above that of school characteristics. For executive function, the F-statistic on addition of school characteristics to the model with household characteristics is significant at the 5 percent level, while for fine motor skills, it is not significant even at the 10 percent level. The latter finding suggests that the joint effect of school characteristics on fine motor skills, over and above that of household characteristics, is somewhat diminished when compared to other domains. 108 Table A10.3 Comparison of nested models for multiple regression of MELQO assessment scores by age and gender, school and socioeconomic (SES) characteristics DOMAIN SOURCE OF F BLOCK DF RESIDUAL PR > F R2 CHANGE IN VARIATION DF R2 NUMERACY Sequence 1 Age and gender 41.9 2 100 0.000 0.078 ADD: School 11.4 9 100 0.000 0.147 0.070 characteristics ADD: SES characteristics 15.0 32 100 0.000 0.239 0.091 Sequence 2 Age and gender 41.9 2 100 0.000 0.078 ADD: SES characteristics 18.4 32 100 0.000 0.219 0.142 ADD: School 6.2 9 100 0.000 0.239 0.020 characteristics LITERACY Sequence 1 Age and gender 26.7 2 100 0.000 0.054 ADD: School 16.2 9 100 0.000 0.167 0.112 characteristics ADD: SES characteristics 7.9 32 100 0.000 0.241 0.074 Sequence 2 Age and gender 26.7 2 100 0.000 0.054 ADD: SES characteristics 11.5 32 100 0.000 0.191 0.137 ADD: School 8.5 9 100 0.000 0.241 0.050 characteristics SOCIAL Sequence 1 Age and gender 2.2 2 100 0.122 0.003 EMOTIONAL ADD: School 14.8 9 100 0.000 0.170 0.168 characteristics ADD: SES characteristics 4.6 32 100 0.000 0.230 0.060 Sequence 2 Age and gender 2.2 2 100 0.122 0.003 ADD: SES characteristics 6.4 32 100 0.000 0.125 0.122 ADD: School 13.9 9 100 0.000 0.230 0.105 characteristics EXECUTIVE Sequence 1 Age and gender 23.1 2 100 0.000 0.045 FUNCTION ADD: School 4.3 9 100 0.000 0.078 0.034 characteristics ADD: SES characteristics 9.1 32 100 0.000 0.145 0.067 Sequence 2 Age and gender 23.1 2 100 0.000 0.045 ADD: SES characteristics 12.7 32 100 0.000 0.135 0.091 ADD: School 2.1 9 100 0.038 0.145 0.010 characteristics FINE MOTOR Sequence 1 Age and gender 20.0 2 100 0.000 0.023 ADD: School 2.8 9 100 0.006 0.040 0.016 characteristics ADD: SES characteristics 4.3 32 100 0.000 0.099 0.059 Sequence 2 Age and gender 20.0 2 100 0.000 0.023 ADD: SES characteristics 5.1 32 100 0.000 0.084 0.061 ADD: School 2.5 9 100 0.013 0.099 0.015 characteristics 109 Annex 11 ECE outcomes in the Save the Children Japan (SCJ) and ger-kindergarten samples: Notes and regression output tables Distribution of ages Figure A11.1 Ages of children in the ger-kindergarten The target age group for the ger-kindergarten and the SCJ samples, Mongolia, 2015 sample in the WB Kindergarten Survey was 5- 25 SCJ Ger year old children. In contrast, most of the children in the Save the Children Japan (SCJ) 20 sample were over 6 years of age. In order to 15 Percent make the two samples comparable, they were truncated to overlapping ages, between the 10 ages of 70 – 77 months (figure A11.1). This 5 reduced the number of usable observations in the ger-kindergarten sample from 534 to 155 0 55 60 65 70 75 80 children, and from 315 to 248 in the SCJ Age (months) sample. Source: Author’s calculations using data from World Bank Kindergarten Quality Survey (2015) and SCJ (2015) Family engagement at home Table A11.1 Comparing at-home engagement index (HEI) scores in the SCJ home-based intervention with the ger-kindergarten sample, controlling for influence of household- and individual-level characteristics Household Household Household Household Household member member member member member engagement engagement engagement engagement engagement VARIABLES index index index index index Child is in SCJ sample 1.680*** 1.705*** 1.685*** 1.679*** 1.727*** (0.0847) (0.0829) (0.0813) (0.0821) (0.0912) Predicted index - national 0.147*** 0.136** 0.121* 0.133** (0.0539) (0.0565) (0.0682) (0.0554) Age (months) 0.0214 0.0227 0.00876 (0.0153) (0.0154) (0.0159) Child is female -0.140** -0.146** -0.168*** (0.0558) (0.0583) (0.0572) Maternal education = High 0.122 -0.0468 (0.0950) (0.0771) Maternal education = Secondary 0.191** -0.0389 (0.0883) (0.0866) Maternal education = College or higher 0.115 -0.107 (0.164) (0.184) Constant -0.502*** -0.359*** -1.862* -2.085* -1.794* (0.0847) (0.0989) (1.110) (1.127) (1.050) Full set of household and individual controls No No No No Yes Observations 403 403 403 402 399 110 R-squared 0.396 0.403 0.407 0.408 0.456 Robust standard errors in parentheses. Household- and individual level controls not shown include paternal education, primary occupation and ethnicity of the household head, and indicators for presence of other children in the household. *** p<0.01, ** p<0.05, * p<0.1 ECE outcomes Table A11.2 Comparing (normalized) MELQO direct assessment scores in the SCJ home-based intervention with the ger kindergarten sample, controlling for influence of household- and individual- level characteristics VARIABLES Cognitive Language Social emotional Executive function Fine motor Child is in SCJ sample 0.466*** 0.652*** 0.400*** 0.772*** 0.302*** (0.112) (0.122) (0.120) (0.113) (0.0986) Household member engagement index 0.119*** 0.181*** 0.157*** 0.244*** 0.0285 (0.0384) (0.0480) (0.0402) (0.0304) (0.0432) Age (months) 0.0360 0.0266 0.0143 0.0457 0.0248* (0.0314) (0.0263) (0.0242) (0.0300) (0.0130) Wealth index score 0.126* 0.0827 -0.0119 -0.0357 0.144** (0.0722) (0.0662) (0.0762) (0.0731) (0.0584) Child is female -0.0807 0.166* -0.0824 -0.0618 0.0517 (0.0664) (0.0877) (0.110) (0.141) (0.175) Maternal education = High -0.317*** -0.0377 -0.104 -0.154** -0.0840 (0.0984) (0.0856) (0.0751) (0.0757) (0.0938) Maternal education = Secondary -0.189 -0.0781 -0.0727 0.0252 -0.0311 (0.150) (0.159) (0.0780) (0.0777) (0.0749) Maternal education = College or higher -0.256 -0.120 -0.0275 0.175 -0.256 (0.199) (0.154) (0.214) (0.266) (0.163) Constant -2.122 -1.630 -1.494 -3.734 -2.041* (2.166) (1.808) (1.879) (2.301) (1.041) Observations 399 399 399 399 399 R-squared 0.290 0.287 0.169 0.338 0.116 Robust standard errors in parentheses. Household- and individual level controls not shown include paternal education, primary occupation and ethnicity of the household head, and indicators for presence of other children in the household. *** p<0.01, ** p<0.05, * p<0.1 111 Annex 12 Determinants of ECE outcomes in Mongolia: Regression output table Relationship of household- and individual-level characteristics with selected early development outcomes among children aged 36-59 months, Mongolia, MICS, 2010 (1) (2) (3) (4) Can Can recognize Kicks, bites, hurts Easily VARIABLES count? numbers up to 10? other children? distracted? Mother's Education Primary 0.139** 0.0163 -0.0164 -0.0753 (0.0627) (0.0928) (0.0892) (0.0921) Basic (lower secondary) 0.0972* 0.171** 0.0926 -0.0437 (0.0543) (0.0801) (0.0772) (0.0803) Upper secondary 0.107** 0.128 0.0337 -0.120 (0.0544) (0.0802) (0.0776) (0.0805) Vocational 0.150** 0.117 0.105 -0.0302 (0.0639) (0.0942) (0.0908) (0.0942) College, university 0.0967* 0.158* 0.0586 -0.0560 (0.0576) (0.0850) (0.0820) (0.0850) Wealth Index Second 0.000526 0.00835 -0.0268 -0.00823 (0.0371) (0.0548) (0.0525) (0.0542) Middle 0.00412 0.00498 0.00471 0.00229 (0.0493) (0.0728) (0.0698) (0.0723) Fourth 0.0496 0.0501 -0.00771 -0.0212 (0.0592) (0.0875) (0.0838) (0.0867) Richest 0.00803 0.158 0.190 0.107 (0.100) (0.148) (0.140) (0.147) Pre-primary Education Enrolled in 2010/2011 0.119*** 0.199*** -0.000310 -0.00218 (0.0231) (0.0342) (0.0327) (0.0341) Gender Female 0.0407** 0.0672** -0.0653** -0.0521* (0.0202) (0.0298) (0.0285) (0.0296) Ethnicity Kazak 0.00166 -0.162* 0.0414 0.0440 (0.0616) (0.0909) (0.0871) (0.0901) Other 0.00886 -0.0530 -0.0546 0.0623 (0.0275) (0.0407) (0.0388) (0.0405) Constant 0.656*** 0.188 0.208 0.124 (0.156) (0.230) (0.220) (0.227) Full set of household and individual controls No No Yes Observations 1,003 997 1,001 957 R-squared 0.092 0.123 0.049 0.070 Standard errors in parentheses. Household- and individual-level controls not shown include indicators for age group, location, “other” ethnicity, religion, material of roof, dwelling type, ownership of dwelling and agricultural land, livestock, bank account, sanitation facilities at home, number of children books in household, level of care, availability of toys, and number of children in the household. *** p<0.01, ** p<0.05, * p<0.1 112