WPS7832 Policy Research Working Paper 7832 Investing in School Readiness An Analysis of the Cost-Effectiveness of Early Childhood Education Pathways in Rural Indonesia Nozomi Nakajima Amer Hasan Haeil Jung Sally Brinkman Menno Pradhan Angela Kinnell Education Global Practice Group September 2016 Policy Research Working Paper 7832 Abstract This paper presents evidence on the cost-effectiveness of paper analyzes the cost-effectiveness of various early educa- early childhood education pathways in rural Indonesia. It tion pathways in Indonesia to show that providing access documents the existence of substantial differences in school to both playgroups and kindergartens to young children readiness between 6 to 9 year old children. Using detailed at developmentally appropriate ages can optimize public enrollment histories, it unpacks whether and how early edu- investments in early childhood education. The paper sub- cation experiences explain these gaps. The analysis considers jects the analysis to a variety of robustness checks, and not only the sequence of services children enroll in, but also concludes that children enrolled in play-based early edu- the age at which they enroll and the duration for which they cation programs (playgroups) at age 3 or 4, followed by the enroll. The differences in primary school test scores between country’s more academically structured programs (kinder- a child who has no early education exposure and a child who gartens) at age 5 or 6, are more likely to be ready for primary completes a full sequence at the developmentally appropri- school than children who do not follow this sequence. ate age are 0.42 standard deviations in language and 0.43 Compulsory pre-primary education policy should con- standard deviations in mathematics, roughly equivalent to sider incorporating both playgroups and kindergartens. an additional 0.9 to 1.2 years of primary schooling. The This paper is a product of the Education Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at ahasan1@ worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Investing in School Readiness: An Analysis of the Cost-Effectiveness of Early Childhood Education Pathways in Rural Indonesia Nozomi Nakajima1, Amer Hasan1, Haeil Jung2, Sally Brinkman3, Menno Pradhan4 and Angela Kinnell5 JEL classification codes: I20, I24, I25, D61 Keywords: early childhood education, sequence, timing, duration, primary school, cost- effectiveness, rural Indonesia 1 Education Global Practice, World Bank, Washington, D.C. USA 2 Department of Public Administration, Korea University, Seoul, South Korea 3 Fraser Mustard Centre, Telethon Kids Institute, University of Western Australia, Perth, Australia 4 University of Amsterdam and VU University, Amsterdam, the Netherlands 5 School of Psychology, University of Adelaide, Adelaide, Australia Corresponding author: ahasan1@worldbank.org Acknowledgements: We would like to thank Dedy Junaedi, Upik Sabainingrum, Anas Sutisna, Lulus Kusbudiharjo and Mulyana for managing the fieldwork. Data collection was partially funded by the Government of the Kingdom of the Netherlands through the Dutch Education Support Program Trust Fund (TF057272). We are grateful for useful comments provided by Marguerite Clarke, Samer Al-Samarrai and Harry Patrinos on earlier versions of this paper. 1. Introduction International evidence shows that investing in high-quality early childhood programs can have large economic returns, especially for children from socially disadvantaged groups (Barnett 2011, Engle et al. 2011, Walker et al. 2011, Yoshikawa et al. 2013). In response, developing countries are looking to increase public investments in early education programs (World Bank 2016a). One of the challenges faced by policy makers is deciding what to fund given the wide range of programs that exist in local settings. In order to invest smartly in early education programs, there is considerable interest in understanding how children’s various early education experiences predict their success as they transition into primary school.1 This understanding is particularly pertinent to on-going policy discussions in Indonesia regarding how best to prioritize investments in pre-primary education to meet the Sustainable Development Goals. Much of the existing literature on early education in developing countries compares children who attended preschool to those who did not, or compares children who attended preschools with improved quality to those who attended non-improved preschools (Engle et al. 2011, Nores and Barnett 2010, and the references cited therein). While these are valid comparisons in understanding whether a specific type of preschool is effective, such comparisons may not necessarily capture the reality of children’s early educational experiences, which are often not as binary. We use rural Indonesia as an example to show that there is considerable heterogeneity in the sequence, timing, and duration of early childhood education participation among children and argue that these factors matter in predicting children’s success in primary school. We use a uniquely rich data set from Indonesia that allows us to link children’s primary school test scores with their entire enrollment histories in early education programs. The data were collected in 2013 and sampled children in 310 poor villages in Indonesia as part of an evaluation of the Indonesia Early Childhood Education and Development project. We focus on test scores in language, mathematics and general cognitive skills of nearly 13,000 children ages 6 to 9. First, we explore the extent to which families self-select into different sequences, timing, and duration of early education programs. Second, we analyze how these different early education pathways correlate with primary school test scores. We show that children enrolled in play-based programs 1 In this paper, “early education” and “early childhood education and development (ECED)” are used interchangeably and refer to the broad range of site-based early learning programs from age 3 until entrance into primary school. 2 at age 3 and 4, followed by more academically structured programs at age 5 and 6 are more likely to succeed in primary school. Third, we describe children’s test scores to demonstrate that substantial learning gaps already exist between the most disadvantaged and the less disadvantaged children in the beginning of primary school—even when comparing children within poor villages. Finally, we estimate the cost-effectiveness associated with various early education pathways to show how returns to investments in early childhood education can vary considerably based on the sequence, timing, and duration of enrollment in early education programs. This paper is organized as follows. In the next section, we describe the literature on early grade assessments in developing countries and on early education sequence, timing, and duration. This is followed by an introduction to early childhood education in rural Indonesia. We describe the data in section 4, present descriptive evidence of gaps in test scores in section 5, and explain our empirical strategy in section 6. Results are presented in section 7 and estimates of costs associated with various early education pathways are presented in section 8. This is followed by a discussion of the findings and their policy implications. 2. Literature Review In developing countries, considerable advances have been made to improve children’s access to education, fostered by the Millennium Development Goal to achieve universal primary education. However, children are not adequately learning in schools (Pritchett 2013). An estimated 200 million primary school children in developing countries are struggling to read even basic words (UNICEF 2012). Others have documented a “twin crisis” in access and learning in schools, whereby high dropout rates are observed in the early grades among children who receive poor quality education (Davidson and Hobbs 2013). Given that education quality (as measured by cognitive skills) has a strong impact on individual earnings and on economic growth (Hanushek and Woessman 2012), the lack of education quality in many education systems around the world has implications for poverty reduction. Converging research supports the importance of ensuring that children acquire basic skills and competencies during the transitional years into primary school, which is defined as grades 1 to 3 (UNICEF 2012). One way to measure whether students are in fact learning these foundational skills and competencies is through early grade assessments. Many early grade assessments administered in developing countries do not collect information about children before 3 they entered primary school since such information is beyond the scope and purpose of these assessments. However, recent work in economics, education, and neuroscience shows that early childhood investments can have large persistent impacts on subsequent education and on later life outcomes (Cunha, Heckman and Schennach 2010, Sylva et al. 2010). For example, results from the OECD’s Programme for International Student Assessment (PISA) show that in countries across the world, participation in quality early childhood education is strongly associated with reading performance at age 15, even after controlling for children’s socioeconomic backgrounds (OECD 2012). As a result, linking early grade assessments with children’s early education experiences can help researchers and policy makers better understand the factors that influence children’s learning during the early years of primary school. The vast majority of research from developing settings that links early childhood education experiences to children’s primary school performance are evaluation studies. Consequently, they compare children who attended preschools with those who never attended, or compare children who attended preschools with improved quality to those who attended non- improved preschools (Engle et al. 2011, Nores and Barnett 2010, and the references cited therein). Such comparisons provide evidence for investing in high-quality preschools but they may not adequately capture the reality of many local settings where various types of early education services exist and where children enroll in different types of early education programs at different stages of development. In contexts like these, it is useful to compare different early education service experiences or pathways to understand how they predict children’s transition into primary school and subsequent academic achievement. In one of the few studies that examine early childhood education type and children’s early learning outcomes in a developing setting, Singh (2014) observes that in the State of Andhra Pradesh in India, enrollment in private preschools is associated with significantly higher test scores at the beginning of primary school relative to those in public preschools. The results highlight the role of preschool type in the emergence of test score gaps at school-entry. Existing research typically does not take into account different types of early education pathways in predicting children’s performance as they transition into primary school given that such data are often unavailable. Our study leverages a uniquely rich data set from rural Indonesia that links children’s test scores in the first few years of primary school with detailed histories on their early education pathways. Moreover, we extend prior research by examining the sequence, 4 timing, and duration of attending different types of early education as predictors of early grade learning. 3. Indonesia’s early childhood education system Over the last decade, the government of Indonesia has been implementing policies and programs to prioritize early childhood education and development (ECED). This has resulted in dramatic improvements in ECED enrollment, with the gross enrollment rate increasing from 24.1 percent to 54.4 percent between 2000 and 2013 (World Bank 2016b). However, access to early education services has historically been unequal, with children from the poorest quintile having significantly lower enrollment rates than those from the wealthiest quintile (Alatas et al. 2013). In response, the government of Indonesia launched an initiative which increased access to early education services in 3,000 poor villages in 50 districts throughout the country (Brinkman et al. 2015, Hasan, Hyson and Chang 2013, Jung and Hasan 2015). Early childhood education in Indonesia consists of a variety of different programs that are overseen by different ministries. Despite the wide range of programs, two types of ECED programs are dominant: playgroups and kindergartens. The Ministry of Education and Culture regulates playgroups (kelompok bermain, KB), which are typically for children ages 3-4 and meet three days per week for three hours each day. Playgroups are characterized as play-based learning environments with a combination of both unstructured and structured play activities, typically facilitated by teachers who have nominal formal early childhood education training. Structured play activities generally include songs and dance, and exposure to paints/pencils and paper, and reading sessions where the teacher reads books to the children introducing them to books, letters and numbers. These community playgroups will often have anywhere between 10 and 40 children in some instances. In contrast, kindergartens are regulated by both the Ministry of Education and Culture (for taman kanak-kanak, TK) and the Ministry of Religious Affairs (for raudhotul atfal, RA). They typically cater to children ages 5-6 and meet five to six days a week for three hours each day. Compared to playgroups, kindergartens emphasize a more academic and structured approach to learning. In addition, the tuition fee for kindergartens is usually higher than playgroups. Although playgroups and kindergartens are intended for specific age groups, these are not always adhered to and families often enroll their children in playgroups and/or kindergartens at various ages before entering primary school at age 7. 5 Given this landscape, we hypothesize that above and beyond the type of ECED service attended, the sequence, timing, and duration of early education play a role in children’s early grade learning. We define sequence as the order in which children enroll in different ECED programs. For example, some children may enroll in playgroup then kindergarten before entering primary school, while others may only enroll in kindergarten before primary school. We define timing as the age at which children enroll in various programs. For example, some children may enroll in playgroup at ages 3-4 while other children may enroll at ages 5-6. We define duration as the length of enrollment in various ECED programs. This paper is particularly timely as the government considers policy options in support of the Sustainable Development Goals. At present, there is no empirical evidence showing what ECED sequence, timing, and duration are associated with better early learning outcomes in Indonesia. On the one hand, it is possible that children will benefit more from continued enrollment in playgroups instead of moving around from playgroup to kindergarten. This hypothesis is partially supported by the fact that research on dosage of early childhood education programs has shown that programs that last 1 to 3 years had average effect sizes of 0.3 S.D. while programs lasting less than 1 year had average effect sizes of 0.2 S.D. (Nores and Barnett 2010). Conversely, it is plausible that attendance in playgroup followed by kindergarten will be associated with better early learning outcomes than attendance in playgroup only. This alternative hypothesis is supported by recent evidence from the United States showing that children who attended Head Start (a federal preschool program) at age 3 followed by Oklahoma Pre-K (a locally-funded preschool program) at age 4 exhibited stronger early reading skills than children who remained in Head Start at age 4 (Jenkins et al. 2016). The authors posit that children who stayed in Head Start were less likely to receive variation in curricula and activities, while those who switched programs were more likely to benefit from new learning experiences, which are critical for early childhood development (Bronfenbrenner 1994). Thus, the objective of this paper is two-fold. First, it explores how the sequence, timing, and duration of early education predict children’s performance in primary school. Second, it aims to describe how well children in rural Indonesia are acquiring initial skills in reading, mathematics, and general cognition. It is important to note that the focus of this paper is to document which early education pathways are prevalent among children in rural Indonesia and what children know in the early years of primary school. Our data do not allow us to make causal claims about the 6 links between early education pathways and children’s primary school test scores. Our study contributes to the literature on early learning in developing countries by showing that even in rural settings various early education pathways exist and that these diverse early education experiences should be accounted for in understanding children’s test score performance in the early years of primary school. This has implications as policy makers consider how best to optimize the allocation of scarce public resources. 4. Data and Measures Data We use data collected in 2013 as part of an evaluation of a government initiative to increase access to ECED services in rural Indonesia. The sample consists of children, families, and ECED facilities in 310 poor villages in Indonesia. The sample is not meant to be nationally representative of the population as a whole; however households in the sample are comparable to the rural sub-sample of Indonesia’s nationally representative SUSENAS household survey (see Hasan, Hyson and Chang 2013). Our study focuses on 12,976 children between the ages of 6 and 9 who were enrolled in primary school in 2013 and took an early grade assessment covering language, mathematics, and general cognitive skills. Of these children, we have complete detailed retrospective information on their ECED enrollment histories from 2008 for 12,949 children.2 Measures Test scores Children in the sample were given an early grade assessment that consisted of three sections: language, mathematics, and general cognitive skills. This early grade assessment was designed specifically for this work and is not a standard early grade assessment used in schools. The language and mathematics items on the test were pooled from a battery of questions that align with the national curriculum for lower primary school grades. Thus, these two sections are meant 2 There are 27 children for whom we have missing data on early education pathways. They have been excluded from the analyses of this paper. 7 to capture early grade learning in relation to what students are expected to acquire in the first few years of primary school in Indonesia. The general cognitive items on the test are based on the Raven’s Colored Progressive Matrices. Two versions of the test were administered: one for younger children ages 6 and 7, which had a total of 52 items and another one for older children ages 8 and 9, which had a total of 64 items. For both tests, the language section tested children’s ability to recognize letters and words, match words to objects, and comprehend short reading passages. The mathematics section tested children’s abilities to add, subtract, and order one to two digit numbers. The items based on Raven’s Progressive Matrices were intended to measure children’s general cognitive skills. We calibrated the test score of children into a common scale using the 6 year-olds as the reference group. We used the mean and standard deviation of the 6 year-old group to normalize the test scores. The advantage of normalizing using the 6 year-olds as a reference group is the ease of interpretation, as we can understand the magnitude of increase in test scores associated with increase in age. The typical child in the sample scored 0.63 SD on language, 0.54 SD on mathematics and 0.28 SD on cognitive skills (see Table 1). Early childhood histories ECED enrollment histories were collected by asking children’s primary caregivers to retrospectively report the types of ECED service a child had ever been enrolled in (including “no ECED” as a type) and the number of months enrolled in each type of ECED during each academic year since 2008-2009. Based on this information, we generate a variable for ECED sequence, which denotes the sequence of ECED programs attended by each child in the sample. In addition, based on a child’s age at the time of the survey in 2013, we are able to extrapolate the age of entry into ECED. We generate a variable that captures timing, which is the age of the child at the time of entry into each ECED service sequence. We also know the months of enrollment in each type of ECED service, which allows us to create a variable for duration. Duration was categorized into three groups: less than one year, between one and two years, and greater than or equal to two years. Table 1 presents the mean and standard deviation of each of these variables. For ECED sequence, we find the largest category to be those who enrolled in kindergarten prior to primary school (35.2 percent). Next are children who did not participate in any ECED programs (19.6 percent), followed by those who enrolled in playgroup prior to primary 8 school (17.1 percent). 13.6 percent of children in the sample enrolled in playgroups followed by kindergarten before entering primary school. The rest (14.5 percent) did not enroll in playgroups or kindergartens but instead, enrolled in one of the other (less common) types of ECED programs. For the timing variable, the most common ECED pathway was to enroll in kindergarten at age 5-6 before primary school (35.2 percent). 9.2 percent of the sample entered playgroup at age 3-4 before primary school and 7.9 percent enrolled in playgroup at age 5-6 before entering primary school. Among those who enrolled in both playgroup and kindergarten before starting primary school, the vast majority entered playgroup at age 3-4 then kindergarten at age 5-6 (12.4 percent of the sample). Only 1.2 percent of children entered playgroup at age 5 then kindergarten at age 6 before enrolling in primary school. For ECED duration, we find the majority of children (71.0 percent) enrolled in playgroup for less than one year. 17.2 percent enrolled in playgroup for one to two years and only 11.7 percent enrolled for two years or more. In comparison to playgroup, the average duration of enrollment in kindergarten was slightly longer. 51.8 percent of children enrolled for less than one year, 26.5 percent enrolled between one and two years, and 21.8 percent enrolled for two years or more. Child and Family Characteristics In addition to primary school test scores and ECED enrollment histories, the survey included a questionnaire administered to the child’s primary caregiver, which gathered extensive information on a range of child and household characteristics. The majority of children in the sample were either age 7 (29.1 percent) or age 8 (37.9 percent) and as a result, they were found in grade 1 (30.4 percent), grade 2 (36.5 percent) or grade 3 (26.1 percent) at the time of the survey. Approximately half were girls (49.2 percent) and 17.9 percent of the sample were stunted.3 At the household level we measured wealth, mother’s education, and parenting practices. Household wealth was constructed using principal component analysis, combining information on ownership of a variety of household assets and materials used in the construction of the respondent’s home. The resulting index was standardized with a mean of zero and standard deviation of one. In our sample, the mean years of mother’s education was slightly over 7 years, suggesting that most mothers had completed 6 years of primary school. We also measured 3 To estimate the proportion of children who were stunted, we use the World Health Organization definition of height- for-age Z-score 2 standard deviations below the median (De Onis 2006). 9 parenting practices based on 24 items adapted from the Longitudinal Study of Australian Children (LSAC) (Zubrick et al. 2008). The primary caregiver of each child was asked a range of questions about their parenting practices that reflect different levels of warmth, consistency, and hostility. The resulting scores for parenting practices were standardized to have a mean of 0 and a standard deviation of 1. ECED Characteristics Lastly, our data also included ECED characteristics averaged at the village level. The Early Childhood Environment Rating Scale-Revised (ECERS-R) – a classroom assessment tool designed to measure the quality of early education through classroom observations – was applied to two ECED centers in each village (see Brinkman et al. 2016 for details of the ECERS-R in Indonesia). Similarly, our data included information on the average monthly fees of ECED services. We used the monthly mandatory fee charged by the services directly to the families, which ranged from zero (free) to 67,500 Indonesian rupiahs, with an average of 10,779 Indonesian rupiahs. Only 303 villages out of 310 were sampled for ECED characteristics; as a result, we have missing observations for ECED characteristics for children in these 7 excluded villages. 5. Differences in enrollment patterns and gaps in test scores Table 2 documents the various enrollment patterns observed in the data and how they vary between children of different backgrounds. 39.7 percent of the children from the bottom quintile report having no ECED at all. This is in contrast to 10.5 percent of children from the top wealth quintile. 7.6 percent of children from the bottom quintile report having attended playgroup and then kindergarten before primary school. By comparison 22.7 percent of the top quintile follow this enrollment pattern. Enrolling in kindergarten before primary school is much more common among the top quintile – 50.4 percent of this quintile does so – while only 28.3 percent of the bottom quintile does so. As Table 3 documents, considerable disparities in test scores already exist in the early years of primary school between children who are socially disadvantaged compared to those who are less so – even among children living in poor villages in Indonesia. Children whose mothers had less than the mean years of education (7.7 years) scored between 0.372 standard deviations (in cognitive skills) to 0.404 standard deviations (in language) less than those whose mothers had more than the mean years of education. The test score gap is particularly striking between children 10 in the bottom and top quintiles of the wealth distribution. Children in the top 20 percent of the wealth distribution scored significantly higher than those in the bottom 20 percent by 0.629 standard deviations (in cognitive skills) to 0.697 standard deviations (in language). In order to interpret the magnitude of these test score gaps, we looked at the incremental increase in test score from one year of primary school in language and math. We focused on these two tests because they align with the curriculum in primary school and, as such, we would expect improvements in test scores as children progress through school. As shown in Table 3, the test score gap between grade one and two was 0.596 standard deviations for language and 0.471 standard deviations for math. Between grade two and three, the test score increases were 0.341 and 0.251 standard deviations for language and math, respectively. Together, this suggests that on average, one year of primary school is associated with a 0.468 standard deviation increase in language and a 0.361 standard deviation increase in math. Based on this assumption, the gap in test scores between those in the top wealth quintile and bottom wealth quintile is equivalent to 1.5 and 1.9 years of primary schooling. This suggests that even among poor, rural families in Indonesia, household wealth is strongly associated with children’s early learning outcomes. Given that we observed substantial disparities in test scores within the first few years of primary school, we sought to understand how children’s early education pathways may play a role in explaining these divergent outcomes. 6. Empirical Strategy We began by estimating who selected into different ECED service pathways using a multinomial logistic regression where ECED sequence was regressed on ECED characteristics averaged at the village level (ECED quality and fees), family characteristics (household wealth, mother’s education in years and parenting quality), child characteristics (age, gender, and stunting), and district fixed effects. Standard errors were clustered at the village level. (1) Pr , 1, … ,5 Where represents the probability that a child will choose a particular pathway j. There are 5 common pathways through which a child might progress through ECED – no ECED; playgroup then primary; kindergarten then primary; playgroup, then kindergarten then primary. 11 Other, less common, combinations are also possible. The results for this regression are presented in Table 4a. Then, we re-estimated this multinomial logit using timing of ECED as the outcome (results in Table 4b) and duration of ECED as the outcome (results in Table 4c). In each of these analyses, we examined which family and individual characteristics strongly predict different early education pathways. We were also interested in how ECED quality and fees predict different early education pathways since families may make decisions about the sequence, timing, and duration of ECED participation based in part on the quality and cost of ECED services in their village. Then, using a multivariate ordinary least squares regression for each primary school assessment, we estimated the impact of different early education pathways on the children’s test scores: (2) where was the test score (language, math or cognitive skills) of child i in primary school living in village j and was a categorical variable indicating the child’s ECED sequence. We also included the entire set of covariates presented in Table 1, where was a vector of child characteristics (age, grade, gender, and stunting), was a vector of family characteristics (household wealth, mother’s education, and parenting quality), was a vector of ECED characteristics averaged at the village level (ECED quality and fees) and was a vector of district fixed effects which was included as a dummy variable for each district excluding one as the base. Standard errors were clustered at the village level in the estimation. The results of this estimation are reported in Table 5a for each test: language, mathematics and cognitive skills. Then, we re-estimated the above equation by replacing with , a categorical variable indicating the timing of entry into each ECED sequence (See table 5b for results): (3) Next, we re-estimated equation (3) by adding a vector of variables for the duration of enrollment ( in playgroups and kindergartens. In doing so we were not only interested in the magnitude and direction of the coefficients of but also in how the coefficients of changed (See Table 5c): 12 (4) In these analyses, we were interested in testing multiple pairwise comparisons for each type of ECED sequence or each type of ECED timing. In this analysis, we tried to mitigate why students went through different early education pathways using a rich set of information on individual and household characteristics. However, it is not possible to completely control for unobserved individual and family characteristics that may affect the early learning outcomes between children who went through different early education pathways. Thus, we were cautious about providing a causal interpretation of our estimates. 7. Results First we examine the extent to which children (and their parents) self-select into various ECED pathways using the multinomial logit of equation 1. The results of the multinomial logit regression with exponentiated coefficients (relative risk ratios) are shown in Tables 4a, 4b, and 4c below. In each case our preferred outcome is the base scenario. A relative risk ratio greater than one indicates that as the value of the variable increases, the risk of the outcome falling into that category increases relative to the risk of the outcome being in the base category. Conversely, if the relative-risk ratio is smaller than one, the outcome is more likely to be in the base category. For early education sequence (Table 4a) and timing (Table 4b), the base category was enrollment in playgroup followed by kindergarten. Thus, we interpret the exponentiated coefficients of each sequence relative to the coefficient for this referent category. Among the child-level variables, we found that older children in the sample were consistently more likely to never enroll in ECED. Conversely, younger children were more likely to enroll in playgroup at age 3-4 then kindergarten at age 5-6. This was not surprising given that access to early education in villages was expanded by the Indonesian government from 2009, which meant that younger children were more likely to have had the opportunity to enroll in playgroups than older children. In addition, our results which take into account the child-level variables in Table 4b suggest that children who are stunted are less likely to enroll in any ECED or the full sequence of early education at the appropriate age. Compared to children who are not stunted, those who are stunted are 1.229 to 1.496 times more likely to enroll only in playgroups and 1.411 times more likely to enroll in playgroup at age 5 followed by kindergarten at age 6. 13 In addition, we found that household wealth and mother’s education were both significant predictors of both early education sequence and timing. As household wealth increased, children were more likely to be enrolled in playgroup at age 3-4 then kindergarten at age 5-6 compared to other possible sequences. More importantly, mother’s education was the only significant family-level variable that predicted enrollment in playgroup at age 5 followed by kindergarten at age 6 (relative risk ratio of 0.923). This means that children whose mothers have higher levels of education are more likely to enroll in playgroup and kindergarten at developmentally appropriate ages rather than enroll in playgroup followed by kindergarten at a later age – holding all other early education, family and child characteristics constant. Thus, mother’s education is a key predictor in ensuring that children not only enroll in the full sequence of ECED but also enroll at the right time. For both early education sequence and timing, the relative risk ratios for early education fees ranged from 0.173 to 0.303 which means that children living in villages that charge fees are less likely to be enrolled in only playgroup or kindergarten, and conversely, more likely to be enrolled in playgroup followed by kindergarten. This is an unexpected result since our estimation controlled for household wealth as well as ECED quality. In other words, if two children had the exact same household wealth and had access to ECED services of the same quality, the child living in a village that charged early education fees would still be more likely to enroll in the full sequence of ECED at developmentally appropriate ages (i.e., playgroup at age 3-4 then kindergarten at age 5-6) than the child living in a village that did not charge any fees. A possible explanation for this result is that the ECED fee variable is a village-level average. Villages that charge higher fees may be able to do so because they have stronger community-wide support for early childhood education, which in turn also promotes families to enroll their children in the full sequence of early education at the appropriate ages. We hypothesized earlier that families make decisions about early education pathways based in part on the quality of the services in their village. We found support for this hypothesis in both the sequence and timing of enrollment in ECED services. For children in villages with high ECED quality, the relative risk ratios were statistically significant and consistently smaller than one, ranging from 0.358 to 0.490. This means that if a child were to move from a village with low ECED quality to high ECED quality, we would expect the child to be more likely to enroll in the full sequence of early education –playgroup at age 3-4 then kindergarten at age 5-6 – rather than 14 playgroups only or kindergartens only. Figure 1 illustrates these results, showing how the predicted probability of enrolling in various early education sequences and timing varies with quality and household wealth. For duration (Table 4c), one to two years of enrollment was the base category. We also found that age was a significant predictor of duration in early education. As age increased, children were more likely to have enrolled for less than one year of playgroup and less likely to have enrolled for two years or more of playgroup. This is consistent with the early childhood education landscape in rural Indonesia, where the government expanded access to early education – particularly playgroups – since 2009, making younger children in the sample more likely to have had the opportunity to enroll in playgroups for longer. For duration in kindergarten, we found slightly different results. The relative risk ratios for age are 0.940 for less than one year of kindergarten and 0.890 for two years or more of kindergarten. This means that among younger children, there was an increasing proportion of children enrolling in kindergarten for two years or more (which is consistent with the results for playgroup duration) but also an increasing proportion of children enrolling in kindergarten for less than one year. This phenomenon can be partly explained by the rapid expansion of playgroups in rural Indonesia, which has resulted in some younger children enrolling in kindergarten for less time and replacing it with more time in playgroup. For family level characteristics, we found that children whose mothers were more highly educated and had better parenting behavior were more likely to enroll in playgroup for at least two years (as shown by the relative risk ratio of 1.031 for mother’s education and 1.108 for parenting quality). We found similar results for duration in kindergarten. A one year increase in a mothers’ education was associated with a 1.054 times increase in the likelihood of the child enrolling in two or more years of kindergarten and a one standard deviation increase in parenting quality increased the chances of enrolling in kindergarten for at least two years by a factor of 1.139. The results showed that as ECED quality increased from low to high, children were more likely to enroll in playgroup and in kindergarten for one to two years and less likely to enroll for less than one year. However, increasing ECED quality did not significantly increase the likelihood of children to enroll in playgroup or in kindergarten for two years or more. In terms of ECED fees, we found that as the amount of fees increased from none to more than 10,000 Indonesian rupiah per month, the likelihood of enrolling in early education for one to two years was significantly 15 greater than enrolling for two years or more. Thus, our findings suggest that parents make decisions about how long to enroll their children in early education based in part on the characteristics of the services in their village. Families were more likely to enroll their children in early education for at least one academic year if the quality of the service was high. In addition, they were less likely to enroll their children in early education for two academic years or more if the monthly fee exceeded 10,000 Indonesian rupiahs. Thus, our multinomial logistic regressions suggest that there is considerable self- selection into different early education pathways. First, children living in villages with higher quality ECEDs were significantly more likely to enroll in playgroup at age 3-4 followed by kindergarten at age 5-6, which was the ECED sequence and timing associated with better primary school achievement in language and math. Higher quality early education services were also associated with increased likelihood of enrollment for one to two years. Second, household wealth was a significant predictor of early education sequence and timing, even after controlling for the cost of ECED fees. Third, children whose mothers had higher levels of education were significantly more likely to enroll in playgroups and kindergarten at the intended age rather than delay their entry into playgroup and kindergarten. As a result, mother’s education was also a significant predictor of enrolling in early education for at least two years. Finally, compared to older children in the sample, younger children were more likely to enroll in the full sequence of early education (playgroup then kindergarten), at developmentally appropriate ages, and for two years or more. This positive trend reflects the improvements made in recent years by the Indonesian government to expand access to early childhood education for children in the country. Next we present the results of the relationship between early education pathways and test scores. These results are presented in Table 5a for sequence, Table 5b for timing, and Table 5c for timing and duration. In each of these tables, the dependent variable in column 1 is the score on the language test, in column 2 it is the score on the mathematics test and in column 3 it is the score on the cognitive skills section of the test. The pairwise comparisons between children who enrolled in different early education pathways are shown at the bottom of Tables 5a through 5c. In table 5a, we found that enrollment in playgroup then primary was associated with a 0.0558 standard deviation increase in language test scores and a 0.0861 standard deviation increase in mathematics tests scores. The point estimate on cognitive skills was 0.0284 but statistically insignificant. Looking at the pairwise contrasts shown in the bottom panel of the table shows that 16 children who enrolled in kindergarten before primary scored higher on the language (0.182 standard deviations), mathematics (0.134 standard deviations), and cognitive skills (0.066 standard deviations) tests than children who enrolled in playgroup before primary. Moreover, those who enrolled in playgroup then kindergarten before entry into primary school had higher test scores than those who enrolled in only kindergarten prior to primary (0.171 standard deviation in language and 0.196 standard deviation in math respectively). This suggests that there is a great deal of synergy between the two types of ECED exposure as it relates to early grade learning; children who enrolled in playgroup then kindergarten performed well above their peers, followed by children who enrolled in kindergarten, and children who enrolled in playgroup. Although children who only experienced playgroup performed significantly lower on the early grade assessment than those who only experienced kindergarten, it is important to note that children who enrolled in playgroups score significantly higher in math (by 0.0861 standard deviations) than those who never enrolled in early education. For general cognitive skills, enrollment in kindergarten (0.132 standard deviations) and enrollment in playgroup then kindergarten (0.198 standard deviations) are both significantly associated with test scores but post-estimation pairwise comparisons show that only enrollment in kindergarten is distinguishable from enrollment in playgroup. In interpreting these relative magnitudes, it is important to note that our analysis does not make any allowance for the fact that playgroups meet for half as many hours per week as kindergartens. Thus the point estimate for playgroups is likely an underestimate compared to that for kindergartens. In Table 5b, we found evidence that in addition to sequence, the timing of entry into ECED was a significant predictor of primary school test scores. Children who enrolled in playgroup at age 3-4 then kindergarten at age 5-6 before entering primary school performed significantly higher in language and math compared to those who enrolled only in kindergarten at age 5-6. The magnitude of this difference was 0.180 standard deviations in language and 0.207 standard deviations in math, both slightly larger than the differences seen in Table 5a. This suggests that enrolling in early education services at developmentally appropriate ages can make a difference to children’s subsequent learning outcomes. We find similar results for general cognitive skills, with a 0.0715 standard deviations difference between those who enrolled in playgroup at age 3-4 then kindergarten at age 5-6 and those who enrolled in kindergarten alone at age 5-6. In Table 5c, we controlled for the duration of enrollment in playgroups and 17 kindergartens. 4 For language test scores, we no longer found that children who enrolled in playgroup at age 3-4 then kindergarten at age 5-6 perform significantly better than their peers who only enrolled in kindergarten at age 5-6. In contrast, for math test scores, we continued to find that children who enrolled in a sequence of playgroup and kindergarten at developmentally appropriate ages yielded significantly higher test scores than their peers who enrolled in other early education pathways. It is worth noting that once we controlled for duration of ECED in Table 5c in addition to timing, the magnitude of the coefficient for playgroup at age 3-4 then kindergarten at age 5-6 dropped for all subjects (in the case of language the point estimate dropped from 0.418 to 0.277 and in the case of mathematics it dropped from 0.477 to 0.238 and in the case of cognitive skills from 0.205 to a statistically insignificant 0.0387). Meanwhile, the coefficients on duration of playgroup and kindergarten showed significantly higher test scores (on average) for those who enrolled for at least two years compared to those who only enrolled for one to two years (Table 5c shows that comparable point estimates for both language and mathematics). This suggests that over and above enrolling in ECED services in the right sequence at the intended ages, the duration of enrollment in playgroups and kindergartens is a significant predictor of children’s subsequent learning outcomes in primary school. We did not find significant results for general cognitive skills (table 5c). The divergent results of the general cognitive skills compared to the language and math results may be explained by the fact that the items in the language and math assessments were pooled from a battery of questions that align with the national curriculum for lower primary school grades while the general cognitive items are based on the Raven’s Colored Progressive Matrices. The matrices measure abstract reasoning and can be regarded as a non-verbal estimate of fluid intelligence. While we would have hoped that participation in ECED services would have shown enhancement of general cognitive skills, none of the programs included specific educational activities to enhance such skills, such as working memory games. Instead the ECED services in Indonesia exposed children to the basics of language and mathematical concepts to prepare them for primary school, in a manner aligned with the national early learning curriculum. Overall, our analysis of early education pathways, as it relates to early grade learning, shows that on average, children who enrolled in playgroup followed by kindergarten scored 4 If age cut-offs of admission were strictly enforced duration and timing would likely be highly correlated. However, in Indonesia, age cut-offs are rarely strictly enforced. 18 significantly higher in language and math tests in the early years of primary school compared to their peers who enrolled only in playgroup or kindergarten. In addition to the sequence of early education service participation, we found that the timing of entry into playgroups and kindergartens was an important predictor of children’s subsequent learning outcomes. Children who enrolled in playgroup at age 3-4 followed by kindergarten at age 5-6 performed significantly higher in language and math tests in primary school than their peers who enrolled in other early education pathways. Finally, over and above the sequence and timing, we found that duration of early education was a significant predictor of children’s math test scores in primary school, as those who enrolled in at least two years of playgroup and kindergarten scored significantly better than their peers who enrolled for shorter periods of time.5 8. Costs of various early education pathways The Indonesian government is currently weighing alternatives for investing smartly in early childhood education to ensure that all young children enter primary school ready to learn. In this section, we estimate the cost of various early education pathways to inform public resource allocation decisions for early childhood education in Indonesia. We do not have causal estimates of ECED pathways on primary school test scores. Thus we cannot calculate actual estimates of the different pathways. Despite this, we feel that our controls and robustness checks validate the presentation of these estimates as cost-effectiveness estimates. 6 We do so by calculating the marginal effect at representative values (MER) of the ECED timing variable at discrete values of ECED duration (shown in Tables 5c). We used the average of the language and math assessments for our effectiveness measure as these two tests were aligned with what children were expected to know in the early years of primary school. The cost of each early education pathway was drawn from the Indonesia ECED Project evaluation data (World Bank 2014) and the 2012 Nomor Unik Pendidik Dan Tenaga Kepandidikan 5 As a robustness check, we re-estimated the model in Table 5c using a sub-sample of children (N=2,874) for whom we have child development measures at age 4 using the Early Development Instrument (EDI). For this sub-group analysis, we included EDI scores at age 4 to control for children’s early childhood developmental outcomes. We find similar results: compared to their peers who enrolled in other early education pathways, children who enrolled in playgroup at age 3-4 followed by kindergarten at age 5-6 performing significantly better in language and math tests in primary schools. The results of this sub-sample analysis are available upon request. 6 Our analysis based on the sub-sample of children for whom we have data on development outcomes at age 4 gives us qualitatively very similar results. (See footnote 6) However, we choose to be conservative in our interpretation and present the cost-effectiveness results as indicative cost-effectiveness. 19 (NUPTK), which is the national teacher database (Ministry of Education and Culture 2012a). Costs were calculated using the ingredients method (Levin and McEwan 2001), which included personnel, facilities, equipment and materials, fees charged to families, and other operational costs of playgroups and kindergartens. In order to estimate annual costs per child, we assumed that the average center size is 31 children per kindergarten and 21 children per playgroup (Ministry of Education and Culture 2012b), and that the average student to teacher ratio was 15:1 in kindergarten and 11:1 in playgroups (as per the Indonesian minimum service standards). Based on these assumptions, we estimated total annual costs to be approximately 150.97 USD per child in playgroups and 256.25 USD per child in kindergartens. We divided the effectiveness of each early education pathway by its cost to calculate indicative effectiveness-cost ratios. This meant that the more effective the early education pathway, the larger the effectiveness-cost ratio. The results are presented in Figure 2, which includes 95 percent confidence intervals to take into account errors in the estimation of the effectiveness of various ECED pathways. Our estimates are of two types – those that are statistically distinguishable from zero and hence cost effective and those that are not. Those pathways whose cost effectiveness is not distinguishable from zero have the following in common: they involve exposure to some playgroup or some kindergarten but not to both. The pathways that are positive and statistically distinguishable from zero have in common the fact that they are some combination of 1-2 years of playgroup with 1-2 years of kindergarten. These estimates range from 0.032 to 0.064 but cannot be distinguished from each other. Thus, our results show that enrolling in a combination of playgroup and kindergarten at developmentally appropriate ages is cost-effective. 7 Focusing on providing access to both playgroups and kindergartens to young children at the appropriate ages can optimize public investments in early childhood education. 9. Discussion As developing countries increase investments in early childhood education, one of the challenges faced by policy makers is deciding what to fund given the wide range of programs that 7 Note that this does not take into account that kindergartens meet for more hours per week. 20 exist in local settings. Our study examined this issue in the context of rural Indonesia, by first analyzing the extent to which families select into different early education pathways and then describing early learning outcomes and exploring how they associate with early education pathways. Finally, we estimated the cost-effectiveness of various pathways to shed light on how investments in early childhood education must take into account the sequence, timing, and duration of such programs. The results of our study show that there is significant self-selection into different early education pathways. The predicted probability of enrolling in playgroup at age 3-4 and kindergarten at age 5-6 significantly increases with household wealth, mother’s education, and availability of high quality ECED services. Such disparities in early education experiences by household characteristics raises the question of how the government of Indonesia can better allocate scarce resources to ensure that children from the most socially disadvantaged backgrounds have an equal chance at success in primary school. Cost-effectiveness analyses from our study suggest that providing a combination of playgroup at age 3 and/or 4 followed by kindergarten at age 5 and/or 6 may potentially be the most cost-effective pathway. This means that it is imperative that access to both playgroups and kindergartens are expanded in poor villages in Indonesia. More broadly, the results of this study highlight the importance of carefully considering how the sequence, timing and duration of different ECED programs support children’s development in the early years when policy makers are faced with the challenge of deciding what type(s) of ECED programs to invest in. Our findings clearly show that there are substantial disparities in early learning outcomes by early education sequence, timing, and duration. Children who enroll in playgroup at age 3-4 followed by kindergarten at age 5-6 scored significantly higher in language and mathematics in the beginning of primary school compared to peers who enrolled only in playgroup (either entering at age 3-4 or at age 5-6) and only in kindergarten (entering at age 5-6). This is consistent with a study from the United States that found that children who transition from one preschool program to another outperform children who remain in the same preschool program (Jenkins et al. 2016). In the Indonesian context, the added marginal effect of enrolling in playgroup then kindergarten is likely due to the different curricula used in playgroups and kindergartens. Children in playgroups predominantly learn through play whereas kindergartens focus on more academic activities to prepare children for primary school. Neuroscience research has shown that secure 21 attachments and stimulation are significant aspects of brain development in the early years and play-based learning helps children develop their fine and gross motor skills, develop language and socialization skills, and become creative problem-solvers. Play not only enhances children’s learning readiness but also can more generally help them adjust to school settings (Zigler, Singer and Bishop-Josef 2004). Child development research has also shown that children’s intellectual development is best supported when children receive increasingly complex, differentiated learning experiences (Bronfenbrenner 1994, Engel, Claessens and Finch 2013). However, playgroups in Indonesia were not designed to provide multiple years of unique, developmentally appropriate learning. They also meet less frequently than other types of services. As a result, children who subsequently enroll in kindergarten are more likely to avoid redundancy in their learning experiences by having exposure to different, more academically focused curricula. We caution against interpreting these results as support for solely academically-focused early education given that child development research shows that children who are exposed to play- based learning in the early years are significantly more likely to have positive socio-emotional development than children who are only exposed to academic preschools (Elkind 2008). Instead, we interpret the results as evidence that early childhood education must support children’s learning at various stages of development and in the context of Indonesia, this is most strongly supported when children enroll in a play-based early education setting (playgroup) followed by a more structured and increasingly academic-based environment (kindergarten) prior to primary school. Given that current policy debates are centered around making a single year of preprimary education compulsory, these findings are timely. It is important to discuss several limitations to our study. First, although our analysis controlled for a rich set of the household background and individual children’s characteristics that should mitigate why students went through different early education pathways, our estimates may be biased if unobserved individual and family characteristics are responsible for the differences in early learning outcomes between children following different early education pathways. For example, parents particularly motivated by education might send their children to the full sequence of preprimary education at the correct timing, which would lead to positive bias in our estimates. In analyses not shown in our results tables, we found that when we controlled for child, family and ECED characteristics, the magnitude of the coefficients on early childhood education pathways became smaller. This seems to indicate that unobserved child, family, and ECED characteristics 22 could potentially be upwardly biasing our estimates for early childhood education pathways. Second, we are not able to causally assess the mechanism through which the combination of playgroup at age 3-4 and kindergarten at age 5-6 may produce higher early learning scores. 8 Further work also needs to investigate early education pathways in other developing countries to better understand the factors that determine the sequence and timing of enrollment more generally. Nonetheless, our results clearly show significant disparities in early learning outcomes by different early education pathways and we found evidence that children from the most socially disadvantaged backgrounds (i.e., low household wealth, low mother’s education, lack of access to high quality ECED) were significantly less likely to receive adequate exposure to a combination of play-based learning and academic preparation to help them succeed in primary school. Yet it is precisely these very children for whom the effectiveness of this intervention is likely to be highest. This means that even if early education pathways themselves are not causing the observed gaps in early learning outcomes, the current ECED policy landscape is not leveling the playing field to ensure that children from the most socially disadvantaged backgrounds have an equal chance at performing well once they enter primary school. 8 Strictly speaking a third limitation is that we do not have baseline child developmental levels at age 3 prior to any ECED attendance across the full sample. As a result in the full sample we cannot rule out that children attending playgroup and/or kindergarten at 3 years of age were not already developmentally advanced in comparison to those who did not attend any service until age 4 or later. However, our robustness check using data on a sub-sample of children for whom we do have baseline child developmental levels suggests that this is not a major concern. 23 Table 1. Summary statistics Mean S.D. Min Max N Test Scores (standardized using age 6 scores) Language 0.634 1.002 -1.435 1.951 12,949 Math 0.539 0.969 -1.513 1.697 12,949 General cognitive skills 0.276 1.071 -1.584 2.878 12,949 Sequence of ECED enrollment No ECED 0.196 0.397 0 1 12,949 Playgroup then primary school 0.171 0.377 0 1 12,949 Kindergarten then primary school 0.352 0.478 0 1 12,949 Playgroup then kindergarten then primary school 0.136 0.342 0 1 12,949 Other combination 0.145 0.352 0 1 12,949 Timing of ECED enrollment No ECED 0.196 0.397 0 1 12,949 Playgroup (3-4) then primary 0.092 0.289 0 1 12,949 Kindergarten (5-6) then primary 0.352 0.478 0 1 12,949 Playgroup (3-4) then kindergarten (5-6) then primary 0.124 0.329 0 1 12,949 Other combination 0.145 0.352 0 1 12,949 Playgroup (5-6) then primary 0.079 0.270 0 1 12,949 Playgroup (5) then kindergarten (6) then primary 0.012 0.109 0 1 12,949 Duration of ECED enrollment Time in playgroup: < 1 year 0.710 0.454 0 1 12,949 Time in playgroup: 1 to 2 years 0.172 0.378 0 1 12,949 Time in playgroup: 2+ years 0.117 0.322 0 1 12,949 Time in kindergarten: < 1 year 0.518 0.500 0 1 12,949 Time in kindergarten: 1 to 2 years 0.265 0.441 0 1 12,949 Time in kindergarten: 2+ years 0.218 0.413 0 1 12,949 Child characteristics Age 6 0.139 0.345 0 1 12,949 Age 7 0.291 0.454 0 1 12,949 Age 8 0.379 0.485 0 1 12,949 Age 9 0.192 0.394 0 1 12,949 Grade 1 0.304 0.460 0 1 12,949 Grade 2 0.365 0.482 0 1 12,949 Grade 3 0.261 0.439 0 1 12,949 Grade 4 0.067 0.251 0 1 12,949 Grade 5 0.001 0.029 0 1 12,949 Special needs 0.001 0.034 0 1 12,949 Girl (1=Yes) 0.492 0.500 0 1 12,949 Stunted (1 = Yes) 0.179 0.383 0 1 12,949 Family characteristics Household wealth (standardized) 0.065 0.873 -3.573 2.248 12,949 Mother's education (years) 7.702 2.647 1 15 12,949 Parenting quality (standardized) 0.000 1.000 -5.216 4.198 12,949 ECED characteristics Average ECED quality in village (standardized) -0.004 0.999 -2.338 2.796 12,710 Average monthly ECED fees in village (IDR) 10,779 11,248 0 67,500 12,710 Note: All variables were observed in 2013. For sequence, timing, and duration of ECED enrollment, retrospective information was collected in 2013 about ECED enrollment as far back as the 2008-2009 academic year. 24 Table 2. Enrollment patterns in early childhood education by select child, family and ECED characteristics Playgroup (3-4) Playgroup (5) Playgroup Kindergarten then Playgroup then No (3-4) then (5-6) then kindergarten (5- Other (5-6) then kindergarten (6) ECED primary primary 6) then primary combination primary then primary N=2,539 N=1,187 N=4,559 N=1,600 N=1,882 N=1,027 N=155 Age 6 (Yes=1) 0.166 0.105 0.295 0.155 0.193 0.081 0.006 7 (Yes=1) 0.156 0.103 0.326 0.166 0.158 0.080 0.011 8 (Yes=1) 0.209 0.094 0.368 0.124 0.104 0.085 0.016 9 (Yes=1) 0.253 0.061 0.401 0.036 0.172 0.067 0.011 Grade i 1 (Yes=1) 0.171 0.090 0.300 0.156 0.183 0.086 0.015 2 (Yes=1) 0.179 0.097 0.350 0.152 0.125 0.082 0.015 3 (Yes=1) 0.227 0.089 0.397 0.079 0.126 0.076 0.007 Gender Boy (Yes=1) 0.213 0.082 0.344 0.117 0.151 0.080 0.014 Girl (Yes=1) 0.179 0.102 0.361 0.131 0.139 0.079 0.010 Anthropometry Stunted (Yes=1) 0.288 0.109 0.365 0.107 0.001 0.116 0.015 Not stunted (Yes=1) 0.176 0.088 0.349 0.127 0.177 0.071 0.011 Wealth Bottom quintile (Yes=1) 0.397 0.122 0.283 0.076 0.002 0.111 0.009 Top quintile (Yes=1) 0.105 0.082 0.504 0.227 0.006 0.061 0.015 Mother's education ii Below mean (Yes=1) 0.241 0.089 0.275 0.075 0.225 0.085 0.010 Above mean (Yes=1) 0.115 0.096 0.489 0.210 0.003 0.070 0.016 Parenting quality Bottom quintile (Yes=1) 0.276 0.096 0.383 0.121 0.001 0.108 0.017 Top quintile (Yes=1) 0.194 0.112 0.433 0.154 0 0.094 0.013 ECED quality Bottom tercile (Yes=1) 0.205 0.099 0.367 0.094 0.148 0.079 0.009 Top tercile (Yes=1) 0.161 0.080 0.355 0.179 0.125 0.081 0.019 Note: i Only grades 1, 2 and 3 are reported since the vast majority of children were enrolled in these three grades as shown in Table 1. ii Mean years of mother’s education is 7.7 years for this sample (see Table 1). 25 Table 3. Test scores by select child, family, and ECED characteristics Language Math Cognitive N Age Mean 0.000 0.000 0.000 6 1,794 95% CI [-0.046 - 0.046] [-0.046 - 0.046] [-0.046 - 0.046] Mean 0.385 0.400 0.308 7 3,762 95% CI [0.352 - 0.418] [0.369 - 0.432] [0.273 - 0.342] Mean 0.849 0.681 0.275 8 4,910 95% CI [0.824 - 0.874] [0.655 - 0.706] [0.246 - 0.305] Mean 1.042 0.856 0.431 9 2,483 95% CI [1.009 - 1.075] [0.823 - 0.889] [0.387 - 0.474] Grade I Mean 0.0892 0.113 0.056 1 3,934 95% CI [0.0576 - 0.121] [0.0809 - 0.144] [0.0246 - 0.0880] Mean 0.685 0.584 0.272 2 4,732 95% CI [0.659 - 0.711] [0.558 - 0.611] [0.241 - 0.303] Mean 1.026 0.835 0.440 3 3,385 95% CI [0.999 - 1.054] [0.807 - 0.862] [0.404 - 0.476] Gender Mean 0.552 0.458 0.276 Boys 6,581 95% CI [0.528 - 0.577] [0.435 - 0.482] [0.249 - 0.302] Mean 0.718 0.622 0.277 Girls 6,368 95% CI [0.694 - 0.742] [0.598 - 0.645] [0.251 - 0.302] Anthropometry Mean 0.507 0.388 0.159 Stunted 2,319 95% CI [0.465 - 0.549] [0.347 - 0.429] [0.116 - 0.202] Mean 0.661 0.571 0.302 Not Stunted 10,630 95% CI [0.643 - 0.680] [0.553 - 0.590] [0.281 - 0.322] Wealth Mean 0.312 0.199 0.004 Bottom quintile 2,591 95% CI [0.273 - 0.351] [0.160 - 0.238] [-0.0343 - 0.0431] Mean 1.009 0.886 0.633 Top quintile 2,577 95% CI [0.975 - 1.043] [0.854 - 0.917] [0.592 - 0.675] Mother’s education ii Mean 0.489 0.396 0.143 Below mean 8,306 95% CI [0.467 - 0.511] [0.374 - 0.417] [0.120 - 0.165] Mean 0.893 0.794 0.515 Above mean 4,643 95% CI [0.866 - 0.919] [0.769 - 0.820] [0.484 - 0.546] Parenting quality Mean 0.575 0.479 0.212 Bottom quintile 2,752 95% CI [0.537 - 0.612] [0.443 - 0.515] [0.173 - 0.251] Mean 0.767 0.681 0.354 Top quintile 2,325 95% CI [0.728 - 0.807] [0.643 - 0.719] [0.311 - 0.398] ECED quality Mean 0.527 0.432 0.157 Bottom tercile 4,051 95% CI [0.497 - 0.558] [0.402 - 0.461] [0.125 - 0.189] Mean 0.717 0.602 0.377 Top tercile 4,351 95% CI [0.687 - 0.747] [0.572 - 0.631] [0.345 - 0.410] Note: i Only grades 1, 2 and 3 are reported since the vast majority of children were enrolled in these three grades as shown in Table 1. ii Mean years of mother’s education is 7.7 years for this sample (see Table 1). 26 Table 4a. Multinomial logistic regression of ECED sequence Sequence (base: Playgroup then kindergarten then primary) Playgroup Kindergarten Other No ECED then primary then primary combination Child characteristics Age 1.644*** 1.254*** 1.566*** 1.293*** (0.0711) (0.0506) (0.0583) (0.0582) Girl 0.749*** 1.007 0.965 0.815*** (0.0588) (0.0690) (0.0581) (0.0614) Stunted 1.237* 1.303** 1.087 0.00229*** (0.136) (0.139) (0.0997) (0.00230) Family characteristics Wealth Z-score 0.405*** 0.588*** 0.863** 0.617*** (0.0368) (0.0511) (0.0578) (0.0496) Mother's education (years) 0.786*** 0.871*** 0.948*** 0.796*** (0.0177) (0.0182) (0.0148) (0.0133) Parenting practices 0.903** 0.996 1.016 0.935* (0.0424) (0.0460) (0.0358) (0.0350) ECED quality (base: Low quality) Medium quality 0.973 0.959 0.860 1.006 (0.287) (0.269) (0.177) (0.257) High quality 0.358*** 0.432*** 0.464*** 0.441*** (0.101) (0.117) (0.0871) (0.101) ECED fees (base: No fees) ≤ 10,000 IDR 0.232*** 0.277*** 0.252*** 0.255*** (0.0830) (0.104) (0.0647) (0.0968) > 10,000 IDR 0.193*** 0.188*** 0.270*** 0.278*** (0.0716) (0.0727) (0.0679) (0.106) Constant 1.300 3.857*** 0.709 6.264*** (0.680) (1.974) (0.294) (3.535) Observations 12,710 Note: Exponentiated coefficients. Standard errors clustered at the village level in parentheses. *** p<0.01, ** p<0.05, * p<0.1 27 Table 4b. Multinomial logistic regression of ECED timing Timing (base: Playgroup (3-4) then kindergarten (5-6) then primary) Playgroup (5) Playgroup Kindergarten Playgroup kindergarten (3-4) then (5-6) then Other (5-6) then (6) then No ECED primary primary combination primary primary Child characteristics Age 1.724*** 1.257*** 1.639*** 1.353*** 1.380*** 1.647*** (0.0752) (0.0606) (0.0624) (0.0616) (0.0667) (0.137) Girl 0.722*** 1.074 0.932 0.787*** 0.866* 0.684** (0.0579) (0.0892) (0.0579) (0.0614) (0.0711) (0.101) Stunted 1.286** 1.229* 1.128 0.00238*** 1.496*** 1.411* (0.144) (0.151) (0.106) (0.00239) (0.179) (0.268) Family characteristics Wealth Z-score 0.398*** 0.576*** 0.850** 0.607*** 0.580*** 0.841 (0.0374) (0.0537) (0.0594) (0.0506) (0.0547) (0.109) Mother's education (years) 0.780*** 0.895*** 0.942*** 0.790*** 0.829*** 0.923** (0.0177) (0.0206) (0.0146) (0.0132) (0.0197) (0.0305) Parenting practices 0.892** 1.041 1.004 0.924** 0.923 0.890 (0.0430) (0.0539) (0.0369) (0.0358) (0.0515) (0.0835) ECED quality (base: Low quality) Medium quality 0.965 0.965 0.853 0.998 0.935 0.916 (0.287) (0.275) (0.176) (0.256) (0.281) (0.287) High quality 0.361*** 0.389*** 0.467*** 0.443*** 0.490** 1.061 (0.103) (0.107) (0.0901) (0.104) (0.149) (0.295) ECED fees (base: No fees) ≤ 10,000 IDR 0.233*** 0.260*** 0.253*** 0.256*** 0.303*** 1.085 (0.0831) (0.0986) (0.0648) (0.0968) (0.117) (0.564) > 10,000 IDR 0.202*** 0.173*** 0.282*** 0.290*** 0.229*** 1.612 (0.0747) (0.0677) (0.0709) (0.110) (0.0911) (0.825) Constant 1.078 1.976 0.592 5.215*** 1.249 0.00399*** (0.568) (1.066) (0.250) (2.960) (0.717) (0.00361) Observations 12,710 Note: Exponentiated coefficients. Standard errors clustered at the village level in parentheses. *** p<0.01, ** p<0.05, * p<0.1 28 Table 4c. Multinomial logistic regression of ECED duration Duration in playgroup Duration in kindergarten (base: 1 to 2 years) (base: 1 to 2 years) <1 year ≥ 2 years <1 year ≥ 2 years Child characteristics Age 1.197*** 0.801*** 0.940** 0.890*** (0.0383) (0.0333) (0.0285) (0.0275) Girl 0.898** 1.117* 0.897** 1.049 (0.0415) (0.0738) (0.0393) (0.0545) Stunted 0.795*** 1.182* 0.795*** 1.015 (0.0585) (0.104) (0.0567) (0.0741) Family characteristics Wealth Z-score 0.839*** 0.823*** 0.576*** 0.985 (0.0379) (0.0441) (0.0318) (0.0568) Mother's education (years) 0.959*** 1.031** 0.877*** 1.054*** (0.0117) (0.0158) (0.0124) (0.0152) Parenting practices 1.007 1.108*** 0.990 1.139*** (0.0299) (0.0437) (0.0288) (0.0340) ECED quality (base: Low quality) Medium quality 1.042 1.234 1.157 1.222 (0.157) (0.181) (0.214) (0.213) High quality 0.753** 1.194 0.748* 1.145 (0.108) (0.164) (0.129) (0.192) ECED fees (base: No fees) ≤ 10,000 IDR 0.596*** 0.853 0.713 0.827 (0.0963) (0.126) (0.166) (0.144) > 10,000 IDR 0.649*** 0.657*** 0.537*** 0.715* (0.109) (0.105) (0.126) (0.124) Constant 2.628*** 2.865*** 14.69*** 1.388 (0.831) (1.003) (5.110) (0.451) Observations 12,710 12,710 Note: Exponentiated coefficients. Standard errors clustered at the village level in parentheses. *** p<0.01, ** p<0.05, * p<0.1 29 Table 5a. Association of ECED sequence and test scores in primary school Sequence Language Maths Cognitive skills (1) (2) (3) Sequence (Base: No ECED [I]) Playgroup then primary [II] 0.0558** 0.0861*** 0.0284 (0.0263) (0.0261) (0.0298) Kindergarten then primary [III] 0.237*** 0.220*** 0.132*** (0.0228) (0.023) (0.0269) Playgroup then kindergarten then primary [IV] 0.408*** 0.416*** 0.198*** (0.0283) (0.0286) (0.0347) Other combination [V] 0.441** 0.305 0.118 (0.175) (0.196) (0.251) Constant -0.559*** -0.511*** -0.353*** (0.0379) (0.0381) (0.0445) Observations 12,710 12,710 12,710 R-squared 0.281 0.233 0.108 Pairwise contrasts Sequence: [II] - [I] 0.0558** 0.0861*** 0.0284 [III] - [II] 0.182*** 0.134*** 0.0660** [IV] - [III] 0.171*** 0.196*** 0.09 Note: Robust standard errors clustered at the village level in parentheses. All regressions include child characteristics (age, grade, gender, and stunting), family characteristics (household wealth, mother’s education, parenting practices), ECED characteristics (average quality, average fees), and district fixed effects. *** p<0.01, ** p<0.05, * p<0.1 30 Table 5b. Association of ECED timing and test scores in primary school Timing Language Maths Cognitive Skills (1) (2) (3) Timing (Base: No ECED [I]) Playgroup (3-4) then primary [II] 0.0854*** 0.116*** 0.0650* (0.0318) (0.0312) (0.0356) Kindergarten (5-6) then primary [III] 0.238*** 0.221*** 0.133*** (0.0228) (0.023) (0.0269) Playgroup (3-4) then kindergarten (5-6) then primary [IV] 0.418*** 0.427*** 0.205*** (0.0289) (0.0292) (0.0358) Other combination [V] 0.440** 0.304 0.117 (0.174) (0.196) (0.251) Playgroup (5-6) then primary [VI] 0.0225 0.0529 -0.0131 (0.0333) (0.0327) (0.0377) Playgroup (5) then kindergarten (6) then primary [VII] 0.320*** 0.308*** 0.141* (0.0716) (0.0676) (0.0844) Constant -0.559*** -0.510*** -0.352*** (0.0379) (0.0381) (0.0445) Observations 12,710 12,710 12,710 R-squared 0.281 0.233 0.108 Pairwise contrasts Timing: [II] - [I] 0.0854*** 0.116*** 0.0650* [VI] - [II] -0.0629* -0.0629* -0.0781* [III] - [II] 0.153*** 0.105*** 0.0680** [IV] - [III] 0.180*** 0.207*** 0.0715** [VII] - [IV] -0.0983 -0.119* -0.0634 Note: Robust standard errors clustered at the village level in parentheses. All regressions include child characteristics (age, grade, gender, and stunting), family characteristics (household wealth, mother’s education, parenting practices), ECED characteristics (average quality, average fees), and district fixed effects. *** p<0.01, ** p<0.05, * p<0.1 31 Table 5c. Association of ECED timing and duration and test scores in primary school Duration Language Maths Cognitive Skills (1) (2) (3) Timing (Base: No ECED [I]) Playgroup (3-4) then primary [II] -0.122* -0.0119 -0.0158 (0.0633) (0.0659) (0.0762) Kindergarten (5-6) then primary [III] 0.216** 0.0822 0.0116 (0.0968) (0.109) (0.124) Playgroup (3-4) then kindergarten (5-6) then primary [IV] 0.277** 0.238** 0.0387 (0.108) (0.119) (0.138) Other combination [V] 0.399** 0.247 0.0668 (0.175) (0.198) (0.246) Playgroup (5-6) then primary [VI] -0.115* -0.0135 -0.0664 (0.0601) (0.0631) (0.0745) Playgroup (5) then kindergarten (6) then primary [VII] 0.231* 0.167 0.000588 (0.126) (0.134) (0.157) Duration (Base: 1- 2 years) Playgroup: < 1 year -0.114** -0.0405 -0.0444 (0.0552) (0.058) (0.0693) Playgroup: ≥ 2 years 0.140*** 0.128*** 0.0553 (0.0324) (0.0319) (0.0379) Kindergarten: < 1 year 0.0198 -0.102 -0.0999 (0.0962) (0.109) (0.123) Kindergarten: ≥ 2 years 0.0925*** 0.0857*** 0.0511* (0.0219) (0.022) (0.0274) Constant -0.461*** -0.366*** -0.206 (0.114) (0.125) (0.144) Observations 12,710 12,710 12,710 R-squared 0.284 0.235 0.109 Timing: [II] - [I] -0.122* -0.0119 -0.0158 [VI] - [II] 0.00669 -0.00153 -0.0506 [III] - [II] 0.338*** 0.0941 0.0273 [IV] - [III] 0.0609 0.156*** 0.0271 [VII] - [IV] -0.0452 -0.071 -0.0381 Note: Robust standard errors clustered at the village level in parentheses. All regressions include child characteristics (age, grade, gender, and stunting), family characteristics (household wealth, mother’s education, parenting practices), ECED characteristics (average quality, average fees), and district fixed effects. *** p<0.01, ** p<0.05, * p<0.1 32 Figure 1. Predicted probability of ECED pathway by ECED quality and household wealth Note: Graphical representation of multinomial logistic regression output in Table 4b. Controlling for fees, family characteristics (household wealth, mother’s education, and parenting practices) and child characteristics (age, gender, and stunting) with district fixed effects. 33 Figure 2. Indicative cost-effectiveness of various early education pathways 1 year of playgroup (3/4), 1 year of kindergarten (5/6) then primary 0.063 1 year of kindergarten (5/6) then primary 1 year of playgroup (5), 1 year of kindergarten (6) then primary 2 years of playgroup (3-4), 1 year of kindergarten (5/6) then primary 0.046 1 year of playgroup (3/4), 2 years of kindergarten (5-6) then primary 0.039 2 year of playgroup (3-4), 2 years of kindergarten (5-6) then primary 0.032 2 years of kindergarten (5-6) then primary 2 years of playgroup (5-6) then primary 2 years of playgroup (3-4) then primary 1 year of playgroup (5/6) then primary 1 year of playgroup (3/4) then primary -0.150 -0.100 -0.050 0.000 0.050 0.100 0.150 0.200 Standard deviation increase in test score for every 100 USD Note: 95% confidence intervals. Test scores are the average math and language test scores of children measured at ages 6 through 9. Costs are assumed to be a total of 150.97 USD per child for one year of playgroup and 256.25 USD per child for one year of kindergarten. Test score improvements are based on the marginal effects of the timing variable in Table 5c. 34 References Alatas, H., Brinkman, S., Chang, M., Hadiyati, T., Hartono, D., Hasan, A., ... & Roesli, R. (2013). Early childhood education and development services in Indonesia. In Suryadarma, D., & Jones, G. W. (Eds.), Education in Indonesia (82-108). Singapore: Institute of Southeast Asian Studies. Barnett, W. S. (2011). Effectiveness of early educational intervention. Science, 333(6045), 975- 978. Brinkman, S. A., Hasan, A., Jung, H., Kinnell, A., & Pradhan, M. P. (2015). The impact of expanding access to early childhood services in rural Indonesia: evidence from two cohorts of children. World Bank Policy Research Working Paper, (7372). Brinkman, S. A., Hasan, A., Jung, H., Kinnell, A., Nakajima, N., & Pradhan, M. P. (2016). The Role of Preschool Quality in Promoting Child Development: Evidence from Rural Indonesia. World Bank Policy Research Working Paper, (7529). Bronfenbrenner, U. (1994). Ecological models of human development. Readings on the development of children, 2, 37-43. Cunha, F., Heckman, J. J., & Schennach, S. M. (2010). Estimating the technology of cognitive and noncognitive skill formation. Econometrica, 78(3), 883-931. Davidson, M., & Hobbs, J. (2013). Delivering reading intervention to the poorest children: The case of Liberia and EGRA-Plus, a primary grade reading assessment and intervention. International Journal of Educational Development, 33(3), 283-293. De Onis, M. (2006). WHO Child Growth Standards based on length/height, weight and age. Acta Paediatrica, 95(S450), 76-85. doi: 10.1111/j.1651-2227.2006.tb02378.x Elkind, D. (2008). The power of play: How spontaneous, imaginative activities lead to happier, healthier children. Cambridge, MA: Da Capo Lifelong Books. Engel, M., Claessens, A., & Finch, M. A. (2013). Teaching students what they already know? The (mis) alignment between mathematics instructional content and student knowledge in kindergarten. Educational Evaluation and Policy Analysis, 35(2), 157-178. Engle, P. L., Fernald, L. C., Alderman, H., Behrman, J., O'Gara, C., Yousafzai, A., ... & Iltus, S. (2011). Strategies for reducing inequalities and improving developmental outcomes for young children in low-income and middle-income countries. The Lancet, 378(9799), 1339-1353. Hanushek, E. A., & Woessmann, L. (2012). Do better schools lead to more growth? Cognitive skills, economic outcomes, and causation. Journal of Economic Growth, 17(4), 267- 321. Hasan, A., Hyson, M., & Chang, M. C. (2013). Early Childhood Education and Development in Poor Villages of Indonesia: Strong Foundations, Later Success. World Bank Publications. Jenkins, J. M., Farkas, G., Duncan, G. J., Burchinal, M., & Vandell, D. L. (2015). Head Start at Ages 3 and 4 Versus Head Start Followed by State Pre-K Which Is More Effective?. Educational Evaluation and Policy Analysis, 0162373715587965. Jung, H., & Hasan, A. (2015). The impact of early childhood education on early achievement gaps in Indonesia. Journal of Development Effectiveness, 1-18. Levin, H.M. & McEwan, P.J. (2001). Cost-effectiveness analysis: Methods and applications. 2nd edition. Thousand Oaks, CA: Sage. Ministry of Education and Culture (2012a). Nomor Unik Pendidik Dan Tenaga Kependidikan [Data set]. Jakarta: Ministry of Education and Culture. Ministry of Education and Culture (2012b). Early Childhood Education and Development Census Data [Data set]. Jakarta: Ministry of Education and Culture. 35 Nores, M., & Barnett, W. S. (2010). Benefits of early childhood interventions across the world: (Under) Investing in the very young. Economics of education review, 29(2), 271-282. OECD. (2012). Education at a Glance 2012: OECD Indicators. Paris: OECD Publishing. Pritchett, L. (2013). The rebirth of education: Schooling ain't learning. Washington, D.C.: Center for Global Development. Singh, A. (2014). Test score gaps between private and government sector students at school entry age in India. Oxford Review of Education, 40(1), 30-49. Sylva, K., Melhuish, E., Sammons, P., Siraj-Blatchford, I., & Taggart, B. (Eds.). (2010). Early childhood matters: Evidence from the effective pre-school and primary education project. Routledge. UNICEF. (2012). School readiness: A conceptual framework. New York: United Nations Children’s Fund. Walker, S. P., Wachs, T. D., Grantham-McGregor, S., Black, M. M., Nelson, C. A., Huffman, S. L., ... & Gardner, J. M. M. (2011). Inequality in early childhood: risk and protective factors for early child development. The Lancet, 378(9799), 1325-1338. World Bank. (2014). Indonesia Early Childhood Education and Development Project (ECED) Implementation Completion and Results Report. Washington, DC: World Bank. World Bank. (2016a). World Bank Group, UNICEF urge greater investment in early childhood development [Press release]. Retrieved from http://www.worldbank.org/en/news/press- release/2016/04/14/world-bank -group-unicef-urge-greater-invesment-in-early- childhood-development World Bank. (2016b). World Bank Education Statistics (EdStats) [Data set]. Retrieved from http://data.worldbank.org/data-catalog/ed-stats Yoshikawa, H., Weiland, C., Brooks-Gunn, J., Burchinal, M. R., Espinosa, L. M., Gormley, W. T., Ludwig, J., Magnuson, K. A., ... & Zaslow, M. J. (2013). Investing in our future: The evidence base on preschool education. Society for Research in Child Development and Foundation for Child Development. Zigler, E. F., Singer, D. G., & Bishop-Josef, S. J. (2004). Children's play: The roots of reading. ZERO TO THREE/National Center for Infants, Toddlers and Families. Zubrick, S., Smith, G.J., Nicholson, J.M., Sanson, A.V., & Jackiewicz, T.A. (2008). Parenting and families in Australia. Canberra: FaHCSIA (Department of Families, Housing, Community Services and Indigenous Affairs). 36 Annex A: Two parameter item response theory results The tests administered to the children in this project followed a common-item design, whereby 39 common items (also known as anchor items or linking items) were included in both tests. We applied a two-parameter item response theory (IRT) model to look at these common items across all the examinees to assess the difficulty and discrimination of the test items. The 2 parameter logistic IRT approach models the probability that a test taker j with a given ability will correctly answer an item i as: exp 1| ~ 0,1 1 exp where exp is the exponential function, is the difficulty parameter, and is the discrimination parameter. The results of the IRT shows that each of the 39 common test items predict student’s latent ability in language, mathematics, and general cognition (full results of the two-parameter IRT are in Table A1).14 Moreover, the items range in difficulty and discrimination, making it an appropriate test to examine students of varying abilities. Overall, the language and mathematics sections were relatively easy while the section on general cognitive skills was relatively difficult. As shown in the test characteristic curve in Figure A1 below, a student with a mean ability ( =0) was expected to answer 7 out of 10 items correctly in the language section, 10 out of 15 correct in the mathematics section, and 5 out of 14 items on the general cognitive skills section. Moreover, the test information function shows that the tests provide maximum information for examinees located at an ability level at the mean (for math and general cognitive skills) or slightly below the mean (for language). As the ability level moves away from the mean, the amount of information decreases and thus, the test estimates the corresponding ability levels with less precision (depicted by the increasing standard errors). For more details of the test information function, see Annex B. Taken together, the results of the IRT shows us that the tests (with the 39 common items) were well designed to measure the language, math, and general cognitive skills of children across the whole sample. 14 We did not use item response theory (IRT) to reweight our scores. 37 Figure A1. Test characteristic curve (TCC) and test information function (TIF) of early grade assessment Test Characteristic Curve Test Information Function Note: TCC shows the expected score (number of correct items) of a mean ability student ( =0) among all examinees in the sample (age 6 to 9). TIF shows how well the test can estimate each ability level. In IRT, information refers to the reliability or precision of a test; the ability level where the information parameter peaks is the ability level at which the test provides maximum information. For equations and interpretation of information in IRT, see Annex B. 38 Table A1. Two parameter item response theory results Language Math Raven’s matrices (cognitive) Difficulty Discrim Rank Rank Difficulty Discrim Rank Rank Difficulty Discrim Rank Rank Item (b) Item (b) Item (b) (a) by b by a (a) by b by a (a) by b by a 1 -0.8897 1.2582 1 1 1 -1.0427 2.3942 3 6 1 -1.0091 2.605 1 13 2 -0.4363 1.5842 6 3 2 -1.455 2.7665 1 11 2 -0.8637 2.7072 2 14 3 -0.3553 1.5788 9 2 3 -1.2386 2.9204 2 12 3 0.0498 2.3912 5 12 4 -0.3882 1.8782 8 6 4 -1.0058 2.5389 4 9 4 -0.3185 1.1128 4 3 5 -0.4286 2.1541 7 8 5 -0.4791 2.3664 6 5 5 0.563 1.8783 9 9 6 -0.5868 1.6645 3 4 6 -0.3154 2.9673 8 13 6 6.063 0.2849 14 1 7 -0.7825 3.1905 2 10 7 -0.2429 3.5732 11 14 7 -0.5376 1.7846 3 8 8 -0.472 2.3082 5 9 8 -0.2702 3.7138 9 15 8 0.509 1.9795 8 11 9 -0.0206 1.6921 10 5 9 -0.2434 2.2931 10 4 9 1.1583 1.6146 12 6 10 -0.5027 2.0769 4 7 10 -0.0479 2.4421 13 7 10 1.0737 1.5654 11 5 11 0.1051 2.5131 14 8 11 0.8939 1.6317 10 7 12 0.1546 2.5506 15 10 12 5.2521 0.3441 13 2 13 -0.3284 1.4113 7 2 13 0.2825 1.9747 7 10 14 -0.1692 1.223 12 1 14 0.0607 1.1286 6 4 15 -0.4793 1.6269 5 3 Notes: The difficulty parameter (b) measures the difficulty of answering the item correctly. For a test of n items, “rank by difficulty” is 1 for the easiest item and n for the most difficult item. The discrimination parameter (a) measures the differential capability of an item (i.e., an item with a high discrimination parameter has a high ability to differentiate subjects’ ability levels). For a test of n items, “rank by discrimination” is 1 for the least discriminatory item and n for the most discriminatory item. 39 Annex B. The test information function – equations and interpretation In item response theory, information is defined as the reciprocal of the precision with which the ability parameter can be estimated (Baker 2001). Specifically, under a two-parameter model, the information function for a test item is defined as: where: is the discrimination parameter for item i, is the ability level of interest, , and 1 . The test information function at a given ability is simply the sum of the item information at that level, which is defined as: where: is the amount of test information at an ability level of , is the amount of information for an item i at ability level , and N is the number of items in the test. The test information function tells us how well each ability level is being estimated by the test. If the amount of information is large, an examinee whose ability is at the corresponding level is estimated with high precision. Conversely, if the amount of information is small, an examinee whose ability is at that corresponding level is estimated with low precision. Theoretically, a test that can precisely estimate ability across the entire ability range would yield a horizontal line at some large value of I across all ability levels. In reality, however, tests have varying degrees of precision in estimating different levels of ability (Baker 2001). The test information function is particularly helpful when designing tests targeted at obtaining precise estimates of an ability at specified intervals. For example, if the purpose of a test was to identify remedial students, a test should comprise of items that allow it to provide more precise estimates of students at lower levels of ability (i.e., below the mean or 0). 40 Annex C. Primary school test score distribution by early education sequence Language 41 Math 42 Cognitive skills 43