Combining Supply- and Demand-side Interventions: Evidence from a Large Preschool Program in Cambodia , Adrien Bouguen† Jan Berkes∗ , Tsuyoshi Fukao§ , Deon Filmer‡ Impact Evaluation Final Report June 12, 2019 ∗ DIW Berlin, Free University of Berlin † Agriculture and Resources Economics Department, University of California, Berkeley ‡ The World Bank § The World Bank 1 Acknowledgments This research project is a joint effort between the Cambodian government, the World Bank, a team of field researchers (Angkor Research), and a team of academic re- searchers. We are particularly grateful to the World Bank for its constant support: special thanks to Simeth Beng for his support. This research was funded by SIEF. Special thanks to Alaka Holla for her useful comments throughout the program’s du- ration. The project could not have been conducted without the collaboration of the Ministry of Education, Youth and Sport in Cambodia. We are particularly grateful to Sok Sokhom and Lynn Dudley, who made this research possible. Data were col- lected in the field by a team of very dedicated field researchers from Angkor Research: special thanks to John Nicewinter, Ian Ramage, Benjamin Lamberet, Kimhorth Keo, and Ratanaksophea Saing for the stellar fieldwork. Finally, many researchers also con- tributed to this research through their very useful comments: Craig McIntosh, Patrick Kline, Christopher Walters, Karen Macours, Diego Vera, Markus Fr¨ olich, Paul Gertler, Supreet Kaur, Edward Miguel, Katja Kaufmann, Elisabeth Sadoulet, Alain de Janvry, Cl´ement de Chaisemartin, Antoine Camous, Harald Fadinger, Luc Behaghel, Clement Imbert, and Marc Gurgand. The findings, interpretations, and conclusions expressed in this paper are those of the authors and do not necessarily represent the views of the World Bank, its Executive Directors, or the governments they represent. 2 Executive Summary The main purpose of this evaluation is to study the impact of a supply-side intervention designed to improve the quality of community preschools, combined with a demand- side intervention designed to stimulate demand for preschools. The evaluation has three treatment arms: the construction of formal community preschools (CPS) and associated activities within villages (T1); the addition of a home visit by a village chief plus a door-to-door campaign to promote awareness about CPS and the value of education (T2); and the further addition of a home-based program consisting of trained “core parents,” who provided more intensive caregiver training sessions that focused on good parenting, the value of nutrition, and the importance of preschool (T3). Villages were randomly assigned either to a control group (C) or one of the treatment arms. The evaluation’s principal finding is that construction of CPS increased partici- pation in preschool. Children in villages with a newly built CPS were about seven percentage points more likely to have ever attended a preschool by the time they were between four and six years old (compared to a counterfactual value of 72 percent). Put differently, they were enrolled, on average, about one month more than children in the comparison group (who by that age range had been enrolled for about 5.4 months). While some CPS enrollments came from households that would not have enrolled absent a newly built CPS, the majority stemmed from a reduction in enrollments at informal preschools (IPS). This was in part by design, as the CPS construction aimed to upgrade informal arrangements and ensure a higher and uniform quality standard. In addition, part of the CPS enrollments came from a reduction in enrollments at state preschools (SPS, which are typically colocated in a primary school). The program designers and research team did not anticipate this. In fact, villages with a preexisting SPS were not supposed to be eligible for the program. The intervention’s occurrence in the context of enrollment shifts (rather than increases) affects the interpretation of its impacts on child development outcomes. Children previously at home who were induced to attend might have better outcomes as a result of the program; children who switched from an SPS might have worse outcomes; and children who switched from an IPS might see only marginal increases if the new quality standards were not substantially higher than the previous one. Enrollment increases were not different across the various treatment arms. Children in villages where the demand-side interventions were deployed were not systematically more likely to enroll in CPS (or any type of preschool). This was despite the fact that 3 caregivers in those villages did recall receiving a leaflet promoting preschool enrollment (in the case of T2 and T3) and participating in home-based program sessions (in the case of T3). While substantial effort went into designing and implementing the demand- side component of this program, it possibly did not have the intensity required to make a big difference, and a more intensive program (e.g., more frequent visits) might have had bigger impacts. At the same time, it is important to note that the construction of the CPS itself had an “information” component, in the sense that it was likely an important village event that households would have known about. It is possible the demand-side interventions deployed as a part of this study did not convey any information over and above that basic level. As discussed below, it is also possible other constraints exist that are more binding for preschool enrollment. The average intent-to-treat impacts of the intervention on child development out- comes (i.e., the simple comparison of children in treatment versus control villages) were small one year after the program started and had disappeared two years after- wards. More specifically, children in treatment villages scored about 0.04 standard deviation (sd) higher on an index of cognitive development (made up of measures of early literacy and numeracy along with executive function), and about 0.09 sd lower on a measure of socioemotional problems (the Strengths and Difficulties Questionnaire) after one year of implementation. After two years, these impacts were smaller and no longer statistically significant. These overall results mask considerable variation, how- ever. Strikingly, the impacts on cognitive development for children from the wealthiest households—those in the richest quartile in the sample—were large and statistically significant: 0.09 sd after one year of the program and 0.13 sd after two years. These results suggest that wealthier families were better able to take advantage of the CPS, perhaps by providing complementary inputs both in the preschool years and after the children had made the transition to primary school. The small size of the average impacts may be due to the fact that much of the enrollment change stemmed from enrollment shifts across preschool types rather than increases. An additional explanation may be that while CPS construction led to sub- stantial increases in infrastructure and material availability, this was not accompanied by a concomitant improvement in instructional quality. Detailed classroom observa- tions show CPS were substantially better equipped than IPS (but less well-equipped than SPS); they also show that curriculum content and quality of pedagogy, as well as the frequency and quality of teacher-child interactions, were similar in CPS and IPS 4 (while those in SPS were generally better). But this finding of small short-term impacts followed by a later (medium-term) fade-out, for example, once children are in primary school, is not unprecedented in the literature. Recent studies in Colombia and Malawi have documented similar pat- terns. Evidence from the United States suggests that medium-term fade-out might nevertheless be consistent with long-term improved outcomes perhaps because other— nonmeasured—aspects of child development improved. In addition, other research sug- gests that reaping the full medium- and long-term benefits of preschool might require complementary investments at later stages of human capital development. The results might seem disappointing in the sense that they do not document a rapid increase in net new enrollments into preschool; nor do they document large aver- age impacts on short (one-year) or medium (two-year) term impacts on child cognitive or socioemotional outcomes.1 The findings highlight that, notwithstanding the ro- bust evidence that high-quality preschool experiences boost child development,2 it is difficult to engender those experiences in a program at scale in a low-income, capacity- constrained environment. The findings have both programmatic and research implications. From a program- matic point of view, they suggest prioritizing a review of the quality of the CPS model and ways to enhance it. Two places to start could be the training of CPS teachers (currently substantially shorter and less intensive than that of SPS teachers) and the very limited duration of the CPS school day (currently two hours long). The goal should be to ensure that CPS offer services tailored to the varied developmental needs of children and therefore have high impact on child development. From a research point of view, the findings suggest more work is required to un- derstand the drivers of preschool quality and enrollment. The demand-side approaches tested in this evaluation were not enough to mobilize much additional enrollment over and above simply building CPS, suggesting that other factors inform the decision to send children to preschool. Direct or indirect costs may play a role, so approaches to reduce those costs—for example, cash transfers or reduced travel distances—might be 1 While the time difference between the follow-ups is only one year in our trial, we use the distinction between short- and medium term because the two-year follow-up captures a very different enrollment scenario than the one-year follow-up. At the two-year follow-up, 26% of the sample is enrolled at primary school, in stark contrast to the one-year follow-up (4%). Since heterogeneous effects show that midline impacts are primarily driven by five-year-olds, it seems likely the fade-out relates to primary school enrollment. Fade-out of IQ gains is common in early childhood interventions and usually occurs near primary school entry age. 2 See discussion in World Bank (2018). 5 necessary to induce higher participation rates. In addition, if the quantity and quality of preschool services do not meet families’ needs, households might have low demand for them. Therefore, increasing the quantity (time spent in preschool per day) or the quality of preschools may increase demand. Further investigation of these factors is required to better understand them. 6 Contents 1 Introduction 8 2 Brief literature review 10 3 Background and Program Design 12 3.1 Early Childhood Development programs in Cambodia . . . . . . . . . . 12 3.2 Supply-side intervention: formal community preschools . . . . . . . . . 13 3.3 Demand-side intervention: door-to-door awareness campaign and par- enting program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4 Evaluation Design and Data 16 4.1 Randomization and sampling . . . . . . . . . . . . . . . . . . . . . . . 16 4.2 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 4.2.1 Survey and instruments . . . . . . . . . . . . . . . . . . . . . . 18 4.2.2 Balance at baseline . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.2.3 Attrition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.3 Quality of CPS and other preschools . . . . . . . . . . . . . . . . . . . 22 5 Empirical Framework 25 6 Results 26 6.1 CPS construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 6.2 Enrollment outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 6.3 Child development outcomes . . . . . . . . . . . . . . . . . . . . . . . . 29 6.4 Demand-side interventions . . . . . . . . . . . . . . . . . . . . . . . . . 30 6.5 Perceived return to schooling . . . . . . . . . . . . . . . . . . . . . . . 31 7 Cost-effectiveness analysis 32 8 Conclusion 34 A Supplemental Material 56 B Child assessments 63 C Cost analysis 71 7 1 Introduction In this evaluation, we study the impact of an early childcare program on preschool en- rollments and child cognitive and socioemotional development. The program comprises three main interventions: a supply-side intervention that includes the construction of (formal) community preschools (CPS) and two demand-side interventions in the form of a door-to-door program (D2D) and a home-based program (HBP) to raise aware- ness of the availability and importance of new preschools. The evaluation design is a randomized controlled trial (RCT) in which 305 villages were assigned randomly to one of four experimental arms before the beginning of the program’s implementation. In the first arm (T1, 120 villages), the Cambodian government built a structure to host the CPS, trained a local teacher, and required the local community to regularly pay the preschool teacher. In the second arm (T2, 63 villages), in addition to the CPS construction and other activities in T1, the village received an awareness campaign conducted jointly by the village chief and our field team (the door-to-door campaign, or D2D). In a third treatment arm (T3, 64 villages), a parent in the community orga- nized monthly meetings with young parents to promote good parenting practices and increase awareness of the preschools (home-based program, or HBP) in addition to the interventions in T1 and T2. A remaining set of 58 villages formed the control group. Importantly, people in both treatment and control villages had access to other forms of preschool that were arguably substitutes to the new CPS. The presence of these al- ternative preschools raised several challenges in terms of identification, interpretation, and external validity of the findings. The inclusion of a demand-side component in this evaluation was based on the hypothesis that the village-level impact of the preschool construction program could be amplified by increasing the number of children who attended these preschools. Whereas the T1 intervention allows us to identify the impact of the simple preschool construction program on enrollments and child development, the T2 and T3 arms test increasingly intensive approaches to boosting attendance at these preschools. We collected three rounds of data to analyze impacts: a baseline survey at the start of the program, a midline survey one year later, and an endline survey two years later. Our results paint a mixed picture in terms of impacts on preschool attendance and child development. The presence of a preschool significantly increased preschool en- rollment. When a new CPS was present, about 41% of the children between three and five years old attended it. The construction of a CPS increased attendance at 8 any preschool by about 10 percentage points. At midline, after one year of implemen- tation (when children were between three and five years old), preschool attendance increased the cognitive and socioemotional development of children. Children in the T1 and T3 branches had cognitive development scores statistically significantly higher than the control group—although this difference is relatively small (less than 5% of an sd). However, children in T2 did not. (Initial imbalance in parental skills and educa- tional infrastructure make the T2 branch less comparable to the control group, and we therefore interpret this finding with caution.) At the same time, we find no evidence that the demand-side interventions included in T2 and T3 affected parental behaviors or attitudes toward education. Parents were already somewhat aware of the value of preschool (and education more generally); the additional information provided was not enough to change their behavior. Detailed classroom observations show that CPS were substantially better equipped than informal (community) preschools (IPS), but less well-equipped than state preschools (SPS). Yet they also show that curriculum content and quality of pedagogy, as well as the frequency and quality of teacher-child interactions, were similar in CPS and IPS (while those in SPS were generally better). At endline (when children were between four and six years old), two years after the program started, we can no longer detect any average impact of the CPS program on the measures of child development. We find this fade-out of the impact more pronounced when children are of primary school age (i.e., when they are six years old). We interpret this result as evidence that the initial benefits from preschool were not sustained once children made the transition to primary school, results that are consistent with those found elsewhere (Ozler et al., 2018; Andrew et al., 2018). Children from the wealthiest households are an exception to this pattern: their cognitive skills are positively affected at both midline and endline, suggesting that parents from these households are better able to take advantage of the new CPS (perhaps by supplying complementary inputs). At the same time, evidence of average fade-out should be interpreted with care. While it may indicate that the studied preschool intervention doesn’t improve medium-term competencies, it is also possible that other, nonmeasured skills were affected that could lead to better long-term accumulation of human capital and better long-term outcomes. Duncan and Magnuson (2013) argue that this could explain the patterns of findings in the United States. Moreover, it is also possible that high-quality primary schools, which 9 may be in short supply in Cambodia, are required to sustain impacts.3 If wealthier households are more able to sustain high-quality inputs through primary school, this might explain the sustained impacts on cognitive development for children from these households. Further research that tracks individuals over longer periods of time is necessary to investigate these various hypotheses. 2 Brief literature review Studies on preschool programs in high-income countries show that the effect of preschool depends on multiple factors such as the quality of the program (Blau and Currie, 2006; Datta Gupta and Simonsen, 2016) and continuous follow-up by later investments (Cunha and Heckman, 2007). Further, the impact of preschool usually depends on ob- served and unobserved characteristics of the children enrolled (Havnes and Mogstad, 2011; Cornelissen et al., 2018). While preschool programs that target disadvantaged children can be highly cost-effective (Kline and Walters, 2016), the economic case for subsidized universal programs tends to be relatively weak as some children might al- ready have a stimulating home environment or might already be enrolled in private preschool (Havnes and Mogstad, 2015; Elango et al., 2015). The existing literature on preschool programs in low- and middle-income countries points to positive effects of preschool attendance, but the evidence is mostly based on nonexperimental study designs (Engle et al., 2011; Rao et al., 2014). Berlinski et al. (2008) exploit variation in preschool attendance within households after a rapid expansion of preschool supply in Uruguay. They find positive effects of pre-primary education on subsequent school attainment and reduced school drop-out rates. For Argentina, Berlinski et al. (2009) consider the effects of a large expansion of univer- sal preschool education on performance in primary school by exploiting variation in treatment intensity across regions and cohorts. They find that preschool attendance significantly increased test scores during primary school. Experimental evidence on the effectiveness of large-scale early childhood education and development (ECED) programs in low-income countries is rare, but the number of studies is slowly increasing. There are only a few studies from low-income countries in which the effectiveness of large-scale preschool or playgroup services is explored in 3 Johnson and Jackson (2018) document such dynamic complementarities between the Head Start program and investments in primary and secondary schools in the United States. 10 an RCT. Martinez et al. (2017) analyze the impact of a preschool program by Save the Children in rural Mozambique. In an environment with few other preschools, they find positive impacts of preschool attendance on school readiness. Two years after the program started, treated children performed significantly better on the cognitive, fine-motor, and socioemotional test domains, although not on language development. The intervention also increased the probability of primary school enrollment at the appropriate age. When it comes to the impact of preschools in an environment where some ECED services are already present, the case is less clear-cut. In a recent paper, Brinkman et al. (2017) assess the impact of community-based playgroups in rural Indonesia. They find little impact on child development based on their experimental design, in which expo- sure to the intervention was relatively short (11 months). They use a nonexperimental control group in a difference-in-difference framework to show that longer exposure, up to three years, had a significantly positive impact on most test domains for children from disadvantaged backgrounds. While the program increased overall enrollment, enrollment into (nonprogram) kindergarten services dropped significantly. Hence, the measured effects are based on a change from a mix of nonprogram kindergarten services and parental care to the new playgroups. Bouguen et al. (2018) analyze the effects of a randomized preschool construction program in Cambodia during 2009–2011. While the construction of preschools did not improve children’s test scores in the short run, the authors present evidence that the randomized intervention even affected some children negatively. Their findings are best explained by the fact that some children who would have registered in primary school before the regular age were going to the new preschools once they were available. The study shows that the introduction of a preschool program can have unintended consequences and that demand-side responses have to be considered when evaluating a program of this kind. A similar finding is described by Blimpo et al. (2019), who evaluate the impact of different randomized ECED services in The Gambia. In a first experiment, community-based centers were randomized among villages. In a second experiment, the authors analyze the impact of a teacher training program in preschools attached to primary schools. The authors find no evidence of improved levels of child development on average. However, they find suggestive evidence that the first inter- vention had negative effects on children from less-disadvantaged households who may have been steered away from a better-quality environment in their homes. Bernal et al. 11 (2019) find negative effects on cognitive outcomes, positive effects on nutrition and no statistically significant effect on socio-emotional development from an intervention that allowed children at the age of 6-60 months to switch from small home-based community nurseries to large childcare centers in Colombia. While the evaluated centers had signif- icantly higher structural quality than the counterfactual nurseries, nurseries performed better at some quality indicators and it remained unclear whether process quality (e.g. pedagocial quality) improved for children enrolling at the new centers. 3 Background and Program Design 3.1 Early Childhood Development programs in Cambodia Despite two decades of robust economic growth, Cambodia remains one of the least developed countries in Southeast Asia, with a GDP per capita estimated at $1,160 in 2015 ($3,300 in PPP terms). The country also faces multiple challenges in the edu- cation sector. With a preschool enrollment rate in 2009 of 40% among five-year-olds (MoEYS, 2014), Cambodia fares poorly in comparison to neighboring Thailand and Vietnam.4 To increase the capacities and quality of its education system, the Cam- bodian government received a first grant from the Global Partnership for Education (GPE I) of $57 million for the period 2008–2012. The government, in cooperation with the World Bank, used part of the resources to invest in the expansion of the national early education system, which is composed of formal preschools, informal preschools, and parenting programs. Over the period of this grant, preschool enroll- ment of five-year-olds increased from 40% in 2009 to 56% in 2012 and 66% in 2016, while enrollment of three- and four-year-olds remained at a low 20% and 37% in 2016, respectively (MoEYS, 2014, 2017). Bouguen et al. (2013) have evaluated the impact of the ECED components of GPE I on child development outcomes. They find no impacts on outcomes overall, although the study period was marked by implementation issues including low individual take-up and delays in program implementation. To improve primary school readiness and be on track with the Sustainable Devel- opment Goals,5 the government of Cambodia, with the support of the World Bank, launched another education expansion program for the period 2014–2018. The plan 4 Source: Data from UNESCO Institute for Statistics. 5 The SDG 4.2 states that all children should benefit from at least one year of pre-primary education by 2030. 12 is financially supported by a second GPE fund (GPE II) of $38 million, still adminis- tered by the World Bank, with the objective of strengthening the existing foundation of Cambodia’s education system. Of this amount, about $20 million were allocated to a component focused on ECED programs. Our study focuses on the subcomponent that includes the construction of formal community preschools. 3.2 Supply-side intervention: formal community preschools Before GPE II, two distinct types of public preschools existed in Cambodia: state preschools (SPS) and community preschools. Since these community preschools lacked uniform quality standards, we refer to them here as informal (community) preschools (IPS).6 7,8 GPE II introduced a new type of community preschool with a uniform quality standard, which we refer to as (formal) community preschools (CPS). SPS are financed by the Ministry of Education, Youth and Sport (MoEYS). SPS teachers benefit from two years of formal training in a MoEYS teacher training center in Phnom Penh. They receive a monthly salary of roughly $250 to teach for three hours a day, five days a week. As almost all SPS are attached to a public primary school, SPS have access to classrooms equipped with teaching and play materials, not to mention better overall infrastructure (including sanitation facilities). In contrast, IPS are typically not attached to a primary school. The local commu- nity establishes the IPS and covers operational costs. This includes the IPS teacher salary, which is at the discretion of the local commune council. Due to council bud- getary constraints, it varied from $30 to $50 per month at the time of our baseline; most IPS teachers rely on other income sources. IPS teachers are trained for about 35 days by provincial education departments before they begin work. Teachers are required to provide a two-hour preschool class five days a week. The quality of IPS can differ substantially across villages as, until recently, communes were required to establish IPS with their own funds. Consequently, IPS classes are often held in a teacher’s home, a community hall, or a pagoda. IPS often lack appropriate equipment, materials, and facilities. In most cases, IPS lack even the most rudimentary equipment such as tables 6 We use this term for expository clarity; this is not the formal or usual nomenclature in Cambodia. 7 According to government data (MoEYS, 2017), out of 7,241 preschool facilities in Cambodia in 2016, 55% were SPS, 39% were IPS, and 6% were private preschools. However, these preschools are not evenly distributed across the country, and 38% of the 1,646 communes in Cambodia had no preschool facility. 8 See Bouguen et al. (2013) for an impact evaluation of each type of preschool developed in the context of the GPE I. 13 and chairs. To increase preschool access and to improve the unsatisfactory quality of IPS, the Cambodian government agreed to use the GPE II grant to establish 500 new CPS. Some of these replaced previously existing informal arrangements; others were estab- lished in villages that previously had no preschool or were too large to be served by one preschool alone. In contrast to IPS, CPS benefit from uniform quality standards such as a standardized building directly financed by the GPE II. CPS have a capacity of 25 children and are fully equipped with tables, chairs, a blackboard.9 MoEYS is re- sponsible for the curriculum, teacher recruitment and training, and ongoing monitoring of the facility, including regular payment of teacher salaries. The 35 days long CPS teacher training includes lessons in pedagogical strategies, curriculum content, testing, and how to operate a CPS. They are also trained in basics of child development, child rights and parental education. All teachers participate in a written examination be- fore and after the training. Further, teachers are provided with a package of teaching materials tailored to the CPS curriculum. The CPS teacher is usually a community member who, after completion of the training, gives a two-hour class each day, five days a week, to children aged three to five years. Irregular payments of IPS teacher salaries and the lack of a specialized building made the IPS model unsustainable, and many schools planned under GPE I either did not open or operated only for a short time (Bouguen et al., 2013). Given these problems with the IPS model and the high costs of SPS, the Cambodian government considered CPS a promising, less costly alternative that—due to a uniform quality standard— could prove similarly effective to SPS. CPS require fewer resources for building con- struction and teacher salaries than SPS, since teachers are relatively less educated and are recruited from within the commune. Further, a similarly quick and large scale-up of SPS would have been significantly more costly. It also would have proven difficult due to the more intensive teacher training program and to the lack of sufficient per- sonnel to train many teachers at once. The Cambodian government therefore decided to scale up CPS as an intermediate model between IPS and SPS, and it supported the evaluation of this model with an eye toward the sustainability of CPS enrollment and impacts on early childhood development. The primary conceptual differences between CPS and IPS are the standardized building, equipment, and monitoring of regular teacher salary payments, but CPS teachers do not receive the same salary, training, 9 See pictures in Figure A.1 for an example of the standardized CPS building. 14 and support as SPS teachers.10 As this evaluation shows, the CPS model is appealing because it presents an affordable approach that can be scaled up. Yet certain design features may have limited its effectiveness (e.g., the length of the school day). In retro- spect, these may seem like clear deficiencies, but when the program was developed, the CPS model was considered a substantial improvement over IPS and therefore worth deploying. 3.3 Demand-side intervention: door-to-door awareness cam- paign and parenting program The door-to-door (D2D) program implemented as a part of this intervention was a demand-side intervention aimed at stimulating demand for ECED programs by speak- ing directly to individual caregivers. The goal was to sensitize them to the value of preschool education and guide them through the enrollment of their children at CPS. An additional component was to provide information about returns to education. Such information has been shown to effectively increase attendance and change the social composition of students in other lower-income contexts (Nguyen, 2008; Jensen, 2010). At baseline, caregivers received a printed leaflet that had information about the newly established CPS. It noted that the teacher had been trained and it suggested how preschool in general could help children improve primary school readiness and, poten- tially, their overall educational attainment. In addition, the leaflet provided informa- tion about average income in Cambodia by educational background, visualized in a graph and using data from the Cambodian Socioeconomic Survey 2009. Caregivers received another informational leaflet about one year later, at midline.11 The local village head and the field staff responsible for the study’s data collection performed the D2D activities.12 10 The Cambodian government intended to significantly increase teacher salaries conditional on good implementation of the CPS model. After completion of this impact evaluation in 2018, the government provided additional funding to communes to significantly increase salaries of 500 CPS teachers. For these teachers, CPS class duration increased from two to three hours and responsibility for the HBP was assigned to CPS teachers. 11 The content of the leaflets was developed in cooperation with MoEYS. After baseline, we re- ceived feedback from village heads that the leaflet contained too much information (see Figure A.2). Therefore, the midline leaflet was simplified to focus only on the advent of a new CPS (see Figure A.3). 12 In the initial concept for this study, the D2D was entirely dependent on the village heads acting as volunteers for this effort. Due to concerns about noncompliance, we decided to have the study field staff target sampled households as well. While this came at the cost of not being able to distinguish 15 The home-based program (HBP) formerly operated in Cambodia as an independent early childhood education service to support parents of children aged zero to five years. However, it was redesigned as supplementary to CPS, aiming to enhance the effect of CPS enrollment. The program is run by local “core parents” who receive initial and ongoing training from MoEYS. The 35 day long training covers a wide range subjects such a child rights, pre- and postnatal care of mothers, hygiene, nutrition, diseases prevention, developmentally appropriate activities for children, school readiness, dis- abilities, health services, child protection and more. Similarly to CPS teachers, core mothers participate in a written examination before and after the training. They are responsible for promoting enrollment of children aged three to five years in CPS and for leading monthly informational meetings with parents of children aged zero to six years. “Core parents” are volunteers who only receive stipends while in training. The HBP was supposed to take place regularly; it was designed as a more intensive demand-side intervention than the D2D. Rather than just raising awareness about the new CPS, the HBP also informs parents why preschool is important. 4 Evaluation Design and Data 4.1 Randomization and sampling This evaluation of the CPS program is based on an RCT.13 All sample villages are situated in the south and northeast parts of Cambodia, as the western part of the country was already covered by GPE I (see map in Figure 1). Eligibility criteria for villages to participate in the study were: expressed demand for a CPS, a high poverty rate, and a large number of children aged zero to five years. The study sample is composed of 305 villages. Before baseline, villages were ran- domly assigned to the control group or one of the three treatment groups: CPS (T1); CPS and D2D (T2); or CPS, D2D, and HBP (T3). Randomization was stratified to obtain a sample for which treatment is balanced within each of the 13 provinces of our between the two parts of the D2D, we believed it would be more informative to learn about the impact of a more intensive and consistently implemented intervention than to rely exclusively on volunteers in a setting where compliance could only have been monitored retrospectively. When this decision was made, we paid particular attention to maintaining the initial character of the D2D: a brief information intervention with no or minimal costs that could be conducted with virtually no training. The task of distributing and explaining information leaflets could easily be assigned to village heads or teachers in the future. 13 The study has been preregistered at AEA’s Social Science Registry (AEARCTR-0001045). 16 sample.14 The design is summarized in Table 1. Table 2 gives an overview of data collection activities and timing of the preschool construction. Our analysis is based on three main waves of data collection: a baseline data collection in May–July 2016, a midline survey in April–June 2017 (midline), and an endline survey in May–July 2018 (endline). In addition, a brief monitoring survey was conducted in late 2016 to confirm that CPS construction was proceeding as sched- uled. With 91% of CPS constructed before follow-up, Table 2 confirms the construction rolled out as planned. Nevertheless, and despite our effort to ensure that the baseline survey preceded CPS construction, a completed CPS building was already available at baseline in 17% of the treatment group villages. It is challenging to conduct an experiment like this with school construction. On one hand, fielding the baseline too early (well before any construction) would have increased the risk, in case of construc- tion delay, that our baseline sampled children would have been too old to attend the newly built preschools.15 On the other hand, fielding the baseline too late would have resulted in baseline measures that were arguably already affected by the program. We discuss the implications of the slight overlap between baseline survey and construction below. During the baseline data-collection exercise in 2016, the survey firm sampled up to 26 eligible households per village, using an adapted version of the EPI walk to sample them.16 This method guarantees the baseline sample is representative for households with children at preschool age within the village.17 Eligible households include at least 14 The randomization was performed with a list of 310 eligible villages provided by the government. Of these, 60 were assigned to the control group, 123 to T1, 63 to T2, and 64 to T3. Unfortunately, the randomization list contained erroneous village names and five of them were duplicated or could not be identified after the randomization even after substantial effort by MoEYS and the data collection firm. Therefore, the total number of villages decreased to 305. We treated this dropout as random and did not replace the villages. The randomization list also contained villages for which a CPS teacher was no longer available or for which no land could be secured for CPS construction. Therefore, less than 100% of treatment group villages received a CPS. Since these factors are potentially endogenous, we do not treat these as random. To maintain ex ante expected balance between control and treatment villages, these villages were therefore not removed from our sample. 15 As described in Bouguen et al. (2013), construction delays occurred in a previously evaluated program in Cambodia. This considerably reduced take-up, exposure time, and statistical precision. 16 EPI refers to the Expanded Programme on Immunization of the World Health Organization; see e.g., Henderson and Sundaresan (1982). 17 The sample is not nationally representative. As mentioned above, villages were selected based on criteria such as expressed interest in the program, poverty rate, lack of a functioning preschool, and teacher availability. Sample households are exclusively from rural areas in southern and eastern Cambodia. This is because western provinces received CPS under a previous preschool construction program. 17 one child between 24 and 59 months old at baseline. Children eligible to enroll in school or preschool were therefore between three and five years old at midline and between four and six years old at endline. For the baseline data collection, 7,053 eligible households were identified. Extra household sampling was conducted at midline in villages where the number of eligible households at baseline was below 10. An additional 53 households were therefore added to the sample at midline. 4.2 Data 4.2.1 Survey and instruments The survey firm recruited interviewers based on their familiarity with data collection and their experience with young children. To ensure that children and parents correctly understood the questionnaire and that the instruments were reliable, the research team pretested every instrument at least three times before collecting data in the sample villages. The survey firm translated the questionnaires into Khmer and an independent third party translated them back into English, which led to further refinements in the instruments. The research team participated in the interviewer training conducted by the local firm’s fieldwork manager. Field staff were organized into six groups, each comprising four interviewers, one supervisor, and one field editor. All supervisors had several years of data collection experience in Cambodia and were responsible for household sampling and quality control procedures. Editors supported supervisors by doing spot checks and interviewer observations, and they conducted independent reinterviews of at least 20% of interviews in each village. For each household, a household survey, caregiver survey, and one assessment per eligible child were conducted.18 At the village level, interviews were conducted with village heads and preschool teachers. In addition, the endline data collection was complemented by a classroom observation survey conducted at all preschools within the sample villages. The household survey includes information about family structure, household wealth, 18 The caregiver is defined as the direct relative (parent, grandparent, aunt/uncle, or adult sibling) who takes care of the child most of the time. In most cases, the caregiver is a biological parent (60.4% at baseline, 58.7% at midline). In the provinces of Kampong Speu, Kandal, Prey Veng, Svay Rieng and Takeo, the caregiver is often a grandparent of the child. These are provinces with relatively high levels of manufacturing industry; mothers who work in factories only occasionally get to spend time with their child during the work day. 18 and other socioeconomic background characteristics. We construct a dwelling quality index and household assets index using Multiple Correspondence analysis (MCA). Each variable was coded using the sign of the coordinate of the first MCA dimension. We then took the average of the standardized version of each coded variable. Both indexes were standardized and summed to create a wealth index that is used to describe wealth quintiles in subsequent analyses. The caregiver survey includes questions regarding the child’s preschool enrollment and socioemotional development (Strengths and Difficul- ties Questionnaire), as well as 25 questions about parenting practices. The latter measure three key dimensions of parenting (cognitive parenting, emotional parenting, and negative parenting). Further, the survey includes information about participation in the HBP, the caregiver’s perceived returns to education, and a short test of the respondent’s nonverbal reasoning ability (based on the Raven’s Progressive Matrices Test). The approximately 45-minute child assessment includes a comprehensive battery of cognitive tests (executive function, language, early numeracy, fine motor development and, at baseline and midline only, gross motor development) as well as anthropometric measures (height and weight). Most of the child tests are based on the Measuring Early Learning Quality and Outcomes (MELQO) toolkit. (See UNESCO (2017) for a description of the measures-of-development process and Raikes et al. (2019) for evidence of validity).19 Additional child tests were added to the MELQO items to increase the sensitivity and breadth of the child assessment. The additional tests included the following: the Dimensional Change Card Sort (Zelazo, 2006)20 ; a receptive vocabulary test based on picture recognition derived from the Test de Vocabulario en Imagenes Peabody (TVIP), a version of the Peabody Picture Vocabulary Test adapted for Spanish-speaking popu- lations that was normed to low-income populations in Mexico and Puerto Rico (Dunn et al., 1986; Dunn and Dunn, 2007); a test for knowledge of reading concepts (based on a monitoring tool used by the Cambodian Ministry of Education’s Early Child Devel- opment Department); and a sustained attention test. Children’s socioemotional devel- opment was measured using the caregiver-reported Strengths and Difficulties Question- naire. Since the overall purpose was to generate a well-functioning test for Cambodia 19 An in-depth discussion of midline child tests, scoring methods, cultural adaptations, pretesting procedures, and questions about parenting practices can be found in Berkes et al. (2019). 20 No cultural adaptation of words or pictures was conducted since the test was working well in the field and changes were not deemed necessary by the local staff. 19 (as opposed to maintaining consistency for other purposes, such as international com- parisons), adaptations were prioritized to ensure adequate fit to the local context. While most tests in the early literacy, numeracy, and fine-motor skill domains relate directly to the CPS curriculum, executive function tests are only indirectly related. Before constructing the composite scores of child test domains, individual tests were first scored and standardized, thus ensuring that all the tests contributed equal variance to the composite score. For almost all the tests, scoring was done by assigning one point for each correct response and summing these points to create an individual score for each test. When a child was unable to complete the practice trial of a test, a score of zero was assigned for this test, as long as the child participated in the other tests. Standardization of each test score was done by subtracting its sample mean and dividing it by its sample sd. All standardized test scores of one domain (e.g., executive function) were then summed into a domain score and standardized again by subtracting its sample mean and dividing by the sample sd of the domain score for better interpretability. The individual tests, their distributions, and the scoring methods are summarized in Appendix B. The village and teacher surveys included questions about the ECED services avail- able in the sample village. This allowed us to monitor implementation of program interventions and alternative services. Parts of the teacher survey and the classroom observation tool are based on the Measuring Early Learning Environments (MELE) module of MELQO. MELE includes key domains for quality in early learning envi- ronments and the sample items used to measure them. During pretests preceding the endline data collection, constructs to be measured were selected and specific items were adapted intensively to take into account culture-specific views on what defines a high- quality learning environment. We divided the final module into five domains: teacher characteristics, equipment, classroom setting, curriculum content and pedagogy, and teacher-child interactions. 4.2.2 Balance at baseline Baseline characteristics of villages, households, children, and their caregivers are sum- marized in Table 3. To test for statistically significant differences, we regress the variables on binary indicators for the different treatment groups and a set of province dummies to account for stratified randomization. Overall, the sample is balanced in child, household, and caregiver characteristics. The characteristics for T1 and T3 have 20 only minor differences with the control group.21 However, the characteristics for T2 have larger differences, in particular for variables that one might expect to be asso- ciated with child outcomes as well. We address this issue by controlling for these characteristics in our main regressions. As expected, there are significant differences in preschool enrollment and take-up of demand-side interventions due to the rollout of the interventions simultaneously to baseline (as shown in the bottommost panel of Table 3). It is therefore possible that test scores are already slightly affected by the treatment at baseline, which would pose a challenge to interpreting our results. A mitigating factor is that (as shown in Table 2) this only potentially affects a small part of the baseline sample. While it is possible that teachers in other treatment villages were already trained and giving preschool classes in a different building or at home, exposure to preschool is one to two months at most.22 The more relevant comparison, therefore, is the fact that all child characteristics are well-balanced at baseline. A potentially greater cause for concern is an imbalance in the availability of state preschools (again, SPS). (This is shown in the baseline panel of Table 8, which describes village infrastructure at baseline, midline, and endline). The control group has the highest share of villages in which an SPS is located and a significantly higher share than group T2. The existence of an SPS was supposed to make a village ineligible for the program, but clearly this guideline was not followed. Whether this imbalance is due to chance or other reasons, we need to consider it when interpreting the results. It will be important to recognize that because children who do not enroll in CPS might benefit from SPS, the imbalance can manifest as an imbalance in the outcomes of children who would never enroll in CPS (“never-takers”). 4.2.3 Attrition To minimize attrition associated with seasonal migration related to the agricultural cycle, all survey waves were completed before the beginning of August of each year. Families often move to different villages or are too busy for interviews due to agri- 21 A multidimensional binary poverty index was constructed for each household using baseline data and an adapted version of the method suggested by Alkire and Santos (2010) with k =0.3; i.e., all poor people must be deprived in at least 30% of the weighted indicators. This multidimensional poverty index is based on the indicators for health, education, and living standards. 22 Since previously existing preschools focused less on children at age three or four, the differences in preschool enrollment are also likely to be high at baseline despite the lower number of CPS since no other preschools existed for this age group in the control group at that time. 21 cultural work during the school breaks in August and September. An overview on attrition patterns is given in Table 4. Most importantly, attrition and the random assignment are not related as shown in columns 1 and 3 of Table 4 for both the midline sample (left three columns) and the endline sample (right three columns). Column 2 shows that attrition is not significantly related to a set of variables strongly associated with child development. There is also no evidence of differential attrition with respect to baseline controls except for T2 at the endline follow-up, where children with a high height-for-age z-score were more likely to attrit from the sample. As evident from col- umn (3), the inclusion of households added to the sample at midline causes differential attrition to become significant for T1 at midline since fewer households were added to the sample in group T1. The overall level of child attrition is around 10% at both midline and endline, and mostly due to seasonal migration.23 While not unreasonably high compared to similar studies in other countries, and while not likely to differentially affect the various treatment arms of the study (and therefore not affect the internal validity of our analysis), we recognize that the external validity of our results is limited to households that didn’t relocate at midline or endline. 4.3 Quality of CPS and other preschools We use the teacher survey and classroom observation tools to assess how new CPS compare to SPS and informal preschools (again, IPS) in our sample.24 While recruit- ment and training procedures are similar for teachers of IPS and CPS, only CPS benefit from a standardized building. In contrast, SPS teachers are recruited through a dif- ferent process and teach in classrooms within the premises of a primary school. As the impact of this evaluation is highly context specific and depends on not only the quality of the intervention itself but also its alternatives, we compare properties of the different preschools to give the results some context. Table 5 summarizes characteristics of the main preschool teacher and equipment of the preschool facility. As shown in columns 4 and 5, teachers of CPS and IPS 23 Reasons for attrition of 597 households at midline: child does not currently live in the village (542), caregiver/child temporarily not at home (34), reason unknown (10), household unknown (5), refusal (3), child passed away (3). 24 Note that this is not representative of IPS and SPS in Cambodia. In particular, it does not represent the sample of IPS after their initial construction, e.g., what was evaluated in Bouguen et al. (2013). Only IPS still operating at the time of endline are represented in this sample. These IPS are positively selected in terms of sustainability and therefore also most likely positively selected in terms of quality. 22 are highly similar in all characteristics. SPS teachers differ significantly: they are 41 percentage points more likely to have at least nine years of education and and 55 percentage points more likely to have at least 12 years of education than CPS teachers. In addition, they perform a large (0.51 sd) higher on the Raven’s Progressive Matrices Test. They are also significantly more likely to have more than eight weeks of teacher training. Surprisingly, SPS also have a higher share of teachers with no training at all. The salary difference between SPS and CPS teachers is substantial, with SPS teachers earning more than four times more. While the SPS teacher salary of $250 is more than twice as high as the monthly GDP per capita, CPS teachers only receive a small compensation. While all SPS and almost all CPS and IPS teachers receive their full salary, SPS teachers are more likely to receive it regularly.25 Both IPS and SPS lack tables and chairs for teachers and students, but almost all CPS have them due to the successful centralized procurement process. CPS are also much more likely than IPS to have writing utensils, art materials, fantasy play materials, and educational toys. However, not all CPS have these materials. Moreover, CPS are least likely to have access to electricity, a functional water source, or toilet facility. While SPS are most often attached to a primary school and IPS often take place in a teacher’s home, CPS tend to be solitary structures without close proximity to a water supply or toilet facilities. Table 6 shows classroom setting and teacher behavior based on the observation of one full preschool class. CPS and IPS classes last approximately two hours, while SPS classes last three hours. Enrollment and attendance numbers differ significantly between preschool types, with the highest attendance rates for SPS classes (76.9%) followed by CPS (70.4%). IPS classes, which tend to have the smallest number of students enrolled, have the lowest attendance rate (66.4%). With an average of 25 enrolled children at CPS, CPS classes are at the maximum capacity allowed. Yet, due to the 70% attendance rate, CPS classes would have the capacity to accept more children. Teachers were asked if they followed a curriculum and if they could show proof of how they tracked children’s development and attendance. Differences between CPS and IPS are significant for these items, and CPS teachers are even more likely than SPS teachers to be able to present a printed curriculum. Curriculum content from the classroom observation is more diverse for SPS than 25 Once again, we note that one caveat of this comparison is that preschools that do not pay a teacher for a longer time are likely to shut down at some point. Hence, this sample of preschools does not capture unsustainable IPS that no longer operate. 23 CPS. While teachers were equally likely to cover math and storybook reading, they were more likely to include all other categories of curriculum content listed in Table 6. For example, they were 23 percentage points more likely to engage in activities supporting the development of literacy skills. This difference might also be driven by differences in the duration of class between the schools. Differences were also observed with regard to the pedagogical approach, the quality of which was assessed for each covered category.26 SPS teachers showed the highest quality for all subject categories; the difference is significant for most categories. The last panel of Table 6 shows that SPS teachers are significantly more likely to use better-quality disciplinary strategies and to provide more feedback. Additionally, they are better able to engage students in class. Overall, Tables 5 and 6 confirm that the transformation of IPS to CPS mostly affected the learning environment by providing better equipment, while teacher quality was only affected marginally in the measured dimensions. In contrast, SPS teachers are significantly more educated and perform better in terms of instructional quality. In other words, while the switch from IPS to CPS is likely to affect children positively, a switch from SPS to CPS would likely affect them negatively. While the majority of substitutions come from IPS rather than SPS due to the replacement of IPS by CPS, this makes findings for group T2 difficult to interpret because SPS are less common in this group due to finite-sample imbalances. Last, we use questions from the caregiver survey to compare differences in missing days, perceived teacher quality, financial contributions, and travel time between the types of preschools in Table 8. Ideally, we would like to compare parents’ perceived quality differences between preschools to validate whether observed quality differences are also known to parents. However, data on perceived quality are limited: they are only available for enrolled children and about the type of preschool the child attends. Hence, the information provided in Table 8 should be interpreted with care. It shows that children in SPS are significantly older than children in CPS. They also are less likely to be from a poor household. While their caregivers at baseline are better educated, this difference is not statistically significant. 26 For example, for literacy skill activities, the lowest score (1) was assigned when teachers only used repetitive activities, close-ended questions, and/or choral responses. A higher score (2) was assigned when the teacher allowed children some choice in how to use materials or carry out an activity. A score of 3 was assigned if the teacher engaged children in a discussion and used open-ended questions. The highest score (4) was assigned when the teacher allowed children some choice in how to carry out an activity, engaged them in a discussion, and used open-ended questions. 24 Table 8 shows at both midline and endline that CPS classes get cancelled signif- icantly more often than SPS classes. Though differences tend to be relatively small, parents of children enrolled at CPS assess the kindness of CPS teachers as higher than that of their SPS counterparts. This could be explained by the greater likelihood that CPS teachers are from the same community as the parents than are SPS teachers. Yet at endline, parents of children enrolled at SPS assess the reliability of SPS teachers as higher than that of CPS teachers. While financial contributions for teacher salary, con- struction, and renovation tend to be almost zero for all types of schools, contributions for school materials are substantial.27 On average, parents of children enrolled at CPS have paid $35 in the current school year at midline and $47 at endline.28 Contributions are even higher at SPS, with $49 at midline and $68 at endline. Finally, (non)pecuniary transportation costs are also likely to be higher for SPS, which are often outside the village boundaries in our sample. Further, distance (measured as caregiver-reported travel time by foot) is about twice as much for SPS than for CPS. Hence, while SPS tend to be of higher quality than CPS for most dimensions of our classroom observa- tion survey, sending a child to an SPS might be associated with higher pecuniary and nonpecuniary costs than sending her to a CPS. 5 Empirical Framework Because villages were randomly assigned to either the control group or to one of the three treatment groups, we can straightforwardly estimate classical intent-to-treat im- pacts (ITT) of the intervention. Specifically, we estimate ITT effects by first pooling all treatment arms and then allowing for different impacts for each of the different treatment groups by using the specification: 27 The exact question asked is “How much money have you spent on school material (paper, board, chalk, cloth, water, food) for your child since the beginning of the school year?” Hence, it does not distinguish between money for food that is used for “pedagogical” reasons and money that is used to buy snacks for consumption. While we do not have data on this, we cannot rule out that teachers sell food as an indirect way of increasing their salaries. 28 At most CPS and IPS, the school year starts in late October or early November. About half of CPS and IPS end their school year at the end of July, while the other half end the school year at the end of August. Most SPS have their school year from late October or early November until the end of August. Since the endline data collection was conducted about one month later than the midline data collection, some caution is warranted with comparisons between the waves. 25 g Yip = α + βg · Zip + Xip δ + µp + uip , (1) where Yip is an outcome (e.g., test score) measured posttreatment for child i in g province p. Zip is a binary variable for any treatment (in the pooled treatment specifi- cation) or a vector with three binary indicators for treatment groups g ∈ {T 1, T 2, T 3} (in the disaggregated specification), with the control group as the reference category. Xip is a vector of control variables which, in the basic specification, includes the base- line test score, child age and child age squared. To improve statistical power and to control for potential imbalances, we also use a model with additional control variables including child gender, baseline height-for-age z-score, household size, household wealth quintile dummy variables, and caregiver baseline age, gender, score on the Raven’s Pro- gressive Matrices Test assessment, and parenting scores. (We note that the precision of the estimates improves only slightly using these additional control variables, and point estimates remain very stable). The specification includes µp , which is a set of province fixed effects to account for stratified randomization on the province level (Bruhn and McKenzie, 2009). uip is an individual error term where within-village correlation will be taken into account for estimation of the variance. The ITT’s are given by βg . For example, βT 1 gives the average effect of CPS construction on children in T1 villages. It is important to note that the interpretation of the ITT is difficult for a variety of reasons, all of which are very salient in this context. ITT is the average effect on all children in the treatment group, which includes children who don’t react to the inter- vention. It also includes the effect on children who attend another preschool prior to the intervention and who decide to switch into a CPS after its establishment. Further, incomplete (or delayed) CPS construction in some of the treatment villages will likely reduce the size of the ITT estimate. 6 Results 6.1 CPS construction Before discussing the impact of the intervention on outcomes, Table 8 presents an overview of the CPS construction status and overall availability of preschools at the time of the midline and endline surveys. Importantly for interpreting our results, 81% of 26 control group villages have some sort of preschool at midline; by endline, this is 84.5%. Nevertheless and unsurprisingly, the number is significantly higher in the treatment group villages, where availability of any preschool is already at 93.3%, 100%, and 98.4% for T1, T2, and T3, respectively, by midline. The patterns over time shown in Table 8 reflect how the construction of new CPS has affected school infrastructure in the villages. Most formal CPS have replaced preexisting IPS arrangements. Some IPS nevertheless remain open in treatment group villages, either due to failure to implement a formal CPS or because they are run independently by a local pagoda or an NGO. Notably, adherence to the CPS construction allocation was high: there were no formal CPS in control villages at midline and only one at endline. As mentioned above, Table 8 makes clear that many of the villages in the sample had an SPS, despite the fact that the program intended to focus on villages without one. Moreover, the table documents the imbalance in SPS availability across treatment arms: T2 villages are statistically significantly (about 13 percentage points) less likely to have SPS. While this should not have occurred under randomization in a sufficiently large sample, nothing about the selection process would have led to such a difference. This means that in the analysis of enrollment effects below, we will not be able to differentiate between a decrease in SPS enrollment due to substitution between SPS and CPS versus an imbalance in the availability of SPS driving results. Another implication is that when analyzing impacts on child development outcomes, the interpretation of results for T2 villages needs to take into account this somewhat different profile with regards to SPS availability in these villages. 6.2 Enrollment outcomes We first analyze child enrollment into preschool—a key government objective for the program. As mentioned above, follow-up data collection took place from April–June 2017 (midline) and May–July 2018 (endline); the school year for all types of preschools begins in late October or early November. At midline, 78.5% of enrolled children were enrolled in 2016 or earlier. At endline, 98.1% of enrolled children were enrolled before 2017 or earlier. Hence, data collection took place five to eight months after the start of the ongoing school year. Table 9 summarizes the impact on enrollment into any type of preschool, into each type of preschool, and into primary school, by treatment status. The measure of enrollment in this table is the child’s status on the day of the midline and endline survey 27 visits. At midline, the interventions increased enrollment into any type of preschool by about 10 percentage points (10.6, 9.5, and 11.9 in T1, T2, and T3, respectively), compared to a control group enrollment rate of 43.5%. In other words, the intervention increased preschool enrollment by about 25%. The impact on enrollment into CPS is considerably higher.29 As shown in column 2 of Table 9, the impact on enrollment in a CPS is 39%–43% at midline and 30%–37% at endline in treatment villages. Differences in impacts across treatment types are not statistically significant, with the exception of T2 villages, which have slightly higher CPS enrollment than T1 and T3 at endline. The table shows that this increase in CPS enrollment is fueled in large part by substitution across various types of preschools. There are two main potential channels: replacement of IPS—which was by design—and crowding-out of enrollment in other preschools, SPS in particular, which was unexpected.30 In most cases, newly established CPS arrangements replaced preexisting IPS arrangements. Therefore, for a majority of affected children, the intervention comes in the form of preschool improvement rather than a change in preschool availability. In addition to enrollment on the day of the survey visit, we analyze two other indicators of enrollment: ever enrolled in preschool by the endline survey, and duration of enrollment (measured at both midline and endline). Table 10 reports results for the program’s impact on ever having been enrolled in preschool. These are consistent with those for the day of the visit reported in Table 9, although the percentage increase in enrollment is smaller (a roughly seven percentage point increase over a control group mean of about 70%, so an increase of about 10%). The cross-preschool-type patterns suggest that much of this increase comes from a reduction in the probability that children are ever enrolled in an IPS or an SPS. As shown in Table 11, the intervention also significantly affected the months enrolled at preschool. By midline, the intervention had increased the average months enrolled at preschool, by 0.8–1.4 months from a counterfactual 3.4 months in the control group. This magnitude of this overall treatment effect is similar at endline, although by endline 29 While we would need to include province fixed effects into this regression for correct inference, we refrain from doing so to ensure that the constant is equivalent to the mean of the control group. No conclusions change when province fixed effects are added to the regression. 30 An earlier evaluation of GPE I (Bouguen et al., 2018) showed that enrollment into newly es- tablished SPS was in large part driven by substitution from underage enrollment in primary school. The results here do not suggest that there is substitution between CPS and primary school. This is despite the fact that at endline, about a third of the children were at least six years old and eligible for primary school. 28 the counterfactual months of enrollment were 5.4 in the control group. The impact on duration of CPS enrollment does grow from one survey to the next (from 3.4–3.9 months at midline, to 4.1–5.1 months at endline). This suggests CPS might have been better able to retain students.31 6.3 Child development outcomes Table 12 summarizes the impact of the program on child development. In the short term (by midline), the program improved child cognitive development by 0.04 sd on average. Effects are slighly larger for T1 (0.048 sd) and T3 (0.045 sd), and small and insignificant for T2 (0.009 sd). While generally statistically significant, these impacts are nevertheless small in magnitude. The program also significantly reduced the occurrence of socioemotional problems in all three treatment groups at midline. But at endline, all these impacts are both smaller in magnitude and not statistically different from zero. Point estimates for the cognitive development index are approximately reduced by half in T1 and T3 and have completely vanished for the socioemotional problems measure. Table 13 presents cognitive scores individually by individual domains, and adds in the impacts effects on gross- and fine-motor development. Impacts are most pro- nounced for early numeracy, followed by language development; they are insignificant for executive function. In the pooled model, only the impacts on early numeracy are statistically significantly different from zero. At endline, language development was sig- nificantly positively affected (T1) while fine motor development and executive function were negatively affected (T2). In the pooled model, none of the impacts is statistically significant, and all are close to zero. To further investigate which set of skills drives the impacts, we look at ITT effects on the individual test scores in Table 14. While estimates vary between treatment groups and waves, impacts on verbal counting skills and number and letter knowledge are above 0.03 sd for the joint treatment group (any T) estimate for both follow-ups. The finding is striking since these tests are directly linked to the curriculum; that is, it emphasizes the development of these skills. Similarly, the initial letter identification test is curriculum based and positively affected (+0.05 sd) at the endline. Yet, other 31 We show how enrollment patterns depend on child age in Figures A.4-A.6. In all three treatment groups, the probability of CPS enrollment is increasing with child age until children begin to enroll into primary school at the beginning of age six. 29 curriculum-based tests, such as the name writing, reading words, and fine motor tests do not show these consistent impacts. In a last step, we look at heterogeneous impacts by running ITT regressions by subgroups, presented in Table 15. While the differences by age are not extremely large, it is striking that children at age five—that is, baseline age four at midline and baseline age three at endline—are most affected in their cognitive development. The same findings hold for impacts on socioemotional problems at midline but not at endline. Furthermore, midline effects are primarily driven by nonstunted children, suggesting that CPS fail to foster development of the most-disadvantaged children and likely increase socioeconomic gaps in cognitive development. This finding is also confirmed by the impacts on cognitive development by wealth quartile, which show that only the richest quarter benefits from CPS and that these preschools fail to foster development of the less-advantaged children. 6.4 Demand-side interventions Self-reported take-up of the demand-side interventions is likely to be less reliable than for preschool enrollment since these were often just a one-time interaction many months in the past. Nevertheless, we report in Table 16 the impacts on indicators for these “take-up” measures in order to explore whether these may partly drive the lack of differential impacts across T1, T2, and T3. We interpret ever having received a home visit to discuss child development, or ever having received an information leaflet, as indicators of D2D activities (T2 and T3). No treatment groups report any additional home visits as a result of the program. It is possible that home visits from the village head might be difficult to capture (and for respondents to recall) due to their infor- mal nature. This interpretation is also consistent with the fact that almost 70% of respondents in control villages report having received a home visit, so it is possible this measure is too blunt. On the other hand, respondents in T2 and T3 were more likely to report having received an information leaflet whereas those in T1 were not (as one would expect given the content of the different treatments). This impact is apparent at both midline and endline. While the result is somewhat reassuring (i.e., coverage is significantly different across treatment arms), the overall percentage of respondents who report receiving a leaflet is small in all groups. The highest take-up by endline is reported in T2 villages, where it only reaches 18%. Even allowing for poor respondent memory, and despite the field team’s effort to ensure that 100% of sample households 30 in T2 and T3 received a leaflet, this value is low and suggests the intensity of the intervention was weak. The cross-treatment-group patterns in the likelihood of having participated in an HBP (either “ever” or “more than once”) are likewise somewhat reassuring. The impact of having been assigned to T3 significantly increases these likelihoods. But the impact of being assigned to T1 is also statistically significant (although about half the magnitude). This is surprising since these villages had no formal HBP deployed as a part of the intervention. At the same time, and consistent with the discussion above, the overall take-up rates implied by these numbers are small. The highest participation rate reported for ever having participated in an HBP by endline is just below 30% (T3), again suggesting the intensity of the intervention was weak. 6.5 Perceived return to schooling The focus on demand-side interventions in this program was designed to affect caregiver perceptions about the value of preschool and of education overall. Table 17 summa- rizes these effects through two main sets of outcome measures: reported optimal age for starting preschool and reported return to education. The results are consistent with the intervention lowering the age that caregivers think is optimal for starting preschool. A significantly higher share of caregivers reported that the optimal age to be enrolled at preschool for the first time is three or four years old. But there is no significant difference across the different treatment groups, suggesting it is likely the CPS construction itself (and any activity or information dissemination surrounding it) that is driving the result—not the specific demand-side interventions. The magnitude of the impact is somewhat larger at midline than at endline, further suggesting that caregivers might change their views on this question as their children grow older or that there is fade-out in the impact of these perception impacts. Initial excitement associated with the CPS construction might fade or caregivers might lower their perceived value of preschool based on their experience with the new CPS. The last two columns of Table 17 show that the distribution of leaflets had virtually no impact on the perceived wage increment to schooling (with the exception of a small impact for T2 at midline). Reported average values for the annual increments associated with schooling are 6.8% for primary school and 8.5% for secondary school (at endline, virtually the same as at midline). These numbers are remarkably similar to those reported by Humphreys (2015) based on Cambodia’s Socio-Economic Survey 31 (CSES) for 2010. In that analysis, the return to a year of primary school was estimated to be 5.8% for men and 3.5% for women, and the corresponding rates for a year of schooling at secondary school were estimated to be 14.2% and 14.6%.32 The implicit values included on the flyer (based on our analysis of CSES 2009) were 8% and 12%.33 One hypothesis for the lack of impact, therefore, is that people had a reasonably accurate perception of the return to schooling, and new information didn’t affect their priors. 7 Cost-effectiveness analysis In conjunction with this analysis of program impacts, a detailed analysis of costs was also carried out (and presented in Appendix C). The analysis describes the cost struc- tures of CPS, SPS, and HBP, all of which were implemented under the Second Educa- tion Sector Support Project (SESSP) financed by GPE II. Cost-effectiveness analyses are often limited to programmatic costs, defined as front- line inputs required for direct delivery of the intervention. Our cost estimates are based on the ingredient methods which aims to cost every resource required to make an in- tervention happen. This includes support costs, such as management, administrative and overhead costs which support the intervention above the level of direct commu- nity level implementation. Therefore, our cost measures are not directly comparable to studies that base their cost-effectiveness analyses on programmatic costs only (e.g. Brinkman et al. (2017)). While we also cite programmatic costs for better comparabil- ity with other studies, we stress that including support costs is important, as they are critical to making any ECED intervention a reality. Programmatic cost estimates are only reported here to provide a quick snapshot of specific cost ingredients with which implementers are more likely to have discretionary control over. The cost data are based on the endline survey, SESSP financial reports, MoEYS M&E data, and other estimates from the MoEYS ECE department, and they consist of two types: (1) data related to SESSP programs only, and (2) data related to all programs found in the 13 provinces where this impact evaluation was conducted. To 32 The secondary school values reported here are averages over those in Humphreys (2015), which are estimated separately for lower and upper secondary. 33 These are implicit because they are based on cell means reported for people who have completed those years of schooling. Note that the rate for primary school is calculated as the difference between “completed” and “not completed” primary school, which we model as six versus three years of schooling in the calculation of the “return.” 32 resolve the mismatch of datasets, two different sets of estimates were prepared: one that focuses on SESSP programs and one that focuses on all programs in the sample provinces. The generation of two estimates aims to account for some of the errors that may be introduced by data from different populations. The two models can also highlight cost efficiencies at small and large scale. Given the uncertainty regarding much of the cost data, a range of assumptions was used for various cost ingredients to produce high-end, mid-range, and low-end estimates. The high-end estimates assume the more expensive assumption for each ingredient, and the low-end estimates assume the less expensive assumption for each ingredient. In this section, we use mid-range estimates of the SESSP model only and refer to Appendix C for province model and the high- and low-range estimates. The total annual costs for running 500 CPS, of which 250 were initially planned to be constructed within the study sample, were $3,136,743. The average annual costs per school were $6,273. Using the average class size of 24.5 from the endline survey, this implies annual costs of $256 per child, which compares to roughly 22% of the GDP per capita ($1,160 in 2015). The mid-range estimate of the same model estimates annual SPS costs at $12,602 per school and $426 per child. Also, under the alternative province model and the high- and low-end estimates, costs per child at SPS were 50%–126% higher than at CPS.34 Whether coverage and capacities of the ECED system in Cambodia are best ex- panded by CPS depends on their cost-effectiveness. While CPS costs can be linked with the impacts of CPS construction on child development, cost-effectiveness for SPS cannot be estimated under the current impact evaluation since it does not include es- timates of causal effects of SPS on child test scores. Yet, incremental cost-effectiveness ratios ICER = (C1 − C0 )/(E1 − E0 ) of CPS can also not be directly calculated without further assumptions since children who benefited from CPS have different counterfac- tual care scenarios. C0 consists of the costs of home care, IPS, and SPS. The total increase in CPS attendance (38.9%) was accompanied by a decrease in IPS attendance (-24.7%) and SPS attendance (-4.1%). Hence the share of children drawn from IPS (24.7/38.9 = 63.5%) and SPS (10.5%) can be derived under the assumption that empty IPS and SPS slots were not filled by other children (Berkes and Bouguen, 2018). To ob- tain an estimate of cost-effectiveness, we further assume that the costs of IPS are equal to the costs of CPS since they mostly differ in terms of their building, and construction 34 Mid-range estimates for annual programmatic costs (excluding support costs) per child are $173 and $321 for CPS and SPS, respectively. 33 costs only make up 3% of the overall costs. To obtain a treatment-on-the-treated (TOT) estimate for T1, we divide the ITT effect on the cognitive development index by the take-up rate: T OT = 0.051/0.131. In this case, the mid-range estimates of the SESSP model imply an incremental cost-effectiveness ratio of a 1 sd improvement one year after the start of the intervention equal to ICER = 256−(0.635∗ 256+0.105∗426) 0.131 = $371.8. 8 Conclusion These results might seem disappointing in the sense that they do not show a large rapid increase in net new enrollments into preschool or large average impacts on short-term (one-year) or medium-term (two-year) child cognitive and socioemotional outcomes. The findings highlight that, notwithstanding the robust evidence that high-quality preschool experiences boost child development,35 it is difficult to engender those ex- periences for all children in a program at scale in a low-income, capacity-constrained environment. Our findings have both programmatic and research implications. From a program point of view, they suggest it should be a priority to review CPS quality with a par- ticular focus on enhancing process quality (curriculum, pedagogy, and teacher-child interactions). Two places to start could be the training of CPS teachers, which is currently substantially shorter and less intensive than that of SPS teachers, and the adequacy of the CPS school day, which is currently limited to two hours. The goal should be to ensure that CPS offer services that are tailored to the varied developmental needs of children and therefore have high impact on child development. From a research point of view, the findings suggest further work is required to un- derstand the drivers of quality and preschool enrollment. The demand-side approaches tested in this evaluation were not enough to mobilize much additional enrollment over and above simply building CPS, suggesting other factors affect the decision to send children to preschool. As direct or indirect costs may play a role, approaches to re- duce those costs—for example, cash transfers or reduced travel distances—might be necessary to induce higher participation rates. In addition, if the quantity and quality of preschool services do not meet families’ needs, households might have low demand for them. Therefore, increasing the quantity (time spent in preschool per day) or the quality of preschools may increase demand. Further research is required to better 35 See discussion in World Bank (2018). 34 understand these factors. 35 References Alkire, S. and M. E. Santos (2010). Acute multidimensional poverty: A new index for developing countries. Technical report, Human Development Report Office (HDRO), United Nations Development Programme (UNDP). Andrew, A., O. Attanasio, E. Fitzsimons, S. Grantham-McGregor, C. Meghir, and M. Rubio-Codina (2018). Impacts 2 years after a scalable early childhood develop- ment intervention to increase psychosocial stimulation in the home: A follow-up of a cluster randomised controlled trial in colombia. PLoS medicine 15 (4), e1002556. Berkes, J. and A. Bouguen (2018). Heterogeneous preschool impact and close substi- tutes: Evidence from an early childcare program in cambodia. Working Paper . Berkes, J., A. Raikes, A. Bouguen, and D. Filmer (2019). Joint roles of parenting and nutritional status for child development: Evidence from rural cambodia. Develop- mental Science , e12874. Berlinski, S., S. Galiani, and P. Gertler (2009). The effect of pre-primary education on primary school performance. Journal of Public Economics 93 (1), 219–234. Berlinski, S., S. Galiani, and M. Manacorda (2008). Giving children a better start: Preschool attendance and school-age profiles. Journal of Public Economics 92 (5), 1416–1440. na, and M. Vera-Hern´ Bernal, R., O. Attanasio, X. Pe˜ andez (2019). The effects of the transition from home-based childcare to childcare centers on children’s health and development in colombia. Early Childhood Research Quarterly 47, 418–431. Blau, D. and J. Currie (2006). Pre-school, day care, and after-school care: who’s minding the kids? Handbook of the Economics of Education 2, 1163–1278. Blimpo, M. P., P. M. Amaro Da Costa Luz Carneiro, P. Jervis, and T. Pugatch (2019). Improving access and quality in early childhood development programs: Experimen- tal evidence from the gambia. Bouguen, A., D. Filmer, K. Macours, and S. Naudeau (2013). Impact evaluation of three types of early childhood development interventions in cambodia (english). Policy Research working paper IE 97 (WPS 6540). Bouguen, A., D. Filmer, K. Macours, and S. Naudeau (2018). Preschool and parental response in a second best world: Evidence from a school construction experiment. Journal of Human Resources 53 (2), 474–512. Brinkman, S. A., A. Hasan, H. Jung, A. Kinnell, and M. Pradhan (2017). The impact of expanding access to early childhood education services in rural indonesia. Journal of Labor Economics 35 (S1), S305–S335. 36 Bruhn, M. and D. McKenzie (2009). In pursuit of balance: Randomization in prac- tice in development field experiments. American Economic Journal: Applied Eco- nomics 1 (4), 200–232. Cornelissen, T., C. Dustmann, A. Raute, and U. Sch¨onberg (2018). Who benefits from universal child care? estimating marginal returns to early child care attendance. Journal of Political Economy 126 (6), 2356–2409. Cunha, F. and J. Heckman (2007). The technology of skill formation. The American Economic Review 97 (2), 31. Datta Gupta, N. and M. Simonsen (2016). Academic performance and type of early childhood care. Economics of Education Review 53, 217–229. Duncan, G. J. and K. Magnuson (2013). Investing in preschool programs. Journal of Economic Perspectives 27 (2), 109–132. Dunn, L. M. and D. M. Dunn (2007). PPVT-4: Peabody Picture vocabulary test. Pearson Assessments. Dunn, L. M., E. R. Padilla, D. E. Lugo, and L. M. Dunn (1986). TVIP: Test de Vo- agenes Peabody: Adaptaci´ cabulario en Im´ on Hispanoamericana. American Guidance Service. ıa, J. J. Heckman, and A. Hojman (2015). Early childhood Elango, S., J. L. Garc´ education. Technical report, National Bureau of Economic Research. Engle, P. L., L. C. Fernald, H. Alderman, J. Behrman, C. O’Gara, A. Yousafzai, M. C. de Mello, M. Hidrobo, N. Ulkuer, I. Ertem, et al. (2011). Strategies for reducing inequalities and improving developmental outcomes for young children in low-income and middle-income countries. The Lancet 378 (9799), 1339–1353. Havnes, T. and M. Mogstad (2011). No child left behind: Subsidized child care and children’s long-run outcomes. American Economic Journal: Economic Policy 3 (2), 97–129. Havnes, T. and M. Mogstad (2015). Is universal child care leveling the playing field? Journal of Public Economics 127, 100–114. Henderson, R. H. and T. Sundaresan (1982). Cluster sampling to assess immunization coverage: A review of experience with a simplified sampling method. Bulletin of the World Health Organization 60 (2), 253. Humphreys, J. (2015). Education premiums in cambodia: Dummy variables revisited and recent data. Econ Journal Watch 12 (3), 339–345. 37 Jensen, R. (2010). The (perceived) returns to education and the demand for schooling. The Quarterly Journal of Economics 125 (2), 515–548. Johnson, R. C. and C. K. Jackson (2018). Reducing inequality through dynamic com- plementarity: Evidence from head start and public school spending. NBER Working Paper Series 23489, National Bureau of Economic Research. Kline, P. and C. R. Walters (2016). Evaluating public programs with close substitutes: The case of head start. The Quarterly Journal of Economics 131 (4), 1795–1848. Martinez, S., S. Naudeau, and V. Pereira (2017). Preschool and child development under extreme poverty: evidence from a randomized experiment in rural mozambique. The World Bank. MoEYS (2014). Education strategic plan 2014-2018. Technical report, Kingdom of Cambodia, Ministry of Education, Youth and Sport. MoEYS (2017). The education, youth and sport performance in the academic year 2015-2016 and goals for the academic year 2016-2017. Technical report, Kingdom of Cambodia, Ministry of Education, Youth and Sport. Nguyen, T. (2008). Information, role models and perceived returns to education: Ex- perimental evidence from madagascar. Unpublished manuscript 6. Ozler, B., L. C. Fernald, P. Kariger, C. McConnell, M. Neuman, and E. Fraga (2018). Combining pre-school teacher training with parenting education: A cluster- randomized controlled trial. Journal of Development Economics 133, 448 – 467. Raikes, A., N. Koziol, M. Janus, L. Platas, T. Weatherholt, A. Smeby, and R. Sayre (2019). Examination of school readiness constructs in tanzania: Psychometric evalu- ation of the melqo scales. Journal of Applied Developmental Psychology 62, 122–134. Rao, N., J. Sun, J. M. Wong, B. Weekes, P. Ip, S. Shaeffer, M. Young, M. Bray, E. Chen, and D. Lee (2014). Early childhood development and cognitive development in developing countries. DFID, UK Government. UNESCO, UNICEF, B. I. W. B. (2017). Overview melqo: Measuring early learning quality outcomes. World Bank (2018). World Development Report 2018: Learning to realize education’s promise. The World Bank. Zelazo, P. D. (2006). The dimensional change card sort (dccs): A method of assessing executive function in children. Nature Protocols 1 (1), 297. 38 Figure 1: Location of treatment and control group villages 39 Table 1: Random treatment allocation to 305 villages Group CPS D2D HBP Villages T1: CPS 120 T2: CPS+D2D 64 T3: CPS+D2D+HBP 63 Control 58 Table 2: Timetable Period Activity CPS construction 03/2016 Begin CPS construction 0% completed 05/2016 – 07/2016 Baseline data collection 17% completed 12/2016 Monitoring survey (by phone) 82% completed 04/2017 – 06/2017 Follow-up data collection 91% completed 05/2018 – 07/2018 Endline data collection 91% completed Note: Percentages refer to share of villages in the three treatment groups for which construction of a new CPS was reported as com- pleted on the day of data collection. 40 Table 3: Balance in baseline covariates Mean: Difference in means: N C T1-C T2-C T3-C F-test (p-val) Child characteristics Cognitive development index 7491 0.030 -0.001 0.006 -0.006 0.993 Executive function score 7491 0.000 0.036 0.020 0.013 0.782 Language score 7491 0.000 -0.010 0.001 -0.026 0.887 Mathematics score 7491 -0.000 -0.033 -0.020 -0.014 0.829 Fine motor score 7491 0.000 0.052 0.012 0.027 0.569 Gross motor score 7491 -0.000 0.031 0.009 0.002 0.735 SDQ: overall problems 7611 -0.003 0.027 0.042 0.002 0.724 Age (yrs) w. decimals 7632 3.476 0.008 0.020 -0.039 0.133 Female 7632 0.506 -0.020 -0.030* -0.004 0.208 Length/height-for-age z-score 7473 -1.607 -0.031 -0.021 -0.062 0.638 Household characteristics Household size 7632 5.966 -0.030 0.064 -0.004 0.869 Multidim. poor 7632 0.437 0.002 -0.012 -0.037 0.370 Monthly income >100 USD 7632 0.464 -0.010 -0.068* -0.034 0.298 Farming activity 7630 0.822 0.002 0.000 0.023 0.728 Caregiver characteristics Caregiver female 7629 0.890 0.024** 0.029** 0.017 0.123 Caregiver age 7629 40.777 0.415 -1.188 0.173 0.091 Caregiver biological parent 7542 0.596 -0.024 0.028 -0.003 0.122 Caregiver years of education 7621 4.681 -0.286 -0.264 0.292 0.498 Caregiver Raven’s score 7558 0.050 -0.070 0.011 -0.006 0.115 Cognitive parenting 7611 -0.006 0.018 -0.012 0.073* 0.222 Socioemotional parenting 7612 -0.008 -0.021 -0.112** -0.022 0.102 Negative parenting 7613 0.002 0.052 0.092* 0.013 0.159 Baseline program attendance Attending preschool 7612 0.153 0.073*** 0.063** 0.108*** 0.001 Attending nonstate preschool 7612 0.123 0.068*** 0.057** 0.101*** 0.001 Attending state preschool 7612 0.030 0.005 0.006 0.008 0.885 Careg. participated in any HBP 7609 0.104 0.031* 0.055*** 0.184*** 0.000 Caregiver received any D2D 7609 0.017 0.003 0.012* 0.021*** 0.009 Differences in means are based on OLS regressions at the child level on treatment group dummies and province fixed-effects using robust standard errors clustered at the village level. F-test reports p-value of F-test for joint significance of treatment group dummies. * 10%, ** 5%, *** 1 % significance level. 41 Table 4: Attrition of eligible children at midline and endline Midline Endline (1) (2) (3) (1) (2) (3) T1 0.019 -0.062 0.021* -0.003 0.067 -0.001 T2 0.008 0.058 0.009 0.008 0.091 0.009 T3 -0.008 0.001 -0.006 -0.002 0.050 0.000 Cognitive development index 0.006 -0.010 Age -0.015 0.008 Height-for-age z-score -0.005 -0.009 Multidim. poor 0.002 0.016 Caregiver years of education 0.001 -0.001 T1 * Cognitive development index -0.015 0.006 T1 * Age 0.014 -0.020 T1 * Height-for-age z-score -0.007 0.006 T1 * Multidim. poor 0.028 0.001 T1 * Caregiver years of education 0.001 0.001 T2 * Cognitive development index -0.002 -0.006 T2 * Age -0.007 -0.007 T2 * Height-for-age z-score 0.009 0.041*** T2 * Multidim. poor -0.011 0.019 T2 * Caregiver years of education -0.002 -0.001 T3 * Cognitive development index 0.001 0.008 T3 * Age 0.001 -0.015 T3 * Height-for-age z-score 0.004 0.005 T3 * Multidim. poor -0.004 0.003 T3 * Caregiver years of education -0.001 0.001 Control group mean attrition 0.102 0.0956 Joint F-test (p-values): Baseline controls (without interaction) 0.847 0.190 T1 interactions with baseline controls 0.525 0.423 T2 interactions with baseline controls 0.676 0.0434 T3 interactions with baseline controls 0.986 0.912 Additional sampling at midline No No Yes No No Yes Observations 7632 7632 7693 7632 7632 7693 Table shows OLS regressions with midline and endline attrition as dependent variable using all children eligible for testing at baseline (columns 1 and 2) plus children that were added to the sample at midline (column 3). All regressions also control for province fixed effects. Robust standard errors are clustered on the village level. Missing baseline control variables are replaced by the control group mean. * 10%, ** 5%, *** 1 % significance level. 42 Table 5: Teacher characteristics and preschool equipment (1) (2) (3) (4) (5) CPS IPS SPS IPS-CPS SPS-CPS Teacher characteristics Age 40.28 40.78 34.05 0.49 -6.24*** Female 0.93 0.95 0.84 0.03 -0.08* Completed primary school (6) 0.81 0.75 0.98 -0.06 0.17*** Completed lower secondary school (9) 0.45 0.37 0.86 -0.08 0.41*** Completed upper secondary school (12) 0.13 0.13 0.68 -0.00 0.55*** No teacher training 0.11 0.16 0.29 0.05 0.17*** 1–8 weeks of teacher training 0.84 0.78 0.30 -0.06 -0.54*** More than 8 weeks of teacher training 0.05 0.06 0.41 0.01 0.36*** Had practical teacher training 0.21 0.22 0.41 0.01 0.20*** Trained as prim./sec. school teacher 0.02 0.05 0.44 0.02 0.42*** Years since first teaching experience 5.95 6.35 5.97 0.40 0.02 Nonverbal reasoning test (Raven’s) -0.10 -0.07 0.41 0.03 0.51*** Salary for teaching position (USD) 60.40 65.11 250.22 4.71 189.82*** Teacher fully paid, regularly 0.75 0.81 0.95 0.06 0.20*** Teacher fully paid, irregularly 0.22 0.19 0.03 -0.03 -0.19*** Equipment Table and chair for teacher 0.99 0.33 0.87 -0.66*** -0.12*** Storage for teacher 0.97 0.19 0.52 -0.78*** -0.44*** Tables and chairs for children 0.95 0.32 0.67 -0.64*** -0.29*** Children’s tables and chairs appropriately sized 0.78 0.29 0.34 -0.48*** -0.44*** Blackboard/whiteboard and markers/chalks 0.95 0.84 0.97 -0.11** 0.02 Electricity access 0.07 0.29 0.24 0.21*** 0.16*** Field, playground, or school yard 0.60 0.60 0.83 0.00 0.23*** Equipment for gross-motor activities on school yard 0.37 0.32 0.30 -0.05 -0.07 First aid kit 0.31 0.11 0.33 -0.20*** 0.03 Functional water source 0.47 0.67 0.81 0.20*** 0.34*** Functional drinking water source 0.61 0.54 0.67 -0.07 0.06 Hand washing facility 0.53 0.37 0.57 -0.16** 0.05 Toilet facility 0.27 0.44 0.90 0.18** 0.64*** Writing utensils 0.85 0.68 0.89 -0.17*** 0.04 Writing utensils used by children 0.55 0.54 0.79 -0.02 0.24*** Art materials 0.82 0.61 0.81 -0.21*** -0.01 Art materials used by children 0.50 0.45 0.70 -0.05 0.20*** Fantasy play materials 0.59 0.25 0.25 -0.34*** -0.34*** Fantasy play materials used by children 0.36 0.16 0.08 -0.20*** -0.28*** Educational toys/math materials 0.69 0.38 0.67 -0.32*** -0.03 Educational toys/math materials used by children 0.39 0.22 0.46 -0.17*** 0.07 Observations 215 63 63 Columns 1–3 show averages by type of preschool. Columns 4-5 show differences between types of preschools. * 10%, ** 5%, *** 1 % significance level. 43 Table 6: Classroom setting and teaching practices (1) (2) (3) (4) (5) CPS IPS SPS IPS-CPS SPS-CPS Classroom setting Length of class (minutes) 113.78 137.53 172.49 23.74 58.71*** Total length of breaks (minutes) 39.07 18.01 44.22 -21.05* 5.16 Number of children enrolled in this class 25.09 22.56 27.57 -2.53** 2.48** Children present 17.66 14.98 21.21 -2.68** 3.54*** Num. of teachers in classroom 1.01 0.98 0.98 -0.03 -0.03 Num. of assistants in classroom 0.03 0.03 0.00 0.00 -0.03** Num. of other adults in classroom 1.03 0.73 0.31 -0.30 -0.72*** Teacher follows curriculum to teach class 0.61 0.39 0.49 -0.22*** -0.12* Teacher documents children’s development regularly 0.34 0.16 0.37 -0.18*** 0.02 Teacher documents children’s attendance 0.79 0.58 0.86 -0.21*** 0.07 Curriculum content and pedagogy Activities supporting development of maths skills 0.74 0.68 0.71 -0.06 -0.02 Quality of maths activities [1–4] 2.68 2.47 2.89 -0.21 0.21 Activities supporting development of literacy skills 0.66 0.65 0.89 -0.01 0.23*** Quality of literacy activities [1–4] 2.66 2.47 3.39 -0.19 0.74*** Activities supporting development of expressive language skills 0.80 0.68 0.78 -0.12* -0.02 Quality of expressive language activities [1–3] 2.37 2.28 2.67 -0.09 0.31** Activity: reading of storybook 0.49 0.41 0.46 -0.08 -0.03 Quality of storybook activities [1–6] 3.54 2.93 3.86 -0.62* 0.31 Activities supporting development of general knowledge 0.76 0.77 0.87 0.00 0.11** Quality of teaching during general knowledge activities [1–6] 3.78 3.89 4.00 0.10 0.22 Activities supporting development of fine motor skills 0.35 0.35 0.57 -0.01 0.22*** Quality of teaching during fine motor skills activities [1–3] 2.04 2.09 2.31 0.05 0.27** Activities supporting development of gross motor skills 0.60 0.49 0.75 -0.11 0.14** Quality of gross motor skills activities [1–3] 1.63 1.56 1.85 -0.07 0.22 Time of gross motor skills activities (minutes) 7.87 8.75 10.42 0.88 2.55** Quality of the teacher’s use of theme [0(no theme used)–4] 2.15 2.26 2.23 0.11 0.08 Teacher-child interactions The teacher enjoyed teaching [1–3] 2.63 2.56 2.65 -0.07 0.02 The teacher showed negative attitudes [0–2.5] 0.09 0.08 0.11 -0.01 0.03 Quality of the disciplinary strategies used by the teacher [0–4] 2.71 2.49 3.23 -0.22 0.52*** Occurences of negative interactions 5.95 6.66 6.15 0.72 0.20 Occurences of encouragements 9.48 8.70 15.34 -0.78 5.86*** Time children wait without any specific activity (minutes) 1.80 1.42 2.18 -0.38 0.38 The teacher correct student’s work and give feedbacks [0–3] 1.99 2.14 2.45 0.16 0.46*** Time children are left without supervision (minutes) 0.72 0.92 1.81 0.20 1.09* Quality of the engagement of children [0–4] 2.70 2.90 3.40 0.20 0.70*** Teacher’s awareness of children’s individual needs [0–3] 1.56 1.47 1.59 -0.09 0.03 Teacher’s behavior with respect to gender equality [0–4] 3.32 3.24 3.44 -0.08 0.12 Number of times the class is interrupted 2.23 2.45 1.81 0.22 -0.42 Presence of disturbing noise [1–3] 1.44 1.56 1.38 0.12 -0.07 Observations 215 63 63 Columns 1–3 show averages by type of preschool. Columns 4–5 show differences between types of preschools. * 10%, ** 5%, *** 1 % significance level. 44 Table 7: Missing days, perceived quality and financial contributions by households (1) (2) (3) (4) (5) CPS IPS SPS IPS-CPS SPS-CPS Midline Cognitive development index 0.25 0.45 0.81 0.20*** 0.56*** Socioemotional problems -0.13 -0.16 -0.26 -0.03 -0.13** Age (yrs) 4.59 4.74 5.01 0.15*** 0.42*** Female 0.52 0.49 0.52 -0.03 -0.00 Caregiver years of education (baseline) 4.60 4.34 5.90 -0.25 1.31 Multidim. poor (baseline) 0.38 0.42 0.32 0.04 -0.06* Regular school days without class taking place (last 30 days) 2.48 1.51 0.92 -0.97*** -1.56*** Days missed due to personal reasons (last 30 days) 4.32 3.51 3.42 -0.81** -0.90*** Perceived kindness of teacher (1–10) 8.74 8.25 8.57 -0.48*** -0.16* Perceived professional knowledge of teacher (1–10) 8.62 8.09 8.56 -0.52*** -0.06 Perceived reliability of teacher (1–10) 8.38 8.02 8.43 -0.36** 0.05 Contribution to teacher salary in current school year (USD) 0.14 0.42 0.16 0.28** 0.02 Contribution to school material in current school year (USD) 35.15 31.70 49.22 -3.44 14.08*** Contribution to construction/renovation since child attends (USD) 0.67 0.67 0.64 -0.00 -0.03 School is within village boundaries 1.00 0.79 0.60 -0.21*** -0.40*** Travel time to school (by foot) 15.78 18.86 29.94 3.08** 14.16*** Observations 2350 568 460 Endline Cognitive development index -0.27 -0.07 0.11 0.20*** 0.38*** Socioemotional problems 0.04 0.03 -0.03 -0.00 -0.07 Age (yrs) 5.19 5.37 5.62 0.18*** 0.43*** Female 0.52 0.48 0.49 -0.03 -0.03 Caregiver years of education (baseline) 4.64 5.50 4.80 0.86 0.16 Multidim. poor (baseline) 0.43 0.42 0.37 -0.01 -0.06** Regular school days without class taking place (last 30 days) 2.13 1.78 1.31 -0.35 -0.82*** Days missed due to personal reasons (last 30 days) 3.47 2.82 3.15 -0.65** -0.31 Perceived kindness of teacher (1–10) 8.87 8.71 8.69 -0.16* -0.18** Perceived professional knowledge of teacher (1–10) 8.72 8.57 8.80 -0.15* 0.07 Perceived reliability of teacher (1–10) 8.65 8.64 8.84 -0.01 0.19** Contribution to teacher salary in current school year (USD) 0.18 0.96 0.12 0.78*** -0.06 Contribution to school material in current school year (USD) 46.74 53.32 68.29 6.58** 21.55*** Contribution to construction/renovation since child attends (USD) 0.74 0.77 0.87 0.03 0.13 School is within village boundaries 0.99 0.82 0.54 -0.18*** -0.45*** Travel time to school (by foot) 15.20 21.57 31.32 6.37*** 16.12*** Observations 1899 645 649 Contributions and travel time are trimmed at the 99th percentile to control for outliers. Columns 1–3 show averages by type of preschool. Columns 4–5 show differences between types of preschools. * 10%, ** 5%, *** 1 % significance level. 45 Table 8: Preschool infrastructure (1) (2) (3) (4) (5) Obs. C T1-C T2-C T3-C Baseline Any preschool in village 305 0.759 0.085 0.003 0.008 Any Community Preschool in village (CPS) 305 -0.000 0.203*** 0.143*** 0.167*** Informal Preschool in village (IPS) 305 0.603 -0.010 -0.032 -0.103 State Preschool in village (SPS) 305 0.224 -0.049 -0.113* -0.052 Midline Any preschool in village 305 0.810 0.123** 0.190*** 0.174*** Any Community Preschool in village (CPS) 305 -0.000 0.858*** 0.984*** 0.938*** Informal Preschool in village (IPS) 305 0.655 -0.564*** -0.623*** -0.546*** State Preschool in village (SPS) 305 0.241 -0.058 -0.130* -0.070 Endline Any preschool in village 305 0.845 0.097* 0.155*** 0.140*** Any Community Preschool in village (CPS) 305 0.017 0.841*** 0.983*** 0.905*** Informal Preschool in village (IPS) 305 0.724 -0.607*** -0.677*** -0.599*** State Preschool in village (SPS) 305 0.259 -0.050 -0.132* -0.040 Table shows preschools available in sample villages. C is the control group mean and constant in a regression of preschool availability on a set of dummy variables for each treatment group. No control variables included in regression model. Estimates correct for heteroskedasticity. * 10%, ** 5%, *** 1 % significance. level 46 Table 9: Enrollment on day of survey by type of school (1) (2) (3) (4) (5) Any preschool CPS IPS SPS Primary Midline (age 3–5) Group: T1 0.102*** 0.389*** -0.247*** -0.041* 0.004 Group: T2 0.094** 0.432*** -0.263*** -0.075*** 0.001 Group: T3 0.128*** 0.426*** -0.232*** -0.066*** -0.010 Constant 0.396 -0.000 0.284 0.112 0.040 p-value: T1=T2 0.832 0.241 0.0879 0.0206 0.699 p-value: T1=T3 0.427 0.340 0.425 0.0991 0.112 p-value: T2=T3 0.377 0.884 0.0736 0.475 0.288 Observations 6992 6992 6992 6992 6992 Endline (age 4–6) Group: T1 0.062* 0.297*** -0.194*** -0.041 0.004 Group: T2 0.079** 0.373*** -0.228*** -0.066** -0.024 Group: T3 0.057 0.307*** -0.191*** -0.058** 0.002 Constant 0.402 0.009 0.258 0.135 0.257 p-value: T1=T2 0.598 0.0226 0.0153 0.201 0.154 p-value: T1=T3 0.864 0.783 0.888 0.364 0.952 p-value: T2=T3 0.533 0.0796 0.0526 0.712 0.223 Observations 7015 7015 7015 7015 7015 Constant shows the control group mean. No control variables included in regression model. Estimates correct for heteroskedasticity and within-village correlations. * 10%, ** 5%, *** 1 % significance level. 47 Table 10: Ever enrolled at type of school (1) (2) (3) (4) (5) Any preschool CPS IPS SPS Primary Endline* (age 4–6) Group: T1 0.061** 0.478*** -0.301*** -0.057 0.004 Group: T2 0.064** 0.595*** -0.357*** -0.106*** -0.023 Group: T3 0.079** 0.528*** -0.286*** -0.088** 0.002 Constant 0.718 0.017 0.413 0.200 0.262 p-value: T1=T2 0.903 0.00324 0.00118 0.0490 0.160 p-value: T1=T3 0.448 0.266 0.574 0.230 0.896 p-value: T2=T3 0.583 0.141 0.00450 0.457 0.253 Observations 7015 7015 7015 7015 7015 Constant shows the control group mean. No control variables included in regression model. Estimates correct for heteroskedasticity and within-village correlations. *Past enrolment spells not available for midline survey. * 10%, ** 5%, *** 1 % significance level. 48 Table 11: Total months enrolled by type of school (1) (2) (3) (4) (5) Any preschool CPS IPS SPS Primary Midline (age 3–5) Group: T1 0.907** 3.421*** -2.088*** -0.426* 0.104 Group: T2 0.797* 3.651*** -2.250*** -0.604*** 0.071 Group: T3 1.386*** 3.872*** -1.945*** -0.541** -0.006 Constant 3.405 0.000 2.411 0.994 0.267 p-value: T1=T2 0.787 0.556 0.0506 0.189 0.693 p-value: T1=T3 0.223 0.270 0.379 0.398 0.156 p-value: T2=T3 0.188 0.613 0.0546 0.652 0.355 Observations 6985 6985 6985 6985 6985 Endline (age 4–6) Group: T1 0.839* 4.131*** -2.879*** -0.413 0.078 Group: T2 0.934* 5.088*** -3.342*** -0.813*** -0.181 Group: T3 1.165** 4.533*** -2.714*** -0.653** 0.050 Constant 5.370 0.102 3.774 1.494 2.053 p-value: T1=T2 0.817 0.0155 0.00135 0.0435 0.134 p-value: T1=T3 0.393 0.362 0.467 0.260 0.873 p-value: T2=T3 0.600 0.224 0.00271 0.414 0.227 Observations 7015 7015 7015 7015 7015 Constant shows the control group mean. No control variables included in regression model. Estimates correct for heteroskedasticity and within-village correlations. *Past enrolment spells not available for midline survey. Therefore, midline outcomes only measure total months of enrollment at the school where child is enrolled at time of midline survey. Endline outcomes measure total months of enrollment at current school and all previous schools. * 10%, ** 5%, *** 1 % significance level. 49 Table 12: Impact of the program on child development outcomes (intention-to-treat impacts) Midline (age 3–5) Endline (age 4–6) Cognitive develop- Socioemotional Cognitive develop- Socioemotional ment index problems ment index problems T1 0.044* 0.048** -0.062 -0.068* 0.023 0.031 0.024 0.013 (0.023) (0.020) (0.038) (0.038) (0.027) (0.024) (0.038) (0.037) T2 -0.000 0.009 -0.097** -0.103** -0.026 -0.015 -0.001 -0.011 (0.027) (0.024) (0.042) (0.042) (0.033) (0.029) (0.043) (0.042) T3 0.050* 0.045** -0.128*** -0.116*** 0.032 0.023 -0.016 -0.005 (0.025) (0.022) (0.045) (0.045) (0.034) (0.031) (0.044) (0.043) Any treatment 0.034* 0.037** -0.088** -0.090** 0.013 0.017 0.007 0.002 (0.020) (0.018) (0.035) (0.035) (0.024) (0.021) (0.035) (0.035) Additional controls N Y N Y N Y N Y 50 Observations 6917 6917 6990 6990 6966 6966 7014 7014 All regressions control for baseline value of dependent variable, child age, child age squared, and province fixed effects. Additional control variables include gender, baseline height-for-age z-score, household size, household wealth quintile dummy variables, and baseline caregiver age, gender, Raven’s score, and parenting scores. Missing baseline covariates are replaced by the sample mean and interacted with a missing covariate dummy. Standard errors clustered on village level. * 10%, ** 5%, *** 1 % significance level. Table 13: Impact of the program on subdomains of child development (intention-to-treat impacts) Midline (age 3–5) Endline (age 4–6) Executive Language Early nu- Fine mo- Gross Executive Language Early nu- Fine mo- function meracy tor motor function meracy tor T1 0.042 0.048* 0.061** 0.070** 0.014 -0.008 0.051* 0.007 -0.018 (0.026) (0.028) (0.029) (0.032) (0.032) (0.025) (0.028) (0.026) (0.031) T2 0.004 0.025 0.014 -0.002 -0.009 -0.046 0.006 -0.013 -0.079** (0.028) (0.036) (0.033) (0.038) (0.037) (0.029) (0.036) (0.030) (0.033) T3 0.038 0.039 0.061* 0.052 0.023 0.015 0.026 0.015 -0.053 (0.027) (0.030) (0.032) (0.034) (0.034) (0.030) (0.035) (0.032) (0.034) Any treatment 0.031 0.040 0.049* 0.046 0.010 -0.012 0.033 0.004 -0.043 (0.023) (0.026) (0.025) (0.030) (0.029) (0.021) (0.026) (0.023) (0.027) Additional controls Y Y Y Y Y Y Y Y Y 51 Observations 6917 6917 6917 6917 6917 6966 6966 6966 6966 All regressions control for baseline value of dependent variable, child age, child age squared, and province fixed effects. Additional control variables include gender, baseline height-for-age z-score, household size, household wealth quintile dummy variables, and baseline caregiver age, gender, Raven’s score, and parenting scores. Missing baseline covariates are replaced by the sample mean and interacted with a missing covariate dummy. Standard errors clustered on village level. * 10%, ** 5%, *** 1 % significance level. Table 14: ITT effects on individual tests Midline (age 3–5) Endline (age 4–6) Any T T1 T2 T3 Any T T1 T2 T3 Executive function Head-knee task 0.004 0.009 -0.028 0.027 0.027 0.010 0.007 0.078** Forward digit-span test 0.029 0.043 0.009 0.024 0.027 0.031 0.009 0.036 DCCS 0.045 0.049 0.043 0.040 -0.018 0.000 -0.030 -0.039 Cancellation task 0.060** 0.075*** 0.040 0.053* -0.033 -0.023 -0.081** -0.005 Language TVIP 0.018 0.023 0.014 0.015 0.009 0.012 0.001 0.011 Naming items 0.027 0.028 -0.009 0.061 0.008 0.017 -0.009 0.007 Short-story 0.033 0.043* 0.017 0.032 0.024 0.022 0.038 0.016 Reading concepts 0.025 0.047 -0.008 0.017 0.049* 0.065** 0.064* 0.004 Letter knowledge 0.050 0.041 0.101 0.015 0.028 0.053 -0.018 0.027 Name writing 0.019 0.031 -0.012 0.025 Initial letter identification 0.047 0.062* 0.028 0.039 Reading words 0.009 0.037 -0.033 -0.001 Early numeracy Measurement concepts -0.014 -0.011 -0.031 -0.003 Verbal counting 0.087*** 0.105*** 0.054* 0.086*** 0.050* 0.063* 0.015 0.060 Quantitative comparison 0.019 0.020 -0.022 0.055* 0.003 0.005 -0.000 0.003 Number knowledge 0.062** 0.071** 0.061 0.046 0.034 0.044 0.000 0.049 Shape recognition 0.005 -0.006 -0.000 0.028 -0.001 -0.018 0.008 0.020 Arithmetic problem -0.009 -0.007 -0.017 -0.004 Spatial vocabulary -0.048* -0.059** -0.027 -0.047 Fine motor Copying 0.038 0.043 0.033 0.035 -0.042 -0.028 -0.053 -0.056 Draw-a-person 0.044 0.072** -0.013 0.050 -0.022 0.008 -0.077** -0.026 SDQ Emotional symptoms -0.021 -0.007 -0.033 -0.034 -0.009 -0.011 -0.042 0.028 Conduct problems -0.063** -0.045 -0.085** -0.074* -0.015 -0.026 -0.015 0.004 Hyperactivity/inattention -0.078** -0.088** -0.040 -0.097** 0.008 0.017 -0.001 -0.001 Peer problems -0.053 -0.028 -0.088* -0.065 0.014 0.044 0.025 -0.054 Prosocial 0.033 0.020 0.003 0.085** 0.026 0.022 0.025 0.034 All regressions control for individual baseline test scores, child age, child age squared, and province fixed effects. Addi- tional control variables include gender, baseline height-for-age z-score, household size, household wealth quintile dummy variables, and baseline caregiver age, gender, Raven’s score and parenting scores. Missing baseline covariates are replaced by the sample mean and interacted with a missing covariate dummy. Standard errors clustered on village level. * 10%, ** 5%, *** 1 % significance level. 52 Table 15: ITT effects by subgroups Midline (age 3–5) Endline (age 4–6) Baseline characteristics Cognitive Socio- Cognitive Socio- development emotional development emotional index problems index problems Age 2 0.024 -0.077 -0.008 -0.026 (0.025) (0.062) (0.020) (0.064) Age 3 0.038 -0.080 0.035 0.027 (0.031) (0.049) (0.034) (0.050) Age 4 0.050 -0.102* 0.018 0.004 (0.038) (0.056) (0.044) (0.065) Stunted -0.005 -0.070 0.011 -0.014 (0.025) (0.052) (0.028) (0.053) Not stunted 0.060*** -0.093** 0.020 0.003 (0.023) (0.041) (0.027) (0.040) Wealth quartile 1 0.035 -0.099 -0.020 -0.042 (0.033) (0.078) (0.029) (0.083) Wealth quartile 2 0.011 -0.134** 0.013 -0.054 (0.033) (0.052) (0.034) (0.054) Wealth quartile 3 0.013 0.009 -0.034 0.097 (0.039) (0.062) (0.042) (0.066) Wealth quartile 4 0.085** -0.099 0.134*** 0.030 (0.037) (0.069) (0.042) (0.064) All regressions control for individual baseline test scores, child age, child age squared, province fixed effects, gender, baseline height-for-age z-score, household size, household wealth quintile dummy vari- ables and baseline caregiver age, gender, Raven’s score, and parenting scores. Missing baseline co- variates are replaced by the sample mean and interacted with a missing covariate dummy. Standard errors clustered on village level. * 10%, ** 5%, *** 1 % significance level. 53 Table 16: Measuring take-up of demand-side interventions Obs. C T1-C T2-C T3-C Midline: Ever received home visit to discuss development of child 6552 0.688 0.012 -0.018 0.027 (0.02) (0.02) (0.03) (0.03) Ever received information leaflet about preschool 6552 0.053 0.020* 0.056*** 0.079*** (0.01) (0.01) (0.01) (0.01) Participated in HBP to discuss development of children (last 12 months) 6552 0.176 0.065*** 0.045 0.190*** (0.02) (0.02) (0.03) (0.03) · · · participated more than once (last 12 months) 6552 0.118 0.040** 0.027 0.133*** (0.02) (0.02) (0.02) (0.03) 54 Endline: Ever received home visit to discuss development of child 6575 0.742 0.001 0.038 0.03 (0.02) (0.03) (0.03) (0.03) Ever received information leaflet about preschool 6575 0.075 0.008 0.105*** 0.083*** (0.01) (0.01) (0.02) (0.02) Participated in HBP to discuss development of children (last 12 months) 6575 0.178 0.056** 0.03 0.100*** (0.02) (0.02) (0.02) (0.03) · · · participated more than once (last 12 months) 6586 0.096 0.047*** 0.035* 0.075*** (0.01) (0.02) (0.02) (0.02) C is the control group mean and constant in a regression of the outcome variable on a set of dummy variables for each treatment group. No control variables included in regression model. Estimates correct for heteroskedasticity. * 10%, ** 5%, *** 1 % significance level. Table 17: Perceived return to education (1) (2) (3) (4) (5) Midline: Optimal Optimal Optimal Return Return preschool age preschool age preschool age primary secondary ≤3 ≤4 ≤5 school school Group: T1 (CPS) 0.062** 0.065** 0.010 0.007 0.000 Group: T2 (CPS+D2D) 0.049* 0.080*** 0.017 0.010** 0.004 Group: T3 (CPS+D2D+HBP) 0.105*** 0.106*** 0.013 0.004 0.002 Constant 0.259 0.631 0.947 0.065 0.086 Group: T2 or T3 0.078*** 0.093*** 0.015 0.007 0.003 p-value: T1=T2 0.595 0.515 0.293 0.557 0.226 p-value: T1=T3 0.123 0.0952 0.663 0.436 0.639 p-value: T2=T3 0.0560 0.329 0.673 0.208 0.496 55 p-value: T1=T2 or T3 0.476 0.162 0.402 0.901 0.290 Observations 6989 6989 6989 6911 6971 Endline: Group: T1 (CPS) 0.031 0.073*** 0.014 0.004 0.001 Group: T2 (CPS+D2D) 0.040* 0.069** 0.011 0.005 0.002 Group: T3 (CPS+D2D+HBP) 0.041* 0.079*** 0.026* 0.001 0.002 Constant 0.188 0.501 0.916 0.068 0.085 Group: T2 or T3 0.041** 0.074*** 0.019 0.003 0.002 p-value: T1=T2 0.669 0.902 0.779 0.756 0.550 p-value: T1=T3 0.620 0.785 0.278 0.410 0.638 p-value: T2=T3 0.949 0.731 0.234 0.318 0.925 p-value: T1=T2 or T3 0.571 0.932 0.627 0.746 0.509 Observations 7021 7021 7021 6992 7004 Constant shows the control group mean. No control variables included in regression model. Row “Group: T2 or T3” shows parameter of regression model with a joint dummy variable for T2 and T3. Estimates correct for heteroskedasticity and within-village correlations. * 10%, ** 5%, *** 1 % significance level. A Supplemental Material 56 Figure A.1: Example of standardized CPS building and classroom. 57 Figure A.2: English version of leaflet used in door-to-door intervention at baseline. Community Preschool Your village has been selected to benefit from an improved community preschool supported by the Cambodian Ministry of Education. To make sure that children have enough space to learn and thrive, the preschool will have its own building and equipment. A trained teacher will prepare children of age 3-5 for primary school for 2 hours per day. If your child is of age 3-5, it is at the right age to benefit from the preschool. Preschool is free for all children. It is important for your child that it constantly learns new things. Preschool education can help children to become more intelligent and well-behaved. The community preschool is a place where children can learn how to interact with each other and learn about honesty, respect, sharing and perseverance. They will also learn about numbers, letters and words. Visiting a community preschool can help your child to stay in school longer and to do well in his/her future. School education is very important for children. Data from the Cambodian Socioeconomic Survey 2009 has shown that children who stay in school longer are likely to earn more in the future. Average Monthly Income by Education 680,000 KHR 410,000 KHR 330,000 KHR 260,000 KHR Primary school not Primary school Lower secondary Upper secondary completed completed 58school completed school completed Figure A.3: English version of leaflet used in door-to-door intervention at midline. Community Preschool A stimulating environment is crucial for optimal development of your child. Preschool education can contribute to a better future for your child by providing new learning experiences every day. At preschool, a trained teacher will help your child to learn important values such as respect, sharing and perseverance. Children will also be prepared for primary school by learning about numbers, letters and words. If you have a child age 3-5, you can enroll your child at preschool! 59 Enrolled: State Preschool Enrolled: any school Enrolled: State Preschool Enrolled: any school 0 .05 .1 .15 .2 .25 .2 .4 .6 .8 1 0 .05 .1 .15 .2 .25 .2 .4 .6 .8 1 3 3 4 4 4 4 5 5 Age Age Age Age width of 3.6 months. 5 5 6 6 7 7 6 6 Enrolled: Primary school Enrolled: formal CPS Enrolled: Primary school Enrolled: formal CPS 0 .2 .4 .6 .8 0 .2 .4 .6 0 .05 .1 .15 0 .2 .4 .6 3 3 4 4 C C T2 T2 4 4 5 5 Age Age Age Age 60 5 5 6 6 7 7 6 6 T3 T1 T3 0 Enrolled: informal CPS .1 .2 .3 .4 T1 0 Enrolled: informal CPS .1 .2 .3 .4 Current enrollment at midline Current enrollment at endline 3 4 4 5 Age Age 5 6 7 6 Figure A.4: Preschool enrollment at day of midline and endline survey. Figure shows separate local polynomial regressions using an epanechnikov kernel and a band- Figure A.5: Ever enrollmed at day of endline survey. Ever enrolled: informal CPS Ever enrolled: formal CPS .8 0 .1 .2 .3 .4 .5 1 Ever nrolled: any school .6 .8 .4 .6 .2 .4 0 4 5 6 7 4 5 6 7 4 5 6 7 Age Age Age Ever enrolled: State Preschool Ever enrolled: Primary school .3 .6 .8 .2 .4 .1 .2 0 0 4 5 6 7 4 5 6 7 Age Age Ever enrolled at endline C T1 T2 T3 Figure shows separate local polynomial regressions using an epanechnikov kernel and a band- width of 3.6 months. 61 Figure A.6: Total months enrolled at day of midline and endline survey. Months enrolled: any preschool Months enrolled: formal CPS Months enrolled: any school 0 2 4 6 8 10 6 6 8 4 4 2 0 2 0 3 4 5 6 3 4 5 6 3 4 5 6 Age Age Age Months enrolled: State Preschool Months enrolled: Primary school Months enrolled: informal CPS 4 2 4 1.5 3 3 2 1 2 .5 1 0 1 0 0 3 4 5 6 3 4 5 6 3 4 5 6 Age Age Age Months enrolled at midline C T1 T2 T3 Months enrolled: any preschool Months enrolled: formal CPS Months enrolled: any school 15 6 3 4 5 6 7 8 10 4 5 2 0 0 4 5 6 7 4 5 6 7 4 5 6 7 Age Age Age Months enrolled: State Preschool Months enrolled: Primary school Months enrolled: informal CPS 0 .5 1 1.5 2 2.5 8 0 1 2 3 4 5 0 2 4 6 4 5 6 7 4 5 6 7 4 5 6 7 Age Age Age Months enrolled at endline C T1 T2 T3 Past enrolment spells are not available for the midline survey. Therefore, midline outcomes 62 where child is enrolled at time of midline only measure total months of enrollment at the school survey. Endline outcomes measure total months of enrollment at current school and all previous schools. Figure shows separate local polynomial regressions using an epanechnikov kernel and a bandwidth of 3.6 months. B Child assessments This section summarizes the individual tests, their distributions and the scoring meth- ods used in this paper. An in-depth discussion of the tests, scoring methods, cultural adaptations and pretesting procedures can be found in Berkes et al. (2019). To ensure that children correctly understood the tests and that the test were reli- able, the research team pretested every instrument at least three times before collecting data in the sample villages. The survey firm translated the questionnaires into Khmer and an independent third party back-translated them into English which led to fur- ther refinements in the instruments. The final child assessments included a total of 15 individual tests at baseline, 17 at midline and 20 at endline. Before constructing the composite scores of child test domains, individual tests were first scored and standardized thus ensuring that all tests contributed equal variance to their composite score. Scoring was done by assigning 1 point for each correct response and summing up these points to create an individual score for each test. When a child was unable to complete the practice trial of a test, a score of zero was assigned for this test as long as the child participated in the other tests. Standardization of each test score was done with the control group mean of the same wave by subtracting the mean and dividing by the standard deviation of this wave. All standardized test scores of one domain (e.g. executive function) were then summed into a domain score and standardized again by subtracting its sample mean and dividing by the sample standard deviation of the domain score for better interpretability. After these steps, we obtained the following composite scores: 1. Executive function: 1.1. The construct inhibitory control is assessed with the head-knee task. The test has two stages. In the first stage, the child stands in front of the enumerator and is asked five times to touch his/her head or knees. In a second stage, the child is asked to do the opposite of what the enumerator says. 1.2. Working memory (short-term auditory memory) is assessed with a forward digit span test in which children have to repeat sequences of digits which increase in length. 1.3. The Dimensional Change Card Sort test is used as a measure of cognitive flexibility. We followed the procedures outlined in Zelazo (2006) using cards 63 with two colours (blue and red) and two pictures (boat and rabbit). To reduce the burden on tested children, we followed the protocol with the exception that children needed to pass the pre-switch phase (at least 5 out of 6 correct) in order to participate in the post switch phase. The border version of the test was only administered at endline. The demonstration phase of the test included one practice trial. As per protocol, this practice trial was not used to determine whether a child is eligible for the test as it could have performed well by chance. 1.4. We use a self-developed cancellation task to measure sustained attention. In this test, children see a printed matrix with different symbols and are asked to cross-out all symbols that match the given one (e.g. cross out all flowers). When completed, a larger matrix is given and a new symbol has to be crossed-out. The test continues until a child has completed 4 matrices, crossed out more wrong than correct images in a matrix, until the child loses attention, or states that it is done. The test was scored by using the difference between correctly and incorrectly crossed out images. 2. Language: 2.1. Receptive vocabulary skills are assessed 78 with a test derived from the TVIP. In this test children are asked to match a word to one out of four pictures. The version used in the Cambodian context was culturally adapted during piloting and validation exercises prior to baseline data collection and with the support of key informants. The final instrument includes 82 pictures with a rule that the test stops after 6 out of the last 8 pictures were wrong. All other language development tests were taken from the MELQO. 2.2. Expressive language skills are assessed by asking children to name up to 10 things that can be eaten and up to 10 animals they know. The final score is the number of recalled items. 2.3. Receptive language is assessed with a listening comprehension test in which a short story (116 words) is read to the child. After reading the story, the child is asked five questions about the content of the story. 2.4. Knowledge of reading concepts is assessed by showing a children’s storybook and asking how the book should be opened and where and in which direction one should start reading the story. 64 2.5. Reading skills are assessed with a letter name knowledge test in which chil- dren have to identify common letters of Khmer script. 2.6. Endline only: A name writing test was conducted to assess whether children were able to write their own name. 2.7. Endline only: An initial letter identification test was conducted in which children were asked to name the first alphabet letter letter of words that were read to the child 2.8. Endline only: Reading skills were assessed by asking the child to read out loud different printed words. 3. Early numeracy: 3.1. Midline only: The tests for early numeracy includes a self-developed test for measurement concepts, e.g. if the child understands concepts such as tallest/shortest, in which the child had to point to different printed objects. 3.2. In a test for verbal counting, children had to count up to 30. 3.3. Numbers and operations are also administered with a self-developed quanti- tative comparison test where children had to compare the number of printed objects on two sides of a page. 3.4. A number identification test analogous to the letter name knowledge test was used. 3.5. A self-developed shape recognition test was used to test if children are able to identify basic geometric shapes. 3.6. Endline only: Children were asked to read printed arithmetic problems and say the correct answer (e.g. 2+1). 3.7. Endline only: A spacial vocabulary test was conducted in which the child was shown 4 pictures with a ball and a chair. The child was asked to point to the correct picture with the ball either on, under, in front or next to the chair. 4. Fine-motor development: 4.1. A drawing test, where children copy shapes, like circles or squares, was used to assess fine-motor skills. 65 4.2. A draw-a-person test 5. Gross-motor development (midline only): The Malawi Developmental As- sessment Tool (MDAT) was used for assessment of grossmotor skills. 6. Socio-emotional problems: The recommended method was used to create a total difficulties score, i.e. summing up scores of the individual subcomponents without standarizing first. The subcomponents are: 6.1. Emotional symptoms 6.2. Conduct problems 6.3. Hyperactivity/inattention 6.4. Peer problems 66 Figure B.1: Distribution of midline cognition composite scores and individual tests. .05 .4 .3 .4 .04 .3 .3 Fraction Fraction Fraction Fraction .2 .03 .2 .2 .02 .1 .1 .1 .01 0 0 0 0 -2 -1 0 1 2 0 5 10 15 20 0 1 2 3 4 5 0 5 10 15 1: Executive function composite 1.1: Head-knee task 1.2: Forward digit span 1.3: DCCS .1 .15 .15 .4 .08 .3 Fraction Fraction Fraction Fraction .1 .1 .06 .2 .04 .05 .05 .1 .02 0 0 0 0 0 10 20 30 40 -2 0 2 4 6 0 20 40 60 80 0 5 10 15 20 1.4: Cancellation task 2: Language composite 2.1: Receptive vocabulary 2.2: Expressive language .6 1 .25 .3 .8 .2 Fraction Fraction Fraction Fraction .2 .4 .6 .15 .4 .1 .1 .2 .2 .05 0 0 0 0 0 1 2 3 4 5 0 1 2 3 0 5 10 15 20 0 1 2 3 4 5 2.3: Receptive language (short story) 2.4: Knowledge of reading concepts 2.5: Letter recognition test 4.1: Shape drawing .1 .5 .4 .4 .08 .4 .3 .3 Fraction Fraction Fraction Fraction .06 .3 .2 .2 .04 .2 .1 .1 .02 .1 0 0 0 0 -2 0 2 4 6 0 1 2 3 4 0 10 20 30 0 1 2 3 4 5 3: Early numeracy composite 3.1: Measurement vocabulary 3.2: Verbal counting 3.3: Quantitative comparison .4 .6 .3 Fraction Fraction .4 .2 .2 .1 0 0 0 5 10 15 20 25 0 1 2 3 4 3.4: Number identification 3.5: Shape identification Figure shows empirical distribution of composite and raw test scores. 67 Figure B.2: Distribution of midline motor-development and socio-emotional composite scores and individual tests. .25 .25 .8 .2 .2 .6 Fraction Fraction Fraction .15 .15 .4 .1 .1 .2 .05 .05 0 0 0 -2 0 2 4 6 0 1 2 3 4 5 0 10 20 30 4: Fine-motor composite 4.1: Shape drawing 4.2: Draw-a-person .15 .1 .15 .08 Fraction Fraction Fraction .1 .1 .06 .04 .05 .05 .02 0 0 0 -2 -1 0 1 2 -4 -2 0 2 4 0 .5 1 1.5 2 6: Socio-emotional problems composite 5: Gross motor composite 6.1: SDQ: emotional symptoms .25 .2 .25 .2 .2 .15 Fraction Fraction Fraction .15 .15 .1 .1 .1 .05 .05 .05 0 0 0 0 .5 1 1.5 2 0 .5 1 1.5 2 0 .5 1 1.5 2 6.4: SDQ: peer relationship problems 6.2: SDQ: conduct problems6.3: SDQ: hyperactivity/inattention 68 Figure B.3: Distribution of endline cognition composite scores and individual tests. .08 .4 .15 .3 .06 .3 Fraction Fraction Fraction Fraction .2 .1 .04 .2 .1 .05 .02 .1 0 0 0 0 -4 -2 0 2 4 0 5 10 15 20 0 5 10 15 0 5 10 15 20 1: Executive function composite 1.1: Head-knee task 1.2: Forward digit span 1.3: DCCS .25 .08 .06 .1 .2 .06 Fraction Fraction Fraction Fraction .04 .15 .04 .05 .1 .02 .05 .02 0 0 0 0 0 10 20 30 40 -2 0 2 4 0 20 40 60 80 0 5 10 15 20 1.4: Cancellation task 2: Language composite 2.1: Receptive vocabulary 2.2: Expressive language .2 .4 .8 .8 .15 .3 .6 .6 Fraction Fraction Fraction Fraction .1 .2 .4 .4 .05 .1 .2 .2 0 0 0 0 0 1 2 3 4 5 0 1 2 3 0 5 10 15 20 0 .5 1 1.5 2 2.3: Receptive language (short story) 2.4: Knowledge of reading concepts 2.5: Letter recognition test 2.6: Letter writing .8 .08 .2 .8 .6 .06 .15 .6 Fraction Fraction Fraction Fraction .4 .4 .04 .1 .2 .2 .02 .05 0 0 0 0 0 1 2 3 4 5 0 .5 1 1.5 2 -2 -1 0 1 2 3 0 10 20 30 2.7: Initial letter identification 2.8: Reading words 3: Early numeracy composite 3.2: Verbal counting .3 .4 .4 .8 .3 .3 .6 Fraction Fraction Fraction Fraction .2 .2 .2 .4 .1 .1 .1 .2 0 0 0 0 0 2 4 6 0 5 10 15 20 0 1 2 3 4 0 1 2 3 4 5 3.3: Quantitative comparison 3.4: Number identification 3.5: Shape identification 3.6: Arithmetics .4 .3 Fraction .2 .1 0 0 1 2 3 4 3.7: Spacial vocabulary 69 Figure B.4: Distribution of endline motor-development and socio-emotional composite scores and individual tests. .06 .1 .5 .08 .4 Fraction Fraction Fraction .04 .06 .3 .04 .2 .02 .02 .1 0 0 0 -2 -1 0 1 2 3 0 5 10 15 0 10 20 30 4: Fine-motor composite 4.1: Shape drawing 4.2: Draw-a-person .1 .2 .2 .08 .15 .15 Fraction Fraction .06 Fraction .1 .1 .04 .05 .05 .02 0 0 0 -4 -2 0 2 4 0 .5 1 1.5 2 0 .5 1 1.5 2 6.1: SDQ: emotional symptoms 6.2: SDQ: conduct problems 6: Socio-emotional problems composite .2 .25 .2 .15 Fraction Fraction .15 .1 .1 .05 .05 0 0 0 .5 1 1.5 2 0 .5 1 1.5 2 6.4: SDQ: peer relationship problems 6.3: SDQ: hyperactivity/inattention 70 C. COST ANALYSIS June 3, 2019 Conducted by the Strategic Impact Evaluation Fund (SIEF) Author: Sam Fishman Abstract: In 2019, SIEF initiated a study to estimate costs for Early Childhood Education and Development (ECED) activities in Cambodia funded under the World Bank’s Second Education Sector Support Project (SESSP) . Retrospective costs were estimated for one school year for each of three interventions: state preschools (SPS), community preschools (CPS) and home based care programs (HBP). SIEF estimated a range of costs for each program based on six models using different data sources and different assumptions about program scale and the value of cost ingredients. The average cost per child per year was estimated to be between $331 and $669 for state preschools, between $156 and $443 for community preschools schools and between $131 and $360 for the home based care program. The precision, accuracy, and applicability of cost estimates can be improved through establishment of a complete cost collection cycle at the Cambodian Ministry of Education, Youth and Sports (MoEYS). Additionally, the impact of state preschools will need to be evaluated before the cost-effectiveness of programs can be compared. Our initial retrospective estimates of the average cost per-child of ECED programs do demonstrate that community preschools may be a more efficient alternative to state preschools, but not necessarily a more cost-effective alternative. The Home Based Care program’s efficiency varied significantly based on the assumptions about the opportunity costs of participation and on the actual rate of participation. Finally, this report breaks down program costs by category, which may point to opportunities for saving and improving the quality of services. SUMMARY The government of Cambodia has been expanding investments in Early Childhood Education and Development (ECED)-related services and activities through state preschools (SPS), community preschools (CPS), and home based programs (HBP). The inter-ministry budgeting process over the next few years will determine new levels of funding for CPS and SPS schools (beginning with the 2019 Budget Strategic Plan (BSP)). Accurate cost estimates for ECED program can help the Ministry of Education Youth and Sports (MoEYS) tie transfers, such as school operational budgets and headcount payments, to local needs and assess their sustainability. Cost estimates can also inform scale-up of ECED services. Community preschools are often used to expand coverage. There are already some indications that the average cost per child in community preschools in Cambodia will prove 71 lower than for SPS. For example, teacher salaries for formal preschool teachers are much higher, as they are often career teachers with significant formal training. Community teachers by comparison undergo a thirty-five day training and are paid a much smaller salary arranged by local Commune Councils. Additionally, SPS buildings are constructed through more expensive contracting services, while CPS buildings are constructed by community working groups which are trained and provided money for construction. However, what would be important to know for policy would be how cost-efficient or cost-effective these schools are. Cost-efficiency here is defined as the cost of providing one child with a year of one of the three ECED programs. Cost-effectiveness is the cost of increasing cognitive development and school readiness outcomes for one child through a year of one of the three ECED programs. An analysis of the average cost per child (total program costs divided by total beneficiaries) of both preschool models, combined with an assessment of the relative quality of services, is necessary to determine if the CPS model in Cambodia is a viable model for expanding preschool coverage. In this study, we focus on cost efficiency. ➢ We estimate that the average cost of providing one year of SPS to one child be is between $331 and $669. ➢ We estimate that the average cost of providing one year of CPS to one child is between $156 and $443. ➢ We estimate that the average cost of providing one year of the home based program to one child is between $131 and $360. This note describes the costing methods used to estimate the costs and cost-efficiency of the two Second Education Sector Support Project (SESSP) funded preschool models, as well as average costs of the home visiting program. The first section lays out the principal research questions that framed our cost analysis. Section 2 presents the costing approach used to estimate costs, including specific assumptions about the costs of certain inputs. The third section describes data collection and some data limitations, while Section 4 lays out the estimation strategy and the rationale for estimating a range of cost estimates. Section 5 presents the estimates of total costs and cost-efficiency, as well as breakdowns of the SPS, CPS, and HBP costs by expenditure categories, such as personnel costs, program resources, and overhead. The final section discusses the results and offers recommendations for costing similar programs in the future. 72 RESEARCH QUESTIONS Table 1: Research questions for SPS, CPS and HBP COST METRIC RESEARCH QUESTIONS WHAT DOES THIS MEAN Total Costs What are the total costs of delivering The total cost to all stakeholders, including community state preschools at scale? members, to deliver SPS at scale. What are the total costs of delivering The total cost to all stakeholders, including community community preschools at scale? members, to deliver CPS at scale. What are the total costs of delivering The total cost to all stakeholders, including community home based care programs at scale? members, to deliver HBP at scale. Cost-Efficiency What is the average cost to The cost of delivering one year of SPS services to one child. (average cost) providing one child with one year of This is calculated by dividing the total cost by the number state preschool? of child beneficiaries What is the average cost to The cost of delivering one year of CPS services to one child. providing one child with one year of This is calculated by dividing the total cost by the number community preschool? of child beneficiaries What is the average cost to The cost of delivering one year of HBP services to one providing one child with one year of child. This is calculated by dividing the total cost by the the home based care program? number of child beneficiaries Cost-Effectiveness How does the average cost per child Cost-effectiveness compares the cost per beneficiary for to improve cognitive and social- different programs to achieve a certain impact/outcome emotional scores by a certain measure.1 amount differ across the CPS program and the CPS variant that also included home-based visits? The goal of collecting costs of the CPS, SPS, and HBP programs is to estimate the total cost, cost-efficiency, and cost effectiveness of these interventions. Cost collection will also provide information on the cost structure of the programs, or the breakdown of costs by cost category (for example, personnel, overhead, and equipment). Total program costs will provide a picture of what aggregate resources were required to maintain these programs at present scale. However, cost-efficiency metrics will better inform the cost of further scaling these programs by measuring the average cost of providing a service to one child for one year. The estimates provided are economic analyses of costs and therefore take into account insight from multiple stakeholders and non-financial costs. Because the community preschool and home visiting programs were embedded within an impact evaluation, it is also possible to examine the cost-effectiveness of adding the home visiting program to the preschool program. 1SIEF could not estimate the cost-effectiveness of Cambodia’s investments of CPS and SPS schools. To have any policy relevance, cost-effectiveness measure must compare two programs where positive impacts were measured. 73 That is, we can compare whether adding the home visiting component generates more benefits for the preschool program than costs. While a comparison of CPS and SPS cost-effectiveness would also be useful, we currently have no way of estimating the impact of the state preschools. Cost studies of preschool services in low and middle income countries have produced a very wide range of estimates for the cost of providing one year of preschool to one child, ranging from $86 per-child per-year unit costs in Niger (Jaramillo & Mingat, 2008) and $264 (Issa & Evans, 2008) in Kenya to $761 in India (Levin & Schwartz, 2012) and $1,598 in Indonesia (Levin & Schwartz, 2012).2 Comparing these across contexts, however, does not tell us much, as ECED programs exhibit a range of complexity in their designs and costs tend to be measured differently as well. Despite these problems with the external validity of ECED cost metrics, answering questions about the in-country costs of different ECED programs remains of critical value. METHODS In Cambodia, SIEF collected cost data by gathering basic information about program implementation and available financial data. Next, using this information, SIEF constructed preliminary cost models, following the ingredients method to list all inputs to the program that had an opportunity cost. 3 For each ingredient identified, these models required data from the MoEYS and other ministries on the unit costs and quantities, as well as the fraction of each input that was used. For some inputs where data was lacking, we had to make assumptions based on the best available information about program implementation and finances. The cost models were then finalized with exchange rate and inflation data, and average costs per beneficiary were calculated based on the total number of beneficiaries. SIEF follows the cost capture process outlined in World Bank (2019).4 Financial data that comes from budgets or spending reports needs to be detailed and disaggregated including specific line items, quantities, and unit prices. Non- financial information is also a critical part of cost data capture. This can come from monitoring and evaluation data, interviews with program implementers, and exercises like time- & effort-tracking. Most importantly, cost data is ideally captured in real-time during program implementation. Collecting the data at project close may result in inaccurate or even missing data. 2Emily Gustafsson-Wright Izzy Boggild-Jones Sophie Gardiner, “The Standardized Early Childhood Development Costing Tool (SECT) A Global Good to Increase and Improve Investments in Young Children,” Brookings Institute, 2017, https://www.brookings.edu/wp-content/uploads/2017/09/standardized-ecd-costing-tool.pdf 3The ingredients method is commonly used to conduct cost analysis, and involves listing all societal inputs required to make an intervention happen and valuing those inputs using quantity and price data. 4World Bank Group (2019). Capturing Cost Data. Retrieved from http://pubdocs.worldbank.org/en/994671553617734574/Capturing-Cost-Data-190314.pdf. 74 Figure 1: Timeline of SESSP activities, impact evaluation, and costing May - July 2018: COST Cost study begins 2014: AND February 2019: April - June 2017: SESSP funded Midline data Endline data Cost estimates activities begin collected conducted finalized May - July 2016: July 2017: July - November Baseline data SESSP closing 2018: collected date Ongoing Cost Collection with MoEYS In Cambodia, because the cost study started near the close of the CPS program that was being evaluated, we had to collect costs retrospectively. In an ideal scenario, we would have used pre-intervention cost models in 2013-2014 to guide real-time data collection throughout the SESSP funding period. In this case, to start collecting costs, we first visited Cambodia in June, 2018 to collect financial data, M&E data, as well as qualitative data from interviews, document reviews, and field observations. In Cambodia, collecting disaggregated data required in depth discussions with MoEYS officials, as most financial data was in aggregate form, typically in national-level totals, which had to be modified to include more detailed line items; to specify quantities, unit costs, and frequency of purchases or activities; and to disaggregate expenditures by province and type of school. Interviews also helped determine which inputs were actually considered part of the intervention, how much of shared inputs like ECE officer salaries and shared trainings to allocate to the ECED interventions, and whether it was possible to measure the cost of each input over specific geographic areas, or for just the SESSP schools and the areas implementing the home visiting program. Inputs for the preschool programs came from four different geographic administrative levels: the community, the district, the province, and the central government. Interviews were conducted at each of these levels to determine what resources different government or community stakeholders were contributing to the program. M&E data and data from the impact evaluation’s end line survey helped determine how many beneficiaries these programs were reaching and how many resources the programs received from community-level sources. Direct observations of the program helped identify the use of certain community inputs , their dosage, and their value. Data from interviews with the Chief of Planning office for the ECE department at MoEYS and the Director for ECE activities in Kapong Chhnang 75 Province identified community funding for the schools, teacher activities and training costs, and whether certain costs at different levels were being double counted. THE INGREDIENTS METH OD To identify the costs of the programs, SIEF followed the ingredients method. The ingredients method is widely used by cost analysts and requires a listing and valuing of every resource required to make an intervention happen. 56 This means that inputs like volunteer time or community contributions or donations are also included as costs. This is important as economic analysis is primarily concerned with estimating the costs of replicating an intervention. If the intervention were to be expanded or implemented elsewhere, it is necessary to understand all the costs to all stakeholders, as in different contexts, certain resources might not be donated or provided by a community and could instead become costs covered by the provider of the service. It is also critical that “support” costs are included in the analysis. Support costs include expenses and efforts required for management and administration of the program. They include indirect expenses such as office rents, use of assets, general resources procured by the government, and salaried personnel who may support many activities. Determining the support costs associated with a government intervention can be difficult, as interventions are often supported by multiple levels of government (district, province, federal), and there often aren’t existing financial practices in place to allocate general administrative funding streams across programmatic activities. Once we have a list of all ingredients needed to make an intervention happen, we need to value them. To do this, you need to collect information on the price of different inputs, the quantity of those inputs applied over the course of a year, and the allocation of shared inputs across different programs. In the absence of real-time data, you will have to collect this information retrospectively. This can be problematic due to recall issues, as program staff often cannot accurately recall information on how to allocate costs across the various activities they work on. For example, regular or real time reporting is often necessary for personnel costs where time and effort data is required to understand the percent of gross salary to allocate to an intervention or activity. Often, this kind of data suffers from recall bias when reported retrospectively. 7 8 SIEF had to capture all data retrospectively. Thus, the list of ingredients is likely to be incomplete and their values may be inaccurate. In costing the community and state preschool programs, SIEF tried to address these constraints by 5 Levin et al., 2018 6 Rice, 1997; McEwan, 2002; Ross et al, 2007; Harris, 2008 7Das, Jishnu, Jeffrey Hammer, and Carolina Sanchez-Paramo. "The Impact of Recall Periods on Reported Morbidity and Health Seeking Behavior." Journal of Development Economics 98 (2012). 86. 8 Vellore Arthia, Kathleen Beegle, Joachim De Weerdt, Amparo Palacios-López, “Not your average job: Measuring farm labor in Tanzania,” Journal of Development Economics 130 (2018), 160-172. 76 1) generating a range of estimates and 2) carefully documenting assumptions and data sources so that future cost analysis can build upon existing cost models. THE COST MODEL SIEF’s cost models list all ingredients for each program (CPS, SPS, and HBP) separately in a master excel document. A fourth list includes all management and administrative ingredients that are shared by all four programs. Each ingredient is then valued by multiplying the three primary variables: units (or quantities), unit prices, and allocation percentages. Each ingredient is also tagged with an ingredient category (see table 2 below). Additional frequency and dosage variables are sometimes added if not all elements of an ingredients value can be described by units, unit prices and allocation percentages. For example, for some trainings, the frequency of the training (times it occurs per year) and the dosage (days the training lasts) are added separately, whereas the quantity describes the teachers trained and the unit price describes the cost per teacher per day. Finally, many ingredients had to be amortized over multiple years to arrive at a yearly cost. Costs for each program are then tabulated in summary tabs that take into account exchange and inflation rates. Summary tabs also disaggregate the costs of each program by ingredient category. Below is a list of the categories of cost ingredients for the SPS, CPS and HBP programs. These are relatively standard cost categories used for ECE interventions. The second columns list the most significant cost ingredients in each category. Table 2: Ingredient categories used for SPS, CPS and HBP CATEGORY INGREDIENTS Personnel – frontline/direct delivery CPS Teachers SPS Teachers Core mothers, Lead mothers, Mother members Personnel – other ECE officers Government personnel Consultants Administration (overhead) Overhead estimate Equipment Furniture Computers Pre-service training Pre-service trainings for CPS teachers Pre-service trainings for HBP core mothers In-service training In-service SPS training 77 In-service CPS training In-service HBP training ECE officer trainings Classroom materials Materials for classrooms Materials for HBP Investments from parents in school resources Investments from community in school resources Construction costs Construction of CPS schools Construction of SPS schools CPS construction management trainings (amortized) To measure these costs, we had to make the following assumptions: ➢ Amortization: SESSP involved some major startup costs and non-yearly costs to support ECE programs, such as construction of CPS and SPS schools, purchase of furniture, hiring of consultants, and pre-service trainings. Because these inputs last beyond the evaluation period, we had to amortize costs to arrive at yearly costs for the intervention. When possible, amortization periods were based on either direct knowledge of the duration of assets and materials (how long before they need to be replaced) or on how often annual expenditures occur. In some cases where this was more difficult to estimate, we used a range of estimates. For example, we amortized the cost of constructing one SPS school ($62,541) over 35 years to arrive at a cost of $1,786 per school per year. In another example, furniture for schools was estimated to have a five year lifespan, so an initial purchase of $619 per school was reduced to a $124 yearly cost. ➢ Shared administration and overhead costs: Administrative and overhead costs are a major component of any government ECE program. However, they are often hard to measure for single activities or interventions because administrative funding streams rarely disaggregate financial data by specific program codes and there usually aren’t other reporting structures that help with allocating these costs. In SIEF’s models, management and overhead costs are split equally between the three treatment arms. This is not entirely arbitrary, as interviews among district, provincial and central level ECE officers suggested that ECE department personnel spend roughly equal time on each intervention. ➢ Costs are measured in USD: Often global cost studies measure costs in local currency and then must factor exchange rates into their analysis. However, Cambodia has a highly dollarized economy, and the majority of accounting and cost data received was denominated in USD. We could not obtain much of the data we needed, and thus we estimated six different cost models to account for areas of uncertainty. Two master models reflect different beneficiary populations: 1) the SESSP funded programs 78 model only refers to schools and programs that were directly established through SESSP funding and; 2) the 13 provinces model, which measures costs for all programs established in the provinces where the impact evaluation occurred. Each of these two models takes a different total number of beneficiaries as the denominator and also draws from different data sources to value ingredients. Within each of these two models, we nest three additional models based on high-range, mid-range, and low-range estimates for costs. These models vary the values of certain unit costs, quantities, allocation percentages, or amortization rates in instances where we were less certain about the accuracy of the initial data and estimates used to value ingredients. DATA COLLECTION Cost capture occurred between June 2018 and October 2018, and Table 5 describes what data could be obtained. Table 3: Data collected for cost study INGREDIENTS DATA OBTAINED DATA SOURCE CPS Teachers Number of teachers, core mothers World Bank / MoEYS SPS Teachers Average teacher salaries; number of teachers Endline Data Core, lead, & member mothers ECE officers Rough estimate for ECE officer salaries MoEYS M&E data Government personnel Quantity of ECE personnel nationwide at POE and Central level. Endline Data Consultants Overhead data Total budget for central level, total education spending ratios World Bank; MoEYS Furniture Unit and unit costs of furniture, laptops, flooring SIEF Consultant Computers In-service SPS/CPS/HBP trainings SESSP training costs MoEYS ECE officer trainings Materials for classrooms Unit cost for Materials for classroom and HBP World Bank Docs Materials for HBP Quantity for materials matched 1x1 ratio per school or program, per year. Investments from parents in school resources Unit cost of parent investments Investments from community in school Investment from community Teacher investments in schools Pre-service trainings for CPS teachers Aggregate costs for CPS and HBP trainings World Bank Research Team Pre-service trainings for core mothers 79 Construction of CPS and SPS schools Unit costs for construction of schools MoEYS dept of finance & ECE dept CPS construction management trainings At this stage, SIEF still lacked data on critical inputs, such as transport, per diems and other travel, equipment, office rents and maintenance, disaggregated training costs, and actual expenditure data on salaries and administrative costs. Other missing information ideally would have been collected in real time, such as time and effort data for government labor inputs and more detailed data on expenditures at the community level (for example, qualitative interviews suggested CPS teachers spent significant out-of-pocket money on school supplies). Additionally, we were not always able to confirm a standard frequency and dosage for certain activities, such as monitoring and evaluation efforts, because implementation varied significantly across districts and an average estimate was not available. The following table maps the quality of data achieved for individual ingredients for all three programs. Each variable (column) for each ingredient category (row) is colored according to the quality of the data obtained (green = good data; yellow = partial data; red = no or very minimal data). 80 Table 4: Data quality map INGREDIENT INGREDIENT EXISTS UNITS UNIT COST FREQUENCY / DOSAGE ALLOCATION RULE CPS and SPS Teachers Core, Lead, and Members Mothers T&E (imputed Cost) ECE officers Other Government personnel Consultants Overhead (management/indirect) Travel, per diems, other gov. M&E / Management costs Furniture Computers In-service SPS training In-service CPS training In-service HBP training ECE officer trainings Materials for classrooms Materials for HBP Investments from parents Investments from community Teacher investments Pre-service trainings for CPS teachers Pre-service trainings-core mothers Construction of CPS/SPS schools CPS construction trainings 9 9 Green = good data; yellow = partial data; red = no or very minimal or unreliable data). 81 The amount of missing data underlines the difficulty of collecting sufficient cost data retrospectively and the necessity for more sustained engagement with program officials in order to clarify the types of data necessary to conduct cost analysis. In future efforts to cost Cambodian ECED programs, expectations about cost collection should be established over a longer timeframe, and these efforts should produce pre-intervention analyses so that a full cost collection and analysis cycle can be established. This would ideally involve MoEYS integrating appropriate ongoing cost reporting measures into their existing M&E and financial reporting processes. ESTIMATION STRATEGY To account for mismatched data sources and data gaps, a range of estimates were created using six different cost models. Each model estimates total cost and cost-efficiency (average cost per child) for each of the three ECED programs. SESSP model vs. 13 provinces model Much of the data made available to SIEF consisted of two types: 1) data related to SESSP funded programs or; 2) data related to all programs found in the 13 provinces where the impact evaluation was conducted. To resolve the mismatch of data sets, SIEF chose to create two different sets of estimates, one that focuses on the SESSP programs and one that focuses on the impact evaluation provinces. These two models offer a very crude comparison between programs at different scale: medium scale for SESSP and large scale for the 13 provinces model. Additionally, again at a very crude level, the SESSP model can be viewed as a look at ECED programs funded under World Bank project funding, whereas the 13 provinces model looks at the programs under the conditions of primarily domestic financing.10 Since SESSP funded HBP at a larger scale, there is a smaller difference between the number of total schools/home based programs evaluated in both models (901 HBP in SESSP, 1587 in the 13 provinces model). However, this analysis does not accurately reflect many of the cost-efficiency differences between the two different scales and funding models. Data mismatches necessitated using frequency, quantity, unit cost and allocation data in both models that may only apply to only one model. For example, unit costs for hygiene materials for classrooms were found in the SESSP grant reporting and applied to both SESSP and 13 provinces model, whereas unit costs for polyvinyl floor investments were found only in the national level data from MoEYS, and were also applied to both models. The main purpose of creating two models is not to estimate the programs at different scale, but rather to reduce the error involved in using data that refers to different populations of beneficiaries. 10The 13 provinces model still includes SESSP programs in its sample. However, the SESSP funded SPS and CPS schools represent a much smaller share of the overall population of schools evaluated in the 13 provinces cost model. The HBP programs were funded on a much larger scale by SESSP, and therefore there is a smaller difference between the overall scale of the HBP program evaluated in the two models. 82 Mid-range vs. high-end and low-end estimates Given data gaps and uncertainty about some of the unit costs, quantities, allocation percentages, and amortization schedules used to value ingredients, we varied our assumptions for some cost ingredients to produce high-end and low-end estimates for both the SESSP model and 13 provinces model. This is similar to the standard practice of conducting sensitivity analyses for cost analyses. 11 The high-end estimates apply the more expensive values for many ingredients, and the low-end estimates assume the less expensive values for these ingredients. The mid-range estimate was generated first, and high-end or low-end estimates vary assumptions from mid-range models. Figure 2 below describes each model and set of estimates. Figure 2: Map of estimation strategy SEPARATE SETS OF VARIED ASSUMPTIONS SIX TWELVE PROGRAMS EVALUATED FOR HIGH END LOW MODELS ESTIMATES END ESTIMATES Total Cost High-End, SESSP Cost-Efficiency Cost model for SESSP Total Cost funded programs High-End, Provinces High-End Cost-Efficiency (medium scale) Total Total Cost Cost Mid-Range, SESSP Mid-range Cost-Efficiency Cost model for all Mid-Range, Provinces Total Cost programs in 13 impact Cost-Efficiency evaluation provinces Low-End (large scale) Low-End, SESSP Total Cost Cost-Efficiency Low-End, Provinces Total Cost Cost-Efficiency DETAILED ASSUMPTIONS Below in tables 7, 8 and 9 are summaries of every major ingredient included in SIEF’s cost models. Allocation percentages are key variables in all models. These are calculations of how much of the ingredient is allocated to the 11Sensitivity analyses are typically used to evaluate the costs of an intervention under different conditions which may hold in the future, or in different contexts. Often separate sets of sensitivity analyses are cited that isolate one changed assumption. For example, sensitivity to exchange rate fluctuations may be evaluated, and a range of potential costs cited for this changing assumption. In SIEF’s high-end and low-end estimates, an aggregate of many changing assumptions is used. Upon request, SIEF can produce disaggregated sensitivity analyses that focus on single assumptions. 83 specific intervention in question. For example, SPS teacher salaries would be allocated 100% to SPS costs, but construction consultants who worked on SPS and CPS schools have 50% of the cost of their fees allocated to the SPS intervention, and 50% to the CPS intervention. In other cases, such as for lump-sum overhead rates, we need to use more complex rules and assumptions for determining allocation percentages. Support costs Support costs, which include management, administrative, and overhead costs required many assumptions, and ultimately SIEF had to rely on rough estimates based on the aggregate central level ECE budget and the overall (?) proportion of provincial to central level spending to make an educated guess at an overhead cost for the programs. We also included other ingredients like ECE officer salaries. Data on other key support costs, however, were incomplete, and then we had to approximate costs like the costs of monitoring and evaluation missions, indirect personnel, administrative assets, and contractor costs. Allocation rules for support costs also had to be approximated. In the 13 provinces model, costs were split equally between the three programs. This is not entirely arbitrary, as interviews with ECE officers at the district, provincial, and central level suggest that ECE officers spend about equal time on all three programs. In the SESSP model, 8.4% of costs were allocated to SPS, 51.2% to CPS, and 40.4% to HBP, in line with the relative number of beneficiaries in each of the three programs. 12 SIEF could access aggregate ECE department costs that covered all ECE programs in Cambodia. Therefore, it was also necessary to calculate what percentage of these total national level costs had to be allocated to SESSP only programs or to just the programs found in the 13 impact evaluation provinces. Tables 5 and 6 below show the rules used to estimate this allocation of national administrative costs. Table 5: Scaling down national administrative costs to fit SIEF’s SESSP model Scale Percent of Ratio of CPS Ratio of SPS Ratio of HBP to All programs equally total Schools to schools to national HBPs. weighted administrative national total national costs allocated schools SESSP-low 17% = [(500 / 3132) X (70/3413) X (901/2763)] X (1/3) SESSP-mid 22% = [(1000 / 3132) X (70/3413) X (901/2763)] X (1/3) SESSP-high 33% = NA X NA X (901/2763)] X NA 12SPSformaula: 8% = (70*29.33)/((70*29.33)+(500*24.46)+(901/2*22.4)); CPS formula: 50.2% = (500*24.46)/((70*29.33)+(500*24.46)+(901/2*22.4)); HBP formula: 41.4% =(901/2*22.4)/((70*29.33)+(500*24.46)+(901/2*22.4)) 84 National 100% = [ 1 X 1 X 1 ] X (1/3) Level Table 6: Scaling down national administrative costs to fit SIEF's 13 provinces model Scale Percent of total Ratio of CPS Ratio of SPS Ratio of HBP to All programs equally administrative Schools to schools to national HBPs. weighted costs allocated national total national schools 13 48% = NA X NA X (1671/2763) X NA provinces- low 13 56% = [(1909 / 3132) X (1629/3413) X (1671/2763)] X X (1/3) Provinces- mid 13 61% = (1909 / 3132) X NA X NA X NA provinces- high National 100% = [ 1 X 1 X 1 ] X (1/3) Level Table 7: Support cost assumptions (by cost ingredient) INGREDIENT ASSUMPTIONS HIGH-END / LOW-END ASSUMPTIONS National TA for Lump-sum fee $99,000 taken from Project Appraisal High-end estimates for SESSP assume supporting ECED Document budget that outlined expected costs for the consultants only assist on SESSP schools SESSP loan provided by the World Bank. Costs amortized (an allocation of 100%). over 3 years. Low-end estimates amortized over 6 years Other consultants World Bank project appraisal document total budgeted cost for project management consultants (PMC),: National IT Officer for the, $49,500; National Translator, $49,500; International Technical Advisor, $396,000; National Procurement Consultant, $132,000. The same method is used for the “National technical assistance supporting ECED,” however allocation percentages in all models divided in half to account for other non-intervention 85 activities. Central level ECE $350 USD per month cited by MoEYS as salary for all ECE High-end central level salaries raised to officer gross salaries officers. 44 Central ECE officers * 12 months. 450 (not likely that all ECE officers are paid the same rate) Provincial level ECE $350 USD per month cited by MoEYS as salary for all ECE High-end provincial level salaries raised officer gross salaries officers. 104 provincial ECE officers used (approx. 8 per to 400 province based on interviews) * 12 months. District level ECE $350 USD per month cited by MoEYS as salary for all ECE officer gross salaries officers. 102 district ECE officers used (based on number of districts in each provinces and assuming 1 officer per district based on interviews) * 12 months.13 Computers Unit cost of $649.28 (based on MoEYS estimates) * 197 purchased under computers. This was an SESSP funded purchase. SESSP National and These ultimately represent a relatively small cost, so for Three of four trainings (non-yearly provincial level brevity, assumptions are left out of this table. There are 4 trainings) amortized at 6 years instead trainings different types of trainings that occurred. of 3 for low-range model. Overhead central Based on the total central level PB budget for 2017 level provided by the finance department = 3,389,100,000 riel, or $847,275. Though some of these costs may include other program costs (trainings, materials, etc…), it is the only number that was provided that can be used to roughly estimate overhead. 14 Though this is a very imperfect method, the PB budget does at least provide a rough sense of the scale of funding required to run these programs from the central level. Overhead – Drawing from a MoEYS Department of Finance power provincial level point on overall education spending, 15 MoEYS provincial 13District ECE officers’ salaries are assumed to be less likely to rise above the cited salary level since they operate at a lower level of government. More senior ECE officers at provincial and central level may be more likely to receive higher salaries. Therefore, we do not vary the high-end salary for the district level ECE officers, whereas we do raise it for central level and provincial level officers. 14Programmatic items that might be double counted using this method would be theoretically offset by other items that the yearly budget might exclude (amortized value of assets, trainings, and other materials required for operation). Additionally, salary costs are not included as part of the ECE budget, as they are distributed by the Ministry of Personnel. Though we count ECE officer salaries separately, the value of other indirect personnel who support ECE programs remains unknown. 15Ministry of Education, Youth and Sport Public Financial Management Reform, PFMR ESWG MEETING, Meeting Room A, National Olympic Stadium, Department of Finance, 06 June 2018, 8:30AM, Phnom Penh, Cambodia 86 level education funding is divided by spending at central level (2086.5/297.5)16 and multiplied by ECED spending at the central level ($847,275) = $5,942,317 Overhead – district This is only used in the high-end models, level as is likely that provincial overhead calculations already account for district level budgets, as provinces distribute most funds to districts. Therefore, district overhead is not counted in the mid and low-end models to avoid double counting of provincial overhead costs. State preschools SPS schools often use primary school teachers who have higher salaries but who do not undergo a pre-service training for preschools. Though these teachers undergo formal education for primary school teaching, they often have no formal training preprimary education. SPS buildings are often larger or adjoined to a primary school building, and the standards for other furniture and equipment appear to be higher. Other large differences between SPS and CPS schools include the construction method (contractors for SPS, and community construction groups who are distributed funds for CPS), and the quantity and quality of equipment and material inputs. Parent contributions to material costs are also a good deal higher for SPS schools than CPS schools. The SESSP model only counts the SPS schools that were funded by the World Bank. Some ingredients, such as in- service trainings for preschool teachers, were provided to a much larger number of teachers through SESSP. In these cases, costs were only calculated for the 70 schools. In the case of trainings, this meant calculating the unit cost for training one teacher, and multiplying this by 70 teachers. Though a standard model is used for SPS schools, very different arrangements exist, including scenarios where SPS classes are held in primary school buildings. Table 8: State preschool assumptions (by cost ingredient) INGREDIENT SESSP ASSUMPTIONS PROVINCES ASSUMPTIONS HIGH-END / LOW-END ASSUMPTIONS Teacher salaries $254.92 USD per month based on $254.92 USD per month based on endline * 70 teachers *12 months = Endline * 1629 teachers (MoEYS $214,135 M&E data) *12 months = $4,983,221 16 100s millions riel 87 Classroom materials $2.25 USD per child based on $2.25 USD per child based on MoEYS yearly estimate * 70 MoEYS yearly estimate * 1629 classrooms * 29.33 children per teachers/classrooms (MoEYS M&E classroom (endline) = $4,629 data) * 29.333 children per classroom= $107,513 Procurement of SESSP Only: $55.22 per class Provinces: $55.22 per class (MoEYS Amortization schedules vary polyvinyl floor (MoEYS estimate) * 70 classes / estimate) * 1629 between 3 and 6 years. covering sheet amortized over 5 years = $733 teachers/classrooms (MoEYS M&E data) / amortized over 5 years = $17,990 SPS furniture $7,990 per class (MoEYS estimate) $7,990 per class (MoEYS estimate)* Amortization schedules vary * 70 classes / amortized over 5 1629 teachers/classrooms (MoEYS between 3 and 6 years. years = $111,865 M&E data) ms / amortized over 5 years = $2,603,175 In-service training for 70 teachers * 2.5 times per year * 1629 teachers (MoEYS M&E data) * pre-school teachers average of 6.5 days per training * 2.5 times per year * average of 6.5 $13.05 per day per trainee = days per training * $13.05 per day $11,874 per trainee (MoEYS data) = $276,334 SPS construction 70 classrooms * $62,521 (SESSP Provinces: 1629 classrooms * High-end vs. low-end: Costs construction data) / Amortized over $62,521 (SESSP construction data) / vary between amortization 35 years = $130,358 (In model, Amortized over 35 years = schedules; 40 years (low) 25 costs are disaggregated by type of $2,909,906 years (high). In low end building, 4 line items) estimate, quantity of schools is multiplied by .75 to account for uncertainty about shared buildings. School operating $100* disbursed 2 times per year * $100 * disbursed 2 times per year * Disbursements per year budget (SOB) 70 classrooms = $14,000 1629 classrooms = $325,800 multiplied by 4 for high-end (MoEYS expressed the desire to substantially increase the frequency of these payments, and that some schools do receive substantially more frequent SOB payments). Low end is based on actual SOB totals for 2017 provided by MoEYS which suggest a lower unit cost. In low-end, quantity of schools is multiplied by .75 to account for uncertainty about shared SOBs with primary schools (based on interviews with central level ECE officers). 88 Construction International Construction Consultant for Department of Construction; Costs are amortized over 3 consultants National Construction Consultant for Department of Construction; years (6 years in low-end National supervision Engineers for Department of Construction; National model). Site Engineers for Department of Construction; 10 National Site Engineers for Department of Construction. all salaries taken from budgeted amount in World Banks’s project appraisal document budget. The allocation percentage for SESSP and provinces model is determined by the same criteria specified at the beginning of the section, with the percentage split in half. Parents yearly Based on endline data that Based on endline data that Low-end does not include contributions to measures average contributions measures average contributions per this, assumes it is part of teachers salary per parent at $0.22 * 29.333 parent at $0.22 * 29.333 students teacher salary reported by students per classroom * 70 per classroom * 1629 classrooms $ teacher. classrooms in province = $213 10,713 (Provinces) (SESSP) Parents yearly Based on endline data that shows Based on endline data that shows contributions to contributions per parent at $76.07 contributions per parent at $76.07 teachers salary *29.333 students per classroom * *29.333 students per classroom * 70 classrooms = $72,116 1629 classrooms = $3,634,957 Parents yearly Based on endline data that shows Based on endline data that shows contributions to contributions per parent at $1.33 contributions per parent at $1.33 renovations *29.333 students per classroom * *29.333 students per classroom 70 classrooms = $1,261 *1629 classrooms =$63,539 Parents time and $4.69 per day is used as the $4.69 per day is used as the High-end and low-end effort to engage in agricultural daily wage (Asian agricultural daily wage (Asian estimates vary the time it parent teacher Development Bank & WFP) *40% Development Bank & WFP) * 40% takes out of a parents day meetings (imputed labor participation (baseline) * labor participation (baseline) * 2.76 to attend the meeting (.25 cost) 2.76 meetings per year (endline meetings per year (endline data) * days/meeting for low-end, data) * 0.5 days (time expended, 0.5 days (time expended, and .75 days/meeting for observational/interview data) * observational/interview data) * high-end) 29.33 students/parents per 29.33 students/parents per teacher/classroom (endline)*1629 teacher/classroom (endline) *1629 classrooms =$2,454 classrooms = $83,710 Community Preschools . Community construction working groups construct community preschools with funds distributed by the MoEYS Department of Construction. Teachers undergo 35 days of pre-service training, and unlike formal preschool are not usually formally trained. Salaries are significantly lower than in formal preschools and are paid by the Commune Council rather than the provincial offices of education. The SESSP model only counts the 500 CPS buildings constructed, and assumes there are as many teachers as there are CPS schools. 89 Table 9: Community preschool assumptions (by cost ingredient) INGREDIENT SESSP ASSUMPTIONS PROVINCES ASSUMPTIONS HIGH-END / LOW-END ASSUMPTIONS Teacher salaries $60.96 salary per month (Endline) * $60.96 salary per month (Endline) * 500 teachers *12 months = $365,742 1909 teachers (MoEYS M&E data) *12 months = $1,396,402 Classroom $125 per classroom (MoEYS estimate) $125 per classroom (MoEYS estimate) materials * 500 classrooms = $62,500 * 1,887 classrooms (MoEYS M&E data) = $235,875 CPS furniture $619 per class (MoEYS estimate) * $619 per class (MoEYS estimate)* Amortization schedules 500 classes / amortized over 5 years 1887 classrooms (MoEYS M&E data) vary between 3 and 6 = $61,900 / amortized over 5 years = $233,611 years. Hygiene materials SESSP Only: $87.16 per class (MoEYS Provinces: $87.16 per class (MoEYS estimate) * 500 classes = $43,581 estimate)* 1887 classrooms (MoEYS M&E data) = $164,475 Pre-service 667 CPS teachers17 * 1620.1 (MoEYS Per-day-per-teacher unit costs vary Amortization schedules training of CPS SESSP reported unit costs) amortized between $29 and $34. Total days for varied between 2 and 6 teachers over 3 years= $360,647 each of the 5 trainings vary between years. approx. 6 and 7.5 days. 3 year amortization schedule. Total costs of all 5 trainings = 653,78118 Pre-service 500 teachers * $369.90 per year per 1887 teachers (M&E data) * 2 days Province models vary training of CPS teacher = $184,948 per month * 4 months * $27.33 per frequency between 2 teachers19: day per trainee (MoEYS training data) and 12 times per year = $412,590 Training to CPS & $1994.47 per trainee * 64 trainees * HBE network 50% allocation percentage (half to (SESSP only)20 HBP and half to CPS) = $63,823 17(due to dropouts, proportion of attendees listed in SESSP unit cost data provided by MoEYS (1335) / by total schools for which teachers trained – 1000 * 500 schools being measured =667) 185 separate courses calculated separately based on more disaggregated MoEYS SESSP training data. Daily per teacher unit costs calculated and number of days are extrapolated from training data. 19 There remains uncertainty regarding the costs of in-service trainings in the provinces model. Officially, 2 trainings are supposed to occur per month, however, it is unclear if this means the district holds 2 trainings per month for different teachers, or each teacher attends 2 trainings per month. Furthermore, disaggregated training data for SESSP shows only one of these trainings covered by SESSP. It is unclear whether this was an “initiation training” or representative of the costs of each bi-monthly meeting. Whether this training occurs bi-monthly for every teacher, or is simply offered by the district twice a month, is also unclear. To account for this uncertainty, the high-end and low- end models vary the frequency of the training dramatically. 90 Training of sub- 310 trainees * $1,315 to train one $38.81 per trainee * 5 days * 412 Low-end estimates national trainers trainee (aggregate SESSP training trainees (all from disaggregated amortized over 5 years. on CPS (or titled data) / 3 year amortization rate = MoEYS SESSP training data) / 3 year CPS train the $135,939 amortization rate =$26,649 trainer)21 CPS construction 1500 trainees * $224 per trainee per 5223 estimated trainees (calculated Amortization schedule management training (aggregate ECE training data), by taking the ratio of SESSP total varied between 20 and training22 amortized over 30 years = $11,206 trainees / 500 schools * 1887 schools 40 years. in 13 provinces) * 2.46 days * $55.92 per trainee per day, amortized over 30 years (ECE training data from MoEYS) = $24,015 CPS construction 500 classrooms * $4,561 average 1887 classrooms * $4,561 average Amortization varied distributed to community for school distributed to community for school between 20 and 40 construction (SESSP construction construction (SESSP construction years. data), Amortized over 30 years = data), amortized over 35 years = $76,018 $286,892 School operating Until possibly 2019, this is not provided for CPS. Therefore, it is only included in Budget (SOB) the high end model at a $121 per school 4 times per year. Construction International Construction Consultant for DOC; National Construction Costs are amortized consultants Consultant for DOC; National supervision Engineers for DOC; National Site over 3 and 6 years Engineers for DOC; 10 National Site Engineers for DOC. all salaries taken from budgeted amount in WBG PAD. The allocation percentage for SESSP and Provinces model is determined based on the same criteria specified at the beginning of the Management / Administration / Overhead section, with the percentage split in half. Parents yearly Average contributions per parent Average contributions per parent Low-end does not contributions to (endline) $0.37 * 24.5 students per (endline) $0.37 * 24.5 students per include this (assumes it school materials classroom * 500 classrooms =$4,469 classroom * 1887 classrooms = $16,865 is part of teacher salary reported by teacher). Parents yearly Endline data shows contributions Endline data shows contributions per contributions to per parent at $47.80 *24.5 students parent at $47.80 *24.5 students per teachers salary per classroom * 500 classrooms = classroom * 1887 classrooms = $584,559 $2,206,128 Parents yearly Endline data shows contributions Endline data shows contributions per contributions to per parent at $1.08 *24.5 students parent at $1.08 *24.5 students per 20This is likely a management training for district or provincial officers working on the program. It shows up in the aggregate training data, but not in the disaggregated training data. Since the SESSP model only uses the aggregate estimates to avoid double counting trainings that may be titled differently, this only appears in the SESSP 21Discrepancies between these trainings crop up between the disaggregated and aggregate versions of SESSP training data. Provinces model uses disaggregated, SESSP uses aggregate. 22 Community teams are trained to carry out the construction of CPS schools. 91 renovations per classroom * 500 classrooms = classroom *1887 classrooms = $50,062 $13,265 Parents time and $4.69 per day is used as the $4.69 per day is used as the agricultural High-end low-end vary effort to engage in agricultural daily wage (Asian daily wage (Asian Development Bank & the time it takes out of parent teacher Development Bank & WFP) *40% WFP) * 40% labor participation a parents day to attend meetings (imputed labor participation (baseline) 2.43 (baseline) *2.43 meetings per year the meeting (.25 cost) meetings per year (endline data) * (endline data) * 0.5 days (time days/meeting for low 0.5 days (time expended, rough expended, rough qualitative data) * end, and .75 qualitative data) * 24.5 24.5 students/parents per days/meeting for high students/parents per teacher/classroom (endline) * 1887 end) teacher/classroom (endline) * 500 classrooms =$105,211 classrooms = =$27,878 Other Community This includes three separate line 3 line items’ unit costs are $73,79, Contributions items: one for community $147.58, $73.79 (interviews with (food, equipment, contributed food ($73,79 per teachers and community members) funds)23 school), one for community $123,051. contributed equipment ($147.58 per school), and one for community contributed funds ($73.79).. These were obtained from interviews with teachers and community members.$73,361. Teacher $72.50 * 500 teachers =$36,250 $72.50 * 1887 teachers $138,403 Unit cost is varied in contributions24 low-end model ($24) Home based program . The Home Based Care program involves trained “core mothers” that can provide intensive training through monthly meetings with parents and caregivers where they focus on good parenting, nutrition, and the importance of preschool. To cost this program, we needed to impute costs for the time core mothers, lead mothers, and mother members spent delivering the program. It is unclear whether major opportunity costs exist for the time they spend participating in the program. It is also unclear from the end line data, intervention observations, and interviews just how much time they spent engaging with the program. Therefore, frontline personnel costs vary significantly between high-end and low-end models. For core mothers, the low-end estimates assume only 2 days of work per year, and 1.4 days of work for lead mothers and mother members. The low-end estimate could be more accurate than the midpoint estimate in the case of the HBP program, and should be considered carefully. If another version of the program was 23These investments are calculated using interview data and endline data that provides a rough estimate for how often these contributions occur. Endline responses suggested that these contributions rarely occur, however the qualitative interviews suggested that, at least when they do occur, they can be substantial. However, due to the unreliability of the qualitative interviews, these are kept at a relatively low quantity in the ingredient valuation. 24 These are also based on qualitative interviews, which suggest that about $72.50 is invested per year by the teacher. 92 implemented, it would likely still use mothers from the community. These mothers may have very few opportunity costs associated with participation in the program, as many of them may not engage in productive activities outside the home. Table 10: Home Based Program Assumptions (by cost ingredient) INGREDIENT SESSP ASSUMPTIONS PROVINCES ASSUMPTIONS HIGH-END / LOW-END ASSUMPTIONS HBP Core mother 24 days per year * $4.69 for 24 days per year * $4.69 for 2 days are in low-end time and effort: agricultural wage labor (WFP / Asian agricultural wage labor (WFP / model, and 48 days are Development Bank) * 40% labor Asian Development Bank) * 40% used in high-end model. participation (baseline)* 901 labor participation (baseline) This accounts for both programs= $40,566 *1671 Provinces –M&E data on uncertainty about how number of core mothers = $75,235 much time core mothers devote to the program, and uncertainty about whether other productive activities are deterred through participation. HBP Lead Mother 4.2 days per year * $4.69 for 4.2 days per year * $4.69 for 8.2 days in high-end model, time and effort (on agricultural wage labor (WFP / Asian agricultural wage labor (WFP / and 1.4 days in low-end top of time spent Development Bank) * 40% labor Asian Development Bank) * 40% model (low-end model attending HBP participation (baseline) *4206 lead labor participation (baseline) assumes very little meetings) mothers (extrapolated from M&E *7801 Provinces =$61,302 additional work is deferred data) = $33,052 to participate in program). HBP Mother 4.2 days per year (Endline estimate 4.2 days per year (Endline 4.2 days in high end model, Member for time in meetings) * $4.69 for estimate for time in meetings) * and 1.4 days in low end agricultural wage labor (WFP / Asian $4.69 for agricultural wage labor model (low end model Development Bank) * 40% labor (WFP / Asian Development Bank) * assumes very little participation (baseline) * 10,124 40% labor participation (baseline) additional work is deferred mother members= $79,550 *18,775 mother members= to participate in program). $147,540 Learning Materials $125 per program (MoEYS estimate) $125 per program (MoEYS * 901 HBP SESSP = $112,625 estimate) * 1587 HBP programs (M&E data); = $198,375 Hygiene and $208.12 per program (MoEYS $208.12 per program (MoEYS stationary materials estimate) 901 HBP SESSP = $330,287 estimate) * 1587 HBP programs (M&E data)= $330,287 93 Pre-Service 901 core mothers * $1287.76 1671 core mothers * ($12 - $35 Only altered in low-end Training25 (MoEYS aggregate unit cost) / per day per trainee, depending on model to amortize over 5 amortized over 3 years = $386,756 which of the 7 trainings) * 3-5.5 years instead of 3. average days per training (depending on training) / Amortized over 3 years= $609,284 Core Mothers 901 core mothers * $226 (MoEYS 1671 core mothers * 17.93 per day Unit cost in high-end Monthly Session on aggregate unit cost) / amortized per trainee * 3 average days per estimate for Provinces Health and over 3 years = $203,726 training (depending on training) * model multiplied by 2. Nutrition 2 trainings per year / Amortized over 3 years= $179,754 Training of Data shows only 384 trained, unclear Aggregate data shows $128 to Provinces model doubles Pregnant Mothers whether training happens outside train one mother and 207 trainees unit cost to account for SESSP. * 3 days per week * $38.13 = $46,805 difference in aggregate and per trainee per day = $43,924 disaggregated training data. Training to CPS See CPS input. Same metrics used, 50% allocated to HBP &HBE Network Training on sub- 831 trainees (aggregate training 1545 trainees (disaggregated national core data) * 3.5 days per training * training data), * 3.5 days per trainer of HBP $49.71 per trainee per day / training * $49.71 per trainee amortized to 3 years = $48,195 / amortized to 3 years = $89,598 National technical $118,800 onetime charge $118,800 onetime charge assistancefor Home amortized over 3 years * 33% amortized over 3 years * a Based Care allocation percentage for SESSP 26= 60% allocation percentage $12,913 for Provinces = $12,913 RESULTS RELATIVE PROGRAM COSTS Tables 11-15 summarize the total costs and cost-efficiency results produced by SIEF’s six models. Estimates for the SESSP and 13 provinces models are included in separate tables. The 13 provinces estimates refer to a model of the programs funded largely by the MoEYS budget at scale. The SESSP estimates refer to a model of the programs with substantial external funding at medium scale (though still implemented within a nationally scaled set of programs). 25Similar to CPS pre-service training, the Provinces model uses disaggregated training data for all 7 courses undertaken by core mothers. The SESSP model uses aggregate data and simply uses a unit cost for training 1 core mother. 26Based on proportion of SESSP HBP to national HBP, and 13 provinces HBP to national HBP, as consultant presumably advised on full national program. 94 However, since much of the data used to value cost ingredients was shared between the models, analysis of costs at different scales and under different funding models is incomplete at best. These are estimates of average costs; they are not estimates of the marginal cost of adding an additional school or program or the costs to expand the programs in future years of the program. Rather, they are retrospective estimates of the total cost and average unit costs for programs implemented in the 2016-2017 school year, with some data taken referring to the whole 2014-2017 funding period for SESSP. However, retrospective costs remain a guide to future costs, and SIEF’s cost models can be adapted to build prospective cost models based on MoEYS budget plans. Tables 9-13 can be used immediately to understand the relative efficiency of the three programs at delivering services to beneficiaries. Further discussions with MoEYS will be required to estimate future years’ costs, and to improve cost data collection to produce real-time estimates that can aid decision makers on an ongoing basis. Since adding the HBP program did not lead to any positive impacts on preschool enrolment or child development, it is clearly not a cost-effective way to enhance the impact of the CPS program. Therefore, we estimate only total costs and cost-efficiency metrics in this note (average cost per child per year). In the SESSP model, the midpoint estimates for average cost per child (cost-efficiency) are $426 per child per year for SPS, $256 per child per year for CPS, and $193 per child per year for HBP. In the 13 provinces model, costs decrease for SPS and CPS to $370 and $186 per child per year respectively. However, the HBP average cost actually goes up in the provinces model to $221 average cost per child per year. This is because of lower enrollment per community at scale. In the SESSP model, average costs range from $355 to $663 for SPS, $200 to $443 for CPS, $131 to $338 for HBP. In the provinces model average costs range from $331 to $497 for SPS, $156 to $292 for CPS, and $172 to $360 for HBP. Cost-efficiency gains are most significant for CPS in the provinces model, are small for SPS, and are negative for HBP. The reason for larger cost savings for CPS is in part due to the difference in the way training costs were estimated for the provinces model vs. the SESSP model. The provinces model looks at disaggregated training data that suggests lower per unit costs for CPS and HBP trainings. 95 Table 11: Total Costs of providing one-year of SPS, CPS and HBP to one child (SESSP Model) Total program Total program Total program Children Average cost per Average cost per Average cost per cost cost cost child27 child child PROGRAM (High-end (Midpoint (Low-end (High-end (Midpoint (Low-end estimate) estimate) estimate) estimate) estimate) estimate) 1,361,164 874,136 729,356 2053 663 426 355 SPS 5,416,300 3,136,743 2,442,978 12,230 443 256 200 CPS 3,421,464 1,955,217 1,323,953 10,124 338 193 131 HBP Table 12: Costs of providing one-year of CPS, SPS and HBP to one child (13 Provinces Model) Total program Total program Total program Children Average cost per Average cost per Average cost per cost cost cost child child child PROGRAM (High-end (Midpoint (Low-end (High-end (Midpoint (Low-end estimate) estimate) estimate) estimate) Estimate) estimate) 23,751,721 17,682,272 15,821,717 47,78328 497 370 331 SPS 13,477,845 8,578,427 7,215,417 46,15729 292 186 156 CPS 6,749,879 4,147,262 3,235,969 18,77530 360 221 172 HBP 27 Total cost / children 28 Number of SPS classrooms in IE provinces (1629) * average number of students per classroom (29.333) 29 Number of CPS classrooms in IE provinces (1887) * average number of students per classroom (24.5) 30 Number of HBP programs in IE provinces (1672) * average number of mothers per program (22.47) 96 Table 13: Cost range – 13 provinces model Table 14: Average cost per child range – SESSP model 97 Table 15: Maximum and minimum estimates for cost per child per year (both models) Maximum Cost per Minimum Cost per Source model for estimate PROGRAM child per year child per year Maximum: SESSP $669 $331 SPS Minimum: IE Provinces Maximum: SESSP $443 $156 CPS Minimum: IE Provinces $360 $131 Maximum: IE Provinces HBP Minimum: SESSP model COST STRUCTURE It is also useful to break down program costs by ingredient category to identify the largest contributors to costs. This can help program managers identify cost saving strategies. Table 16 describes each cost category, and tables 17-18 show the percentage breakdown of costs for the midpoint model for SESSP funded programs and programs in the 13 impact evaluation provinces. Frontline personnel costs are higher for SPS in both models, at 25% (SESSP) and 29% (provinces). CPS and HBP have relative low frontline personnel costs by comparison: 15% (CPS) and 8% (HBP) in SESSP, and 20% (CPS) and 7% (HBP) in the provinces model. At small scale, other cost factors represent a larger portion of costs for SPS. At larger scale, frontline personnel costs for SPS become a larger portion of overall costs due to the high salary rates of SPS teachers. However, training costs are essentially non-existent for SPS schools. There are no pre-service training costs for SPS teachers, whereas CPS and HBP have quite high training costs: 19% (CPS) and 27% (HBP) for SESSP, and 8% (CPS) and 18% (HBP) in the provinces model. SPS teachers are fromally educated as teachers, and have usually undergone some tertiary education (This cost of this education is not included in this cost model). CPS and HBP teachers are often trained for the first time in these ECED programs. They also undergoe more extensive in-service training, which is also reflected in the cost structure pie charts. However, CPSand HBP training costs decrease at scale as other items like program materials and frontline personnel costs take on a larger portion of total program costs. Normally you would expect administrative/overhead costs to decrease as a percentage of total program costs at scale, however this only occurs for SPS schools. For CPS schools, administrative/overhead costs stay the same as a portion of total program costs, and for HBP overhead costs actually increase as a percentage of overall program costs. This is because HBP is less efficient overall at scale due to lower beneficiaries, and administrative/overhead costs as 98 calculated in our model are affected by the number of total HBP programs, not total beneficiaries. This highlights the need for better data on administrative and overhead costs. Future efforts at cost collection may find that administrative costs do not increase in a linear fashion with the number of HBP programs under management. It is more common to find that there are marginal administrative/overhead efficiency gains as a program is scaled. As might be expected, construction costs for SPS schools are much higher than for CPS schools. This suggests that aside from teachers salary, community construction methods generate substantial cost savings over privately contracted construction schemes. Materials on the other hand appear higher for CPS schools, but this is just as a proportion of overall costs. Classroom materials were estimated by MoEYS to cost about the same for SPS and CPS schools, and parents actually contribute more to classroom materials for FPS schools than for CPS schools. Additionally, SPS schools have higher equipment costs (this is largely due to furniture and polyvynal flooring installations which we counted as equipment). Table 16: Cost categories Personnel – Frontline All personnel in communities where programs are delivered who work directly to provide the service Administration / Overhead All assets, resources, services and facilities without required to implement program Equipment Cars, machines, fixtures, other fixed assets used to deliver program Monitoring and Evaluation and in-service Trainiings for ECE officers or ongoing trainings for school training personnel Materials Materials, ongoing procurement of resources for month- to-month operation Training – Frontline/direct delivery Pre-service trainings, other formal trainings for frontline personnel Construction Costs All costs related to construction of permanent buildings 99 Table 17: Cost structure of SESSP programs 31 31 Midline estimates only 100 Table 18: Cost structure of programs in 13 provinces 32 32 Midline estimates only 101 DISCUSSION SUMMARY SIEF undertook cost collection efforts for SPS, CPS and HBP programs with the goal of estimating the total cost and cost-efficiency of these programs. This report summarizes the total cost and cost-efficiency (average cost per child) of all three programs. The estimated costs relied on all available budget and financial data, M&E data, program implementation data, and data from field observations and discussions with MoEYS officials over the course of six months. To account for data limitations, we estimated a range of costs. We separately estimated costs for SESSP funded interventions and costs for all SPS, CPS and HBP schools and programs in the 13 provinces corresponding to the experimental sample of an impact evaluation of the CPS program. Varying our assumptions about missing data, we generated additional high-end and low-end estimates. This report also presented breakdowns of the cost structure of programs by spending category. The cost-efficiency results show that it costs $331-$699 per year to offer preprimary instruction to a child in SPS schools, while it costs $156-$443 per year in CPS schools. Adding the HBP program to the CPS services added an additional annual cost of $131-$360 per child. The wide-range of estimates suggests costs could vary based on a number of factors, such as the frequency of construction needs, the willingness of communities to invest in programs, the existence of external funding, the scale of the program, and the administrative resources expended by central, provincial and district governments. The current cost models do not isolate startup costs. Construction costs are amortized over the lifespan of the projectand therefore appear quite small. Knowing average costs per child estimates can support the government of Cambodia objective to expand ECED services to a larger population. Programs with lower average costs, such as CPS, might prove an efficient route to enrolling a greater number of children in preschool, though the relative effectiveness of CPS and SPS schools remains unknown, and decisions should ultimately seek to understand the cost-effectiveness of improving child development outcomes. These estimates can also be used to help the ECE Department at MoEYS to ensure program sustainability, and to make programmatic decisions that help improve quality and efficiency. Cost structure pie charts and estimates of specific cost ingredients can help ECE officials’ budget for their programs and make strategic decisions about how to adjust implementation to save money and improve quality. For example, training costs for HBP and CPS are very high and may actually present an opportunity for cost savings. SPS on the other hand has almost no trainings costs and could potentially benefit from participation in some preschool training as many of the teachers are only trained as primary school teachers. Additionally, the relatively low cost of frontline personnel for CPS at scale could represent an 102 opportunity for quality improvement investments. Perhaps additional investments in teachers’ salaries, or pay for performance could improve program quality for relatively little cost. Finally, these cost estimates can inform the process of determining federal transfer levels to different ECE programs. Specifically, MoEYS officials can review the community and other frontline costs cited in this report and evaluate whether school operational budgets are covering a large enough proportion of the costs incurred by communities to operate ECE programs. For programs to remain sustainable, it is important that school operational budget transfer levels take into account the real financial burdens faced by communities implementing ECED programs. RECOMMENDATIONS The current set of estimates does not tell us which programs are most cost-effective at delivering ECED services. Though the estimates do compare cost-efficiency, per –child unit cost estimates should also be considered a first step toward a more complete cost collection and evaluation cycle. However, some preliminary recommendations can be offered regarding the scalability and sustainability of these programs, as well as future efforts at cost estimation: CPS schools are a cost-efficient alternative to SPS schools, especially at scale. Lower construction costs, teacher salaries, and parent investments are a few reasons the program is likely to achieve some efficiency advantages over SPS schools. However, it is critical that learning outcomes in SPS schools are measured. Cost-efficiency does not tell us which program has the best potential per dollar spent for improving child school readiness and developmental outcomes in Cambodia. Additionally, considering CPS was originally designed as a lower cost model, MoEYS officials should consider whether the estimated cost savings meet their expectations for more efficient preschool services. The Home Based Care Program may be unsustainable or prohibitively inefficient at low rates of participation . HBP does risk cost-inefficiencies if insufficient mother members are enrolled in the program. It appears that enrollment happened at a higher rate under SESSP funded programs compared to the larger sample of all home based programs in 13 provinces. Higher enrollment rates may be required to keep the program sustainable. MoEYS should make efforts to ensure that HBP program participation is at a sustainable level. Additionally, the very high cost of pre-service training for HBP, and the negative impacts measured in the impact evaluation, suggests that MoEYS should consider either ending HBP investments or overhauling the design of the program. Over $1 million dollars was spent to train 901 core mothers under SESSP over three years, which is a large reason for the programs high unit costs. MoEYS should consider adapting M & E systems to more routinely collect cost data. SIEF’s costing exercise should be considered a first step in an ongoing cost collection effort. These cost estimates should not be considered definitive, but rather a first step toward generating real-time and actionable data for ECED practitioners in Cambodia. Further engagement would not only help produce more accurate and precise cost estimates, but could also help institute tools and processes for producing disaggregated and intervention specific data that could help estimate the costs of other 103 MoEYS services. Additionally, future cost models could be pared down significantly in complexity if costs are measured for all programs nationally, which would necessitate only one cost model necessary instead of six. Future cost collection for ECED interventions should begin before program implementation and costs should be measured in real time. Engagement with the ECE monitoring and evaluation teams and financial teams is required to produce a cost collection system that can produce future iterations of these cost estimates with better accuracy. This could potentially be achieved through addition of a small number of new M&E questions, addition of a minimum number of codes or processes for tracking financial data, and periodic collaboration on costs from program managers. The ECE department could benefit from reviewing cost ingredients and the cost structure of programs to determine appropriate transfer levels for School Operational Budets (SOBs), as well as opportunities for cost savings and quality improvements. Cost structure breakdowns can also help program managers to strategically identify areas where they can save costs, or alternatively improve program quality through new investments. Breakdowns by cost category show that materials and other investments from the community are an expensive part of each program, whereas CPS and HBP frontline personnel costs are somewhat less expensive than would be expected at scale. Additionally, training costs are quite high for CPS and HBP schools. Though administrative/overhead costs are also significant, more research is needed to understand the value of support costs. One potential conclusion is that more increases in school operational budgets for CPS and FPS schools are necessary to purchase resources and materials. High costs of material resources are in part due to purchases made by teachers, parents, and community members for schools. Inputs supported by School Operational Budget costs might include parent and community investments and tuition payments, renovation costs, recurring program material costs, and other inputs to school maintenance and classroom support. Program managers could offset higher School Operational Budget costs by looking for cheaper unit prices and unnecessary expenses regarding material support for schools and HBPs. 104 COST ANALYSIS SUPPLE MENTAL MATERIAL CHALLENGES TO COST COLLECTIONS Retrospective data: As mentioned, much cost data was collected retrospectively in 2018 after the close of SESSP activities in 2017. Usually, the nature and structure of the data needed for economic cost analysis requires that research teams analyze the cost structure of a program before the start of an intervention (ex-ante cost analysis). Additionally, it is often necessary to collect real-time data on some cost ingredients, and to assist implementing organizations to adopt new reporting structures prior to the start of an intervention. For example, collecting costs retrospectively limited SIEF from identifying most shared administrative resources required to implement ECE programs. Intervention specific costs of vehicles, travel expenses, office spaces, equipment, personnel time and effort, and other major costs to district, provincial, and central level offices had to derived from “best guess” estimates. The only way to effectively measure the proportion of these costs attached to each of the three ECE interventions in question is to ask personnel to report on time and effort, usage rates, and other financial and non-financial data required to gauge the administrative and community resources required to run ECE programs. Additionally, some community costs had to be guessed, as no cost specific survey was conducted to identify data on community level resources. However, the impact evaluation did collect some critical cost information that allowed for measurement of a number of key cost ingredients, such as teacher salaries, and parent contributions to school resources. Complexity of administration and data: Estimations of total program costs are complicated by the fact that in Cambodia government investments and implementation occurs both vertically (geographic - community, district, province, federal offices) and horizontally (central level departments) across administrative levels. For example, at the federal level the Ministry of Personnel has data relevant to salaried employees working on ECE programs, while the MoEYS has data relevant to implementation and procurement for those programs. Within the MoEYS, there are 3-4 departments each with information necessary to complete a cost analysis of ECE programs. And below the central level, provincial, district and community level data is also necessary to conduct an informed cost analysis of these programs. The following table highlights the most significant ingredients where SIEF did not collect sufficient data. Future cost reporting should focus on collecting more comprehensive data on these ingredients in particular: Table 19: List of inputs with significant data gaps INGREDIENT CHALLENGES / METHOD OF ESTIMATION USED Overhead & Administration (direct and indirect) Overhead rate used. Range of estimates using best guesses / proportional resources in other categories. (Most gov. assets, travel expenses, other resources) 105 Best guess / qualitative interviews used to equally proportion costs to each intervention High end and low end estimates vary assumptions Pre-service Trainings Challenges matching to population of analysis Amortization rate / teacher dropout rates best guesses High end and low end estimates vary assumptions Time and Effort of Parents / Community members Endline data + best guesses High end and low end estimates vary assumptions Door-to-door visits No individual cost ingredients estimated for this. Some costs may be captured in other ingredients. Community resources / donations Endline data provides some data. However, overlap / double counting of some resources uncertain. Low end and high end estimates vary assumptions SPS costs Very little data was available for SPS 106 PROGRAMMATIC COSTS Programmatic costs are defined as frontline inputs required for direct delivery of the intervention, such as teachers, classroom materials, and in-service trainings. Programmatic costs exclude support costs, which include management, administrative and overhead costs which support the intervention above the level of direct community level implementation. Measuring programmatic costs separately might be helpful for ECE department practitioners involved in budget drafting and program design. Though support costs are critical for any economic analysis, they may not be as relevant for ministry practitioners who manage a frontline budget consisting of mostly programmatic costs. Programmatic costs are cited here to assist ECE department practitioners in better understanding the frontline costs that they engage with more frequently. One additional advantage of citing programmatic costs separately is that they may be more accurate. The inputs that SIEF had the most difficulty collecting largely consisted of support costs in the administration / overhead category. However, these estimates should not be cited as the full cost of the programs, nor should they be cited as the full government cost of the program. A complete analysis of intervention costs must include support costs, as they are critical to making any ECED intervention a reality. Programmatic costs are only cited her to give ECE department practitioners a quick snapshot of specific cost ingredients with which they are more likely to have discretionary control over. The following inputs are excluded in order to estimate program costs: 1) National and international consultants and advisors; 2) Training of ECE officers; 3) Overhead at central level; 4) Overhead at district and provincial level; 5) ECE officer salaries 107 Table 20: Total Costs of providing one-year of SPS, CPS and HBP to one child (SESSP Model) Total program Total program Total program Children Average cost per Average cost per Average cost per cost cost cost child child child PROGRAM (High-end (Midpoint (Low-end (High-end (Midpoint (Low-end estimate) Estimate) estimate) estimate) estimate) Estimate) 885,983 659,317 616,698 2053 431 321 300 SPS 2,876,007 2,110,828 1,747,768 12,230 235 173 143 CPS 1,366,494 1,185,771 780,342 10,124 135 117 77 HBP Table 21: Costs of providing one-year of CPS, SPS and HBP to one child (13 Provinces Model) Total program Total program Total program Children Average cost per Average cost per Average cost per cost cost cost child child child PROGRAM (High-end (Midpoint (Low-end (High-end (Midpoint (Low-end Estimate) Estimate) Estimate) Estimate) estimate) Estimate) 19,193,843 15,206,700 13,521,201 47,783 402 318 283 SPS 8,907,898 6,090,787 4,956,825 46,157 193 132 107 CPS 2,279,897 1,750,175 1,015,228 18,775 121 93 54 HBP 108 Table 22: Cost structure of programmatic costs for SESSP programs 109 Table 23: Cost structure of programmatic costs of programs in 13 provinces 110