Early Childhood Education Impact Evaluations for the Middle East and North Africa Summary of workshop discussions for PALESTINE Prepared by: Sonya Krutikova and Maja Capek Participants Najah Abusharkh, Supervisor of Kindergarten, Ministry of Education, Palestine Maja Capek, Development Economics Consultant, World Bank, US Rabiha Elyan, Director of Monitoring and Evaluation, Ministry of Education, Palestine Samar Hamad, Supervisor of Kindergartens, Ministry of Education, Palestine Sonya Krutikova, Associate Director, Institute for Fiscal Studies, UK Policy Problems The government of Palestine is committed to achieving universal access to high-quality public pre- schools for all children in Gaza and West Bank. While a gradual expansion has been ongoing since 2012, public pre-school coverage remains low with only 12% of all pre-school children attending a public pre-school. Further, until recently there was no rigorous, context appropriate curriculum which meant that there was significant variation in substance and quality of what was being done across different pre-schools. A new curriculum was launched in 2017 with the aim of streamlining activities across pre-schools and ensuring that all pre-school children irrespective of whether they attend a private or public pre-school receive high quality ECCE. This new curriculum promotes a holistic view of child development emphasizing several key dimensions including socio-emotional, cognitive and linguistic skills. All activities in the new curriculum map onto these different domains. Delegates emphasized that a key challenge to successful implementation of the new curriculum is that it relies on effective interaction and engagement between teachers and children. Based on observations of the Ministry, reports of teachers as well as preliminary findings of a USAID study by RTI International, pre-school children lack age-appropriate communication and interaction skills. A number of potential reasons were identified. First, many pre-school teachers apply traditional techniques which do not engage kids effectively. Many parents are in favour of traditional approaches with focus on development of “hard skills� including literacy and numeracy. They lack awareness of the importance of socio-emotional skills for child development and can even put pressure on pre-schools to pursue a rigorous hard-skills focused curriculum. Many parents also face especially stressful day-to-day circumstances and are unable to engage in the type of interaction which is productive for children’s socio-emotional development; children may therefore have incurred deficits in socio-emotional skills even before they start pre-school. Delegates emphasized that the lack of socio-emotional and interaction skills among pre-school children is a critical issue which needs to be addressed in order to ensure that the new curriculum is implemented successfully. There is potential to integrate training that targets quality of teacher interaction with children into the national teacher training programme as well as make quality of interaction one of the criteria on which teachers performance is assessed. However, (1) no such programme currently exists; and (2) it is unclear whether it is sufficient to target the teachers only when there are strong indications that this problem starts at home. The focus during the workshop was therefore on the design and rigorous piloting of a programme that would improve the quality of teacher and parent interaction with children and, therefore, children’s development and school readiness. Summary of evaluation design The team were interested in developing, piloting and evaluating a training programme for pre-school teachers that would focus on building their skills for interaction and productive engagement with children. Recognising that the home plays a critical part in child development, as well as, often, in shaping priorities of the pre-schools, the team felt that engagement of not only pre-school teachers but also the parents may be crucial for the success of the programme. They are, therefore, interested in testing two different approaches to designing this programme; (1) targeting pre-school teachers only; and (2) targeting pre-school teachers and parents. The training programme for both teachers and parents would require significant development, though the aim is to heavily adapt materials utilised in other contexts in collaboration with an expert with relevant experience. Though the precise intensity would be determined during this development and adaptation process, for the purposes of initial planning we discussed a programme that would last 2 months and consist of weekly sessions for pre-school teachers in local training centres and weekly session for parents at their children’s pre-schools. The experimental design discussed consisted of a two armed randomised controlled trial (RCT) with one arm receiving pre-school teacher training only, while the other an integrated pre-school teacher and parent training programme. The main final outcome of interest will be child development and school readiness. A comparison of the impacts in the two arms will allow to determine which approach is more effective at boosting these outcomes. The population of interest in this trial consists of pre-school children attending KG1 and KG2. The sampling unit is a pre-school and the sampling frame would include all of the 1,267 pre-schools across the 17 districts of West Bank. First, a stratified random sample of schools would be drawn from the complete list of pre-schools and randomly assigned to one of the two treatment groups or the control group. Given the small geographic area covered by the West Bank it may be feasible to draw schools from each of the 17 districts. However, the level of logistical complexity that this would entail would need to be considered carefully. A random sample of pre-school children would then be drawn from each of the sampled pre-schools. Ballpark sample sizes discussed during the workshop were a total of 300 pre-schools with 10 children per pre-school. Detailed power calculations are necessary before final sample sizes are determined. During the discussions the team identified a number of core features of the project. First, there is an interest not only in direct effects on targeted children but also in determining whether the programme has spill-over effects on siblings of these – especially in the integrated arm. The sample would therefore include, for instance, a younger and older sibling of the “target child�. Second, delegates emphasised the importance of learning about mechanisms for programme impacts and, therefore, the need for measuring not only child development but also potential mediators of the impacts including home and pre-school environments as well as quality of teaching and parent-child interaction. Finally, the team agreed that the project should rely on existing well-validated measures and training materials rather than inventing entirely new ones, recognising, however that many of the best existing measures and training materials will need to be adapted to the Palestinian context. Relevance to ongoing World Bank operations The Bank is preparing a new operation entirely focused on ECD in Palestine. The project is likely to include financing for the design and roll-out of a teacher training program and a parental education program—the impact of which can be evaluated through the proposed IE. A number of preparation missions for the Bank operation will take place between August and October. One suggestion is for the PI to join one or more of these missions to lay the groundwork for finalizing the evaluation design, and ensuring its full integration into the WB project. Client interest The delegates have extensive expertise in the field of education, knowledge of ECCE landscape in Palestine and awareness of key priority areas. The group worked very effectively together on exploring opportunities for productively embedding evaluation in ongoing policy development and implementation. The delegates were clear that the project developed in the course of the workshop was of primary interest and expressed strong interest in taking it forward, conditional on securing appropriate funding. They were confident that the proposed project would get Ministerial endorsement and sign-off. Next Steps In order to take this project forward, key next steps include: 1. Researcher trip to West Bank to gather further information, including mapping and analysis of relevant existing administrative and survey data, as well as continuation of project planning with the delegates (e.g. visiting a sub-set of districts to identify and work through potential barriers to implementation). Key to work with the delegates to deepen their understanding of what the evaluation project would entail at every stage, scale of survey work etc. 2. Review the literature on related training programmes and conduct discussions with experts in the child development field to identify appropriate partners for development of the training programme. 3. Review assessments adapted to comparable contexts and identify the extent of adaptation that would be necessary to measure final outcomes and mediators of primary interest. 4. Refine power calculations based on literature review and selected outcome measures. Finalise design and associated detailed project plan and budget.