92391 Learning in the Face of Adversity The UNRWA Education Program for Palestine Refugees Conference Edition: October 2014 i The findings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the International Bank for Reconstruction and Development / The World Bank Group and its affiliated organizations, or those of the Executive Director of the World Bank Group or the governments they represent. The World Bank Group does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply and judgment on the part of the World Bank Group concerning the legal status of any territory or the endorsement or acceptance of such boundaries. ii PREFACE Since 2009, as part of the World Bank–supported Middle East and North Africa (MENA) Regional Network for Education Research, the Palestinian Authority Ministry of Education (covering the West Bank and Gaza) has been examining learning outcomes at United Nations Relief and Works Agency (UNRWA) schools. It found that in 2007 UNRWA students outperformed students attending public schools by more than a year’s worth of learning in the International Association for the Evaluation of Educational Achievement (IEA)’s Trends in International Mathematics and Science Study (TIMSS) assessment. This was also found to be true in Jordan. While in 2011 both the UNRWA and public systems reflected declines in TIMSS scores, UNRWA schools still showed a significant student performance advantage. Given the resource-constrained administration and the various sources of adversity facing students and teachers — especially refugees, who often suffer disproportionate physical, emotional, and psychological disadvantages—the findings raise questions about what lies behind such results. This mixed methods research study was undertaken to better understand the reasons for success at UNRWA schools and their positive variation from comparable public schools. Econometric techniques were used to analyze international (TIMSS and Programme for International Student Assessment, or PISA) and national learning achievement data. Pedagogical practices and classroom time-on-task were observed using structured methods (Stallings model). Systems Approach for Better Education Results (SABER) tools were used to better understand the policies and implementation strategies for school and teacher management and for monitoring and evaluation. Additionally, qualitative data were collected and analyzed in line with an education resilience conceptual framework to better uncover factors that help students develop the skills to learn despite the adversities they face. The goal of this study is to allow us to better understand how a school system can operate efficiently under adversity. The results of this work will be useful in identifying relevant policies in the region. These findings will also be shared with other countries in conflict or post-conflict situations to share findings on how schools can remain effective, relevant and perform well in difficult contexts. iii Our community is interested in and appreciates learning. The school ensures the students like going there and enjoy the learning that happens. The goal of the school is to create well-educated students. My family also has a big role in learning, and they will be happy when I finish my learning and graduate from the university with high grades. Female student, Gaza iv TABLE OF CONTENTS EXECUTIVE SUMMARY: THE UNRWA STORY .............................................................................. 1 Main takeaways from the research ........................................................................................................... 3 Chapter 1: A Brief Introduction to UNRWA......................................................................................... 14 Chapter 2: The UNRWA Effect: Evidence from Test Scores ............................................................... 16 At baseline (without controls) UNRWA schools increase student performance .................................... 18 Do UNRWA schools increase student performance? ............................................................................. 21 This difference is confirmed under additional approaches .................................................................... 26 At the end, not all the difference is explained ......................................................................................... 28 Chapter 3: Supporting Teachers and Managing Schools ..................................................................... 30 UNRWA teachers are better supported .................................................................................................. 31 School autonomy and accountability represent areas for improvement for both................................... 35 Promoting high quality teaching and classroom time ............................................................................ 37 Conclusion .............................................................................................................................................. 43 Chapter 4: Learning and Resilience ....................................................................................................... 44 Education has a collective meaning and purpose for Palestine refugees in the face of adversity ......... 45 UNRWA staff understand and are able to model a positive identity and well-being ............................. 47 The school system supports students’ competence in the midst of adversity ......................................... 48 Learning is supported by many actors including teachers, students, peers, and family members ......... 51 UNRWA’s close partnership with the refugee community creates shared accountability ..................... 53 What is resilience in the UNRWA Education System? ........................................................................... 54 RELEVANCE OF FINDINGS ................................................................................................................ 58 REFERENCES ......................................................................................................................................... 60 ANNEX: METHODOLOGY................................................................................................................... 64 LIST OF FIGURES Figure 1: Sequential methodology for performance difference between UNRWA and public schools ........................ 2 Figure 2: Parents are more involved in UNRWA schools ............................................................................................. 3 Figure 3: Teachers at UNRWA schools are more satisfied with their jobs (percentage satisfied or highly satisfied) .. 4 Figure 4: Advantage in test scores for UNRWA over public schools, controlling for stated factors ............................ 5 Figure 5: Cost per student in Jordan’s UNRWA and public schools (US dollars) ........................................................ 5 Figure 6: More interaction and less lecturing in UNRWA schools ............................................................................... 9 Figure 7: Students benefit from a wide range of support that helps them learn despite adversity............................... 12 Figure 8: Comparison of TIMSS scores over time ...................................................................................................... 18 Figure 9: Comparison of PISA Scores between public and UNRWA schools for Jordan .......................................... 19 Figure 10: Magnitude of the difference in mean scores relative to public schools in NAfKE (Jordan) ...................... 20 Figure 11: Predicted mean score difference between UNRWA and public schools,................................................... 20 v Figure 12: Explained and unexplained portions of TIMSS scores based on regression decomposition ...................... 29 Figure 13: International comparisons in teaching time ............................................................................................... 38 Figure 14: Difference in activities between public and UNRWA classrooms ............................................................ 39 Figure 15: Important difference regarding the use of materials by school type .......................................................... 40 Figure 16: Percentage of classes in which students work on measurable activities .................................................... 41 Figure 17: Percentage of classes with quizzes or test preparation ............................................................................... 41 Figure 18: Incidence of students being off-task .......................................................................................................... 42 Figure 19: Rating of teacher competency by region and subject ................................................................................. 42 Figure 20: Overall lesson quality by region and subject ............................................................................................. 43 Figure 21: How students make sense and find purpose through education ................................................................. 46 Figure 22: The type of goals high performing students relate with doing well in school ............................................ 47 Figure 23: Types of academic guidance provided to students ..................................................................................... 49 Figure 24: Different sources of support on learning outcomes as perceived by UNRWA students ............................ 52 Figure 25: The five education resilience supportive mechanisms identified in UNRWA schools .............................. 57 Figure 26: The different ways in which schools support the resilience process .......................................................... 59 LIST OF TABLES Table 1: School management & autonomy, UNRWA and public systems comparison ................................................ 6 Table 2: Student background characteristics, based on TIMSS 2007 data .................................................................. 18 Table 3: Comparison of time use and level of Parental support and involvement ...................................................... 21 Table 4: Difference in performance between high and low values in characteristics related to teacher and school ... 22 Table 5: Results of the regression models (Jordan) ..................................................................................................... 24 Table 6: Results of regression models in WBG........................................................................................................... 25 Table 7: Comparison of Test Scores for TIMSS controlling for other factors using PSM .......................................... 26 Table 8: Comparison of Test Scores for PISA controlling for other factors and using PSM (Jordan) ........................ 27 Table 9: Comparison of test scores for NAfKE using PSM (Jordan) .......................................................................... 27 Table 10: Recruiting and attracting high quality teachers, UNRWA and public schools comparison ........................ 32 Table 11: Teacher expectations and guidance, UNRWA and public schools comparison .......................................... 32 Table 12: Teacher motivation, accountability & performance, UNRWA and public schools comparison ................. 33 Table 13: Minimum Training Requirements for Teachers, UNRWA and public schools comparison ....................... 34 Table 14: Opportunities for teacher career development, UNRWA and public schools comparison.......................... 34 Table 15: Quality of School Principals, UNRWA and public schools comparison .................................................... 35 Table 16: Management layers and accountability for student outcomes, UNRWA and public schools comparison 37 LIST OF BOXES Box 1: Who are the UNRWA Palestine refugees? ...................................................................................................... 14 Box 2: Data modeling method used in the study ......................................................................................................... 17 Box 3/4: The Model; Oaxaca-Blinder ......................................................................................................................... 28 Box 5: Systems Approach for Better Education Results (SABER) ............................................................................. 31 Box 6: What Matters in School Autonomy and Accountability? ................................................................................ 35 Box 7: Stallings Classroom Observation ..................................................................................................................... 37 Box 8: Qualitative data approach ................................................................................................................................ 45 vi ACKNOWLEDGEMENTS Many have provided valuable inputs and suggestions for this work. Particular thanks are due to the concept note peer reviewers: Raja Bentaouet-Kattan, Tazeen Fasih, and Suhas Parandekar. Mourad Ezzine, Ernesto Cuadra, Elizabeth King, and Robin Horn provided support. Emilio Porta was instrumental in setting the groundwork for this study by collecting important in-country data, Emily Brearley provided support in the literature review, and Shaista Baksh provided the logistics support for the study. Special thanks also to Halsey Rogers for his comments and feedback on the analysis and the write-up. Special thanks are also due to the staff and students of the UNRWA Education Program in the West Bank, Gaza, and Jordan, as well the public education systems of the Palestinian Authority and the Hashemite Kingdom of Jordan, who greatly facilitated the implementation of this research. This study would not have been possible without the support of the local field data collection team. Students, teachers, and school principals graciously allowed us into their classrooms and provided valuable insights into their often difficult lives and education experiences. A significant part of this work was possible only because of their willingness to share important insights and feedback about what helped them learn. Special thanks to Veronica Grigera, Sarah Mintz, Namrata Saraogi, Jessica Cross, and Nicky Corbett for reviewing and editing the final document. Finally, this work was made possible by the Rapid Social Response Program, supported by Norway, the Russian Federation and the UK Department for International Development (DFID) and the DFID–World Bank Partnership for Education Development. The team include Harry Anthony Patrinos, Husein Abdul-Hamid, Joel Reyes, Jo Kelcey and Andrea Diaz Varela. vii Acronyms EFA Education for All IEA International Association for the Evaluation of Educational Achievement MDG Millennium Development Goals NAfKE National Assessment for Knowledge Economy Skills NCHRD National Centre for Human Resources Development PISA Programme for International Student Assessment SABER Systems Approach for Better Education Results SES Socioeconomic Status TIMSS Trends in International Mathematics and Science Study UNESCO United Nations Educational, Scientific and Cultural Organization UNRWA United Nations Relief and Works Agency WBG West Bank and Gaza viii EXECUTIVE SUMMARY: THE UNRWA STORY Palestine refugees are achieving higher-than-average learning outcomes in spite of the adverse circumstances they live under. Their education system—the United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA)—operates one of the largest non-governmental school systems in the Middle East. It manages nearly 700 schools, has hired 17,000 staff, educates more than 500,000 refugee students each year, and operates in five areas, including the West Bank, Gaza, Jordan, Lebanon, and Syria. Contrary to what might be expected from a resource-constrained administration serving refugee students who continually face a multitude of adversities, UNRWA students outperform public schools in the three regions—the West Bank, Gaza, and Jordan—by a year’s worth of learning. This study uses a mixed methods research approach1, incorporating both quantitative and qualitative research to address the complexity of the research question and its exploratory nature, namely how do UNRWA schools continually and consistently outperform public schools? This study was prepared using the following tools, techniques, and data collection:  Econometric techniques were used to analyze learning achievement data, including international (TIMSS and PISA) and national student assessment data.  The Systems Approach for Better Education Results (SABER) tools and rubrics were used to assess different system components, such as teacher effectiveness, school autonomy, and student assessments.  Stallings classroom observations provided a structured method to compare teachers’ and students’ interactions.  Qualitative data collected through interviews captured the lived experiences of a sample of UNRWA students. These tools were applied through a concurrent research process (Figure 1), constituted through a mixed methods research design that led to integrated findings. It is important to recognize the methodological and practical limitations of this study to establish its relevance to other education systems and contexts of adversity. The UNRWA system covers five regions, of which this study examines three: West Bank, Gaza, and Jordan. Thus, the findings represent factors that appear to be working within a system, but they do not imply that the system as a whole is achieving positive results. That would require additional data collection and analysis for Lebanon and Syria. Nor do the findings attempt to negate or discount the incidence of falling standards in UNRWA schools in recent years. Moreover, the international assessment data point to UNRWA’s performance in relation to the public system in some of the host countries. But the data do not cover the inputs and processes in public school systems, which may differ from those in UNRWA schools, so they cannot be used to pass judgment on what these public systems may face. 1 Please see annex for detailed methodology. 1 Figure 1: Sequential mixed methods methodology design Diagnostics Analysis of and classroom Apply UNRWA Policy analysis econometric using SABER observations resilience over analysis using using Stallings framework Public learning data method Understand factors Analyze system Connect linkages Explore what is associated with the related issues to with environment Lessons happening in the difference in student identify policy surrounding the Learned classroom performance differences learner Source: Authors. International assessment data from 2007 was analyzed to assess learning outcomes of these schools while the explanatory data, to which the bulk of this study is dedicated, was collected in 2011. Results of 2011 test scores, however, demonstrated a decrease in learning outcomes for both the public sector and UNRWA schools. While overall standards have decreased, a controlled analysis of these results adjusted for the same range of variables as the 2007 results still illustrates UNRWA’s learning outcomes to be significantly higher for the West Bank, Gaza, and Jordan. This situation aligns to our understanding of resilience not as an outcome in itself but as a process in contexts of adversity. Both the 2007 and 2011 data suggest that the UNRWA system is gauging and promoting a set of assets and opportunities which better support its students to ‘navigate’ the adversities they face. Even in rapidly changing contexts, and with concomitant results, assessing these assets and opportunities is crucial to understanding why outcomes are achieved and how to support such processes. 2 Main takeaways from the research UNRWA schools continually and consistently outperform public schools by a margin equivalent to more than one additional year of learning. This is achieved as a result of the way these schools recruit, prepare, and support teachers; because of instructional practices and pedagogy in the classroom; and because of school leadership, accountability, and mutual support. This has created a distinguished learning community centered on the student. Of note:  UNRWA selects, prepares, and supports its education staff to pursue high learning outcomes.  Time-on-task is high in UNRWA schools, and this time is used more effectively than in public schools.  UNRWA schools have a world-class assessment and accountability system.  UNRWA schools are part of a wider community and culture of learning that supports the child and ensures that the education received is meaningful and relevant. 1. UNRWA schools increase student performance. Analysis points to significantly higher scores in relation to the mastery of curriculum content and cognitive learning domains. Moreover, more UNRWA students achieve the international benchmarks in math and science. On average, UNRWA students in Jordan, West Bank, and Gaza achieve scores 23 to 80 points higher than their peers at public schools, even after controlling for student characteristics and for urban or rural contexts. Performance of UNRWA students is higher than their peers despite two important parental and teacher characteristics. One would expect that socioeconomic status or parental education would be highly associated with performance, as documented in the education literature. But UNRWA students outperform their peers in public schools despite their socioeconomic disadvantages. Differentiating factors include student self- confidence, as well as parental support and involvement in school activities. Figure 2 shows how increased parental involvement and support leads to improved student performance. Figure 2: Parents are more involved in UNRWA schools 51% 24% 12% 9% High parental support High parental involvement in school activities Public UNRWA 3 Source: TIMSS Study 2007. One would also expect that teachers’ academic qualifications would be associated with student performance, but this is not the case. Students at UNRWA schools outperformed students at public schools despite having teachers with the same years of service and degrees completed as their counterparts in public schools. The most important factors in explaining this performance gap were teachers’ confidence in teaching the subject matter, job satisfaction, and on-going professional development and training. At UNRWA schools, 75 percent of teachers are either satisfied or highly satisfied with their jobs, compared with roughly 50 percent in public schools (Figure 3, below). Figure 3: Teachers at UNRWA schools are more satisfied with their jobs (Percentage satisfied or highly satisfied) 75% 51% Public UNRWA Source: TIMSS Study 2007. From the perspective of UNRWA students, teachers demonstrate their satisfaction and pedagogical confidence through their interaction with students inside and outside of the classroom. This is epitomized by the responses of two students in Gaza who, when asked what helps them to learn, responded: The teacher, the teacher is a role model. He is teaching us the right things and does so in an excellent way, using worksheets, media, and group work. Male student, Gaza Teachers always support me and make me participate in competitions such as math competitions in Gaza and Khan-Younis. And my English teacher always asks me to write essays in English because she wants to motivate me. I also always get rewards from school, which encourages me to work harder and to study harder: When you love your teacher, you always try your best to get high marks in her class. Female student, Gaza The UNRWA advantage in learning outcomes—despite apparent commonalities with public schools—is 25 points, a quarter of a standard deviation, or about a year’s worth of learning. Once other factors are controlled for, the differential declines but remains at 26 points, which is equivalent to a full year of learning (as shown in Figure 4, below). 4 Figure 4: Advantage in test scores for UNRWA over public schools, controlling for stated factors Controlling for general Controlling for school Controlling for motivation characteristics characteristics Source: TIMSS Study, Author’s calculations. These higher learning outcomes are achieved teaching the same curriculum as the public schools in their host countries, at a lower unit cost and with unstable funding that could decline from one year to the next. The cost per student in UNRWA schools is 20 percent less than in public schools in Jordan. Figure 5 demonstrates that the cost per student in UNRWA schools is much less as compared with public schools. Figure 5: Cost per student in Jordan’s UNRWA schools and public schools (US$) $800 $742 $700 $700 $600 $571 $573 $500 $400 $300 $200 $100 $0 2008 2009 UNRWA Public Source: Public Expenditure Review and UNRWA Annual Financial Reports 2008 & 2009. 5 2. UNRWA schools provide critical support for teachers and management. UNRWA schools use the curriculum of the host authority in a given Field of Operation. Therefore, teachers and associated school management structures, as the main mechanism for implementing the curriculum, can provide a source of variation to understand differences in learning outcomes. Based on this, we analyzed data on selection, preparation, and support of teachers as key learning support attributes of the UNRWA system. UNRWA is more successful at attracting and recruiting high quality teachers. UNRWA teachers’ colleges attract the best high school graduates to enroll free of charge and with guaranteed employment after completion and upon meeting the high standards set. Prior to recruitment, UNRWA teachers are also interviewed and are required to pass a written exam. As a result, the UNRWA system appears to have created a sound built-in quality control that is lacking in the public system. Clear expectations are set for UNRWA teachers, who receive guidance and mentoring services. UNRWA teachers are provided with explicit standards regarding what students must know and be able to do, and receive direction on how to achieve these standards. Teachers also benefit from guidance on how to effectively use their time in the classroom, and there is a clear and official focus on time on task. Teachers additionally support the development and implementation of school plans, design curricular material, and take part in the evaluation of activities. UNRWA has more mandated ongoing professional development and orientation for teachers than public schools. Although both public and UNRWA school systems require at least four years of college education to become a primary school teacher, UNRWA also requires classroom experience (built into the curriculum at their teachers’ colleges). In addition, all new teachers must complete a mandatory two-year intensive and structured training program focused on classroom instruction after they are hired. Teachers at UNRWA schools are supported by well-qualified, well-prepared, and experienced school principals. The UNRWA system invests heavily in preparing and developing qualified school principals through programs designed to support school-based leadership through coursework and mentoring. School principals are required to evaluate teacher performance and provide guidance for curriculum and teaching-related tasks. Student narratives point to a keen awareness of their principal’s abilities. A female student in Gaza spoke of her principal in this way: When we have difficulty understanding a teacher, she listens to us and talks to the teacher. She tries her best to solve the problem. She also maintains peace at school. She provides the atmosphere necessary for us to learn. She urges teachers to come to school early to give reinforcement classes. She also urges 6 students to be kind to each other and help each other…in addition to giving us advice about how to prepare for exams… Female student, Gaza 3. The UNRWA system has fewer management layers and is more accountable for student outcomes than public schools. UNRWA and public schools are similar in many aspects; both have limited autonomy with regard to budget, financing, and personnel. But they differ significantly in terms of assessment and accountability—UNRWA has a world class assessment system and is more accountable than the public schools (as shown in Table 1, below). Table 1: Comparison between UNRWA and Public Systems policies in school management & autonomy Dimension Public Jordan UNRWA Public Palestine Budget autonomy ●○○○ ●○○○ ●○○○ Personnel autonomy ●●○○ ●●○○ ●●○○ Participation finance ●○○○ ●○○○ ●○○○ Assessment ●●○○ ●●●● ●●○○ Accountability ●●○○ ●●●○ ●●○○ ●○○○=latent, ●●○○=emerging, ●●●○=established, ●●●●=mature or best practice. Source: World Bank SABER School Autonomy & Accountability 2011. UNRWA has established a well-defined and carefully implemented accountability system to assess and support teaching and learning. Assessments are a priority and are disseminated to teachers (to inform lesson plans and instructional practices) and to policy makers. Teachers are also required to participate in internal and external monitoring and evaluation. While both public and UNRWA evaluation systems include classroom observations and multiple criteria, UNRWA evaluations are more rigorous and frequent. Professional development and performance evaluations are requirements to remain in the teaching profession within the UNRWA system, with incentives for good performance and sanctions for poor performance. 4. UNRWA schools promote high-quality teaching and classroom time. UNRWA teachers, in addition to being well prepared and supported, operate differently in the classroom than those in the public system. The proportion of time spent on learning activities in UNRWA schools compares favorably with successful systems in developed countries. UNRWA schools appear to use this time to engage students through confidently led collaborative activities, discussions, and assignments, all promoting greater student participation. 7 There is less wasted time in UNRWA schools. Quality teaching and instruction are the main focuses at UNRWA schools. In Jordan, 90 percent of teachers’ working time is dedicated to teaching, compared with less than 60 percent at public schools. Teachers in UNRWA schools are also supported by mentors to improve their instructional capacities. There are fewer incidences of students being off-task in UNRWA classrooms as a result. Teacher practices at UNRWA schools reflect a high level of confidence to teach various subjects through a variety of methods. Teachers in UNRWA schools seem to have confidence to engage with multiple approaches rather than a single pedagogical tool. They facilitate learning through innovative techniques supported by multiple classroom materials and teaching aids. Illustrative materials relevant to subject content are also posted in classrooms. Teachers in UNRWA schools implement more interactive learning activities, discussions, and assignments. UNRWA classrooms provide greater opportunities for discussions and debates, as well as for asking questions than in public schools (Figure 6). There are more cooperative activities on projects and group work. Students are more engaged working on assignments and classwork such as solving problems. The focus is on interaction and problem solving, rather than on lecturing, as at public schools. 8 Figure 6: More interaction and less lecturing in UNRWA schools UNRWA PUBLIC Science Jordan 8% Gaza 5% Math West Bank 7% 6% Classroom Mgmnt Assignment 5% Jordan 12% Discussion 5% 13% Lecture Gaza Reading Aloud 9% 15% Arabic West Bank 12% 16% Jordan 7% 10% Source: World Bank Classroom Observations 2011. Students in UNRWA classrooms participate at higher levels than in public schools through structured activities. Students in UNRWA schools spend more time taking notes and collaborating on learning activities. In this way they are able to interact further with the subject material and may be more encouraged to synthesize, analyze, and prioritize. They are also able to interact more with each other. Indeed, peer academic support was an important theme in the student narratives. As one explained: … teaching is excellent; they give information in a good way to the students and nothing is left out of our learning. They ask us if there is any need to explain things again. The teachers make peer groups and each group has an excellent student in it who teaches the others. I was in one of these groups and had four students with me. Male student, Gaza One incentive for collaborative learning may be that graded activities assessed by the teacher are more common in UNRWA schools. Students in UNRWA schools are not only engaged in a variety of functions during class, but the percentage of classes in which students work on graded activities is higher. Student work is assessed, and feedback expected. Quizzes and exams are common, and teachers are more likely to 9 review material when preparing for a test in UNRWA schools. Such formative assessments were observed in about half of the classes in UNRWA schools in Gaza. 5. UNRWA schools successfully create a community and culture of learning. UNRWA’s positive learning outcomes seem to be supported by a close interaction between school and community actors, as revealed in the qualitative data for this study. UNRWA schools have become key entities that form part of a broader culture of learning for Palestine refugees. Education provides meaning and relevance for children and youth in contexts of adversity. This is grounded in close interactions among parents, family members, teachers, and principals. Such school-community structures seem to evolve organically and are supported by classroom practice, school management, and system priorities (through explicit policies, rules, institutions, and dedicated resources). The resilience lens used to analyze data from student interviews identified five mechanisms in UNRWA schools that promote learning in the face of difficult living situations. Education provides a collective meaning and purpose for Palestine refugees in the face of adversity. For the Palestine refugee community, with over 60 years of experience facing various causes of adversity, parental and community participation in school seems to be collectively valued and expected. This collective participation in quality education appears to have evolved as a way to face the difficulties experienced as a community. Students referred time and again to the social, cultural, and economic importance of being well-educated, as noted in the following indicative responses: What motivates me is that I have a goal in my life, which is to become a doctor and work here and serve my country, Gaza-Palestine. I want to become an important person in the society, and being a doctor means being an important person in our society. Male student, Gaza What drives me to learn is that I want to be an engineer or a doctor; and I don't want to be like my father, without a job or a certificate; and to improve the situation in my home, and increase my country's level. Male student, Jordan School is a place where we come to learn in order to achieve our goals in life, especially since we are under occupation. I tell all occupied countries that knowledge is the only way to face occupation. Female student, West Bank UNRWA staff understand and are able to model a positive identity and well-being for their students in the midst of challenge. The sense of community appears to be strengthened by the fact that UNRWA teachers originate from the same at-risk population, are part of the same communities as students, and have themselves been through 10 the UNRWA system. In sharing these difficult living situations, teachers are accessible role models for their students, providing them with motivation, a sense of responsibility, and kindness and support in times of need. Students recognize and value their teachers’ understanding and assistance in times of need. Teachers, therefore, provide real-life examples to demonstrate that education can contribute to a person’s identity and well-being in times of adversity; as one girl in Jordan explains: For me, when I see my teachers, I wish to be one of them…I wish to be one of the teachers. When I look at a teacher, I ask: What made her reach this level? I’m sure she studied, and it was hard. Our computer teacher… always encourages us; she gives us advice on how to study, and when I follow it, I achieve what I want…I want to say that when I look at my teacher , I wish to become like her. Female student, Jordan The school system supports students’ competence in the midst of adversity through academic guidance and socio-emotional support. Effective teaching strategies require socio-emotional support and a caring and supportive environment in which to learn. This dual integrated focus on academic and affective guidance provides students with a sense of control and competence through education, which they see as relevant to their contexts. A non- dualistic approach to education that integrates knowledge and values also appears to reflect the longer- term guiding principles espoused by UNRWA (a vision of empowerment and human development). This was aptly expressed by a female student in Jordan: Teaching is an essential thing. I have been at this school since first grade now. Teaching is really important and so is meeting other classmates and making friends. When it comes to the headmistress, she cares about discipline, psychological support, and students being responsible. When it comes to teachers, they always support us psychologically, teach us how to study, help us with our homework, help our mothers, set plans for exams, and prepare our lessons. In general, they guide us in many useful ways so we can study well. Female student, Jordan Learning is supported by many actors including teachers, students, peers, and family members. Because UNRWA serves a refugee community, relations between school staff and families extend beyond the boundaries of the school campus. Students report multiple sources of support in their learning through these relationships, especially from family members and peers. This echoes much of what is well known about healthy child development (Figure 7). Such community participation in the UNRWA system implies a wider organic process—beyond traditional top-down structures and models for school management. 11 Figure 7: Students benefit from a wide range of support that helps them learn despite the adversity (noting the number of times a topic was mentioned in student interviews) Sources of support for UNRWA students Teachers' Support 724 Self Support 437 School Support 437 Peer support 307 number of times mentioned by students Family and community support 263 Principal's Support 189 Counselor's Support 85 0 100 200 300 400 500 600 700 800 Source: World Bank interviews with students 2011. UNRWA’s close partnership with the refugee community creates shared accountability for learning outcomes. Within the explicit references made by students on the varied roles school and community actors played to ensure a quality learning experience, they also pointed to a sense of mutual accountability across the classroom, school, household, community, and even society. In this way, UNRWA exemplifies a shared and reciprocal accountability for education by monitoring and supporting each other’s actions: Interviewer: Do you have any thoughts, notes, or recommendations that will help you reach the highest levels of education, regardless of the difficulties that you talked about? Student: Provide a safe environment for the whole society—family, teachers, classmates, and neighbors. Simplify the lessons by explaining them in detail, and through hard work, I will learn and reach the highest levels of education. UNRWA’s performance in an adverse context (occupation, armed conflict, restricted movement, and protracted displacement) presents important lessons for other school systems. Specifically, the effective classroom practices of teachers, strong school leadership, sound assessment system, and shared accountability for learning are strong features. Equally important is recognizing the risks and vulnerabilities that students face and fostering relevant interactions across school and community actors to support learning and protection of students in such challenging contexts. 12 The five main lessons stemming from the study include: 1. Adequate teacher preparation appears to be a differentiating factor, but more can be done to expand on pedagogical innovations in the classroom. 2. Public schools in Jordan and Palestine can learn from the UNRWA system of assessment and accountability, as well as its teacher preparation and mentoring. 3. The fluctuating performance of UNRWA schools (as seen in the TIMSS 2011 results) is affected by rapidly changing contexts of adversity and continues to be of concern. While UNRWA is successful in many ways and appears to provide relevant classroom and system level support, more is needed to strengthen this process in order to manage (downward) performance fluctuations over time. This also points to the fact that addressing the external sources of adversity is paramount. 4. Other countries can build on the UNRWA model to create a community and culture of learning that recognizes the adversity of vulnerable groups and promotes collaboration amongst the school, the teacher, the parent, and the community, all focusing on student achievement and well- being. 5. The UNRWA example highlights the need to understand education resilience as a process phenomenon which may be gauged and subsequently promoted by a set of assets and opportunities across a school-family-community partnerships. Resilience theory offers a better understanding of how education systems can achieve this for a given at risk-population, while concentrating efforts to end any structural causes of adversity. 13 Chapter 1: A Brief Introduction to UNRWA The United Nations Relief and Works Agency (UNRWA) is a regional United Nations (UN) agency with a quasi-state function to provide basic education, health, relief, shelter, micro-finance, and community support to about 4.7 million registered Palestine refugees in the West Bank, Gaza, Jordan, Lebanon, and Syria. In the aftermath of the 1948 Arab-Israeli war, which forcibly displaced more than 700,000 Palestinians, the UN General Assembly (UNGA) sought to Box 1: Who are the UNRWA Palestine refugees? mitigate the humanitarian impact of the crisis and work toward a long-term political solution. A refugee status for UNRWA responds to a very UNRWA, established as part of this response, specific definition. In 1952, UNRWA defined its began operations in 1950. beneficiaries as “any person whose normal place of residence was Palestine during the period from June 1, 1946 to May 15, 1948 and who lost The Agency was envisaged as a temporary “stop both home and livelihood as a result of the 1948 gap” measure to cater to refugee needs until a conflict.” UNRWA registered refugees are political solution could be reached. It was also officially known as ‘Palestine refugees.’ This established as one component of a wider special definition excludes those who fled, were not deemed to be in need, or who fled to areas regime for assistance, protection, and reparations outside of UNRWA’s five areas of operations. for the refugees as defined under UNGA Res. 194 Nor are refugee women able to retain or pass on (III). Its specific task was to implement “public their status to their children if they marry a non- refugee. Only refugees in 1948 are officially works” programs aimed at the economic integration covered by the Agency, despite successive of refugees. But it began operations by taking over waves of displacement since then. Descendants the ongoing emergency relief activities for the of male UNRWA refugees do however inherit refugees, and 60 years later, it remains the only UN UNRWA’s administrative title. agency dedicated to meeting the needs of a specific Source: Bocco 2010. population of refugees in a given region. Over the years, adaptations of UNRWA’s services have been made possible by its flexible mandate, which facilitates rather than restricts the agency in its activities. UNRWA’s education program shares and in many ways defines this history because it was quickly established as a priority for UNRWA, and it remains the most significant program. It was also set up through a partnership with UNESCO that continues to this day, with UNESCO providing technical support through the funding of several key staff posts. The education program is managed by the Administration and Governance Unit at UNRWA Headquarters. Implementation is decentralized and based on project or program cycle management. Planning occurs in a fairly short timeframe, and the various components are strategically linked to each other through a common and harmonized framework of results. The current strategy for service delivery (for 2011–15) is set out in UNRWA’s Medium Term Strategy, the “blueprint for programmes and field operations .” The strategy provides the objectives, priorities, reforms, and new approaches for a six-year period, broken down into three biennia. These are in turn operationalized through Field Implementation Plans and Headquarter Implementation Plans. Across all of the strategy documents, the baselines, targets, and goals 14 are expressed in terms of human development. Reflecting UNRWA’s temporary status, it is funded through voluntary contributions of any amount from member states, which account for 95 percent of UNRWA’s income. 15 Chapter 2: The UNRWA Effect: Evidence from Test Scores UNRWA schools’ results in international tests present something of an education system paradox that merits further study. Specifically, an econometric analysis of available achievement data concurs that UNRWA students have a competitive advantage over students in public schools. Observed characteristics related to schools, teachers, and students using an estimated education production function explain some of the differences in learning outcomes. Differences nevertheless persist even after matching. Finally, regression decomposition supports the conclusion that while contextual factors explain part of the variation in scores, a significant part remains unexplained and more ‘latent’ factors need to be uncovered to support our understanding. UNRWA schools continually and consistently outperform public schools by a margin equivalent to more than one additional year of learning. Econometric analysis shows that there is a significant difference in test scores between UNRWA schools and public schools in Jordan and Palestine (West Bank and Gaza, or WBG). Evidence for the analysis 16 comes from several years2 of TIMSS, PISA, and national assessment results. In all cases, UNRWA schools in Jordan and WBG outperformed public schools. Notably, on the 2007 TIMSS assessment, UNRWA students in Jordan scored 494 and 541 points in math and science, respectively, compared to 427 and 482 points by public school students, more than two-thirds of a standard deviation higher and representing a significant difference in education outcomes. Also in 2007, UNRWA students in Jordan achieved close to the international mean in math and above the international mean in science. In WBG, UNRWA students scored 381 and 414 points in math and science, respectively, compared to 367 and 404 by public school students and showed more significant differences in Gaza. Given UNRWA’s status as an education system run for refugees, many of whom live in socioeconomic difficulty and contexts of ongoing violence and conflict, this merits further exploration and understanding. Specifically, the assessment data was analyzed with modeling techniques in order to identify factors which contribute to the difference in performance between the two school systems, as well as to measure the levels of explained and unexplained variation between them. Regression analysis was used to first test the degree to which school factors, teaching characteristics, and family factors explain the differences observed in the student achievement of UNRWA and public schools. Iterative regression steps were then used to demonstrate the association between school type and performance. After each iteration ˗ controlling for school location and socioeconomic characteristics, the change in the magnitude and significance of the coefficient for school type was assessed (Box 2). Box 2: Data modeling method used in the study Mathematics and science scores are modeled against school type and characteristics, as in equation: Y2007 = β0 + β1Type + β2 School +β3 Teaching + β4 Student + β5 Interaction + ε Where Y represents individual student test scores in TIMSS 2007, βo is the constant, β1 is the coefficient for attending UNRWA schools, β2 represents the coefficient for school characteristics, β3 represents the coefficient for teacher characteristics, β4 is the coefficient for student’s characteristics such as mother education or socioeconomic characteristics, β5 is used to assess how the school type is interacting with the different factors to affect scores, and ε is an error term. It is important to keep in mind the UNRWA student selection criteria, as UNRWA students have to possess a refugee identification card in order to enroll. Students in possession of a refugee identification document cannot be refused enrollment in UNRWA schools, though they also have the option to enroll in public schools in Jordan, West Bank, and Gaza. However, Jordanians and non-refugee Palestinians whose families are not registered as having been displaced in 1948 cannot enroll in UNRWA schools except in special circumstances (occasionally made if there are no public schools available in the area for non-refugees). Thus, there is no “selection” into UNRWA schools. 2 2011, 2007, and 2003 TIMSS for West Bank Gaza, while Jordan also participated in 1999. Both Jordan and Palestine also implement national assessments. In addition, Jordan participated in PISA in 2006, 2009, and 2012. 17 Beyond this criterion, the 2007 results showed that there are no major differences in background characteristics between UNRWA and public school students. This is corroborated by similarities in the wealth index3 based on overall family possessions and the level of education of students’ mothers. Overall, in the WBG, UNRWA students are slightly poorer but their mothers are slightly more educated than their peers in public school (Table 2). Table 2: Student background characteristics, based on TIMSS 2007 data Jordan WBG Comparisons Public UNRWA Public UNRWA Age (in years) 13.39 13.40 14.01 14.05 Female (%) 50 51 49 53 Mothers Education (%) Primary/no school 19 17 25 12* Lower secondary 14 11 19 19 Upper secondary 35 35 34 42* Vocational 18 24* 9 12 Tertiary 14 14 13 15 Wealth Index -0.22 -0.26 3 -0.07* * indicates statistical difference, P<0.05 Source: TIMSS study 2007. At baseline (without controls) UNRWA schools increase student performance In terms of the curriculum, UNRWA students consistently outperform their peers in math and science at public schools in Jordan and in the WBG (Figure 8). These results are based on comparative evidence from TIMSS, which helps demonstrate the extent to which eighth grade students have mastered curriculum content in mathematics and science that is common across participating countries. Only in 2003, in Jordan, was the difference between UNRWA and public schools not statistically significant.4 Figure 8: Comparison of TIMSS scores over time UNRWA PUBLIC 600 550 500 450 400 350 300 1999* 2003 2007* 1999* 2003 2007* 2003* 2007* 2003* 2007* Math Science Math Science Jordan WB and Gaza Source: TIMSS studies 1999, 2003, 2007. * statistically significant 3 For constructing the wealth index, we used the calculation methodology developed by Filmer (1988). 4 Focus group participants indicated that the drop in UNRWA’s 2003 performance could be due to a management crisis that happened around the implementation of the field test. 18 In terms of skills and competencies, UNRWA students consistently outperformed their peers in public schools in Jordan based on PISA (Figure 9). PISA assesses whether 15-year-old students in participating countries can apply their knowledge and competencies in reading, mathematics, and science to real world situations. WBG does not participate in PISA. Figure 9: Comparison of PISA Scores between public and UNRWA schools for Jordan 460 UNRWA PUBLIC 440 420 400 380 360 340 320 Math Science Reading Math Science Reading 2006 2009 Source: PISA 2006, 2009. Finally, in terms of knowledge economy abilities, UNRWA students demonstrate their advantage in terms of a Jordanian national assessment instrument to test students’ abilities in skills required for a knowledge economy (known as NAfKE).5 NAfKE assesses student skill domains related to communication and problem solving. The results of NAfKE’s 2006 and 2008 implementation for students in grades 5 and 9 (UNRWA does not have grade 11) were considered and scoring relates to the percentage of test items answered correctly. Findings were similar to those of TIMSS and PISA; though, with exception of math 2006 scores, the major differences occurred at higher grade levels. Students in UNRWA schools significantly outperformed their peers in public schools especially at grade 9 (end of cycle for UNRWA) in all content domains: Arabic, math, and science. On average, UNRWA students had scores one-fifth of a standard deviation higher than the scores of their peers in public schools. This suggests that students graduating from the UNRWA system have higher work-related skills than students graduating from public schools. Figure 10 shows the differences in test scores between the two systems in the two years of NAfKE implementation. Similar results are present in the national assessment in Palestine. 5 NAfKE was developed in Jordan by the National Center for Human Resources Development in collaboration with the Ministry of Education. 19 Figure 10: UNRWA advantage: Magnitude of the difference in mean scores relative to public in NAfKE (Jordan) 25% 20% 15% 10% 5% 0% Math Science Arabic Math Science Arabic Math Science Arabic Math Science Arabic -5% 5th Grade 9th Grade 5th Grade 9th Grade 2006 2008 Source: National Assessment for Knowledge Economy (NafKE) and National Centre for Human Resources Development (NCHRD), Jordan 2008. Performance of UNRWA students is higher than their peers despite two important characteristics. According to PISA 2009, globally 60 percent of within-country variance in reading performance is explained by the economic, social, and cultural status of the student. Contrary to international evidence, the difference in achievement between UNRWA and public schools goes far beyond this. While the general perception is that refugees have relatively less well-off socioeconomic backgrounds than non- refugee peers, resulting in lower learning outcomes, the opposite is true in the Palestinian context. Notably, the school type coefficient continues to be statistically significant and UNRWA schools had a 40-point advantage after controlling for socioeconomic status of children. Similarly, the significant advantage continued after controlling for other characteristics, such as location and student gender (Figure 11). Figure 11: Predicted mean score difference between UNRWA schools and public schools, controlling for general school characteristics 42 40 39 Location Socio-Economic Status Gender (F) Source: TIMSS, author’s calculations. 20 To what degree can this difference be explained once other school, teacher, and student characteristics are controlled for? Furthermore, how much of the difference in performance, to the advantage of UNRWA, is associated with UNRWA’s teaching and learning environment, given that they teach the same curriculum? Possible reasons include that UNRWA schools tend to enrich the curriculum with programs and activities that are not implemented in public schools (as revealed by field visits to schools). Family commitment and participation in the learning process may be another reason (as hinted to by mothers’ education status). Do UNRWA schools increase student performance? While UNRWA schools use the same curriculum as public schools, there are differences in teachers’ morale, training, support, and supervision. Using the teacher and the school questionnaires that accompany the TIMSS assessment, the differences between UNRWA and public schools in TIMSS 2007 were examined.6 To ensure systematic variable selection, school visits and focus groups were conducted in order to then build a hypothesis for testing. There are no significant differences between UNRWA and public school teachers’ completed degrees. But, on average UNRWA teachers report higher job satisfaction (75 percent compared to about 51 percent) than their counterparts in public schools and slightly more educators have a teaching certificate (81 percent compared to 76 percent). Another important area of difference was the instructional time devoted to math and science at school. UNRWA students on average have slightly more hours per week of instructional time studying math and science. They dedicate more time to solving problems and are assigned more homework than their peers in public schools. Also, all UNRWA schools acknowledged that they offer enrichment or remedial classes. This is not the case for public school students. More significant differences were found in terms of school principals’ perceptions related to parental support for education. UNRWA students have higher levels of parental support for student achievement and higher levels of parental involvement in school activities (Table 3). Table 3: Comparison of time use and level of parental support and involvement Comparison of time use and parental support Public UNRWA Math time as percentage of total instruction time 13.5 15.0 Percentage of time spent in problem solving 31.8 37.0* Math class hours per week 3.7 3.9 Frequency of math homework (weekly) 3.3 3.4 Parental support for student achievement (%) .13 .51* Parental involvement in school activities (%) .09 .24* * indicate statistical significance Source: TIMSS Study 2007. Various contextual factors that influence student learning were also tested to see if they explain some of the difference in scores between UNRWA and public schools. These factors include school, teacher, 6 We utilized 35 teacher questions and 22 school questions. 21 pedagogy, student, and home characteristics. In addition to location and general characteristics, overall school climate was considered (including the percentage of students who come from economically disadvantaged or affluent homes), whether or not the school offers enrichment and remedial courses in mathematics, and the frequency and severity of various problematic student behaviors occurring at the school. Teacher qualifications were also tested relative to their perceptions of job satisfaction, parental support and involvement, students’ desire to do well in school, regard for school property, how well prepared they feel to teach topics included in the TIMSS exam, and their expectations for student achievement. Pedagogical factors included percentage of time spent on problem solving, frequency with which the teachers ask students to do various content-related activities in mathematics/science, frequency and amount of homework, and frequency with which the teachers give tests or examinations. As expected, these variables are positively correlated with learning outcomes. All correlations are statistically significant (Table 4 shows results based on separate regressions while controlling each for location, gender, and SES). Table 4: Difference in performance between high and low values in characteristics related to teacher and school Differences in performance Coefficient Teacher has high expectations for students 29.9* Teacher has license or certificate 10.6* Participated in math training in the last 2 years 0.7* High job satisfaction 22.7* Math time as percentage of total instruction time 1.1* Percentage of time spent on problem solving 1.6* Math class hours per week 6.8* Frequency of math homework 10.6* High parental support for student achievement 50.0* High parental involvement in school activities 37.5* School offers enrichment classes 33.3* School offers remedial classes 19.0* *indicate statistical significance at P<0.05 Source: Regressions based on TIMSS 2007. Finally, student factors included motivation in terms of their affinity for school, their perception of other students’ motivation in school, of teachers’ expectations for them, of how much they like and feel competent in math and science, the importance and value they attribute to math and science, and the level of education they expect to complete. Student engagement and perceptions of feeling safe in terms of whether or not they were subject to problematic behaviors by other students were also included, along with the frequency with which they completed various learning activities and out-of-school activities (summarized by frequency with which a student participates in various nonacademic activities and homework outside of school). 22 Each of the contextual factors was divided into one of two sub-factors: one that is changeable and could be adjusted by the education system for improvement and the other sub-factor which is exogenous and beyond school influence. A stepwise regression method was implemented to monitor the changes in the magnitude and significance of school type (UNRWA or public) on performance. Tables 5-6 show the different models for Jordan, West Bank, and Gaza, and the following represents the overall findings: 1. School type continued to be significant with additions of each of the contextual variables. Controlling for additional factors related to student, school, teacher, and pedagogy reduces the magnitude of the coefficient, but it continues to have a sizable and significant magnitude (between 23 and 42 points). 2. Student motivation and proactivity play a significant role in achievement. 3. Student behavior at school is crucial and the increase in incidence of problematic misconduct has a major association with low performance at the school level. 4. Teacher qualification, in terms of the degree completed, did not demonstrate a significant association with student performance. However, their level of confidence to teach the subject was associated with an increase of about six points on the average student score. 5. Teacher job satisfaction played a significant role in explaining the differences in student achievement. A student taught by a teacher with high job satisfaction tended to achieve close to 10 points higher than a student taught by a teacher with low job satisfaction. 6. Although schools were not expected to have high variation in terms of instructional time, the difference in time allocated for math or science at the school did not indicate significance effects on achievement. 7. However, pedagogical activities, such as a focus on problem solving, testing, and practical activities in class showed a significant association with student achievement. 23 Table 5: Results of the regression models (Jordan) Model Model Math 0 1 2 3 4 5 Science 0 1 2 3 4 5 * * * * * * * * * * * School Type 84.0 77.0 70.3 63.2 51.1 48.6 School Type 73.8 68.0 61.3 55.4 48.7 43.2* (UNRWA=1, public=0) (UNRWA=1, public=0) Student SES 5.7 6.6 9.1* 7.3 6.4 SES 8.2* 7.9 10.8* 8.5 7.6 Gender 18.2* 20.1* 23.4* 28.1* 26.5* Gender 25.1* 26.6* 29.0* 31.3* 28.5* Motivated 5.8* Motivated 8.2* Proactive 4.9 Proactive 6.7* School High percentage of -3.9 High percentage of -9.2* students from students from economically economically disadvantaged homes disadvantaged homes School offers enrichment 6.7* School offers enrichment 6.2 or remedial or remedial Severity of problematic -13.2* Severity of problematic -16.3* student behaviors student behaviors Teacher Qualification 2.5 Qualification 3.3 Confidence 5.8* Confidence 6.1* Job Satisfaction 7.8* Job Satisfaction 9.4* Pedagogy Time as percentage of 3.2 Time as percentage of 2.6 total instructional time total instructional time Problem solving 6.4* Problem solving 5.2 Assessment 7.8* Assessment 7.2* Activities 4.4* Activities 8.5* Homework 3.6 Homework 4.0 R-Square .10 .13 .16 .19 .28 .34 .11 .14 .19 .22 .29 .37 Source: TIMSS 2007. Note: Coefficients for student motivation, student proactive, school, teacher and pedagogy are from the full model (model 5) * Refers to statistical significance, p<0.05. 24 Table 6: Results of regression models in WBG Model Model Math 1 2 3 4 5 Science 1 2 3 4 5 * * * * * * * * * School Type 35.1 32.9 30.3 25.8 23.4 School Type 40.2 37.3 35.8 33.1 28.6* UNRWA (UNRWA=1, public=0) (UNRWA=1, public=0) SES 4.2 5.4 7.2 5.6 5.9 SES 6.1 6.9* 5.8 4.7 7.2* Gender 20.6* 23.4* 24.6* 26.1* 25.4* Gender 28.5* 27.4* 26.0* 30.2 27.7 Student * * Motivated 6.4 Motivated 9.8 Proactive 5.6 Proactive 6.2* Percentage of students from -4.3 Percentage of students from -7.5* economically disadvantaged economically disadvantaged homes homes School School offers enrichment or 8.7* School offers enrichment or 6.2 remedial remedial Severity of problematic student -15.1* Severity of problematic student -14.0* behaviors behaviors Qualification 3.6 Qualification 2.7 * Teacher Confidence 6.6 Confidence 5.7* Job Satisfaction 8.6* Job Satisfaction 7.8* Time as percentage of total 1.2 Time as percentage of total 2.3 instructional time instructional time Problem solving 7.3* Problem solving 4.3 Pedagogy Assessment 6.1* Assessment 5.0* Activities 4.1* Activities 6.6* Homework 2.5 Homework 4.7* R-Square 0.12 0.14 0.19 0.29 0.35 0.15 0.17 0.23 0.36 0.41 Source: TIMSS 2007, Note: Coefficients for student motivation, student proactive, school, teacher and pedagogy are from the full model (model 5) * Refers to statistical significance, p<0.05 25 This difference is confirmed under additional approaches Propensity score matching (PSM) method was used to construct a matching group for every student attending UNRWA schools (possible observation under treatment). Then, performance on the different assessments was compared with the non-treatment group (for students attending public school). School and student characteristics were used to construct scores with which the matching is implemented.7 The positive effect of attending UNRWA schools decreases but remains statistically significant in Jordan, while in the WBG the positive effects are higher than the ones observed in a linear regression (Table 7). It is important to note that the magnitude of the association is considerably high, especially in Jordan in 2007, when in math the estimate of the average treatment effect was 72 points while in science it was 65. Table 7: Comparison of Test Scores for TIMSS controlling for other factors using PSM Controlling for Without Control Other Matching Country Subject Year Characteristics MEAN P>|t| MEAN P>|t| MEAN P>|t| Jordan Math 1999 62.30 0.00 60.88 0.00 48.90 0.00 Science 1999 37.72 0.00 35.18 0.00 31.23 0.00 Math 2003 -0.25 0.98 -1.90 0.82 -5.62 0.63 Science 2003 2.09 0.79 0.64 0.91 -0.12 0.83 Math 2007 84.02 0.00 76.97 0.00 72.40 0.00 Science 2007 73.76 0.00 68.04 0.00 64.65 0.00 WBG Math 2003 24.02 0.00 20.73 0.00 12.79 0.00 Science 2003 19.32 0.00 15.00 0.00 20.56 0.00 Math 2007 25.21 0.00 17.50 0.04 20.63 0.01 Science 2007 19.52 0.02 13.84 0.10 17.11 0.02 Source: TIMSS, author’s calculations. ( ) 7 The scores are constructed by applying standard probability logic model: ( } ( ) ; where h(Xi) is a function of covariates. The average treatment effect is then assessed given the propensity score. 26 Similar results were seen for PISA in 2006; however, the difference disappeared in 2009 for math and reading (Table 8). Table 8: Comparison of test scores for PISA controlling for other factors and using PSM (Jordan) Difference UNRWA – Public Year Subject Without control Control Matching MEAN P>|t| MEAN P>|t| MEAN P>|t| 2006 Math 33.33 0.00 29.79 0.00 22.32 0.00 2006 Science 37.18 0.00 35.54 0.00 27.89 0.00 2006 Reading 27.75 0.00 24.37 0.01 15.20 0.02 2009 Math 27.05 0.07 23.09 0.07 17.09 0.26 2009 Science 27.93 0.06 26.46 0.03 26.93 0.09 2009 Reading 18.8 0.09 19.22 0.03 17.38 0.19 Source: PISA 2006, 2009. When using propensity score matching for NAfKE, the difference remains significant in 2006, and in 2008, it is only significant in science for grade 9. Table 9: Comparison of test scores for NAfKE using PSM (Jordan) Difference UNRWA – PUBLIC Year Subject Grade Without Controls Controls for Family Matching Background MEAN P>|t| MEAN P>|t| MEAN P>|t| 2006 Math 5 1.69 0.20 0.77 0.56 2.39 0.14 2006 Science 5 0.87 0.59 -1.89 0.29 -2.17 0.34 2006 Arabic 5 0.99 0.04 -5.38 0.01 -3.06 0.17 2006 Math 9 6.20 0.22 6.28 0.00 6.74 0.00 2006 Science 9 8.06 0.01 6.44 0.00 6.74 0.00 2006 Arabic 9 6.14 1.00 4.60 0.00 5.17 0.00 2008 Math 5 1.00 0.11 -0.58 0.62 -1.22 0.22 2008 Science 5 0.55 0.24 2.26 0.20 1.94 0.18 2008 Arabic 5 -1.22 0.03 0.23 0.91 1.08 0.72 2008 Math 9 3.43 0.77 3.79 0.00 1.74 0.26 2008 Science 9 7.36 0.43 7.62 0.00 5.3 0.00 2008 Arabic 9 4.2 0.00 3.29 0.03 1.9 0.17 Source: NCHRD Jordan 2009. 27 Thus the results of the linear regression and the Box 4: Oaxaca-Blinder propensity scores matching analysis suggest that the This technique was originally used in labor difference in learning outcomes between UNRWA and economics to decompose earning gaps and public schools remain after controlling for other observed to estimate the level of discrimination. It has been applied since to other social issues, contextual factors. In order to estimate the magnitude of including education. It is used in this study the explained variance associated with the observed to assess how much of the achievement gap characteristics relative to the unexplained difference is due to differences in characteristics between UNRWA and public scores, the Oaxaca-Blinder (explained variation) and how much is due to the school and institutional characteristics decomposition method was utilized (Box 3). that differentiates UNRWA from public schools (unexplained variation). This began Box 3: The Model with a specification and estimation of the The model specification used to estimate the production cognitive achievement production functions function for cognitive achievement is: that relate student achievement to individual and socioeconomic characteristics. The ( ) difference between UNRWA and public schools is then decomposed into an where Tij is the observed test score (from the TIMSS explained component (to account for student Math/Science test) of student i in school j, Fij is a vector of and family characteristics) and an individual student characteristics, Sij is a vector of school unexplained component (that represents the school and institutional variables), using the inputs, and єij is an additive error, which includes omitted traditional Oaxaca-Blinder decomposition variables including those that relate to the history of past method. inputs, endowed mental capacity, and measurement error. Source: Blinder 1973, Oaxaca 1973. At the end, not all the difference is explained The observed test score differential can be decomposed as: ( ) ( ) Where T is the standardized test score, Xi is a vector of student, and school related characteristics for the individual, β is a vector of coefficients, and UNRWA and public subscripts are identifiers of the TIMSS test score in Math/Science in year 2007. The overall increase in test scores can, therefore, be decomposed into two components. One is the portion attributable to differences in characteristics ( ) or to the performance of the UNRWA group of students , while the other portion is attributable to differences in the effects on performance ( ) of Public and UNRWA students derived from the same characteristics. The second component is the unexplained portion of the test score difference that may reflect certain unobserved family characteristics that are correlated with achievement, possibly related to household wealth or perhaps due to school and institutional characteristics. As shown in Figure 12, the explained portion of the difference in test scores between UNRWA and public schools due to characteristics is between 30 and 40 percent. Hence the majority of the difference is due to institutional characteristics and other factors. This leads us to believe that the largest part is most likely 28 due to other institutional and individual factors beyond the observed inputs and socioeconomic characteristics of students. Figure 12: Explained and unexplained portions of TIMSS scores based on regression decomposition 100% 90% 80% 70% 60% 50% Unexplained by observed characteristics 40% Endowments (observed 30% characteristics) 20% 10% 0% Science Math Science Math Jordan WBG Source: TIMSS Study 2007. To this end, further investigation of the overall system is needed. The next chapter identifies some of these factors that might shed light on the other piece of the explained variation in scores. An investigation of what happens in the classroom in the UNRWA schools relative to what occurs in the public schools is covered in Chapter 3, and in Chapter 4 those social and emotional supports (individual, community, and systematic) that support academic outcomes and the education resilience among refugee students are presented. Collectively, the findings present a more holistic view of UNRWA’s learning model, which may explain some of the differences noted here. 29 Chapter 3: Supporting Teachers and Managing Schools 30 The Systems Approach for Better Education Results (SABER) Box 5: Systems Approach for (Box 4) and Stallings classroom observations method were Better Education Results used to compare public and UNRWA systems in the policy (SABER) areas of school autonomy and accountability, as well as SABER is a World Bank tool used to compare education policies and teachers. institutions. Data collected can help countries systematically strengthen their UNRWA teachers are better supported education systems, as it uses diagnostics to evaluate the quality of education SABER-Teachers collected information on 10 core teacher policies against evidence-based global policy areas by administering a set of questionnaires. The standards. Two SABER domains were used in this study to assess policies on findings were then analyzed and education systems classified teachers and school management and according to eight core teacher policy goals: accountability and allowed for the 1. Setting clear expectations for teachers; classification of UNRWA and public education systems relative to their level 2. Attracting the best into teaching; of development around a set of policy 3. Preparing teachers with useful training and goals. experience; Source: World Bank 2013. 4. Matching teachers’ skills with students’ needs; 5. Leading teachers with strong principals; 6. Monitoring teaching and learning; 7. Supporting teachers to improve instruction; and 8. Motivating teachers to perform. Classifications were provided according to the following metrics. The results show that overall, in terms of teacher policy and its implementation, the UNRWA system differentiated itself from the two public systems in four areas: (1) leading teachers with strong principals; (2) establishing strong and clear expectations for teachers; (3) monitoring teaching and learning; and (4) preparing teachers with useful training and experience. UNRWA is more successful at attracting and recruiting high quality teachers. While the public and UNRWA systems require the same level of education for recruited teachers, the UNRWA system differentiates itself in two areas, attracting the best into teaching and stringent requirements to become a teacher. UNRWA attracts the best high school graduates to enroll in the UNRWA teacher colleges free of charge but with rigorous selection requirements. Attending the UNRWA teaching programs is very competitive as it targets high achievers on the high school national exam (Tawjihi). Also, before teachers are recruited at UNRWA schools they are interviewed and required to pass a written exam. 31 Table 10: Recruiting and attracting high quality teachers, UNRWA and public schools comparison West Bank Comparison of successfully recruiting and and Gaza attracting high quality teachers Jordan Public UNRWA (WBG) Public Overall goal ratings ●●●○ ●●●● ●●●○ Lever 1: Entry requirements set up to attract Emerging Advanced Established talented candidates? Lever 2: Teachers’ pay appealing for talented Established Established Established candidates? Lever 3: Working conditions appealing for Established Advanced Established talented applicants? Lever 4: Attractive career opportunities? Established Established Established Source: World Bank SABER Teacher Assessment Study 2012. There are clear expectations about the work expected of UNRWA teachers, who receive guidance and mentoring services. UNRWA teachers are provided with explicit standards regarding what students must know and be able to do, and the tasks that teachers are expected to carry out are officially stipulated. There is helpful guidance on the use of teachers’ working time. Ninety percent of teachers’ working time is dedicated to teaching while less than 60 percent of teachers’ working time at public schools in Jordan is devoted to teaching. At the same time, at UNRWA schools teachers’ official responsibilities include tasks related to instructional improvement with the help of a teacher mentor. The system also encourages teachers to support each other on school plans, design of curriculum material, and to take part in the evaluation of activities. Table 11: Teacher expectations and guidance, UNRWA and public schools comparison Comparisons Jordan Public UNRWA WBG Public Overall goal ratings ●●●○ ●●●● ●●○○ Lever 1: Clear expectations for teachers? Established Advanced Emerging Lever 2: Useful guidance on the use of Established Advanced Emerging teachers’ working time? Source: World Bank SABER Teacher Assessment Study 2012. 32 The focus on teachers also results in teachers who are more motivated to perform than their public school counterparts in Jordan and the WBG. Table 12: Teacher motivation, accountability & performance, UNRWA and public schools comparison Comparisons Jordan Public UNRWA WBG Public Overall goals ●●○○ ●●●○ ●○○○ Lever 1: Career opportunities linked to Emerging Established Latent performance? Lever 2: Mechanisms to hold teachers Emerging Established Latent accountable? Lever 3: Teacher compensation linked to Emerging Established Latent performance? Source: World Bank SABER Teacher Assessment Study 2012. The UNRWA system has a structured mechanism to hold teachers accountable. Teacher performance evaluations and professional development are requirements to remain in the field. Upon hiring, teachers are placed in a probation period during which mentoring and training is mandatory. After successful completion of the probation period, teachers are offered open-ended appointments and are granted extra compensation based on performance. Teachers can also be dismissed for poor performance, absenteeism, or misconduct such as child abuse. In the UNRWA system, teachers’ compensation is linked to performance through advancement and promotions from one level to another derived from an established ranking ladder. In the public system in Jordan, teacher classification exists (e.g., beginner to master); however, promotion is based mainly on attending training programs and less on classroom performance and student learning as found in the UNRWA system. UNRWA has more mandated ongoing professional development and orientation for teachers. Although both public and UNRWA systems require at least ISCED5A 8 to become a primary school teacher, UNRWA also requires classroom experience. After recruitment, all new teachers must go through a mandatory intensive two-year structured training program that is focused on teaching and instructional practice. 8 Bachelor degree or higher. 33 Table 13: Minimum training requirements for teachers, UNRWA and public schools comparison Comparisons Jordan Public UNRWA WBG Public Overall goal Ratings ●●●○ ●●●● ●●●○ Lever 1: Minimum standards for pre-service Advanced Advanced Established teaching education programs? Lever 2: Teacher-entrants are required to be Established Advanced Established familiar with classroom practice? Source: World Bank SABER Teacher Assessment Study 2012. The SABER assessment also found that UNRWA teachers are better prepared in the area of improving their instructional practices. Of note, support for teachers is central in the supervision system established by UNRWA, especially during the beginning of their service. School principals are also mandated to ensure that teachers who need support are directed towards appropriate professional development programs or assigned mentors. Table 14: Opportunities for teacher career development, UNRWA and public schools comparison Comparisons Jordan Public UNRWA WBG Public Goal Ratings ●●○○ ●●●○ ●●○○ Lever1: Are there opportunities for professional Emerging Established Emerging development? Lever 2: Is teacher professional development Emerging Established Emerging collaborative and focused on instructional improvement? Lever 3: Is teacher professional development Emerging Established Emerging assigned based on perceived needs? Source: World Bank SABER Teacher Assessment Study 2012. With regards to matching teachers’ skills with students’ needs, no major differences were noted between the public systems in Jordan and the WBG and UNRWA. Each of the systems was found to be ‘established,’ offering similar incentives for teachers to work at hard-to-staff schools. Teachers at UNRWA schools are supported by well-qualified, well-prepared, and experienced school principals. The UNRWA system invests heavily in preparing and developing qualified school principals. Structured programs prepare and support school-based leadership through specific coursework activities and mentoring programs. UNRWA school principals tend also to have extensive experience before becoming school leaders, more so than in public schools. They are explicitly required to evaluate teacher performance and provide guidance for curriculum and teaching-related tasks. 34 Table 15: Quality of school principals, UNRWA and public schools comparison Comparison of well-qualified, well-prepared and experienced school principals Jordan Public UNRWA WBG Public Overall Goal Ratings ●●○○ ●●●○ ●●○○ Lever 1: The education system invests in Emerging Established Emerging developing qualified school leaders? Lever 2: Principals are expected to support and Emerging Established Emerging improve instructional practice? Source: World Bank SABER Teacher Assessment Study 2012. School autonomy and accountability represent Box 6: What Matters in School areas for improvement for both Autonomy and Accountability? The assessment of autonomy and SABER-School Autonomy and Accountability was accountability is based on the depth and scope of policies and programs linked to school- implemented to identify the depth and scope of programs level control of financial and human and policies for school autonomy, learning assessment, and resources, the depth and scope of parent and school accountability across UNRWA and the public community participation in school system. management, and on the depth and scope of school assessment and school accountability. Overall, the SABER findings suggest little difference in Emerging consensus on school performance terms of school autonomy and accountability between indicates that school autonomy and UNRWA schools and their public counterparts. Notably, accountability are key enabling factors for with regards to financial autonomy,9 both UNRWA and mobilizing individual incentives to teach and public systems have a centralized budget over which to learn (Barrera, Fasih, and Patrinos, 2009). By deepening school autonomy and individual schools do not have much control (except for a accountability, schools can redefine their small part of the operational budget). Nor were any major incentive structure to create better conditions differences between the two systems noted in the area of for learning and teaching (Bruns, Filmer, and school autonomy in personnel management. School Patrinos, 2011). Autonomy and accountability are also linked to assessment practices related principals do not have authority in hiring and firing to teachers and learning (Banerjii, et al, decisions of teachers, as it is not managed by the school 2010). directors. Similarly, school councils (may include the school principals) do not have legal authority to hire and fire teachers, nor to hire and fire a school director (Box 5). Finally, in terms of the participation of school councils in school finance matters, there are no major differences in relation to the role of school councils (where they exist) in budget planning and management, operational efficiency, and teacher incentives. This is because budgets are prepared at the central level outside of the influence of schools, and school councils do not extensively participate in assisting the school in the preparation of the school budget, nor do they have legal authority to approve 9 School autonomy is defined as an increasing degree of local control of central fiscal transfers in terms of schools having authority to manage its operational budget, set and manage staff and teacher salaries, or raise other funds in addition to the transfers received from national or sub- national sources. 35 budgets. At the same time, school councils have no legal authority to supervise the implementation of the school budget, as it is the responsibility of central administration. In all areas, UNRWA and public schools were found to have a ‘latent’ status. The UNRWA system has fewer management layers and is more accountable for student outcomes. Despite the generally weak picture both systems present in terms of school accountability, some differences were noted in the UNRWA focus on the assessment of student learning, which was higher than that of the public system. During their orientation, new UNRWA teachers are trained to assess student achievement, and overall assessment data is monitored by UNRWA ‘policy’ makers. All major exams are prepared at the central level and are common across UNRWA schools in all fields of operation. Assessments are carried out in several ways including through standardized testing, inspections, and follow-up advice regarding identified issues at schools. Assessment findings are disseminated to teachers and used to identify schools and teachers who require additional support. The assessment data is also used to inform the development of teachers’ lesson plans and support provided by the system in terms of instructional practices. The UNRWA system also integrates a rigorous evaluation process as teachers are required to participate in internal and external monitoring and evaluation. While both public and UNRWA evaluation systems include classroom observations and multi-criteria reviews – covering knowledge of subject matter, teaching methods, student assessment methods, and student academic achievement – the evaluations are more rigorous and more frequent within the UNRWA system. In addition to their own assessment process, UNRWA schools also participate in national and international assessments, and the results of the school and student performance assessments are generally made available to parents. Some of these differences also extend to the ways in which the UNRWA system assesses the mechanisms in place to render parents, local governments, and society accountable for education outcomes. UNRWA schools have manuals regulating the use of the results of the yearly assessments of school and student performance by school administrations or councils. Moreover, the assessment of school and student performance is part of a national and in some cases regional assessment system with results occasionally being used to compare school performance with schools in similar conditions. 36 Table 16: Management layers and accountability for student outcomes, UNRWA and public schools comparison Comparisons Jordan Public UNRWA WBG Public Overall Goal Ratings ●●○○ ●●●○ ●●○○ Lever 1: Are there systems in place to assess Emerging Established Emerging student learning in order to inform teaching and policy? Lever 2: Are there systems in place to monitor Emerging Established Emerging teacher performance? Lever 3: Are there multiple mechanisms to Emerging Established Emerging evaluate teacher performance? Source: World Bank SABER School Autonomy &Accountability 2011. Conversely, in neither public nor UNRWA systems do school councils have the legal authority to hire external auditors to perform financial audits at the school. Given the important interconnections between different critical components of education systems in achieving quality learning outcomes, it will be important for UNRWA to build on these relative strengths. Promoting high quality teaching and classroom time The Stallings Classroom Observation snapshot was used to study and compare instructional practices in UNRWA and public schools (Box 6). Box 7: Stallings Classroom Observation This data collection tool samples short time segments of classes in process. Known as the Classroom Snapshot, (snapshot for short) it records every person in the classroom, what activities they are engaged in, and it also shows with whom they are engaged. Trained observers make a visual sweep of the classroom, view everybody and what they are doing, and with whom they are doing it. The results are then recorded in a grid that includes all the possible adults (teacher, teacher aide, and visitor), all possible combinations of students (one, two, small groups, and large groups), and 15 categories of activities (including discussion, lecturing, assignments, social interaction, discipline, classroom management, etc.). The snapshot takes place 10 times during a class period. The data collected shows that relative to advanced education systems (for example those in OECD countries), the time spent on learning activities in both UNRWA and public systems in Jordan and Palestine is positively comparable. Significant class time (about 85 percent) is spent on learning activities. In the three regions, teachers in public schools spend similar amount of time on learning activities as in UNRWA schools (Figure 13). Classroom instruction at UNRWA schools as well as at public schools includes a considerable amount of time on learning activities. Differences were however noted with regard to how this time is used. 37 Figure 13: International comparisons in teaching time 100% Learning Activities 90% Classroom Management 80% Teacher Off-Task 70% 60% 50% 40% 30% 20% 10% 0% Source: For Public Schools and UNRWA in West Bank and Gaza, authors’ calculation based on classroom observations; For OECD & LAC, Bruns 2011; OECD data from Abadzi 2009. There is less waste of teaching and learning time at UNRWA schools. UNRWA classes dedicate more time for discussion and assignments than classes in public schools. Conversely, classes in public schools spent more time on lecturing and reading than in UNRWA schools. The UNRWA model includes more opportunities for students to engage with activities and writing. The trend at the public schools is more focused on the teacher and the textbook. Teachers in UNRWA schools are using teaching aids and Information and Communications Technology (ICT) more than in public schools and teachers utilize the displayed relevant material in the classroom (on the walls and tables). For example, all science classes in UNRWA schools are conducted in a Science Lab. Science is taught in regular classrooms in most public schools. Though less often in UNRWA schools, in both systems some students are off-task roughly a quarter to a third of class time. Both the UNRWA and public systems include schools at different levels of instructional effectiveness. Signs of system effectiveness are highlighted by the higher percentage of competent teachers, as perceived by the observers, at the UNRWA schools. Additionally, snapshot observations reveal that UNRWA schools have less variation between schools and are more consistent in terms of instructional activities. This implies that pedagogical practices are more standardized in the UNRWA systems (likely through the preparation of teachers). This might also indicate effectiveness in teacher selection criteria. 38 Figure 14: Difference in activities between public and UNRWA classrooms UNRWA PUBLIC Science Jordan 8% Gaza 5% Math West Bank 7% 6% Classroom Mgmnt Assignment 5% Jordan 12% Discussion 5% 13% Lecture Gaza Reading Aloud 9% 15% Arabic West Bank 12% 16% Jordan 7% 10% Source: Author’s calculations based on Classroom Observations. While more could be done to improve the level of interactions in the classroom within UNRWA schools, observable indications suggest that the current pedagogical training and practices are something to build on in this respect. Teacher practices at UNRWA schools reflect a high level of confidence to teach the subject. Teachers and students in public schools read more from textbooks than in UNRWA schools. Students are writing and taking notes more in UNRWA schools than in public schools. Teachers in UNRWA schools use teaching aids and ICT more than in public schools while teachers in public schools spend more time writing on the blackboard. Figure 15 illustrates material use by region and school type. 39 Figure 15: Important difference regarding the use of materials by school type UNRWA PUBLIC 5% JOR 5% Science 9% GZ 8% 6% 16% WB 7% ICT Learning Aid JOR 14% 13% Blackboard Math 6% Notebook WB 14% 18% Textbook No Material JOR 5% 7% 5% Arabic GZ 7% 9% WB 6% Source: Author’s calculations based on Classroom Observations. Teachers in UNRWA schools implement more activities, discussions, and assignments. Observable differences at the classroom level included higher levels of student engagement in UNRWA class activities (as a percentage of class time). This was most common in Jordan and Gaza across the three core subjects. However, there appears to be large variation within UNRWA schools, particularly West Bank versus Jordan and Gaza, which could be due to a variety of factors, including 1)UNRWA schools are more effective in some areas than others, or 2) public schools in the West Bank are becoming closer to UNRWA in these practices than in the other areas. 40 Figure 16: Percentage of classes in which students work on measurable activities 70% 60% 50% 40% 30% 20% 10% 0% WB GZ JOR WB GZ JOR WB GZ JOR WB GZ JOR WB GZ JOR WB GZ JOR Public UNRWA Public UNRWA Public UNRWA Arabic Math Science Source: Author’s calculations based on Classroom Observations. In some cases it was noted that UNRWA teachers provided students with practice tests or dedicated time to review and test preparation. This was most frequent in Gaza, where it was observed in about half the classes in UNRWA schools (Figure 17). Figure 17: Percentage of classes with quizzes or test preparation 60% 50% 40% 30% 20% 10% 0% WB GZ JOR WB GZ JOR WB GZ JOR WB GZ JOR WB GZ JOR WB GZ JOR Public UNRWA Public UNRWA Public UNRWA Arabic Math Science Source: Author’s calculations based on Classroom Observations. This may help account for the lower incidence of students being off task in UNRWA schools. The snapshots also measured the incidence of students who were not engaged in learning and were either socially interacting with other students or remained completely uninvolved in any classroom-based activity. Results show that overall the incidence of the latter two possibilities was 10 percent higher in public schools as compared to UNRWA schools. The difference between UNRWA and public schools was highest in Gaza and Jordan (Figure 18). 41 Figure 18: Incidence of students being off-task 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Source: Author’s calculations based on Classroom Observations. Furthermore, at the end of each classroom observation, researchers were asked to provide an overall rating of the class and the teacher. The results presented in Figure 19 show that in the areas of teacher competency and mastery of class content, UNRWA schools scored higher in comparison to public schools for all subjects observed. Figure 19: Rating of teacher competency by region and subject 100% 90% 80% 70% 60% 50% Very High 40% High 30% Low 20% Very Low 10% 0% WB GZ JORWB GZ JOR WB GZ JORWB GZ JOR WB GZ JORWB GZ JOR Public UNRWA Public UNRWA Public UNRWA Arabic Math Science Source: Author’s calculation. 42 Researchers also rated the overall lesson quality. This resulted in a lower rating for public schools, while the percentage of schools achieving a superior rating was higher in UNRWA schools (Figure 20). Figure 20: Overall lesson quality by region and subject 100% 90% 80% 70% 60% 50% Very High 40% High 30% Low 20% Very Low 10% 0% WB GZ JORWB GZ JOR WB GZ JORWB GZ JOR WB GZ JORWB GZ JOR Public UNRWA Public UNRWA Public UNRWA Arabic Math Science Source: Author’s calculation. Conclusion Thus, based on our research we find that the UNRWA system is more successful at attracting and recruiting high-quality teachers; there are clear expectations about the work expected of UNRWA teachers, who receive guidance and mentoring activities. UNRWA also has more mandated ongoing professional development and orientation for teachers. These schools are well supported by well- qualified, well-prepared, and experienced school principals. The UNRWA system has fewer management layers and is more accountable for student outcomes. There is less waste of teaching and learning time, more interaction and student involvement in instruction, and teachers exhibit high level of confidence in teaching their students. Overall, school autonomy and accountability represent areas for improvement across both UNRWA and public schools. 43 Chapter 4: Learning and Resilience 44 The Palestine refugee youth interviewed for this study are aware of the adversities they face and expressed Box 8: Qualitative data approach that education helps them maintain hope for the future To understand the underlying mechanisms that UNRWA students considered important to (Box 7). Finding a purpose to their learning is not only learn in a context of adversity, the resilience an individual choice but promoted and supported by evidence comes from a purposeful sample of 74 the wider community. Students, with the support of interviews. The students were selected for their good academic performance in spite of their their teachers and their families, set feasible and relatively more difficult living situations in pertinent personal and community-related goals for comparison to their peers. their education. School staff, family members, and peers provide academic and socio-emotional support The interviews provide important insights into and are mutually accountable for results. Communities how the students succeed and the individual, community, and system level features that help do not do this alone. The UNRWA education system – them. The data is qualitative; the counts noted imbued with the Palestinian refugee context – strives to in some graphs are only contextual references offer relevant learning opportunities and equitable to the number of times a topic was mentioned opportunities in pursuit of these goals. We call this across interviews (it is not intended for statistical analysis). multi-layered process for achieving positive education outcomes in the midst of adversity “education resilience.” Five education resilience supportive mechanisms were identified in UNRWA schools. Education has a collective meaning and purpose for Palestine refugees in the face of adversity Resilience starts from adversity. A wealth of data on the adversities faced by Palestine refugee students was collected through interviews. Students recognized the range of adversities they live under, which affected them in proximal ways when it came to learning. In other words, while the external context of armed conflict, violence, or being a refugee was well understood by the young people, its negative impact was felt in more tangible terms at home and in school. At home, students were affected by socioeconomic hardships—such as unemployed parents, incarcerated family members, and limited access to basic resources. At school, the external adverse context also impacted the day-to-day interactions they had with teachers and other students. Paradoxically, adversity was also a motivating factor to do well in school. Indeed, when faced with particular episodes of adversity, many of the students noted how this merely provided an added impetus to succeed in their studies: Because my parents' divorce affected my learning, when I overcame this problem I decided that I wouldn't let any problem distract me from studying, and I made some rules to use if I encounter any problems so I won't let it affect my learning. I must solve these problems without allowing them to affect my education. (Male student, West Bank) I can't describe the fear that we lived in… I was the only one at home who was studying for final exams. My father used to ask me when he saw me studying why I wasn't afraid. We all used to sit in the same room during the days of the war, and my father didn't go to work so we followed his 45 lead, but studying for my exams was the most important thing for me to do especially during the days of the war. (Female student, Gaza) Education, in turn, played an important role in helping Figure 21: How students make sense and students to make sense of the adverse conditions within find purpose through education which they lived. They understood difficult life events Setting goals as opportunities and education as protection. Education Separating hardship from education became the primary means through which they set Taking charge of problems tangible and realistic goals for themselves and in some Finding something Positive in Difficulties cases in relation to the wider Palestine refugee community (Figure 21). As one boy in Gaza noted: My motivation is to get a good future, and to know my future lies and to put a goal in front of me. And I've put education as a goal in front of me and I have to do it, so I have to learn and to know all the subjects and to pay my attention to the teacher's explanation, and to have a better future. Common goals were to become well-educated professionals such as doctors, engineers, and, significantly, teachers. These personal goals were often linked to a desire by students to be positive role models in their communities. This was neatly expressed by one female student in Gaza, who in recounting her experiences of the war noted: I'm a good student although when I was at the fifth grade, the war began and my uncle died in that war; I felt that these days were like a test for us, the students, to work and study hard regardless the difficult days that we faced. From the analysis of their narratives, a pattern emerged of students’ awareness of the adversities associated with being Palestine refugees, but also of education helping them to maintain hope for the future. The UNRWA education system seems to offer learning opportunities that are meaningful to the lives of these students (both relevant to their situation as a marginalized refugee population, and as individual students who have faced particular personal difficulties). Such positive values shape the identity of Palestine refugees and promote a sense of personal well-being and control, connection with others, and social accountability (Figure 22). I have one goal: to be a doctor or an engineer, so that we can show our enemy that even though we lived in difficulties, we became successful in our lives. (Male student, Jordan) Interviewer: Does being a Palestinian refugee have a positive or negative impact on you? Student: It urges me to study harder since we are refugees who had to leave their land and have to work hard to improve our status and prove our identity. (Female student, West Bank) 46 Figure 22: The type of goals high performing students relate with doing well in school (Disaggregated by gender and noting the number of times it was mentioned in student interviews) Personal purpose Personal or social pride and respect Improve living conditions Academic & Career Motivation Support family financially Family purpose Make Parents Proud Female students Male students Have Future Family Education above parents' Community To Help Others purpose To contribute to Palestinian society 0 10 20 30 40 50 60 70 Source: Author’s calculations based on student interviews 2012. UNRWA staff understand and are able to model a positive identity and well-being in the midst of the same challenges their students face The high cultural premium for learning reflects a clear and unwavering prioritization of education, which is promoted at the school level by teaching staff. Teachers can provide effective support to students in adversity, especially when they understand the challenges faced. Many of the student narratives provided evidence of how teachers, head-teachers, and counselors understood the challenges students confronted and were dedicated to addressing the potential negative impacts on learning. School staff were often noted to be very approachable and went above and beyond their professional duties to support students and ensure they were able to continue learning despite living and learning in a difficult context. This included a significant amount of out-of-class support as described by this female student in Gaza: We had a teacher in the sixth grade; she used to come to our home and teach me math every day. I was the smartest student in her class and always get the highest marks in math, and I'm still good at math thanks to her. Whenever I need help with any school subject even if it's not math, I go to her house. Another boy in the West Bank spoke of how his English teacher was always on hand to help outside of class hours: He likes to joke with us during the class, but at the same time he’s very good at teaching. He knows how to teach us and make us all understand, and if we have a difficulty with anything he would explain it to us during the class or after the class; we can go to him any time. The motivation of UNRWA teachers and their ability to understand and connect with their students is not surprising. UNRWA teachers are themselves Palestine refugees and as such hail from the same at-risk population, are part of the same geographical communities as the youth, and have themselves been 47 educated in the UNRWA system. They are committed to a community project of educating their youth and share and promote this goal in close proximity with family members. For example, when asked about his parents checking on his school performance, one boy in the West Bank told us: Yes, my father always calls my teachers and the principal, and one of my teachers…knows my father and uncle very well, because he went to school with my father, so he always talks to my uncle about my performance at school, and my father sometimes calls him and visits him at school. That's how he checks on my performance at school. A major benefit of school staff that share or have shared many of the same adversities as their students is that they have useful insights into their students’ lives. Teachers can serve as role models of personal well-being and a positive identity. This was alluded to by one girl in Gaza, who told us: The best thing about my teachers is that they treat us like any normal student; they treat us like citizens who live in their own country; they always talk to us and tell us about our rights. Students were acutely aware of the influence of teachers on their lives and told us that they often saw their teachers like a second family, while schools were also described in terms of being a second home. Students refer to this positive influence from their teachers as follows: For me, when I see my teachers, I wish to be one of them…I wish to be one of the teachers. When I look at a teacher, I ask: What made her reach this level? I’m sure she studied and it was hard. Our computer teacher… always encourages us, she gives us advice on how to study, and when I follow it, I achieve what I want…I want to say that when I look at my teacher, I wish I to become like her. (Female student, Jordan) Interviewer: What do you like in a teacher? Student: The way they educate the students and treat the student as his son or little brother. (Male student, Gaza) As Bocco has stated relative to the UNRWA system, ‘the practice of teachers is embedded in their local experience: They themselves are refugees and share the same experiences of most pupils’ parents’ (Bocco 2010). Yet this connection between students and teachers is not necessarily unique to the UNRWA system. Rather UNRWA may be using this to its benefit as an Agency that was specifically set up to manage the adversity experienced by the refugees. Its approach to education seems to recognize that context impacts the educational experiences of students, and vice versa. The school system supports students’ competence in the midst of adversity through academic guidance and socio-emotional support Students reported that effective teaching in adversity comprised academic guidance and socio-emotional support. As was noted in earlier chapters, academic guidance in UNRWA schools included effective teaching and learning practices (such as time on task and the use of supplementary materials and practical experiments), assessments (including practice exams) and accountability for results (which was achieved through high expectations for all students and recognition of their efforts). Interviewed students also noted the availability of equity-focused programs (which students referred to as remedial or catch up 48 classes), and an orderly learning atmosphere (through discipline and a well-maintained learning environment). For students, a good teacher, in terms of academic guidance, was one who provided multiple techniques to take students beyond a passive transfer of knowledge. A good teacher engaged students in critical and creative thinking and ensured that there was a sense of equality of opportunity among students. This was exemplified by the response of a male student in Gaza. When asked about what made a good teacher, he responded: …that he can provide the information clearly and he doesn’t differentiate between the students, and he should give us more than what is in the textbook to accommodate our ideas. He should give everything that is useful for the students, and help us with creativity and thinking, not only memorizing and following the textbooks word for word. Other students stressed the importance of teachers being able to simplify or clarify topics for the class, ensuring that they did not depart from a lesson until everyone understood. For example, a male student in the West Bank explained why he liked learning English: …it's an easy subject to learn; our teacher simplifies it for us and makes us like it. I like his teaching method; he explains the lesson and makes sure we all understand, and if there was anyone who didn't understand anything, he would explain everything to him. Perhaps not surprisingly, students drew clear links between a teacher’s ability and thus motivation to do well in a particular subject. Figure 23 presents the varied forms of academic guidance that the students reported, including components that took place both within regular class time and outside of it (again speaking to the high levels of commitment and motivation of UNRWA teachers). Figure 23: Types of academic guidance provided to students (noting the number of times a topic was mentioned in student interviews) Providing extra curricular activities Outside of classroom support Providing extra classes Out of hours academic support expectation In classroom practices recognition Learning Rewarding and recognising good performance s and Academic encouragement and expectations Good teaching methodology (general) Teacher provides practice exams Providing learning resources and facilities Peer-to-peer organized learning 0 50 100 150 200 250 Source: Author’s calculations based on student interviews 2012. 49 However, the above references to academic and cognitive skills were also complemented by the socio- emotional support and important values students saw in a good teacher. Empathy, respect, and trust were common themes in the students’ descriptions of effective teachers and head teachers . Students provided numerous examples of teachers providing them with personal guidance, acting as confidents, and helping them to address particularly difficult moments in their lives. Students respected this commitment, and it furthered their motivation to study hard and find purpose in education. For example, when speaking about the impact of conflict in Gaza, one girl explained: Teachers were with us all the time. They even visited us at home. That gave me a great motivation. It really feels good when you see that someone cares when you are in an ordeal. Reflective of this as well, a male student in the West Bank spoke of the dedication of his head teacher in supporting a fellow student whose father was in jail: One of my classmates' father was in prison for six years, and he was released nine months ago, and he didn't have a job but now he's working but he does not get paid enough. So the principal used to help him all the time, and sometimes when he [my classmate] had to visit his father at the prison, he would have to miss classes that day, so the students would go explain the lessons to him that he missed; they helped him with school and with his personal life. Another illustration of social and emotional support comes from a female student in Jordan who, when asked about what teachers offered their students, responded: They give moral support. Concerning financial support, our religion teacher helps poor students; she raises money and gives it to the poor. Whatever donations come to school she would distribute them to the poor. Our English language teacher provides us with the necessary advice to become effective members in our country. Yet, while it may be analytically possible to distinguish two types of support—academic guidance and socio-emotional support— in practice they both formed an integrated approach to effective teaching and learning. For students this translated into a sense of protection, affable feelings towards their teachers, and sentiments of competence, motivation, and responsibility towards their studies. For example, a female student in Jordan remarked: When I get a good mark like 30 out of 40, my teacher encourages me positively to get full marks for the next time and praises me before others. Thus, my participation becomes better. Moreover, whenever my answer is unique and not everyone is capable to answer the question, she rewards me with a simple gift, not necessarily an expensive one. Every teacher has her own style in motivating students and when a teacher is good with students, they love her in exchange and study well. Students indeed noted that socio-emotional interactions actually enhanced students’ cognitive abilities to excel academically. This is captured in the following interview segments regarding how teachers help their students: She is one of a kind. She says yes whatever you ask her for. She helps a lot whether you are sick or you have a problem at home. She also goes to give condolences along with teachers when a member of a student’s family dies. That really affected students and encouraged the sense of belonging. They felt they have a value at their school and that makes them do better. There was a student who had lots of help from teachers and benefited a lot and did much better. (Female student in Gaza describing her school principal) There are many things that I like; you can see most of these things through the way they treat the students, through the ways they explain the lessons, through the way they run the class and manage it, and through the way they talk to students, get them to study, and get those students to 50 come talk to them whenever they're facing problems. (Male student, Gaza, discussing what he likes about his teachers) Teaching is an essential thing. I have been at this school since first grade now. Teaching is so important as well as meeting other classmates and befriending them. Concerning the headmistress, she cares about discipline, psychological support, and students' honoring. Concerning teachers, they always support us psychologically, teach us how to study, do our homework, help our mothers, set plans to exams, and prepare our lessons. In general, they guide us to many useful ways concerning studying well. (Female student, Jordan) The integrated nature of academic guidance and socio-emotional support points to the importance of not simply addressing these needs through parallel psychosocial programs but rather through incorporating non-cognitive skills and values in the teaching and school management processes. Indeed, in the case of UNRWA, it appears that this integrated support is not simply born of individual initiative but is explicitly supported by the Agency. This is, for example, reflected in the institutional prioritization of cross cutting concerns such as gender, disability, youth, environment, and protection. Particularly germane to this study is UNRWA’s focus on ‘protection’ within its policies for “safeguarding and protecting the Rights of Palestine refugees” (UNRWA 2008). Within this approach, protection is conceived as an integral part of UNRWA’s service delivery for which partnership with the community is an important element.10 The emphasis on providing socio-emotional support as a key component of quality education is also referenced in a recent education program communique entitled ‘education, learning together,’ in which UNRWA states: Providing a quality education for all young people will impart an understanding of their place in the world and convey a common set of key values such as dignity, tolerance, a sense of cultural identity, gender equality, human rights, and respect for the environment; while strengthening coping strategies, especially in a context where poverty pervades daily life (UNRWA, n.d.).11 Learning is supported by many actors including teachers, students, peers, and family members Support for students’ learning seems to expand beyond the UNRWA schools. The 72 UNRWA students interviewed noted that, in addition to their relationships with teachers, principals, and counselors, home and peers were important sources of support for their learning outcomes (Figure 24). 10 What protection means for UNRWA: consultant report dated 31st March 2008. 11 As UNRWA lacks legislative or authoritative power, implementing such values must be done at the classroom level, through the teaching staff. Given, the importance of this approach for students, additional studies that explore UNRWA’s teacher training and professional development programs and associated incentives could reveal important findings. 51 Figure 24: Different sources of support on learning outcomes as perceived by UNRWA students (noting the number of times it was mentioned in student interviews) Positive relationships with teachers 233 Peer-to-Peer Relationships 199 Families-Students Relationships 175 Counselor-Student Relationship 46 Principal-Students Relationships 38 0 50 100 150 200 250 Source: Author’s calculations based on Student Interviews 2012. These multiple sources of support aligned well to theories of child development within different levels in society (or ecologies)12 and have important implications for supporting community-school partnerships. Alongside teachers, families and peers play a key role in the well-being and education process for at-risk children. Interviewed UNRWA students provided many examples of how parents, siblings, and peers reinforced their learning through: additional opportunities to seek information, knowledge, and clarity on a particular topic; by encouraging mutual support and even competition among peers; and by ensuring the most viable learning environment possible, or simply providing motivation and encouragement. In the students’ own words: Interviewer: How is your father creating the suitable environment [for your studies]? Student: He focuses on ensuring the house is quiet, that there is no noise, and I sit in one of the corners of the house and study. (Male student, Gaza) My uncle encourages me when he talks to me, and always tells me stories about college, that some students have educated parents but are not that smart, while there are other students whose parents are not educated but they are very smart. Also, my aunt is a doctor and lives in London, and she always encourages me to become a journalist and to go visit her. (Female student, Gaza) Perhaps the most illustrative example of support for learning is the one among students themselves. For example, UNRWA schools in Gaza tapped into this additional source of learning support by formalizing peer learning techniques in the classroom. The benefits of this are explained by one student: Interviewer: What do you think of the academic support programs that your school has been holding? Student: They are so useful and beneficial to weaker students who can ask high performers to help them understand hard materials or test them with some questions. That saves 12 Of note is Urie Bronfenbrenner’s ‘Developmental research, public policy, and the ecology of childhood’ (1974). Child Development, 45, 1 -5. 52 embarrassment from asking this of a teacher. You know that girls feel relaxed when dealing with each other…We gather with high achievers to read and study and ask about the things that are hard to understand and the high achievers help us in that. If we all don’t understand something, we ask our teacher to help us. Sometimes we conduct workshops in the school breaks to discuss issues related to Arabic, math, or science or we broadcast some programs on the useful ways of reading, studying, and obtaining marks. We may print all that and distribute it among the students. (Female student, Gaza) By incorporating strategies to bring schools, families, and peers together, education policies and programs are more likely to be relevant for both quality of learning and protecting students from the adversities they face. Community-school partnerships within the UNRWA system may not be fully structured through formal ‘school participatory management’ models, but they remain organically embedded in the mutual support mechanisms among Palestine refugees. The importance of community-school partnerships for learning outcomes in contexts of adversity merits further study. UNRWA’s close partnership with the refugee community creates shared accountability for learning outcomes The participation of many stakeholders in supporting the learning process has created a strong matrix of accountability for learning outcomes that stretches across the classroom, school, household, and community of Palestinian refugees. While students benefit from high expectations placed on them by teachers, head teachers, and parents, students also have high expectations and a clear vision of what the school, their families, and the community can provide. Several students were able to articulate this, as in the following: Our community is interested in and appreciates learning. The school ensures the students like going there and enjoy the learning that happens. The goal of the school is to create well-educated students. My family also has a big role in learning, and they will be happy when I finish my learning and graduate from the university with high grades. (Female student, Gaza) Interviewer: What helps you get high grades? Student: First, my determination and will; second, the help of my teachers; third, the encouragement of my family. They suffered a lot because their learning stopped and this incentivizes and encourages me to achieve the dreams of my father and my mother. There is also religion that helps us to learn. (Female student, Gaza) This wider network of support for quality learning in the face of adversity is accountable to each other. Parents checked in on students or were contacted by the school in cases where there was concern about a student’s behavior or performance. When students were experiencing problems keeping up with their studies, teachers informed their families and requested help at home. One boy in Gaza explains this: My father [not only] used to ask me about my study; he also used to come ask about me at school. Teachers used to tell him that I wasn’t doing well, which made him shed light on my studying at home and I started to improve. Another boy from the West Bank recounted: Yes, they know about every grade I get at school, 53 and my father sometimes calls the teachers to ask about my grades, and when I get home he would ask me and tell me not to lie because he knows exactly how much I get. Also my mother comes to school every two weeks to ask about my grades; they do the same for all of us. Thus, the high expectations and high value placed on learning was strengthened by a shared and mutual school-community accountability to support the learning and well-being of students in a difficult living situation. What is resilience in the UNRWA Education System? People who exhibit resilience engage in a dynamic process whereby they are able to ‘navigate’ and ‘negotiate’ adversities with the support of relevant opportunities and services .13 Education resilience – or positive education that results in the face of adversity – is therefore a process and not necessarily a fixed outcome. Results can fluctuate, especially in fast-changing contexts. However, the resilience process can inform how to navigate and negotiate difficult situations. This study points to five education resilience supportive mechanisms identified in UNRWA schools: (i) Students make sense of adversity and find purpose in education; (ii) school staff understand the challenges students face and model well-being and a positive identity; (iii) student competence in the face of adversity is sustained through academic guidance and socio-emotional support; (iv) partnerships are fostered across schools, families, and communities; and (v) accountability for learning and protection of students is shared and mutual. These education resilience strategies at the school and community level are not intended to justify leaving at-risk education communities unaided; rather they are identified as areas that need to be supported by education systems. Interviewed UNRWA students mentioned some of the ways schools and the education system can support their education resilience process. They pointed to six types of school activities including general interpersonal relationships; teaching and learning practices; school management; material and financial resources; and assessment, accountability, and discipline. It can be inferred from the comments of interviewed students that an education resilience perspective appears to be embodied in several of UNRWA’s approaches. The UNRWA system triggers resilience- fostering opportunities by a collective sense-making of adversity and a holistic vision of education as a meaningful purpose for Palestine refugees. All actors are accountable to this aim and to each other. In practice, UNRWA has been able to tap into a high cultural premium for learning and enhance it with the support of students, parents, teachers, head teachers and other education staff. The Agency has played an important role in fostering positive attitudes towards education and subsequently embracing and enhancing them for the benefit of the wider refugee community through a reinforced network of support for students. For the students, their refugee status or the daily adversity they face has not made their education dreams less realistic or likely. Instead, these difficulties have made their education more relevant because the system itself is focused on addressing them. UNRWA explicitly articulates its vision of education as a service that balances the specific needs of the population it serves, aligned approaches to the contexts 13 For more on the resilience process, see Ungar, M. (Ed.) (2012) The social ecology of resilience: A handbook. New York, NY: Springer and Liebenberg, L. & Ungar, M. (Eds) (2008), Resilience in Action Toronto, ON: University of Toronto Press. 54 within which they live, and the need for compatibility and competitiveness within the global education priorities. Indeed, UNRWA states that its vision for education is in accordance with the broader UN vision, and it therefore situates its educational developments and priorities within the global frameworks of the Millennium Development Goals (MDGs) and the Education for All (EFA) initiative.14 Yet, it also provides its own Agency-level interpretation of this, which translates to the skills and competencies it would like to transmit to its students. According to the Agency’s Commissioner General: The fundamental goal of UNRWA’s Education programme is to ensure that the children who come out of our schools know how to think in a critical, independent, intelligent manner; And to do this, we need to review the manner in which they learn.15 One commentator suggests that UNRWA seeks to meet this challenge by implementing locally defined curriculum content within a model of ‘western, secular, liberal education’ (Bocco 2010, p.245), based on a specific and articulated UN vision for education to achieve sustainable positive impact in the face of adversity. The resilience data collected for this study complements this and suggests that a strong commitment at the systems level and varied accessible forms of support for its students support the attainment of positive learning outcomes in the midst of a difficult context. This is what Maya Rosenfeld’s historical perspective alludes to: …the consolidation of UNRWA as an education-centered agency facilitated a transformation in the educational and occupational profiles of second- and third-generation refugees of both genders…UNRWA’s pioneering efforts to universalize a basic education of nine years and to promote the attainment of secondary and professional education enabled young Palestinian refugees to achieve an “educational advantage” over their peers in the Arab host countries… 16 Specifically, for the Palestine refugees who participated in this study, learning in adversity is about much more than a simple classroom dynamic, but rather comprises an active engagement of the whole community around education, and the commitment and vision of the service provider to provide them with locally relevant and accessible opportunities through which to fulfill their potential and, crucially to make transformative contributions to their communities. In the words of UNRWA Commissioner General, Filippo Grandi, speaking at the 30th Anniversary Conference of the Refugee Studies Centre, Oxford University, 7th December 2012: UNRWA’s work, in spite of multiple challenges, was made easier by the exceptional resilience and resourcefulness of the Palestinians. Their accomplishments have become part of the folklore instilled in each succeeding generation of refugees. They reveal the depth of the refugee will – the universal will of refugees, if I may say so – to self-improvement, to shape their lives against overwhelming odds, and to contribute positively to the dynamic of development in their communities, in the societies hosting them, and the world at large. 14 UNRWA programme budget for 2012 – 2012. Accessed 8th August 2012 (http://unrwa.org/userfiles/file/financial_updates/2011/Blue percent20Book percent202012-2013.pdf). 15 UNRWA education: learning together, accessed 8th August 2012 (http://www.unrwa.org/userfiles/2012080244052.pdf). 16 Data collected for this study is contrary to Rosenfeld’s claim in this article that the UNRWA “educational advantage” persist ed until the 1980s, but has since not been reproduced. Yet this does not negate her central argument that adverse conditions and development in the region may have resulted in decreases in declining student performance in the recent past. 55 UNRWA, and the international community that sustains it, share credit in the continuing contribution to the human capital of the refugees. It is, however, the refugees that have seized opportunities and made their own way into the firmament of business, skilled professions, civil society, and public service, in every continent around the globe. In exercising social mobility despite the many handicaps against them, Palestine refugees – just like so many other refugees with whom I have had the privilege to work in my professional life – call our attention to their ambitions, resourcefulness, and creativity… UNRWA’s experience in providing education is an important one to help us better understand what education resilience means at the systems level. Resilience research takes as its starting point a context of risk or adversity. From this, however, assets – strengths, opportunities, and capacities – already existing within the system are identified. Resilience-relevant interventions can then build on these assets to provide vulnerable students with more mechanisms and opportunities to ‘navigate’ and ‘negotiate’17 the risks they face. They are not left alone. This chapter considered this process relative to the UNRWA system and was indeed able to identify important supportive processes embedded within the system. These lessons could support UNRWA to adopt a more explicit resilience approach to its work and may also provide useful lessons for other education systems in similar contexts. To conclude, Figure 25 provides a schematic overview of the education resilience lessons learned by the insightful comments of 72 UNRWA students who themselves exemplify learning in the face of adversity. 17 Using the terminology of Michael Ungar (2008) and (2011). 56 Figure 25: The five education resilience supportive mechanisms identified in UNRWA schools 57 RELEVANCE OF FINDINGS UNRWA has delivered quality education, significantly outperforming two public school systems, under challenging circumstances. Such resilience does not imply that UNRWA is immune to crisis. Indeed, since 2007, several significant events have complicated the service delivery climate in West Bank, Gaza, and Jordan. These include: the blockade on Gaza that began in 2006, economic sanctions, withholding of tax revenues, ongoing military and militant operations, Palestinian internal politics, and the events of Arab Spring. Although exogenous to the education system, these factors are likely to have had – and continue to have – an impact on both refugees and non-refugee education systems, and may help account for the overall general decrease in learning outcomes.18 Moreover, UNRWA’s sustained competitiveness in such adverse contexts (occupation, armed conflict, restricted movement, and protracted displacement) still presents important lessons for other school systems. Specifically, the effective classroom practices of teachers, strong school leadership, assessments, and shared accountability for learning are strong features in the UNRWA system that can support organizational adaptability and performance in the face of adversity. Equally important is recognizing the risks and vulnerabilities that students face and fostering relevant interactions across school and community actors to support learning and protection of students in such challenging contexts. A resilience approach does not imply that schools and communities at risk are left to fend for themselves, but calls for an alignment and institutionalization of education systems to support the resilience processes of which students, teachers, and families avail themselves. As such, the UNRWA experience merits further exploration and investigation to sustain in a difficult context the quality learning opportunities for all children and youth. Figure 26 shows some identified ways in which schools could support the resilience process. 18 A recent World Bank report concluded that the prevailing situation was resulting in lasting damage and that ‘much bolder efforts to create the basis for a viable economy need to be made to prevent the continued deterioration that will have lasting and costly implications for economic competiveness and social cohesion.’ See: http://www.worldbank.org/en/news/press-release/2013/03/11/palestinian-economy-losing-long-term- competitiveness. Accessed 25th March 2013. 58 Figure 26: The different ways in which schools support the resilience process (noting the number of times it was mentioned in student interviews) Supportive relationships 691 Teaching and learning practices 480 School management 163 Material and Financial Contributions to education 138 Assessments and Accountability 72 Disciplinary approaches 32 0 100 200 300 400 500 600 700 800 Source: Author’s calculations based on student interviews, 2012. 59 REFERENCES Anderson, H. H., & Brewer, H. M. (1945). Studies of teachers’ classroom personalities. I. Dominative and socially integrative behavior of kindergarten teachers. Applied Psychology Monographs, 6. Anderson, H. H., & Brewer, J. E. (1946). Studies of teacher’s classroom personalities. II. Effects of teachers’ dominative and integrative contacts on children’s classroom behavior. Applied Psychology Monographs, 8. Anderson, H. H., Brewer, J. E., & Reed, M. F. (1946). Studies of teachers’ classroom personalities. III. Follow-up studies of the effects of dominative and integrative contacts on children’s behavior. Applied Psychology Monographs, 11. Barr, A. S. (1929). Characteristic differences in the teaching performance of good and poor teachers of the social studies. Bloomington, IL: Public School Publishing. Blinder, A. (1973). “Wage discrimination: Reduced form and structural estimates”. Journal of Human Resources 8(4): 436–455. Carroll, J. (1963). “A model of school learning”. Teachers College Record, 64, 723-733. Carroll, J. (1989). “The Carroll model: A 25 year retrospective and prospective view”. Educational Researcher, 18, 26-31. Coker, J. G., & Coker, H. (1979). Classroom observations keyed for effectiveness research: Observer training manual. Carrollton, GA: National Institute for Effectiveness-Directed Education. Cowley, K. S., & Meehan, M. L. (2001). “Perceptions of Kentucky’s extended school services program by district and school coordinators”. Charleston, WV: AEL. Cowley, K. S., Meehan, M. L., Finch, N., Chadwick, K., Howley, C., Riffle, J., & et al. (2002). Comprehensive evaluation of the Kentucky extended school services program. Charleston, WV: AEL. Emmer, E., Sanford, J., Clements, B., & Martin, J. (1981). The design of the junior high management improvement study, report no. 6150. Austin, TX: The University of Texas, Research and Development Center for Teacher Education. Engelhart, M. D. (1972). Methods of educational research. Chicago: Rand McNally. Evertson, C. M., & Green, J. L. (1986). Observation as inquiry and method. In M. C. Wittrock (Ed.), Handbook of research on teaching (pp. 162-213). New York: American Educational Research Association. Evertson, C., & Burry, J. (1989). Capturing classroom context: The observation system as lens for assessment. Journal of Personnel Evaluation in Education, 2, 297-320. Evertson, C., & Emmer, E. (1980). Effective management at the beginning of the school year in junior high classes. Austin, TX: The University of Texas, Research and Development Center for Teacher Education. 60 Evertson, C., Emmer, E., & Clements, B. (1980). Report of the methodology, rationale, and instrumentation of the junior high classrooms organization study. Austin, TX: The University of Texas, Research and Development Center for Teacher Education. Evertson, C., Emmer, E., Clements, B., & Sanford, J. (1980). Observer training manual: Classroom management improvement study (CIMS). Austin, TX: The University of Texas, Research and Development Center for Teacher Education. Flanders, N. A. (1949). “Personal social anxiety as a factor in learning.” Unpublished doctoral dissertation, University of Chicago. Flanders, N. A. (1970). Analyzing teaching behavior. New York: Addison-Wesley. Flanders, N. A. (1962). “Using Interaction Analysis in the Inservice Training of Teachers ”. The Journal of Experimental Education. Vol. 30, No. 4 (Jun., 1962), pp. 313-316. Deon, F., & Pritchett, L. (1988). “Estimating Wealth Effects without Expenditure Data-or Tears: An Application to Educational Enrollments in States of India”. World Bank Policy Research Working Paper No. 1994. Washington, DC: Development Economics Research Group (DECRG), The World Bank. Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Thousand Oaks, CA: Sage. “Handbook of Research on Teaching (pp. 683-714)”. New York: American Educational Research Association. Horn, E. (1914). Distribution of opportunity for participation among the various pupils in classroom recitations. Contributions to Education, 67. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage. Medley, D. M. (1977). “Teacher competence and teacher effectiveness: A review of process product research.” Washington, DC: American Association for Teacher Education. Medley, D. M. (1982). Systemic observation. In H. E. Mitzel, J. H. Best, & W. Rabinowitz (Eds.), Encyclopedia of educational research (5th ed., Vol. 4, pp. 1841-1851). New York: The Free Press. Medley, D. M., & Mitzel, H. E. (1958). “A technique for measuring classroom behavior”. Journal of Educational Psychology, 49, 86-92. Moore, J. R. (1968, February). “An analysis of teacher and pupil verbal behavior and teacher procedural and evaluative behavior in relation to objectives unique to PSSC and non-PSSC physics curricula”. Paper presented at the meeting of the National Association for Research in Science Teaching, Chicago, IL. Nesselrodt, P. S., & Schaffer, E. C. (1993, April). “The ISERP programme: A revised classroom observation instrument”. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA. Nesselrodt, P. S., & Schaffer, E. C. (2000a). “External evaluation of Kentucky’s extended school services, spring 2000”: Phase 1 – final report. Carlisle, PA: Author. 61 Nesselrodt, P. S., & Schaffer, E. C. (2000b). “External evaluation of Kentucky’s extended school services, spring 2000”: Phase 1 – final report, part 2. Carlisle, PA: Author. Oaxaca, R. (1973). “Male-female wages differentials in urban labor markets”. International Economic Review 14(3): 693–709. OECD (2010), PISA 2009 Results: Overcoming Social Background – Equity in Learning Opportunities and Outcomes (Volume II) http://dx.doi.org/10.1787/9789264091504-en .Accessed October 9, 2014. Padron, Y. N., Waxman, H. C., & Huang, S. L. (1999). “Classroom behavior and learning environment differences between resilient and nonresilient elementary school students.” Journal of Education for Students Placed at Risk, 4(1), 65-82. Puckett, R. C. (1928). “Making supervision objective”. School Review, 36, 209-212. Rosenshine, B. (1970). “Evaluation of classroom instruction”. Review of Educational Research, 40(2), 279-300. Rosenshine, B., & Furst, N. (1973). The use of direct observation to study teaching. In R. M. W. Travers (Ed.), Second handbook of research on teaching (pp. 122-183). Chicago: Rand McNally College Publishing. Schaffer, E. C., & Nesselrodt, P. S. (1992, April). “The development and testing of the Special Strategies Observation System”. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Schaffer, E., Nesselrodt, P., & Stringfield, S. (1991). “The groundings of an observation instrument: The teacher behavior-student learning research base of the Special Strategies Observation System.” Paper presented at the International School Effects Research Workshop, Kaohsiung, Taiwan. Simon, A., & Boyer, E. G. (Eds.). (1967-1970). Mirrors for behavior: An anthology of classroom observation instruments (Vols. 1-14). Philadelphia: Research for Better Schools. Slavin, R. (1987). “A theory of school and classroom organization”. Education Psychologist, 22, 89-108. Slavin, R. (1989). A theory of school and classroom organization. In R. Slavin (Ed.), School and classroom organization. Hillsdale, NJ: Erlbaum. Stallings, J. (1980). “Allocated learning time revisited, or beyond time on task”. Educational Researcher, 9, 11-16. Stallings, J. A. (1977). Learning to look: A handbook on classroom observation and teaching models. Belmont, CA: Wadsworth Publishing. Stallings, J. A. (1978, March). “The development of the contextual observation system”. Paper presented at the annual meeting of the American Educational Research Association, Toronto, Canada. Stallings, J. A. (1983). “The Stallings observation system”. Unpublished manuscript, George Peabody College, Vanderbilt University at Nashville. 62 Stallings, J. A., & Mohlman, G. G. (1990). “Issues in qualitative evaluation research: Observation techniques”. In H. J. Walberg & G. D. Haertel (Eds.), The international encyclopedia of educational evaluation (pp. 639-644). New York: Pergamon Press. Stringfield, S., Winfield, L., Millsap, M. A., Puma, M. J., Gamse, B., & Randall, B. (1994). Urban and suburban/rural special strategies for educating disadvantaged children: First year report. Washington, DC: U. S. Department of Education. Sullivan, D. K., & Meehan, M. L. (1983). “The development, pilot test, and field test of an instrument for the systematic observation of classroom interruptions”. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada. Todd, P.E. and I. Wolpin (2003). “On the specification and estimation of the production function for cognitive achievement”. Economic Journal 113: F3-F33. UNRWA. (2008). “What protection means for UNRWA in concept and practice”. http://www.unrwa.org/userfiles/20100118155412.pdf. Accessed: October 9, 2014. UNRWA. n.d. “Education learning together”. http://www.unrwa.org/userfiles/2012112174018.pdf. Accessed: October 9, 2014. Wiersma, W. (2000). Research methods in education (7th ed.). Boston, MA: Allyn and Bacon. Withall, J. (1948). “The development of a technique for the measurement of social emotional climate in classrooms”. Unpublished doctoral dissertation, The University of Chicago. Withall, J. (1949). “Development of a technique for the measurement of socio-emotional climate in classrooms”. Journal of Experimental Education, 17, 347-361. Withall, J., & Lewis, W. W. (1963). Social interaction in the classroom. In N. L. Gage (Ed.), World Bank. (2011). “SABER School Autonomy &Accountability”. Website. Human Development Network/Education, Washington, DC. http://saber.worldbank.org/index.cfm?indx=8&tb=4. Accessed: October 9, 2014 World Bank. (2012). “SABER Teachers”. Website. Human Development Network/Education, Washington, DC. http://saber.worldbank.org/index.cfm?indx=8&tb=1. Accessed: October 9, 2014 World Bank. (2013). “The What, Why, and How of the Systems Approach for Better Education Results (SABER) (Conference Edition).” World Bank, Washington, DC. Wrightstone, J. W. (1934). “Measuring teacher conduct of class discussion”. Elementary School Journal, 34, 454-460. 63 ANNEX: METHODOLOGY (a) SABER SABER-Teachers and SABER-School Autonomy were implemented systematically. Information on a set of questions was obtained and then aggregated into pre-established indicators—and corresponding sub- indicators— to assess outcome in each policy area. The strength of the indicators is tied to the depth and scope of the programs and policies associated with it. The degree of indicator strength reflects what is implemented on the ground. At the low level of indicator strength, the policy is not implemented, while at the high level there is complete implementation of the policy. SABER-Teachers’ 10 core policy areas are:  Requirements to enter and remain in teaching  Initial teacher preparation  Recruitment and employment  Teachers’ workload and autonomy  Professional development  Compensation (salary and non-salary benefits)  Retirement rules and benefits  Monitoring and evaluation of teacher quality  Teacher representation and voice  School leadership SABER-School Autonomy and Accountability has identified five core policy goals that are important in assessing school-based management policies:  School autonomy in the planning and management of the school budget  School autonomy in personnel management  Role of the school council in school governance  School and student assessment  Accountability to stakeholders (b) Stallings Classroom Observation19 The Stallings observation system consists of three main instruments: (1) the Physical Environment Information, (2) the Classroom Snapshot, and (3) the Five-Minute Interaction. Only the classroom snapshot was used for this study. 19 Several classroom observations exist. For more on the history of classroom observation, see: Lewin, Lippitt, and White (1939); Anderson & Brewer (1945); Thelen (1951); Flanders (1962), Withall (1963); Simon and Boyer (1967); and Medley (1982), Stallings (1977); and Stallings and Mohlman (1990). Stallings and Mohlman (1990) addressed the delineation of common elements across all “observational techniques”. These common elements include: (a) purpose, (b) a set of operational definitions, (c) a means to train observers, (d) a focus on observation, (e) a setting, (f) a unit of time, (g) observation schedule (timing of data collection, not the observation form itself, (h) a method to record data, and (i) a method to process and analyze data. 64 The Classroom Snapshot records all the participants in the classroom, what they are doing, and with whom they are engaged (Stallings 1977). It is completed by a classroom observer and recorded in a grid that includes all adults in the classroom (teacher, teacher aide, and visitor), all possible combinations of students (one, two, small groups, and large groups), and 15 categories of activities (such as arts/crafts, tables/games/etc., math, reading, social studies, science, dramatic play, practical skills, social interaction, discipline, transitional activities, classroom management, etc.). There are a total of 22 rows of such activities in the snapshot, and the observer marks what the adults are doing and the corresponding arrangement of students. The snapshot is completed five times per class period, just prior to completing the interaction codes (below). Stallings reports that the snapshot form looks formidable, but that it is rather efficient and easy to use, with teachers and administrators learning to use it satisfactorily in just two half-day training sessions (1983). The tool was implemented in a stratified sample (based on TIMSS 2007 school performance, type, location, and size) of 50 UNRWA schools and 50 public control schools and was selected following the 2007 TIMSS sampling framework. The stratification is based on school location in terms of population concentration and proximity to other schools as a proxy for competition. Implementation was carried out in each school in the sample by a six-person team of observers three times in the winter term of the 2011/2012 school year. Observers were unaware of the classification of school to be visited. An eighth grade class was also visited and observers collected data using the snapshot and five-minute interaction tools. At the end of the observations, UNRWA schools were compared to public schools using the different domains of the instruments. To carry out this process, a research team was recruited and trained to conduct the field work. A regional supervisor followed up on implementation and worked with coordinators in each of the three fields. In each selected school, three classes (math class, science class and reading class) were visited three times during the fall session (September to December 2011) by two observers. The same observers who visited the UNRWA schools also visited the public schools in each field of operation. The fieldwork was managed and coordinated by an international consultant based in Washington, D.C., while a regional coordinator supervised the data collection in Jordan and coordinated with two supervisors: one in the West Bank and one in Gaza. Each member of the research team received a structured training to familiarize them with the research methodology and use of the Stallings instruments. This process was carried out by two international experts with prior training experience. Six researchers and a field supervisor were recruited from each of the regions to conduct the classroom observations. All 18 field researchers were retired school teachers. Training and certification workshops were conducted for each of the three regions. Three local researchers with doctorate degrees and extensive field experience supervised the field work (and were trained and certified on the Stallings tool, as well). In each school selected in the sample, three eighth-grade classes were selected to be observed covering Arabic language, mathematics, and science subjects. Each school visit also included a school instrument to be filled by the researcher from the school principal before the observation. The observers filled a class inventory survey before the beginning of the snapshots. Then from each class, they filled 10 snapshots. After the training, the team piloted the instruments in four schools in West Bank, four schools in Gaza, 65 and three schools in Jordan. Data was collected during the month of November 2011. In the end, 1,800 snapshots were collected. All data were sent to a central location in Jordan and data specialists entered the data using WinDEM 20 software. In addition to the coded field checking, each Snapshot was entered twice for data confirmation before finalizing the entry. Finally, all data was converted to STATA for analysis. In order to compare with other countries in which the Snapshot was implemented, activities were grouped into: learning activities, classroom management, and teacher off-task. The table below shows the classification of activities into these categories. Table A1: Teacher time categorization a) Learning activities: Reading aloud, demonstration/lecture, discussion, practice & drill, assignment/classwork, copying. c) Teacher off-task: Teacher social interaction or teacher uninvolved, teacher out of the room. b) Classroom management Verbal instruction, discipline, classroom management, classroom management alone. d) Student off-task: Social interaction, student(s) uninvolved, discipline. (c) Qualitative data approach The UNRWA resilience component was undertaken using qualitative data collection and analysis methods. Seventy-two high performing UNRWA students of refugee families in most difficult situations in and outside camps were interviewed in three of UNRWA’s fields of operations: the West Bank, Gaza , and Jordan. This was a purposeful sample of eighth and ninth graders who were selected according to pre- determined resilience criteria: (i) They had higher-than-average school grades and (ii) came from families which received additional school and social services due to difficult life situations. All students remained anonymous and provided their assent to be interviewed along adult consent. To understand the context of adversity that affects all Palestinian youth, 24 additional interviews with similar youth in public schools in the three fields were also conducted. This complementary sample of interviews from outside the UNRWA system helped us better understand the contexts of adversity of Palestinian youth in general. All students were interviewed in Arabic by local researchers for half an hour during which time they were asked about the challenges they had faced and how they had performed academically in spite of this. All interviews followed a similar format: Children were probed about the challenges they faced and the support mechanisms that they utilized in the home, community, and school environment. In addition to talking about their individual approaches to pursuing education success, the Palestinian students also shed light on those environmental factors that promote learning despite adversity (in other words, they identified sources of support for the resilience process). Their understanding and reasoning regarding the 20 Open source data entry tool by IEA data processing center. 66 risks and obstacles they encounter, as well as their stated educational goals and purpose, were an important starting point from which to subsequently identify which aspects of the school system propel them towards resilience. Following completion of the data collection, the interviews were entered into NVivo, qualitative data Box A1: Interviewee Demographic analysis software –. NVivo was chosen owing to its Data appropriateness for qualitative data collection and the  53 percent of those interviewed were male and 47 percent female ability to use it to support the construction of a  All students were aged 13-16 years and grounded theory of the education resilience mechanisms were studying in grades 8-10 in the UNRWA system. Given the open-ended interview  Each field was attributed a third of the questions, there was significant complexity in the total interviews  10 different neighborhoods were included coding (from general open coding, to more focused in the study, 3 in Gaza, 3 in Jordan, and 4 categories, to axial relations within and between in the West Bank categories identified). NVivo allowed for greater ease in  44 percent of the interviewees lived in the identification of themes and patterns within and refugee camps while 52 percent lived outside of camp settings between codes and supported us to adhere to a rigorous process through which theories and concepts could emerge and be interpreted from the qualitative data set available. This was especially important given that the study was neither of a representative nor statistically significant sample of students, but rather of a purposefully selected group of students with the lived experiences that could yield light into the topic of interest: learning in context of adversity. Once the data was uploaded into the software, we began a process of coding references made within each and every one of the interviews in accordance with the following broad categories:  Types of adversity  Meaning of education  Emerging thematic dimensions  Sources of support Within these categories, many more sub-categories were identified. Thus, ‘types of adversity’ was subcategorized into adversity present in different spheres of the students’ social context (home, school, community, or stemming from the conflict). ‘Meaning of education’ was attributed from the purpose that students accorded to their studies: self-improvement, financial motivation, family support, community betterment, or national goals. Similarly, ‘sources of support’ were broken down based on their genesis (from members of the family, from the students themselves, from different actors within the education system, or from the peer group). These three levels of analysis resulted in over 130 different codes. Some segments of the text were coded at multiple codes The key themes as they relate to both adversity and sources of support provided an important foundation upon which to conduct our subsequent analysis. Once these have been presented, we will then consider how those supportive school level approaches21 – be they implicitly or explicitly mentioned in the narratives of the youth – address the adversities identified and relate to resilience theory and literature. At this stage, theoretical elements and key findings within the resilience literature helped us interpret the 21 Here we will draw upon school effectiveness theory in order to better structure and subsequently analyze school-based approaches. 67 students’ narratives in a more integrated way. We also sought to make use of the study’s mixed methods approach to bring together and even note the differences across our findings from the different data collection tools. As Bazeley (2002) posits, ‘In determining how best to present the ideas and evidence generated through the completed study, the issue becomes one of the degree to which quantitative and qualitative components can or should be integrated.’ In this way, our analysis for the 72 UNRWA students and follows a three-tier, or leveled process: (i) The codification of themes regarding adversities experienced (this included 24 additional interviews with public system students), the related purpose or reason for education success, and the sources for support for learning in spite of adversity. (ii) The qualitative analysis led us to five key education resilience mechanisms (making sense and finding purpose; seeking identity and well-being; competence and control through academic and socio-emotional support; connecting with others; and committing and being accountable. For presentation purposes, narrative segments were selected to substantiate these five mechanisms. (iii) Grounded on the UNRWA students’ stories of education resilience, we develop a theory or hypothesis of the possible relationship between education system support and academic outcomes in spite of adversity. (d) Methodological limitations Our choice of mixed methods was intended to allow for a more comprehensive evidence base that captures the process aspects of resilience mechanisms. Nevertheless it is important here to state the limitations that our chosen data collection tools present. In other words, what inferences are we able to make from the evidence base and what are we not? With regard to the classroom observations, although subjectivity was highlighted in training and certification sessions of the researchers before the beginning of the task, the classroom observers and interviewers were likely to know if the school was an UNRWA school or a public school. With regards to the qualitative data, the sample size used for interviews is always small, allowing the researcher to probe deeper into complex relationships and new information. Thus, this type of ‘small n’ cannot be held as representative for all UNRWA schools or the three fields of operation. Rather, qualitative data seeks to identify important characteristics to facilitate both exploration and explanation of the research questions. Theoretical analysis and conceptualization of the new topics identified in the students’ narratives allow us to make inferences regarding the processes and mechanisms that support education resilience – learning in a context of adversity – in addition to those pre-identified by the quantitative measures. The sample of students in public schools is not meant to be representative of the public system, nor “a control group” for the UNRWA resilience findings. This much smaller sample of public schools was selected and is intended not as a point of comparison but rather as a check against which to compare the context of adversity across Palestinian youth – in and outside of the UNRWA system. There was a strong convergence between UNRWA and non-UNRWA students on their reports of the proximal and external risks experienced by Palestinian youth in West Bank, Gaza, and Jordan. In this way, the bulk of the analysis for the education resilience component centers on the data collected from 68 UNRWA schools alone. Finally, and to reiterate, we are not looking to establish whether the students in our sample are ‘resilient’ but rather what process mechanisms and inputs help them in their resilience. 69 70