89994 Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education Jennifer Parsons and Jennifer McCleary-Sills Gender Group, World Bank Group I know education is what separates a girl who is trapped in a cycle of poverty, fear, and violence from one with a chance at a better future. –-Malala Yousafzai (July 13, 2014, The New York Times) Progress toward gender equality over the past two decades is most evident in education goals, as primary and secondary school enrollment rates are now the same for boys and girls in many countries. Yet structural, social, and financial barriers continue to prevent enrollment, attendance, and school completion for millions of young people around the globe. And gender imbalances persist. Improving access to and retention in primary and secondary education is a critical element in achieving development objectives, including targets for gender equality. Addressing education quality for all students is also critical. Drawing on lessons from 27 impact evaluations (IEs) supported by the World Bank Group (WBG), this brief highlights opportunities to improve education outcomes and suggests strategies to reduce gender disparities in this area. The findings indicate that strategies that combine structural interventions with individual and family level financial incentives show the greatest promise for improving education outcomes and leveling the playing field for girls. intergenerational benefits: children of better educated women are more likely to receive an The value of education is both intrinsic and education and less likely to be stunted, as instrumental. Educational achievement leads to improved health outcomes and improves economic earning potential in the short and long term. Recent research highlighted girls’ education as a linchpin to promoting greater gender equality (World Bank Group, 2014). Around the world, better educated women are more able to make and implement decisions, even where gender norms are restrictive. Women with more education also tend to marry later and have fewer children (Raj, 2010; WBG, 2014). Lower fertility can increase women’s life expectancy and has Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 1 educated mothers have greater autonomy in making decisions and more power to act for In recent years, the World Bank Group has their children’s benefit (WBG, 2014). increased investments in gender-informed Yet, as recent World Bank Group research programs, research, and impact evaluation. underscores, structural and systematic Given the large investment and attention to disadvantages, particularly access to education programing globally, numerous education, are linked to deprivation of agency World Bank IEs measure progress in formal and constraints for women and girls (WBG, education. This brief explores the most 2014). Three major domains of agency where commonly measured education outcomes this deprivation is connected to a woman’s across a large number of education education level are: limited control over initiatives: school enrollment, school household resources, child marriage, and attendance, and school retention rates (often intimate partner violence. In addition, there is a measured by dropout rates). Eight of these clear divide among women’s experiences programs targeted girls specifically, while the based on the level of education they have other 15 were provided for girls and boys. attained. About nine in ten women with a Programs that measured impact among girls primary education or less experience at least and boys separately or included different one of these deprivations and almost one in programming components for girls and boys five experiences all three. This finding met our criteria for inclusion as gender IEs. contrasts with almost seven in ten and one in twenty, respectively, of women with a Based on these outcomes of interest, from the secondary education or higher. Almost 1 in 5 161 IE papers within the enGENDER IMPACT rural women with a primary education (eGI) database, we identified 27 that met the experience all three deprivations compared inclusion criteria. These represent research in with 1 in 100 urban women with a higher 16 countries across the six WBG regions—four in Latin America and the Caribbean (LAC), two education (WBG, 2014). in East Asia and the Pacific (EAP), three in Given the impact education has on health and South Asia (SAR), five in Africa (AFR), one in economic outcomes for girls and women, Europe and Central Asia (ECA), and one in the Middle East/North Africa (MENA). Details of evaluation of strategies aimed at increasing school enrollment, attendance, and retention is each IE are discussed further below (see critical to promoting evidence-based Table 1 for summary). Five of the identified IEs investments in education. also measured test scores as a component of the evaluation. Given the mixed nature of the evidence regarding standardized tests as valid measures of education quality and student performance (Moon et al., 2003), that outcome is not discussed here. Two IEs identified in this search measured labor force participation as an outcome. However, as this outcome can be influenced by many individual- and community- level factors not directly linked to education attainment or accounted for in the evaluation, it is not included in this brief. There is growing evidence that education completion is protective against child marriage and early pregnancy, and a few IEs also looked at these outcomes. However, these topics are explored in a separate brief in this series: Preventing Child Marriage: Lessons from World Bank Group Gender Impact Evaluations. Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 2 The IEs discussed in this brief include 23 day. This method does not account for programs using nine different strategies aimed students who may have high attendance rates at improving education outcomes. Eighteen of but happen to be out of school on the day of the programs used individual or family-level the observation. In addition, self-reported data strategies including cash transfers, school fee were used in some IEs (1, 4, 17, 19, 22, 27) to reduction, school vouchers, and nutrition and assess attendance, a measure that is subject school feeding initiatives. Five of the programs to social desirability bias. were large-scale community- or country-level initiatives, such as public-private partnerships, Twelve of the IEs (1, 4, 12, 13, 17, 19, 20, 22, school management reform, or infrastructure 24, 25, 26, 27) measured attendance, defined assessment, and analyzing labor demands as the percentage of days the student was determined their impact on school attendance recorded as present during days the school rates. Three additional IEs focused on was in session, or if a student was present in increasing vocational training and technical an eighth-grade class at the time of the visit by skills, but those are outside of the scope of this enumerators. Multiple methods were used and brief, which explores outcomes related to only cross-referenced, including household surveys, formal education. school records and ledgers, and site visits to schools. It is unclear if the same validation was done for non-attendance. The single IE that assessed a program for girls and boys was in Kenya, where an intervention aiming to School Enrollment promote attendance and retention provided One of the most common outcomes for school uniforms to students free of charge education programs is school enrollment, (12). In addition, treatment schools received measured by 17 of the eGI evaluations. support in a variety of ways: a pair of nurses Measures of enrollment were drawn from a visited the school several times a year, variety of sources, including household agricultural representatives organized student surveys and school census data, with school- clubs to grow crops, and each school received based surveys often used to corroborate self- a grant for classroom construction. reported data from household surveys. Among the eight IEs of programs targeting girls only, all were cash transfer programs contingent upon school attendance rates of between 80–90%. Programs in Bangladesh (7, 8), Cambodia (13), and Malawi (26, 27) targeted girls in secondary schools while the program in Pakistan (5, 6, 16) targeted primary as well as secondary education. School Attendance School attendance is perhaps of more importance to education outcomes because it reflects the level of exposure to instruction School Retention and Dropout rather than mere enrollment. However, School retention is a third common measure of measurement of school attendance faces the impact of education programs, and it is methodological limitations. For example, in six frequently measured by its inverse, i.e., the of the included IEs (12, 13, 19, 20, 24, 27), rate of school dropout. In the included IEs, a evaluators performed random on-site checks to variety of measures was used to assess determine whether currently enrolled children retention and dropout rates, including were in attendance on a given observation household surveys, Multiple Indicator Cluster Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 3 Survey (MICS) data, and teacher reports. For (10). The second was a nutritional program in five of the IEs (5, 21, 23, 26, 27), household Burkina Faso that provided free lunch to all surveys recorded whether children were still primary school children and nutritional enrolled at the end of the school year after a subsidies to girls who had 90% attendance program was implemented. In one evaluation rates (25). (5), middle school completion, middle to high school transition, and high school completion School Attendance were measured using pre- and post- Eleven of the 12 IEs that measured attendance intervention MICS data. Dropout was also were shown to have increased attendance measured by teacher-reported enrollment at rates overall. Three of those IEs were the end of the program’s two-year duration. connected to programs for girls only (13, 26, 27), and of the eight remaining IEs, only one Eight IEs measured outcomes of retention, (5, showed a stronger impact among girls than 11, 14, 20, 21, 23, 26, 27). Three of these boys (12). evaluated programs targeting girls only, all of which were cash transfer programs (5, 26, 27). The programs targeting girls and that were effective in increasing attendance rates were all cash transfer programs, contingent upon school attendance rates of between 80–90%. In one arm of a cash transfer program in Multiple approaches and strategies were used Malawi (26), a significant increase in school across the 23 programs evaluated within the attendance was seen only among girls in the scope of this analysis to improve enrollment, conditional cash transfer (CCT) arm, but not in attendance, and school completion. The the unconditional arm, compared with control following highlights programmatic components groups. This indicates that the transfer alone of interventions and their effect on education may be insufficient to help families overcome outcomes measured by the 27 IEs. other social barriers to girls’ regular attendance and eventual school completion. School Enrollment All 17 of the IEs measuring school enrollment had a statistically significant positive impact for boys and girls at both the primary and secondary level, with increases ranging from 2% to 51% across the studies. Five IEs measured primary school attendance only (9, 15, 18, 20, 24), five measured secondary school attendance only (3, 7, 8, 13, 16), and seven measured both primary and secondary school attendance (2, 6, 10, 17, 25, 26, 27). Eight of these assessed programs that were for girls only (5, 6, 7, 8, 13, 16, 26, 27), and of the nine remaining IEs, two showed a stronger impact among girls than boys (10, 25). An IE in India assessed a program focused on school management and found the increase in Two IEs had greater programmatic impact school attendance was greater among boys among girls. The first was a public-private than among girls (4). While the IE of a partnership program in Pakistan, where private nutrition-based program in Burkina Faso (25) entrepreneurs were paid by the government found an increase in enrollment (greater per student enrolled, and given the task of among girls), it also found a decline in school establishing and running primary schools attendance for both boys and girls who offering free enrollment for children ages 5 to 9 received the intervention. These findings Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 4 suggest that there are other factors more influential in family decisions to send a child to Impacts of programs on education vary school beyond simply the cost of education depending on the outcome measured, as well and the provision of food. For example, absenteeism increased in households with as whom the intervention targets. Across the fewer children available for child labor, while three outcomes included in this analysis, CCT absenteeism decreased for households with a programs targeted at girls only had significant relatively large number of children. These impacts on school enrollment, attendance, and results are consistent with documented child retention. Programs that provided labor practices in Burkina Faso at the time of unconditional cash transfers, even when paired the intervention. with the suggestion that the money be used for education, did not have the same effect, School Retention and Dropout indicating that the conditionality of a cash Eight IEs measured outcomes of retention, all transfer is a key part of incentivizing girls’ of which showed improvement as a result of enrollment and continued participation in the program (5, 11, 14, 20, 21, 23, 26, 27). school. Transfer payments in CCT programs Three of those programs (5, 26, 27) targeted went to the girl’s parents (5, 6, 13, 16), to the girls only and were all CCT programs girl directly (7, 8), or were split between the two contingent on school attendance rates of (26, 27). All of these programs had a positive between 80–90%. impact on their intended education outcomes, regardless of the direct recipient of the cash Of the five remaining IEs, one evaluation of a transfer. program in Uruguay (21) showed that providing pre-school education had a larger effect on reducing dropout rates and increasing primary and secondary school completion rates for boys than for girls, although the difference was not statistically significant. An evaluation of a program in the Philippines provided a package of macro-level educational investments, including school construction and renovation; textbooks; teacher training; school-based management; and other facility and equipment support. This strategy was shown to improve completion rates overall, with a greater impact among girls (23). For programs in which improved education Conditional cash transfers, school vouchers outcomes were greater for girls than boys, a that can be used in public or private schools, number of additional programming and investment in school infrastructure were all components were present. Through public- components of programs that were effective in private partnerships, newly established schools increasing retention rates and decreasing provided additional space for girls and boys to school dropout rates for primary as well as attend at no cost, increasing access to secondary school students. Results from the education for many families and resulting in included IEs suggest that a combination of higher enrollment. Providing uniforms and improving schools’ physical infrastructure, supplies in addition to school vouchers or providing school supplies to students, as well CCTs increased attendance and retention, as as incentivizing attendance through cash did provision of meals and medical care at transfer programs is the most effective way to school. Investing at the macro level (training keep students in school and reduce dropout. teachers, improving school infrastructure and materials, and increasing accountability of Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 5 schools to the Ministry of Education) all limited to only those carried out or supported improved rates of attendance and retention, by the World Bank Group and subsequently especially among girls. This suggests that included in the enGENDER IMPACT database. sustained impact on education—from higher The IEs selected for this database apply a rates of enrollment to increased attendance gender lens to either the programming content and lower dropout rates—requires multi-level or the data analysis and evaluation process. programs. Interventions should invest in school We therefore acknowledge that it is not infrastructure and personnel in addition to inclusive of all evaluations conducted on all using CCTs or voucher programs to incentivize educational programs and interventions, which individual families or girls to attend through may also offer valuable lessons. secondary school completion. To achieve gender equality and other key development objectives, programs must seek to improve access to and retention in both primary and secondary education. And they must address persistent issues with the quality of education, including by promoting more gender-responsive schools and curricula. This is essential for girls, who face persistent gendered structural and social barriers that prevent many girls around the world from attending and completing school. In addition, families’ decisions around education are made within the context of financial struggles and One finding in an IE of a CCT program in labor market factors. Based on the results of Bangladesh (8) shows the complexity of these eGI evaluations, we see that strategies measuring the impact of education initiatives in combining structural and systematic the context of labor supply and demand. The interventions with individual and family-level outcome measured was school attendance, financial incentives are most effective in specifically in areas where garment factories improving education outcomes and narrowing were present. Introduction of a garment factory the education gap between boys and girls. in a community resulted in increased school These findings also highlight the importance of enrollment for girls ages 5 to 16, while improving how educational attainment and enrollment for girls ages 17 and 18 decreased. quality are measured. Governments should do This may indicate that younger girls begin everything they can to ease the financial enrolling in school to prepare themselves for burdens families face to ensure higher levels of jobs, while older girls leave school to work in school completion among all children. the factories. These findings highlight the need Particular emphasis should be given to to consider labor supply and demand in the eliminating the gender gap, as girls’ context of analyzing school enrollment and educational achievement leads to improved completion. health and development outcomes in their adult lives and in the lives of future generations. Given the large number of While these findings point to what works to programs and interventions designed to improve education outcomes, there are some improve education outcomes, more rigorous limitations to this analysis. One limitation within evaluations are needed to determine which the IEs is the data sources for some of the components of programs are most effective in indicators measured, as discussed above. The increasing school enrollment, attendance, and universe of IEs included in this analysis is retention, particularly among girls. Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 6 enGENDER IMPACT enGENDER IMPACT is an online gateway for Gender-Related Impact Evaluations. At www.worldbank.org/engenderimpact you will find profiles summarizing key information about World Bank Group funded gender-related impact evaluations. These profiles are organized around priority areas for policy action, including: reducing health disparities, shrinking education and skills gaps, increasing economic opportunities, boosting voice and agency, and addressing gender-based violence. enGENDER IMPACT aims to share knowledge from previous evaluations and encourage more and better evaluations in key gender topics. Included Impact Evaluations: Angrist, J., Bettinger, E., and Kremer, M. (2006). Long-Term Educational Consequences of Secondary School Vouchers: Evidence from Administrative Records in Colombia. The American Economic Review, 96(3), 847-862. Baez, J. E., Alam, A., and Del Carpio, X. V. (2011). Does Cash for School Influence Young Women’s Behavior in the Longer Term? Evidence from Pakistan. Baird, S., McIntosh, C., and Özler, B. (2011). Cash or condition? Evidence from a cash transfer experiment. The Quarterly Journal of Economics, 126(4), 1709-1753. Baird, S., Chirwa, E., McIntosh, C., and Özler, B. (2010). The short�term impacts of a schooling conditional cash transfer program on the sexual behavior of young women. Health economics, 19(S1), 55-68. Barrera-Osorio, F., Blakeslee, D. S., Hoover, M., Linden, L. L., and Raju, D. (2011, April). Expanding Educational Opportunities in Remote Parts of the World: Evidence from a RCT of a Public Private Partnership in Pakistan. In Third Institute for the Study of Labor (IZA) workshop, “Child Labor in Developing Countries,� Mexico City. Barrera-Osorio, F., Linden, L., and Urquiola, M. (2007). The Effects of User Fee Reductions on Enrollment Evidence from a quasi-experiment. http://www.leighlinden.com/Gratuidad%20Draft%202007-01.pdf Benhassine, N., Devoto, Fl, Duflo, E., Dupas, P, and Pouliquen, V. (2013). "Turning a Shove into a Nudge? A ‘Labeled Cash Transfer’ for Education". Berlinski, S., Galiani, S., and Manacorda, M. (2008). Giving children a better start: Preschool attendance and school-age profiles. Journal of Public Economics, 92(5), 1416-1440. Blimpo, M. P., and Evans, D. K. (2011). School-based management and educational outcomes: Lessons from a randomized field experiment. Unpublished manuscript. Chaudhury, N., and Parajuli, D. (2010). Conditional cash transfers and female schooling: the impact of the female school stipend programme on public school enrolments in Punjab, Pakistan. Applied Economics, 42(28), 3565-3583. Duflo, E., Dupas, P., Kremer, M., and Sinei, S. (2006). Education and HIV/AIDS prevention: evidence from a randomized evaluation in Western Kenya. World Bank Policy Research Working Paper. Evans, D., Kremer, M., and Ngatia, M. (2008). The Impact of Distributing School Uniforms on Children’s Education in Kenya. Ferreira, F. H., Filmer, D., and Schady, N. (2009). Own and Sibling Effects of Conditional Cash Transfer Programs. World Bank Policy Research Working Paper, (5001). Filmer, D., and Schady, N. (2008). Getting girls into school: evidence from a scholarship program in Cambodia. Economic Development and Cultural Change, 56(3), 581-617. Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 7 Hasan, A. (2010). Gender-targeted conditional cash transfers: enrollment, spillover effects and instructional quality. World Bank Policy Research Working Paper (No. 5257). Heath, R., and Mobarak, A., (2012). "Does Demand or Supply Constrain Investment in Education? Evidence from Garment Sector Jobs in Bangladesh". Jalan, J., and Glinskaya, E. (2003). Improving primary school education in India: An impact assessment of DPEP-Phase I. Indian Statistical Institute (New Delhi) and The World Bank. Kazianga, H., De Walque, D., and Alderman, H. (2009). Educational and health impacts of two school feeding schemes: Evidence from a randomized trial in rural Burkina Faso. World Bank Policy Research Working Paper No 4976. Khandker, S., Pitt, M., and Fuwa, N. (2003). Subsidy to promote girls' secondary education: the female stipend program in Bangladesh. Khandker, S. R., Bakht, Z., and Koolwal, G. B. (2009). The poverty impact of rural roads: evidence from Bangladesh. Economic Development and Cultural Change, 57(4), 685-722. Kremer, M., and Vermeersch, C. (2005). School Meals, Educational Achievement and School Competition: Evidence from a Randomized Evaluation. Lokshin, M., and Yemtsov, R. (2004). Combining longitudinal household and community surveys for evaluation of social transfers: Infrastructure rehabilitation projects in rural Georgia. Journal of Human Development, 5(2), 265-277. Macours, K., and Vakis, R. (2009). "Changing Households' Investment and Aspiration Through Social Interactions: Evidence from a Randomized Transfer Program in a Low-Income Country" World Bank Policy Research Working Paper No. 5137. Martinez, S., Naudeau, S., and Pereira, V. (2012). The promise of preschool in Africa: a randomized impact evaluation of early childhood development in rural Mozambique. Oosterbeek, H., Ponce, J., and Schady, N. (2008). The impact of cash transfers on school enrollment: Evidence from Ecuador. World Bank Policy Research Working Paper Series, Vol. Ravallion, M., and Wodon, Q. (2000). Does child labour displace schooling? Evidence on behavioural responses to an enrollment subsidy. The Economic Journal, 110(462), 158-175. Yamauchi, F., and Liu, Y. (2013). Impacts of an early stage education intervention on students' learning achievement: evidence from the Philippines. The Journal of Development Studies, 49(2), 208-222. Additional References Cited: Moon, T. R., Callahan, C. M., and Tomlinson, C. A. (2003). Effects of state testing programs on elementary schools with high concentrations of student poverty-good news or bad news? Current Issues in Education, 6(8). Available at http://cie.ed.asu.edu/volume6/number8/ Raj, A. (2010). When the Mother Is a Child: The Impact of Child Marriage on Health and Human Rights of Girls,� Archives of Diseases in Childhood 95, 931–35. World Bank Group (2014). Voice and Agency: Empowering Women and Girls for Shared Prosperity. Photo Credits: 1. Lessons in a rural school. Uzbekistan. Photo: Matluba Mukhamedova 2. School Lets Out. Bangladesh. Photo: Scott Wallace 3. Pupils in a school canteen, having a break from classes. Moldova. Photo: Michael Jones 4. Girls during lunch break at school. Romania. Photo: Flore de Préneuf 5. Children listen to their teacher in Villa Nueva, Guatemala. Photo: Maria Fleischmann 6. School children attending parade. South Africa. Photo: Trevor Samson Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 8 Table 1: Summary of Education outcomes measured by WBG Gender IEs Outcomes of Interest School School School Enrollment Attendance Retention Program Title Country Intervention Impact Evaluation Category References Atención a Crisis Nicaragua Cash Transfer  1.Macours and Vakis, 2009 Bono de Desarrollo Humano Ecuador Cash Transfer  2.Oosterbeek et al., 2008 Cambodia Education Sector Support Program Cambodia Cash Transfer  3.Ferreira et al., 2009 Scholarship Program District Primary Education Program India School  4.Jalan and Management Glinskaya, 2003 Female School Stipend Program (as a part of the Pakistan Cash Transfer   5.Baez et al., 2011 Punjab Education Sector Reform (PERSP) 6.Hassan, 2010 Female Secondary Stipend Program Bangladesh Cash Transfer  7.Khandker et al., 2003 8.Heath et al., 2012 Food For Education Program Bangladesh School Fee  9.Ravallion and Reduction Wodon, 2000 Gratuidad Colombia School Fee  10.Barrera-Osorio, Reduction 2007 ICS-Africa, Child Sponsorship Program (CSP) Kenya School Supplies   11.Duflo et al., 2006 12.Evans et al., 2008 Japan Fund For Poverty Reduction Scholarship Cambodia Cash Transfer   13.Filmer and Schady, Program 2008 PACES Program Colombia School voucher  14.Angrist et al., 2006 Promoting Low Cost Private Schooling in Rural Pakistan Public Private  15.Barrera-Osorio, Sindh Partnership 2011 Punjab Education Sector Reforms Programme Pakistan Cash Transfer  16.Chaudhury and Parajuli, 2010 Rural Infrastructure Rehabilitation Georgia Infrastructure   17.Lokshin and Projects Yemtsov, 2004 Save the Children Early Child Development Mozambique Early Childhood  18.Martinez et al., Program Development 2012 School Meals Kenya School Feeding  19.Kremer and Vermeersch, 2005 Tayssir Morocco Cash Transfer    20.Benhassine, 2013 Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 9 The program implemented by the ANEP Uruguay Preschool  21.Berlinski et al., (Administración Nacional de Educación Pública) 2008 The Rural Development Project and the Rural Bangladesh Infrastructure  22.Khandker et al., Roads and Markets Improvement and 2009 Maintenance Project Third Elementary Education Philippines Education  23.Yamauchi and Liu, 2013 Whole School Development The Gambia School   24.Blimpo and Evans, Management 2011 World Food Programme: School Meals and Take Burkina Nutrition   25.Kazianga et al., Home Rations Faso 2009 Zomba Cash Transfer Program Malawi Cash Transfer    26.Baird et al., 2011 27.Baird et al., 2010 Leveling the Playing Field: Lessons from World Bank Group Gender Impact Evaluations on Education 10