Report No: AUS5476 Republic of Togo Togo Service Delivery Indicators Education 2013 29 June 2016 GHN07 AFRICA i Standard Disclaimer: This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Copyright Statement: The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750- 4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. ii REPUBLIQUE TOGOLAISE ………………………. MINISTERE DES ENSEIGNEMENTS PRIMAIRE, SECONDAIRE ET DE LA FORMATION PROFESSIONNELLE ……………………… DIRECTION DU CABINET ……………………… SECRETARIAT GENERAL ………………………. CELLULE NATIONALE DE L’ÉVALUATION Togo Service Delivery Indicators 2013 Technical Report – Education iii EXECUTIVE SUMMARY ....................................................................................................................................... viii I. INTRODUCTION ...............................................................................................................................................2 II. Methodology and Implementation .......................................................................................................................6 III. RESULTS ............................................................................................................................................................ 9 A. Teacher effort ...................................................................................................................................................9 Absence from school ..........................................................................................................................................9 Absence from class ............................................................................................................................................ 11 Time spent teaching per day .............................................................................................................................. 12 B. Teachers with minimum knowledge ............................................................................................................... 14 French ................................................................................................................................................................ 16 Mathematics ....................................................................................................................................................... 16 Pedagogy ............................................................................................................................................................ 17 Availability of inputs at the school ......................................................................................................................... 18 Functioning school infrastructure....................................................................................................................... 19 Availability of teaching resources ...................................................................................................................... 20 Availability of textbooks .................................................................................................................................... 20 Pupil-teacher ratio .............................................................................................................................................. 21 Assessment of pupil learning ............................................................................................................................... 22 What does this mean for Togo? ................................................................................................................................. 27 Comparing Togo with other countries that have done SDI ................................................................................ 29 Annex 1. Sampling ...................................................................................................................................................... 30 Annex 2. Definition of the Service Delivery Indicators in Education ......................................................................... 32 Annex 3. Additional Results ........................................................................................................................................ 34 At-a-glance tables for schools: non-striking and all ............................................................................................... 34 School breakdowns ................................................................................................................................................. 35 Individual breakdowns ............................................................................................................................................ 36 REFERENCES ............................................................................................................................................................ 51 iv Tables and Figures Table 1. Members of the National evaluation commission ......................................................................................................... vii Table 2. Service Delivery Indicators at-a-glance .............................................................................................................................. ix Table 3. Comparison of SDI results across countries ........................................................................................................................i Figure 2. Relationships of accountability between citizens, service providers, and policymakers ........................... 4 Table 3. Education Indicators .................................................................................................................................................................... 5 Table 5. Survey modules .............................................................................................................................................................................. 7 Table 6. School attendance by socio-economic status (percent) ............................................................................................... 7 Table 7. SDI Education sample .................................................................................................................................................................. 8 Table 8. Teacher Effort................................................................................................................................................................................10 Figure 5. Distribution of teacher effort indicators .........................................................................................................................13 Table 9. Teacher assessment....................................................................................................................................................................15 Table 10. Teacher French scores (percent) .......................................................................................................................................16 Table 11. Teacher mathematics scores (percent)...........................................................................................................................17 Table 12. Teacher pedagogy scores (percent) .................................................................................................................................17 Table 13. School input indicators ...........................................................................................................................................................19 Table 14. Sanitation access details (percent) ...................................................................................................................................19 Figure 7. Availability of inputs ................................................................................................................................................................22 Table 15. Student performance results (percent) ..........................................................................................................................24 Figure 8. Student learning correlates (augmented models without poverty) ...................................................................25 Figure 9. Pupil evaluation distribution by section and school ownership ..........................................................................26 Table 16. Pupil evaluation: gender and location breakdowns (percent) .............................................................................27 Table 17. Pupil evaluation: gender and school ownership breakdowns (percent) .........................................................27 Table 18. Education at a glance for non-striking schools and teachers ................................................................................34 Table 19. Education at a glance for all schools and teachers .....................................................................................................34 Table 20. Teacher effort in all schools with all teachers..............................................................................................................36 Table 21. Teacher effort excluding schools with striking teachers and volunteer teachers .......................................36 Table 22. Teacher absence rates, by teacher status (percent) ..................................................................................................37 Table 23. Public school teacher absence rates, by region (percent) ......................................................................................37 Table 24. Teacher effort and knowledge for all schools and non-volunteer teachers ...................................................38 Table 25. Teacher effort and knowledge for all schools and teachers ..................................................................................38 Table 26. Teacher evaluation breakdowns ........................................................................................................................................39 Panel A: Teacher evaluation: ownership and urban-rural within public breakdown (percent) ...............................39 Panel B. Teacher evaluation: contractual status breakdown (percent) ...............................................................................40 Panel C. Teacher evaluations: academic training breakdowns (percent) ............................................................................41 Panel D. Teacher evaluations: grade taught breakdowns (percent) ......................................................................................42 Panel E. Teacher evaluations: teacher training breakdowns (percent)................................................................................43 Panel F. Teacher evaluations: teacher training college degree breakdowns (percent) .................................................44 Table 27. Teacher characteristics (absenteeism sample) ...........................................................................................................45 Table 28. Correlates of teacher effort ...................................................................................................................................................46 Figure 11. Pupil evaluation distribution by section and school location .............................................................................47 Table 30. Pupil performance details (percent) ................................................................................................................................48 Table 31. Correlates of pupil performance in language ...............................................................................................................49 Table 32. Correlates of pupil performance in mathematics .......................................................................................................50 v Figure 1. Incidence of strikes in Togo 3 Figure 10. Education infrastructure co-availability 35 vi ACKNOWLEDGEMENTS The team wishes to acknowledge support from the Ministère des Enseignements Primaire, Secondaire et de l’Alphabétisation throughout the process, including numerous senior staff during consultations on the methods and the results. In particular, the ministry’s Conseil National d’Evaluation was involved at each step including supervision in the field. The time and effort of teachers and students is gratefully acknowledged. Data collection and processing was carried out by a team of consultants managed by TIMS Services. Table 1. Members of the National evaluation commission Name and first name Institution Function Mme ADANDOGOU-AGOUNKE Inspectorate general of Education General inspector, education Kékéli Jeannine M. AMAVI Ayité Ayi-Koutou Université of Lomé General inspector, education M. AYRAKOU P. Koffi Amégbo CNDP-TICE Database manager Mme. AZANLEDJI Améyo Primary education Inspection Pedagogical advisor Lomé-Sud Mme. BALI Mèhèza Primary education Inspection Inspector, Pre-primary and primary Lomé-Ouest education M. BAMAZE Badaba DexCC Head, Professional examinatinos unit M. DARA-AHATO Yawo Dotsè Training directorate Trainer of trainers Mme. DJAYOURI Noulokmey DAENF Head, faire-faire partnership Catherine strategy unit M. KAGNITOM Koffi Primary education Inspection Principal secretary Lomé-Ouest M. OGOUWA Komivi Primary education Inspection Head, evaluation unit M. OURO-BANGNA Fousséni Secondary education Inspection - Inspector, secondary education Golfe Lomé-Est (French) M. PERE Talaki Primary education Inspection - Inspector, secondary education Golfe Lomé-Est (mathematics) M. TSALI Kossi Kpomegni DPEE Head, statistics unit The World Bank Togo SDI team was led by Christophe Rockmore, with important contributions from Deon Filmer, Gayle Martin, Ezequiel Molina, Obert Pimhidzai, and Waly Wane (SDI team), Johannes Hoogeveen (Country Economist), Hervé Assah (World Bank Country Manager for Togo), Hyacinthe Gbaye (Educational and Institutional Strengthening Project Team Leader), and Pamela Mulet (Education Specialist), Brian Stacy (Education Economist), Dosseh Ihou (Resource Management Analyst), Nadia Mireille Zenia Amoudji Agnegue, Esinam Hlomador-Lawson, and Tatiana S. Mensah (Assistant). Seydou Garba Hamidou, technical assistant working for the Educational and Institution Strengthening Project, provided substantial technical inputs and support during the preparation and data collection phases. Jud Shearer and Eleni Papakosta edited the report, and Raihona Atakhodjayeva supervised the printing. Data entry was done by a consortium led by the National Opinion Research Council with the participation of the Unité de Recherches démographiques (URD) of the University of Lomé (Togo). Funding for this activity was provided by the William and Flora Hewlett Foundation, the World Bank and the Global Partnership for Education (under the Educational and Institution Strengthening Project). vii EXECUTIVE SUMMARY This report presents the findings of the Service Delivery Indicators in the education sector in Togo in 2013. Survey implementation was preceded by extensive consultation with Government and key stakeholders on survey design, sampling, and adaptation of survey instruments. Pre-testing of the survey instruments, training of field staff, and field-work took place in 2013. Information was collected from 200 primary schools, 1,141 teachers, and 1,938 grade four pupils in Togo. The results provide a representative snapshot of the quality of service delivery and the physical environment within which services are delivered in public pr i ma r y schools. The survey provides information on three levels of service delivery: measures of (i) teacher effort; (ii) teacher knowledge and ability; and (iii) the availability of key inputs, such as textbooks, basic teaching equipment, and infrastructure (such as sanitation, quality of lighting etc.). The results indicate an adequate number of teachers to serve the population’s ne eds, but they lack the necessary skills and inputs. Absence rate is a factor, although relatively lower than in other countries, except during teacher strikes. The reliance on volunteer teachers also creates challenges as only the present discounted value of future earnings can be considered a source of motivation for them.1 Efforts are needed in all major dimensions surveyed (competence, absence rate, inputs), although recent textbook provision efforts appear to have already made an impact. What providers know Pupils cannot learn more from their teachers than what the teachers know. On average, Togolese primary school teachers mastered 45% of an exam set at the lower primary level and only 2.5% of fourth grade teachers (1.8% of primary school teachers) scored 80% or higher. Among fourth grade teachers in 2013/14 and/or third grade teachers in 2012/13, the average score was 43 percent overall, and 2.7 percent of teachers had a score of 80 percent or higher. Teachers in private and urban public schools did better than their public or rural public peers. What providers do (teacher effort) Overall, 21.6 percent of teachers were not in school during an unannounced and a further 15 percent were at school, but not in the classroom. When in the classroom, teachers taught 79.2 percent of the time, meaning that nearly one-fifth of the time was devoted to other activities. Cumulating the sources of lost teaching time, pupils have roughly 48.9 percent of the scheduled teaching time. What providers can use (availability of key inputs) The pupil-teacher ratio averaged 43 pupils per teacher, slightly higher than the 40 pupils that are the norm in Togo, but 29 pupils per class were observed during the survey on average. However, there are important input deficiencies that make teaching more difficult. Roughly one-quarter (22.8 percent) of schools had the minimum infrastructure, primarily because one school in four (24.4 percent) has functional, private, and accessible latrines. Similarly, 14.6 percent of schools have the minimum teaching equipment, although public schools are more likely to have textbooks for their pupils than private schools. Does Ownership Matter? In most areas, private schools performed better than public schools. The strongest differentials were in infrastructure and classroom teaching time, while the absence rate among non-volunteer teachers was generally similar. Although public school teachers devote a greater share of class time to instruction, private school teachers spend more time teaching and are more likely to be in class (one-half the rate of classrooms with pupils but no teacher). This adds up to 34 minutes of extra teaching time per day for private school pupils. At the fourth-grade level, this is approximately an extra class of French or mathematics. Not only is there more teaching, but private school teachers are, on average, more competent that their public school 1 Their earnings may have been as low as 5,000 FCFA/month, or roughly $5. viii peers. However, their absolute levels remain low; this is a major concern for the state of the educational system in Togo. Geographic and gender differences Aside from class sizes, public school pupils in rural areas are at a disadvantage compared to their urban peers. Combined with other factors, this translates into significantly lower scores on the learning evaluation, despite relatively closer non-verbal reasoning skill levels (a proxy for intelligence). Likewise, girls do worse than boys in the tests. These effects are also cumulative: rural boys do worse than urban ones and rural girls do the worst. How does Togo compare with other countries? Table 3, below, compares Togo with other countries for the public sector. 2 In terms of teacher effort, Togo has a slightly higher absence rate than the average of the other counties, but the absence from the classroom was lower, despite the strikes. Togo and Senegal are tied for the highest number of hours of effective instruction time per day. However, teacher knowledge is particularly low; it is the greatest differential among all the indicators, despite the positive correlation between student and teacher performance (Table C14 and Table C15). Compared to the other SDI countries, Togolese schools have the highest level of textbook availability, which is positively correlated with student learning outcomes in French and mathematics (Figure 8). Togo has the second-highest infrastructure gap (Nigeria is worst) and the worst availability of pedagogical materials. Infrastructure lags primarily due to electricity and latrines, while the gap on pedagogical materials is principally linked to the low number of schools that had all three items simultaneously (even using a 90 percent threshold for pens and exercise books). Table 2. Service Delivery Indicators at-a-glance Urban Rural All Public Private Public Public School absence rate 21.6 23.0 18.5 14.0 24.3 Classroom absence rate 37.2 38.9 33.3 25.3 41.5 Classroom teaching time 2h 40 min 2h 38 min 2h 44 min 3h 13 min 2h 33 min Minimum knowledge among teachers 2.7 1.4 4.8 4.5 0.8 5 h 29 mi 5 h 28 mi 5 h 33 mi 5 h 28 mi 5 h 28 mi Scheduled teaching time n n n n n Observed pupil-teacher ratio 29.1 31.0 24.8 33.5 27.5 Pupils with textbooks (%) 68.5 76.0 52.6 73.3 66.6 Teaching equipment availability (%) 26.4 24.3 30.8 9.4 27.5 Infrastructure availability (%) 22.3 14.4 39.2 18.6 13.6 Note: definitions of indicators are in Annex B. 2See Annex C for more details on comparisons across SDI countries for private schools only (Table C16) and for all schools (Table C17). ix Table 3. Comparison of SDI results across countries Togo SDI Kenya Mozambique Nigeria Senegal Tanzania Uganda Public schools only 2013 average 2012 2014 2013a 2011 2014 Tanzaniea2011 2013 Teacher knowledge Minimum knowledge (% Not teachers) 2.7 12,7 34,8 0,3 2,4 comparable 15,6 Not comparable 10,1 Not Test score (out of 100) 43,0 42,0 55,6 26,9 30,5 comparable 46,6 Not comparable 43,3 Teacher effort School absence rate (% teachers) 21,6 20,1 15,2 44,8 16,9 18,0 15,3 23,0 29,9 Classroom absence rate (% teachers) 37,2 42,1 47,3 56,2 22,8 29,0 46,7 53,0 56,9 Time spent teaching per day 2 h 40 min 2 h 53min 2 h 30min 1 h 41 min 3 h 10 min 3 h 15 min 2 h 57 min 2 h 04 min 2 h 56 min Scheduled teaching time per day 5 h 28 min 5 h 31 min 5 h 31 min 4 h 17 min 4 h 44 min 4 h 36 min 5 h 54 min 5 h 12 min 7 h 13 min Resource availabiltity Student-teacher ratio (observed) 29,1 42,1 39,3 21,4 21,5 34,0 40,6 74,0 53,9 Textbook availability (% Not students) 68,5 37,2 44,5 68,1 33,7 comparable 25,9 Not comparable 6,0 Teaching equipment Not availability (% classrooms) 14,6 57,8 74,3 76,8 48,2 comparable 62,4 Not comparable 79,5 Infrastructure availability Not (% schools) 22,8 36,2 60,2 29,1 13,4 comparable 36,0 Not comparable 57,2 Student learning outcomes Combined math/language Not score (%) 45,1 45,4 69,4 20,8 25,1 comparable 49,2 Not comparable 45,3 Not French score (%) 44,9 44,8 72,5 18,7 23,3 comparable 47,9 Not comparable 43,4 Not Math score (%) 43,9 45,2 57,4 25,1 28,2 comparable 57,5 Not comparable 41,7 Notes: a. Values for Nigeria are the weighted average of the four states surveyed : Anambra, Bauchi, Ekiti,, and Niger. b. These statistics reflect the updated SDI methodology. More detailed information on the methodology is avaialable at www.SDIndicators.org. c. Complete indicator definitions are in Annex B. i I. INTRODUCTION Located in the Gulf of Guinea between Benin and Ghana, Togo is a country with an estimated population of 6.2 million which is growing at 2.8 percent annually and likely to double in 25 years (DGSCN, 2011). With a population density of 109 people per km2, it is much denser than the Sub-Saharan African average (35 people per sq. km). Per capita income (US$589 in 2009) is low compared to Sub-Saharan Africa (US$1,675) and Low Income Countries (US$676) averages. Poverty decreased from 62 percent in 2006 to 59 percent in 2011, although extreme poverty rose during this same period. Poverty is most pronounced in rural areas and in the Savanes region, where 73 percent of people lived in poverty in 2011 (64 percent in 2006). 3 Since 1990, Togo experienced a long period of socio-political crisis that ended with the parliament elections in October 2007. Encouraged by the success of those elections and the new government’s reform platform, donors reengaged with Togo after more than 15 years of very limited assistance. Presidential elections were held in March 2010 and were well received by the observers. Togo reached the Heavily Indebted Poor Countries (HIPC) Initiative decision point in November 2008 and reached the Completion Point in 2010. Togo’s general education system is divided into four levels: (i) a three-year pre-school cycle designed for 3-5 year olds, (ii) a six-year primary cycle designed for 6-11 year olds, (iii) a seven-year secondary education cycle designed for 12-18 years old, consisting of a four-year junior level and a three-year senior level and (iv) a higher education system. There are also (i) technical and vocational education at the junior and senior secondary levels and (ii) literacy programs. Enrollment and progression. Although gross primary enrollment has increased nearly 18 percent (from 113 percent to 133 percent) over the period 2000-2012, net primary enrollment has increased only 8 percent in 2000-9 (last year with data). Although repetition has decreased nearly 35 percent in the between 2000 and 2012, the primary completion rate has increased 15 percent, primarily due to a nearly 30 percent increase for girls. This represents a reduction of 59 percent in the disparity between girls and boys in this time period. The gross enrollment rate disparity between students from the poorest and the richest families has also reduced by 11 percent over 2006-2011. There are five factors to keep in mind when analyzing the results of the education survey. First, for pupils not sitting for examinations, the school year was nearly over. Some teachers were also absent, proctoring or grading the training exams (“CEPD blanc”), which could not be avoided. 4 As Figure 1 shows, there were a number of strikes, both by civil servants and employees of faith-based schools that were felt differently in rural and urban areas (Figure 1). Consequently, the SDI results reflect these realities. First visits were reprogrammed to the extent possible so that teams conducted the first visits on days when there were no strikes. However, second visits were allowed to fall on strike days to reflect the reality faced by pupils. Indicators affected by this are presented with and without striking teachers (respectively, Table 2 and Table 19, for all schools and only schools without strikes). Finally, 44 of the 172 fourth-grade classrooms of schools that had strikes in neither visit were taught by a volunteer teacher, with more in rural (N=37; 77 percent) than in urban and in public (N=36; 68 percent) than in private. Unless otherwise stated, the main text analyzes all schools. 3Coulombe, H. and C. Malé (2012), “Togo: Profil de pauvreté 2006-2011”. 4Where this is noted as a reason for absence, it is coded as “approved”, but it will bias upwards the rates of both overall an d approved absences. Twelve schools’ records mention CEPD-related disruptions. 2 Figure 1. Incidence of strikes in Togo Teacher strike rates Percent of absences in the second visit Rural Urban 63.0 61.4 33.3 31.3 28.6 25.0 22.2 17.6 14.8 11.4 4.5 5.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ay ay ay ay ay ay ay ay ay ay ay n n n n ay ay ay ay ay ay ay ay ay ay ay Jun Jun Jun Jun Jun Jun M M M M M M M M M M M 3 Ju 4 Ju 5 Ju 6 Ju M M M M M M M M M M M 3 4 5 6 7 10 15 16 21 22 23 24 27 28 29 30 31 15 16 21 22 23 24 27 28 29 30 31 Note: estimates are based upon the sample, not the population. The report is structured as follows: Figure 1 below presents the analytical underpinnings of the indicators and the way in which they are organized. Figure 2 provides additional information on the SDI program and a detailed description of each indicator. Section 2 covers implementation and the SDI education methodology. Results are presented and analyzed in Section 3 and Section 4. The report concludes with analyses of implications for Togo and a comparison between the Togo results and those of other countries that have done SDI. 3 Box 1. Analytical Underpinnings Service delivery outcomes are determined by the relationships of accountability between policymakers, service providers, and citizens (Figure 2, World Bank 2004). Human development outcomes are the result of the interaction between various actors in the multi‐step service delivery system, and depend on the characteristics and behavior of individuals and households. While delivery of quality education is contingent foremost on what happens in classrooms, a combination of several basic elements have to be present in order for quality services to be accessible and produced by teachers at the frontline, which depend on the overall service delivery system and supply chain. Adequate financing, infrastructure, human resources, material, and equipment need to be made available, while the institutions and governance structure provide incentives for the service providers to perform. Figure 2. Relationships of accountability between citizens, service providers, and policymakers Service Delivery Production Function Consider a service delivery production function, f, which maps physical inputs, x, the effort put in by the service provider, e, as well as his/her type (or knowledge), θ, to deliver quality services into individual level outcomes, y. The effort variable e could be thought of as multidimensional and thus include effort (broadly defined) of other actors in the service delivery system. We can think of type as the characteristic (knowledge) of the individuals who are selected for a specific task. Of course, as noted above, outcomes of this production process are not just affected by the service delivery unit, but also by the actions and behaviors of households, which we denote by ε. We can, therefore, write y = f(x,e,θ) +ε To assess the quality of services provided, one should ideally measure f(x,e,θ). Of course, it is notoriously difficult to measure all the arguments that enter the production, and would involve a huge data collection effort. A more feasible approach is therefore to focus instead on proxies of the arguments which, to a first‐order approximation, have the largest effects. Indicator Categories and the Selection Criteria There are a host of data sets available in education. To a large extent, these data sets measure inputs and outcomes/outputs in the service delivery process, mostly from a household perspective. While providing a wealth of information, existing data sources (like Living Standards Measurement Survey (LSMS), Welfare Monitoring Surveys (WMS), and Core Welfare Indicators Questionnaire Survey (CWIQ)) cover only a sub‐sample of countries and are, in many c ases, outdated. 4 Box 1. Analytical Underpinnings (continued) The proposed choice of indicators takes its starting point from the recent literature on the economics of education and service delivery, more generally. Overall, this literature stresses the importance of provider behavior and competence in the delivery of education services (as opposed to water and sanitation services and housing that rely on very different service delivery models). Conditional on service providers exerting effort, there is also some evidence that the provision of physical resources and infrastructure has important effects on the quality of service delivery. The somewhat weak relationship between resources and outcomes documented in the literature has been associated with deficiencies in the incentive structure of school and education systems. Indeed, most service delivery systems in developing countries present frontline providers with a set of incentives that negate the impact of pure resource‐based policies. Therefore, while resources alone appear to have a limited impact on the quality of education in developing countries, it is possible inputs are complementary to changes in incentives, so coupling improvements in both may have large and significant impacts (see Hanushek, 2006). As noted by Duflo, Dupas, and Kremer (2011), the fact that budgets have not kept pace with enrollment, leading to large pupil‐teacher ratios, overstretched physical infrastructure, and insufficient number of textbooks, etc., is problematic. However, simply increasing the level of resources might not address the quality deficit in education without also taking providers’ incentives into account. SDI proposes three sets of indicators: (i) provider effort; (ii) knowledge of service providers and (iii) availability of key infrastructure and inputs at the frontline service provider level. Providing countries with detailed and comparable data on these important dimensions of service delivery is one of the main innovations of the Service Delivery Indicators. Additional considerations in the selection of indicators are (i) quantitative (to avoid problems of perception biases that limit both cross‐country and longitudinal comparisons), (ii) ordinal in nature (to allow within and cross‐country comparisons); (iii) robust (in the sense that the methodology used to construct the indicators can be verified and replicated); (iv) actionable; and (v) cost effective to collect. Table 4. Education Indicators Teacher Effort School absence rate Classroom absence rate Time spent teaching per day Teacher Knowledge and Ability Minimum knowledge in mathematics Minimum knowledge in English Minimum knowledge in pedagogy Availability of Inputs Minimum infrastructure availability Minimum equipment availability Share of pupils with textbooks Observed pupil-teacher ratio 5 Box 2. The Service Delivery Indicators (SDI) Program A significant share of public spending on education is transformed to produce good outcomes at schools. Understanding what takes place at these frontline service provision centers is the starting point in establishing where the relationship between public expenditure and outcomes is weak within the service delivery chain. Knowing whether spending is translating into inputs that teachers have to work with (e.g. textbooks in schools), or how much work effort is exerted by teachers (e.g. how likely are they to come to work), and their competency would reveal the weak links in the service delivery chain. Reliable and complete information on these measures is lacking, in general. To date, there is no robust, standardized set of indicators to measure the quality of services as experienced by the citizen in Africa. Existing indicators tend to be fragmented and focus either on final outcomes or inputs, rather than on the underlying systems that help generate the outcomes or make use of the inputs. In fact, no set of indicators is available for measuring constraints associated with service delivery and the behavior of frontline providers, both of which have a direct impact on the quality of services that citizens are able to access. Without consistent and accurate information on the quality of services, it is difficult for citizens or politicians (the principal) to assess how service providers (the agent) are performing and to take corrective action. The SDI provides a set of metrics to benchmark the performance of schools in Africa. The Indicators can be used to track progress within and across countries over time, and aim to enhance active monitoring of service delivery to increase public accountability and good governance. Ultimately, the goal of this effort is to help policymakers, citizens, service providers, donors, and other stakeholders enhance the quality of services and improve development outcomes. The perspective adopted by the Indicators is that of citizens accessing a service. The Indicators can thus be viewed as a service delivery report card on education. However, instead of using citizens’ perceptions to assess performance, the Indicators assemble objective and quantitative information from a survey of frontline service delivery units, using modules from the Public Expenditure Tracking Survey (PETS), Quantitative Service Delivery Survey (QSDS), and Staff Absence Survey (SAS). The literature points to the importance of the functioning of schools and more generally, the quality of service delivery. The service delivery literature is, however, clear that, conditional on providers being appropriately skilled and exerting the necessary effort, increased resource flows for health can indeed have beneficial education outcomes. The SDI initiative is a partnership of the World Bank, the African Economic Research Consortium (AERC), and the African Development Bank to develop and institutionalize the collection of a set of indicators that would gauge the quality of service delivery within and across countries and over time. The ultimate goal is to sharply increase accountability for service delivery across Africa, by offering important advocacy tools for citizens, governments, and donors alike; to work toward the end goal of achieving rapid improvements in the responsiveness and effectiveness of service delivery. More information on the SDI survey instruments and data, and more generally on the SDI initiative can be found at: www.SDIndicators.org and www.worldbank.org/SDI, or by contacting SDI@worldbank.org. II. Methodology and Implementation The SDI indicators draw information from a stratified random sample of 200 schools, comprised of 148 public, 28 faith-based, and 24 private non-denominational schools. This sample provides a representative snapshot of the learning environment in both public and private schools. The details on the sampling 6 procedure are in Annex 1. The education work was implemented as part of the ongoing work with the Government of Togo on improving educational quality and development of the Ministry of Education’s capacity to produce, analyze, and use statistical information for policy formulation and evaluation. The standard SDI survey instruments were adapted to the Togolese context through a participatory process involving technical discussions, training, and piloting with the Ministry of Education’s National (Education) Evaluation Commission (Commission nationale d’évaluation; CNE).5 The education survey was also coordinated with the Global Partnership for Education (GPE) project’s PASEC - inspired survey. A single team that undertook both surveys went to each school and the supervisors were from the CNE. The survey was implemented by the CNE with support and supervision from the World Bank’s Service Delivery Indicators (SDI) team. The modules of the survey instrument are shown in Table 5 below. Table 5. Survey modules Module Description 1. School Administered to the director to collect information about school type, facilities, information school governance, number of pupils, and school hours. Includes direct observations of school infrastructure by enumerators. 2. Teacher Administered to director and individual teachers to obtain a list of all school information and teachers, to measure teacher absence and to collect information about teacher absence characteristics. Includes an unannounced visit (Module 2B) to evaluate absence using the best known method from research on the subject. 3. School finances Administered to the director to collect information about school finances. 4. Classroom Information on teaching activities, classroom conditions; collected through observation direct observation in the classroom. 5. Pupil assessment Test of randomly selected grade-four pupils to measure their learning outcomes in mathematics and language. 6. Teacher Evaluation of teachers’ subject knowledge (mathematics and language) and assessment teaching skills. The sample of schools used in the SDI survey was the same as the PASEC-inspired survey. The sample chosen closely reflects the distribution of school usage across facility types and poverty status, as shown in Table 6 and Table 7. Table 6. School attendance by socio-economic status (percent) Expenditure quintile (percentage) School type Poorest Second Third Fourth Richest Total Public 80.8 77.1 69.0 65.1 41.3 68.8 Faith-based school 8.7 9.4 12.2 13.6 17.3 11.8 Private, non-faith based 1.5 4.3 12.6 19.0 39.3 13.3 Community 8.8 8.9 6.2 2.3 2.1 6.1 Other 0.1 0.3 0.0 0.0 0.0 0.1 Total 100.0 100.0 100.0 100.0 100.0 100.0 Source: author’s calculations based upon the Questionnaire des indicateurs de base du bien -être (QUIBB) 2011. Table 7 provides details of the sample for the Education Service Delivery Indicators. In total, 200 primary 5 The members were drawn from the Ministry of Education, an observer from UNICEF, and a technical advisor from the Global Partnership for Education project. One member of the commission was a union representative. 7 schools, of which 74 percent were public schools and the remaining 26 percent either private for-profit or private not-for-profit schools. The survey assessed the knowledge of 831 primary school teachers, surveyed 1,141 teachers as part of the study of the absence rate, and observed 192 grade four lessons.6 In addition, learning outcomes were measured for 1,938 grade four pupils. Survey implementation was preceded by extensive consultation with Government and key stakeholders on survey design, sampling, and adaptation of survey instruments. Pre-testing of the survey instruments, training of field staff, and field-work took place in 2013. Table 7. SDI Education sample Sample Weighted Variable Pct. of Total Distribution Total Ownership 200 100 100 Public 148 74 69 Private 52 26 31 Faith-based 28 14 16 Non-denominational 24 12 15 Location Rural 136 69 73 Urban 64 31 27 Urban public 35 17 18 Rural public 113 56 82 Teachers 1,224 100 100 Public 862 70.4 66.8 Private 362 29.6 33.2 Pupils 1,938 100 100 Note: subtotals may not add to the totals due to rounding. Five primary school teacher types were captured in the questionnaire: civil servants ( fonctionnaires), auxiliaries (auxiliaires), volunteers (volontaires), private sector contractuals (contractuel du privé), and community (financé par la communauté). Auxiliary teachers are in the probationary period and will either be confirmed as civil servants or else dismissed. Their earnings are lower than civil servants. Volunteer teachers are individuals who do not have the necessary qualifications to be recruited as auxiliaries and therefore work as volunteers in hopes of being integrated to the civil service rolls at a later date. These individuals earned less than 5,000 FCFA/month ($10) in 2013. Community teachers are supported by the community, but have no expectation of integration into the civil service. 6Two schools were not completed due to the strikes that were frequent during the survey’s field period. See Error! Reference source not found. for more on this. 8 III. RESULTS A. Teacher effort Absence from school During the first announced visit, a maximum of ten teachers are randomly selected from the list of all teachers who are on the school roster. The whereabouts of these ten teachers are then verified in the second, unannounced, visit. Teachers found anywhere on the school premises are marked as present. This is limited to teachers who are not volunteers; details on volunteer teachers are in Annex 3. The statistics are weighted, with the interpretation of the indicator being the percentage of teachers who are absent during an unannounced visit. The indicators relating to teacher effort (Absence from school, Absence from class, and Time spent teaching) and the differences in outcomes between public and private, and urban and rural schools, respectively, are presented in Table 8. They are summarized in Figure 3, which shows that out of 100 teachers only 53 percent were in class teaching on a given day. Roughly one teacher in five (21 percent) is absent from school and a further 15 percent were at school, but not in the classroom. 7 Figure 3. Teacher activities during the school day (percent) 0.7% 21.0% 52.2% 15.2% 10.9% Absent from school At school, not in class In class, not teaching Teaching in class Teaching outside Table 8 shows that less than one teacher in five was absent on any given day in Togo. 8 Among public schools, teachers are nearly twice as likely to be absent from school in rural relative to urban areas (24 percent and 17.7 percent, respectively), although the adherence to the wave of strikes that took place during the survey fieldwork was stronger in rural than in urban areas. Excluding schools that had strikes during the second visit, the pattern remains broadly similar, although the absence rate in public school falls to 19.4 percent in 7 These statistics differ from those in Table 8, because they are calculated directly based on the teachers rather than first aggregating at the school level. The figure does not include volunteer teachers. 8 This excludes volunteer teachers, who are not contractually bound to be present in class. When including them, the absence rate from school rises to 21.4 percent. Annex 3 contains additional tables for all schools and all teachers (Table 20) and only non- volunteer teachers in schools without strikes (Table 21). 9 rural and the difference with urban (2.7 percentage points) is reduced further. Whether or not schools with strikers are included, the rural-urban difference is between six and 11 percentage points (p<0.05 for the universe of all schools). In general, non-volunteer teachers are present in school even in the strike environment. Fully 61 percent of schools have no teachers absent, 70 percent of schools have one teacher in six absent (normally, there are six active teachers in a Togolese school), 80 percent have one in three teachers absent, and 10 percent have more than two-thirds of teachers absent. Reasons for absence are varied (Figure 4), but strikes were clearly the most important reason. Excluding schools where at least one teacher was on strike, slightly more teachers are present on average. Absence from the classroom follows similar trends. Table 8. Teacher Effort Difference Urban Rural Difference All Public Private (%) Public Public (%) Absence from school 21.6 23.0 18.5 24.2 17.7 24.0 35.7 Absence from class, but present in school 15,7 16,0 15,1 5,6 15,8 16,0 -1,5 3h 2h Time spent teaching 2h 40min 2h 38min 2h 44min -3,9 08min 33min -35,4 Scheduled lesson time 5h 5h per day 5h 29min 5h 28min 5h 33min -1.6 28min 28min 0,0 Note: this table includes all teacher contracts. Significance levels: *** p<0.01; ** p<0.05; * p<0.1. Figure 4. Reasons why teachers are absent from school (percent) 24.2 12.8 12.1 10.7 10.3 8.5 4.3 3.1 2.6 2.4 1.2 1.1 1.1 1.1 y l r n e ve er e ow e e te s ng ra be am nc rik nc av r io La th la a ni ne kn em iss le se le se Sa O St Ex ai Fu ab ab M ty ck m 't Tr on ni Si d ed ily er e D ov ov m at fa pr pr M ap ap kc un Si er th er O th O 10 Absence from class The indicator is constructed in the same way as the school absence rate indicator, with the exception that now the numerator is the number of teachers who are either absent from school, or present at school but absent from the classroom. A small number of teachers may be found teaching outside, and these are marked as present for the purposes of the indicator. The interpretation of the indicator is the percentage of teachers who are absent during an unannounced visit. The behavior of teachers is generally the same whether or not the striking schools are included or excluded, so what follows is for all schools. Given the importance of volunteer teachers in Togo, they will be included in the following discussion. On any given day, 35.8 percent of teachers are not in the classroom, with rural teachers 6.6 percentage points more likely to be absent than their urban counterparts (Table C3). Of those teachers at school but not teaching, almost all were not in the classroom at all. Relative to private school teachers, public school teachers are less likely to be at school and marginally less likely to be in class when at school. The comparison of absence from school and absence from class shows that roughly 58 percent of the absence rate of teachers from the classroom is driven by absence from the school. This phenomenon is invariant with respect to the schools with strikers. Estimates for absence from school, absence from class, and absence from class, at school, broken down by teacher type are in Table C5. Unsurprisingly, private school (contractual) teachers are statistically significantly more likely to be present than any other teacher category, however their absence from the classroom is primarily due to teachers who are at school, rather than those who do not show.9 Those closest in presence are auxiliary teachers, who have strong future earnings incentives from integration into civil service, then civil servants, then volunteers.10 Table C7 reports regression models of absence from school and the classroom for teachers in all schools. There are four broad categories of variables in the models: (i) teacher-specific variables, (ii) school-specific variables, (iii) geographic effects, and (iv) poverty levels. Looking at the results, individual and supervision incentives seem to be the most important aspects. Individual incentives relate to contractual status and self-selection (teacher’s training college degree). Those with less stable contract types (private school teachers) or greater long-term incentives (auxiliary teachers; p<0.1 for absence from school) are more likely to be present than those with low incentives (volunteer teachers). Teachers in poorer cantons are somewhat less likely to be absent (p<0.05). 11 Supervision incentives are those exerted by hierarchical superiors, whether in the facility or in the ministry. An absent school director (over one standard deviation increase) and strikes during the second visit (0.7-0.8 standard deviation increases) are among the strongest predictors of teacher absence from school. They are significant at the one percent level in both models. Efforts by the ministry to reduce the absence rate in the Savanes region are difficult to measure precisely. Although the absence rate module was done both in January and in May, there is not information both times about the number of visits by supervisors, so it is not possible to estimate whether or not the resources for increased supervision intensity had an effect upon the absence rate. Within the limitations of the sample size, it appears that the public school absence rate is relatively higher in the Savanes region relative to other regions, as reported Table C8, and shown in the regression results. 9 For more information on the characteristics and absence rates of teachers, see Table C6. 10 This excludes two community-financed teachers found in the sample. 11 Poverty information is from Coulombe, H. and C. Malé (2012), “Togo: Profil de pauvreté 2006 -2011”, UNDP and DGSCN, and merged at the canton (first level below the region) level. 11 Time spent teaching per day This indicator combines data from the staff roster module (used to measure absence rate), the classroom observation module, and reported teaching hours. The teaching time is adjusted for the time teachers are absent from the classroom, on average, and for the time the teacher teaches while in classrooms based on classroom observations. While inside the classroom distinction is made between teaching and non-teaching activities. Teaching is defined very broadly, including actively interacting with pupils, correcting or grading pupil's work, asking questions, testing, using the blackboard, or having pupils working on a specific task, drilling, or memorization. Non-teaching activities is defined as work that is not related to teaching, including working on private matters, maintaining discipline in class, or doing nothing and thus leaving pupils not paying attention. The interpretation of the weighted indicator is the percentage of time taught in fourth-grade classrooms in an average day when accounting for absence from the classroom. This indicator measures the amount of time a teacher spends teaching in a school during a normal day. It is calculated by recording the reported scheduled time of a teaching day from school records, i.e., five hours and 29 minutes after break times (Table 8). This number is multiplied by the proportion of teachers absent from the classroom. The idea being that if 10 teachers are supposed to teach five hours and 23 minutes per day, but 3.3 of them are absent from either the school or the class at any one time, then the scheduled teaching time is reduced to 2 hours and 56 minutes (five hours and 29 minutes x 0.489). On average in Togo, a teacher will teach for two hours and 55 minutes (Table 8). That is, teachers teach approximately 48.9 percent of the scheduled time (the r ep o rt e d scheduled time for grade four pupils is five hours and 29 minutes after break times). Table C9 reports some intermediate inputs used in the calculation of the indicator. Even when in the class, however, teachers may not necessarily be teaching. The percentage of the lesson lost to non-teaching activities is measured through observation of a grade four lesson.12 As reported in Table C9, 48.9 percent of a typical day is devoted to teaching, while the remainder to non-teaching activities, such as interruptions, and the teacher being absent from class. To account for this, the effective time for teaching is multiplied by the proportion of time that teaching occurs in the classroom. From the initial five hours and 29 minutes, this leaves two hours and 40 minutes. The distributions of the underlying variables are shown in Figure 5. Taken independently, it appears that there is a group of teachers who are frequently absent from the classroom and a group who teach relatively little while in the classroom; combined, these effects significantly reduce the overall teaching time for Togolese pupils. Comparing Table C9 with Table C10, it is clear that volunteer teachers are more frequently absent, teach less than the others, and perform worse on the evaluation. When considering these tables, it is worth keeping in mind that there are very different incentive, accountability, and compensation frameworks for different teacher categories, particularly volunteer teachers. However, Table 8 also shows that there is some variation between public and private schools in teacher effort. While the scheduled teaching time and the overall proportion of time taught are close, there is a 12 This is most likely an upper bound on the time devoted to teaching during a lesson; since presumably a teacher is more likely to teach when under direct observation (i.e. Hawthorne effects will bias the estimate upward). During the observation, enumerators first had to judge whether the teacher was teaching or not. If they judged the teacher to be teaching, they were supposed to indicate how much time the teacher spent on any of the following teaching activities: teacher interacts with all children as a group; teacher interacts with small group of children; teacher interacts with children one on one; teacher reads or lectures to the pupils; teacher supervises pupil(s) writing on the board; teacher leads kinesthetic group learning activity; teacher writing on blackboard; teacher listening to pupils recite/read; teacher waiting for pupils to complete task; teacher testing pupils in class; teacher maintaining discipline in class; teacher doing paperwork. 12 significant difference in the use of classroom time. Private-school teachers spend four percentage points less time teaching (76 percent and 81 percent, respectively) than do public-school teachers. The variation within public schools across rural and urban is statistically insignificant. In Togo, public schools are far more likely to have orphaned classrooms. 13 Overall, 19 percent of classrooms are orphaned, with strong patterns among the different breakdowns. Public schools are more than twice as likely as private schools to have such unsupervised environments (p<0.05). This phenomenon is generally more pronounced in rural areas than in urban ones (92 percent more likely; p<0.05). Figure 5. Distribution of teacher effort indicators Absent from school Absent from classroom 2.5 1.5 2 1 Density Density 1.5 1 .5 .5 0 0 0 .25 .5 .75 1 0 .25 .5 .75 1 Percent Percent Time spent teaching .004 .003 Density .002 .001 0 100 200 300 400 Minutes per day Excludes volunteer teachers. 13This is measured by inspecting the school premises, counting the number of classrooms with pupils, and recording whether a teacher is present in the classroom or not. The share of orphaned classrooms is then calculated by dividing the number of classrooms with pupils but no teacher by the total number of classrooms that contained pupils. 13 B. Teachers with minimum knowledge This indicator measures teacher's knowledge and is based on mathematics and language tests covering the primary curriculum administered at the school level to all mathematics or language teachers that taught grade 3 in the previous year or grade 4 in the year the survey was conducted. It is calculated as the percentage of teachers who score more than 80 percent on the language and mathematics portion of the test. In Togo, the teachers of grades 2 and 5 whose classrooms were included in the PASEC sample were included in the knowledge evaluations. This increased the number of grade 2 and 5 teachers in the sample. An effort was made to include all possible teachers and the results reported in The indicator Box is representative 3. Assessment of knowledge of average of the teachersteacher in the universe of teachers in the country rather than the average This indicator at the average teacherteacher's measures school. knowledge and is based on mathematics and language tests covering the primary curriculum administered at the school level to all mathematics or language teachers that taught grade 3 in the previous year or grade 4 in the year the survey was conducted. It is calculated as the percentage of teachers who score more than 80 percent on the language and mathematics portion of the test. In Togo, the teachers of grades 2 and 5 whose classrooms were included in the PASEC sample were included in the knowledge evaluations. This increased the number of grade 2 and 5 teachers in the sample. An effort was made to include all possible teachers and the results reported in The indicator is representative of the average teacher in the universe of teachers in the country rather than the average teacher at the average school. The objective of the teacher test is to both evaluate mastery of basic and more advanced reading, writing, and mathematical skills and to evaluate mastery of pedagogical skills. Taken together, these are critical elements in the pupils’ acquisition of subject knowledge. The basic reading, writing, and arithmetic skills that lower primary pupils need to have in order to progress further with their education are interpreted as the minimum knowledge required for the teacher to be effective and serve as the basis for the "Share of teachers with minimum knowledge” indicator. While in some Anglophone countries, primary school teachers were specialized in language or mathematics, in Togo, teachers taught all subjects in primary school. The test was validated against the Togolese primary curriculum and 12 other Sub-Saharan primary school curricula.14 The minimum knowledge indicator was calculated as the percentage of teachers who scored more than 80 percent on the lower primary part of the language and mathematics test (Table 9). The test also contained more advanced questions in both subjects, as well as a pedagogy section. Satisfactory mastery of content knowledge among Togolese teachers was quite low as only 2.7 percent of the teachers in grade 4 (2013/14 school year) and grade 3 (2011/12 school year) achieved a score of 80 percent or higher in the combined French and mathematics sections. With a score of 7.2 percent, private school teachers were nine times more likely to achieve proficiency than the public school teachers (0.8 percent). The results are generally similar when all evaluated teachers are included, even though the average score rises to 45 percent from 43 percent. As the cumulative density plots in Figure 6 show, there was a steep gradient in performance when using all teachers, independently of the class taught. See “Teaching Standards and Curriculum Review“, prepared as background document for the SDI Program by David Johnson, 14 Andrew Cunningham and Rachel Dowling. 14 Figure 6. Teacher evaluation cumulative distribution (all teachers) 100 80 Togo: 45.0% Public: 43.3% Private: 48.7% 60 40 20 0 0 20 40 60 80 100 Grade in math and French (%) Togo Public Private Note: based on 833 teachers (611 in public and 222 in private). Weighted results. The heavy reliance on volunteer teachers in the Togolese context reduces the overall competence, although not as much as might be thought. Comparing the average scores on French and math with civil servants as the reference category (see Panel B of Table C11), auxiliaries (11.7 percentage points higher, p<0.05) and private school teachers (9.9 percentage points higher, p<0.10) do better, while volunteer teachers (9.7 percentage points lower, p<0.05) do worse. 15 Table 9. Teacher assessment Rural Difference Urban Publi Differen Percent All Public Private (%) Public c ce (%) Teachers with minimum knowledge 2,7 0,8 7,2 -823,1* 2,4 0,4 -81,7 Average score French, math, pedagogy 34,9 33,2 39,3 -18,5*** 36,8 32,4 -12,0** French and math 43,0 40,6 48,6 -19,6*** 46,5 39,4 -15,3** Sensitivity analysis (French and math) Minimum knowledge: 100% 0,3 0,0 1,1 0,0 0,0 Minimum knowledge: 90% or higher 0,3 0,0 1,1 0,0 0,0 Minimum knowledge: 80% or higher 2,7 0,8 7,2 -823,1* 2,4 0,4 -81,7 Minimum knowledge: 70% or higher 4,7 2,5 10,1 -310,5* 10,8 0,7 -93,5** Note: Results based on observations from 363 teachers (273 public and 90 private) in 185 schools. Breakdowns by grade taught are in Table 26. Differences are relative to public and rural public, respectively. 15 Additional details are in Table 26. 15 French As Table 26 shows, no matter the breakdown, teachers do better in French than they do in mathematics. Looking specifically at the French section of the assessment (Table 10), private school teachers perform 14.6 percentage points better than their public school counterparts (p<0.01) and urban public school teachers perform 6.6 percentage points better than their rural counterparts (p<0.01). Teacher performance declines as the difficulty level rises, although the decrease is less marked for public school than for private school. On the grammar assessment, teachers averaged 74 percent. 16 Teachers in private schools, urban public schools, and urban schools all performed better than their comparators at the one percent significance level (p<0.01). In the Cloze task, private and urban public teachers performed far better than their comparison groups.17 Finally, the composition task, the only significant difference was recorded in public schools, where urban teachers scored roughly five percent higher than their rural counterparts.18 Panels C and D of Table 26 show that teachers with more years of academic training do better, as do teachers who teach higher grades. Those with a teaching diploma from a teacher’s training college do better than those without such a diploma (Panels E and F of Table 26). Interestingly, the performance is not a monotonic function of the diploma level as those teachers without diplomas score better, on average, than teachers with the first diploma. Table 10. Teacher French scores (percent) Difference Urban Rural Differenc All Public Private (%) Public Public e (%) French 49,4 47,4 54,3 -14,6*** 51,9 46,4 -10,6** Grammar 74,1 71,5 80,4 -12,4*** 78,1 70,1 -10,3*** Cloze task 29,3 27,6 33,3 -20,5 30,4 27,0 -11,0 Composition 24,3 23,8 25,5 -7,3 27,1 23,1 -14,8 Note: these results are for teachers who taught grade 3 in the year prior to the survey and/or grade 4 the year of the survey. Differences are relative to public or rural private, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Mathematics Teachers who taught grade 3 in the year prior to the survey or grade 4 in the year of the survey, correctly completed 30.2% of the basic mathematics examination (Table 11). As with French, teachers in private, urban, and urban public schools outperformed their comparator groups at the p<0.01. In almost all sub- categories of mathematics, urban public school teachers perform at or above (particularly for Venn diagrams) the level of private school teachers. The patterns in the sub-categories of the math section are consistent with the notion that more difficult items should have lower scores. Taken together, two reasons explain why fractions are not an exception to this rule (13.2 percent average score): fractions are only taught in sixth grade and teachers tend to teach the same class each year. Hence, most teachers would have learned about fractions while in school, but may not have taught them in years. Two other areas that proved quite difficult are the Venn diagram and graph- 16 This asked teachers to complete sentences with the proper verb forms 17 This activity assesses vocabulary and text comprehension. It consisted of a short story with blanks where certain words were removed, which the teachers had to complete in a meaningful way. 18 This section of the assessment required teachers to correct a letter written by a pupil, which contained grammatical, punctuation, and spelling errors. 16 reading exercises, where rural/urban breakdowns overall and within public showed significant differences (p<0.01 and p<0.05, respectively). Table 11. Teacher mathematics scores (percent) Difference Urban Rural Difference All Public Private (%) Public Public (%) Math 30,2 27,7 36,3 -30,9** 35,2 26,1 -25,9** Basic math 36,5 33,9 42,9 -26,6** 41,0 32,3 -21,2* Advanced math 18,5 16,2 24,0 -48,0** 24,4 14,5 -40,7** Fractions 13,2 11,4 17,6 -54,7 18,7 9,8 -47,6* Venn diagrams 18,2 15,9 23,8 -49,9* 30,1 12,9 -57,2** Graphs 11,1 9,4 15,0 -59,2 16,1 8,0 -50,4* Note: these results are for teachers who taught grade 3 in the year prior to the survey and/or grade 4 the year of the survey. Differences are relative to public or rural private, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Pedagogy Although this component is not included in the minimum knowledge variable, it does provide some insight into teachers’ ability to prepare and deliver lessons and evaluate students. The overall score in pedagogy was quite low, with fourth grade teachers averaging 19.6 percent (Table 12). There are no major differences across ownership, rural/urban within public, or rural/urban school teachers, suggesting that this is a general weakness for teachers. However, other teacher characteristics show generally monotonic patterns: teachers with more years of schooling do better, as do those who attended teachers’ training college, hold higher teacher training diplomas, and those teaching higher grades. Looking at the sub-categories of the pedagogy section, teachers performed poorly on the lesson preparation (average score: 28.1 percent) and did worse at pupil comparison (15.8 percent). The evaluation of pupil learning showed poorest performance (6.5 percent). Both sections of the evaluation (math/French and pedagogy) had the same allotted time, but teachers often failed to complete entire sections of the pedagogy evaluation. Panels A to F of Table 26 show that increased academic learning, teacher training qualifications, diplomas from teacher training institutions, and grade taught are all positively associated with better pedagogical performance. Although the teacher evaluation does not directly measure classroom performance, it measures elements necessary for the preparation of teaching such as lesson planning, ability to compare learning performance, and assessment of learning progress. Taken together, there are clear problems with teacher mastery of subject matter and of key aspects of teaching. Table 12. Teacher pedagogy scores (percent) Difference Urban Rural Difference All Public Private (%) Public Public (%) Pedagogy 19.6 19.3 20.2 -4.7 20.1 19.1 -5.2 Lesson preparation 28.1 28.1 28.0 0.0 28.6 28.0 -2.1 Pupil comparisons 15.8 15.1 17.2 -13.9 16.5 14.8 -11.3 Pupil evaluations 6.5 6.0 7.5 -13.4 6.2 6.0 -3.3 17 Note: Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Teacher knowledge and use of instruction time The use of instruction time has gained increasing attention as a key element of learning. The information on teacher subject and pedagogical knowledge provides an opportunity to examine the situation in Togo in various ways, including by defining performance quartiles for the different knowledge elements (overall, French and mathematics, and pedagogy). At the beginning of class, teachers with higher pedagogy scores are more likely to provide an introduction (p<0.10 for quartiles 2 and 4 relative to quartile 1). They are also twice as likely to close class with a summary of what was learned (p<0.10 for quartiles 2 and 4 and p<0.05 for quartile 3). Aside from these results, there are few other areas that prove to be significant across student engagement and teacher behaviors. Statistically speaking, teacher behavior in Togo is relatively similar across various dimensions of teacher classroom behavior and engagement of students. Availability of inputs at the school Functioning school infrastructure Minimum infrastructure resources is a binary 0-1 capturing availability of: (i) functioning toilets operationalized as being clean, private, and accessible; and (ii) sufficient light to read the blackboard from the back of the classroom. Functioning toilets: Whether the toilets were functioning was verified by the enumerators as being accessible, clean, and private (enclosed and with gender separation). Electricity: Functional availability of electricity is assessed by checking whether the light in the classroom works and gives minimum light quality. The enumerator places a printout on the board and checks (assisted by a mobile light meter) whether it was possible to read the printout from the back of the classroom. Availability of teaching resources Equipment availability is a binary variable equal to 1 if (i) the randomly selected grade 4 classroom has a functioning blackboard and chalk, (ii) the share of pupils with pens is equal to or above 90 percent, and (iii) the share of pupils with notebooks in that classroom is equal to or above 90 percent. Functioning blackboard and chalk: The enumerator assesses if there was a functioning blackboard in the classroom, measured as whether a text written on the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. Pencils/pens and notebooks: The enumerator counts the number of pupils with pencils or pens and notebooks, respectively. By dividing each count by the number of pupils in the classroom, one can then estimate the share of pupils with pencils or pens and the share of pupils with notebooks. Availability of textbooks The indicator measures in one randomly selected grade 4 class the number of pupils with the relevant textbooks (mathematic or language conditional on which randomly selected class is observed), and divided by the number of pupils in the classroom. Pupil-teacher ratio The indicator of teachers’ availability is measured as the number of pupils per teacher in one randomly selected grade 4 class at the school based on the Classroom Observation Module. 18 The indicators Availability of teaching resources, Functioning school infrastructure, Pupil-teacher ratio, and Pupils per textbook are all constructed using data collected through visual inspections of a grade four classroom and the school premises in each primary school. Table 13 summarizes the findings. Table 13. School input indicators Difference Urban Rural Difference Indicator (% unless noted) All Public Private (%) Public Public (%) Minimum teaching equipment 28.4 27.1 30.8 -13.9 9.4 30.8 69.6*** Pupils with pencils 88.7 88.2 90.0 -2.0 82.9 89.2 7.1 Pupils with exercise books 74.3 71.7 79.5 -10.9 71.2 71.9 1.0 Classroom with board 100 100 100 0 100 100 0 Classroom with chalk 96.0 96.8 94.2 2.6 96.7 96.8 0.1 Contrast to read the board 32.1 34.1 28.0 18.1 10.5 38.8 72.8*** Minimum school infrastructure 22.8 14.9 39.2 -162.9*** 20.3 13.8 -46.6 Minimum visibility by 87.0 91.0 78.7 13.5* 87.1 91.8 5.2 enumerator Toilet functioning and available 24.4 18.5 37.6 -102.9** 22.4 17.7 -26.3 Toilet clean 34.4 23.5 58.2 -147.8*** 30.6 22.0 -39.3 Toilet private 38.4 29.6 57.6 -94.8*** 36.5 28.1 -30.0 Toilet available 50.7 38.7 76.9 -98.6*** 55.1 35.2 -56.5* Observed pupil-teacher ratio 29.1 31.0 24.8 20.0** 40.2 29.1 -38.0*** Textbook availability 68.5 76.0 52.6 30.8*** 76.8 75.9 -1.1 Pupils with math textbook (%) 66.1 75.4 41.3 45.2*** 76.5 75.2 -1.8 Pupils with French textbook (%) 73.9 77.9 68.8 11.7 77.5 77.9 0.5 Note: Results based on observations in 195 schools with data are collapsed at the school level. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Functioning school infrastructure Minimum infrastructure resources is a binary 0-1 capturing availability of: (i) functioning toilets operationalized as being clean, private, and accessible; and (ii) sufficient light to read the blackboard from the back of the classroom. Functioning toilets: Whether the toilets were functioning was verified by the enumerators as being accessible, clean, and private (enclosed and with gender separation). Electricity: Functional availability of electricity is assessed by checking whether the light in the classroom works and gives minimum light quality. The enumerator places a printout on the board and checks (assisted by a mobile light meter) whether it was possible to read the printout from the back of the classroom. As Table 14 shows, the inequality of the distribution of toilets is the most marked among the infrastructure indicators. Only one school in two has a toilet. Averaging across all schools, one school in four has functioning toilets that are accessible to its pupils, with cleanliness and privacy the primary constraints. Pupils in private schools are roughly twice as likely (p<0.01) as their public school counterparts to have any dimension of toilets (clean, private, and accessible). In general, public schools are similar, although there are significantly more private latrines in urban than in rural schools. Table 14. Sanitation access details (percent) Difference Public Public Difference All Public Private (%) Rural Urban (%) Functioning and available 24,4 18,5 37,6 -103,2** 17,7 22,4 -26,6 19 Clean 34,4 23,5 58,2 -59,6*** 30,6 22,0 28,1 Private 38,4 29,6 57,6 -48,7*** 36,5 28,1 23,0 Available 50,7 38,7 76,9 -38,2*** 35,2 55,1 -56,5 Note: Weighted means. Differences are relative to public and rural public, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Equipment availability is a binary variable equal to 1 if (i) the randomly selected grade four classroom has a functioning blackboard and chalk, (ii) the share of pupils with pens is equal to or above 90 percent, and (iii) the share of pupils with notebooks in that classroom is equal to or above 90 percent. Functioning blackboard and chalk: The enumerator assesses if there was a functioning blackboard in the classroom, measured as whether a text written on the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. Pencils/pens and notebooks: The enumerator counts the number of pupils with pencils or pens and notebooks, respectively. By dividing each count by the number of pupils in the classroom, one can then estimate the share of pupils with pencils or pens and the share of pupils with notebooks. Availability of teaching resources SDI measures whether or not all elements of the minimum teaching resources are simultaneously present in a given school rather than individual or average presence of the elements. Minimum teaching resources is a pressing constraint. A visual presentation of the co-availability of the inputs is in Figure 10. Although the general level is particularly low for contrast (32 percent) and insufficient for exercise books (68.5 percent), there are strong rural-urban differences in public schools. Rural schools are 28 percentage points more likely to have blackboards with suitable contrast than in urban schools (p<0.01). Public school students are 30.8 percent more likely to have textbooks than their private school counterparts (p<0.01). As a check of the field staff’s evaluations of the blackboard contrast, which is part of the equipment indicator, a sufficient level of light (measured with a lux meter) is more than twice as common in rural than in urban schools, with a difference (20.2 percentage points; p<0.05) that is larger than the urban average.19 While chalk and boards are generally available, pupils’ learning resources are somewhat less available. Most pupils have access to a pen and an exercise book, however all pupils have a pen in only 47 percent of classrooms and an exercise book in only 28 percent of classrooms. These are the factors that drive down the overall infrastructure indicator. In many classrooms, Togolese schoolchildren use slates and chalk as a substitute for paper and a pen, but the average availability of such items is far lower (19 percent of schoolchildren) than for exercise books (74 percent). There is no statistical difference in the availability of exercise books across ownership status or location. However, the use of slates is 2.3 times more likely in the public (22.8 percent of pupils) than in the private sector (p<0.05) and 2.7 times more likely in rural than in urban environments (p<0.01). Availability of textbooks The indicator measures in one randomly selected grade 4 class the number of pupils with the relevant textbooks (mathematic or language conditional on which randomly selected class is observed), and divided by the number of pupils in the classroom. 19Values above 300 lux are considered sufficient light. Since the rainy season had started, classrooms with small or poorly placed windows were often at risk of having poor light if storms were in the offing or if it was raining. 20 At the end of the observation in French and mathematics classes, textbook availability was directly measured by asking pupils to raise the textbook for the subject observed by the field teams. Government’s recent efforts to increase textbook availability in Togo are visible when comparing public and private schools. In Togo, 68.5 percent of pupils have access to a textbook, although there are some disparities. Private school pupils are 70 percent as likely to have a textbook as their public school counterparts (p<0.01) and only 55 percent as likely to have a math textbook as their public counterparts (p<0.01). 20 The general level of textbook availability is not significantly different across public schools in rural and urban areas. Pupil-teacher ratio The indicator of teachers’ availability is measured as the number of pupils per teacher in one randomly selected grade four class at the school based on the classroom observation module. Fourth grade pupils in Togo have nearly 42 classmates on average, compared to the average of 39 classmates from the government’s education statistics. Private school pupils have nearly seven fewer classmates (p<0.05) than their public school counterparts. While urban classrooms are generally larger than rural ones, this hides the extra ten classmates that an urban public school pupil has relative to his rural counterpart (p<0.01). This represents an extra 24 percent of pupils for the teacher and may contribute to reducing teaching effectiveness. 20The Global Partnership for Education-financed operation in Togo provided 2.9 million textbooks in the years immediately prior to the survey, which may explain this result. 21 Figure 7. Availability of inputs Assessment of pupil learning It is instructive to think of the Service Delivery Indicators as measuring key inputs, with a focus on what teachers do and know, in an education production function. These inputs are actionable and are collected using objective and observational methods at the school level. The outcome in such an education production function is pupil learning achievement. While learning outcomes capture both school-specific inputs (for instance, the quality and effort exerted by the teachers) and various child-specific factors (for instance, innate ability) and household- specific factors (e.g. household welfare), and thus provide, at best, reduced form evidence on service provision, it is still an important measure to identify gaps and to track progress in the sector. Moreover, while the Service Delivery Indicators measure inputs (learning outcomes are not part of the Indicators), in the final instance we should be interested in inputs not in and of themselves, but only in as far as they deliver the outcomes we care about. Therefore, as part of the collection of the Service Delivery Indicators in Togo, learning outcomes were measured for grade four pupils. This section reports on the findings. The objective of the pupil assessment was to assess basic reading, writing, and arithmetic skills. The test was designed by experts in international pedagogy and based on a review of primary curriculum materials from 13 African countries.21 The pupil assessment also measured nonverbal reasoning skills on the basis of Raven’s matrices, a standard IQ measure that is designed to be valid across different cultures. This measure complements the pupil test scores in French and mathematics and can be used as a rough measure to control for innate pupil ability when 21For details on the design of the test, see Johnson, Cunningham, and Dowling (2012) “Draft Final Report, Teaching Standards and Curriculum Review”. 22 comparing outcomes across different schools. Thus, the pupil assessment consisted of three parts: mathematics, French, and non-verbal reasoning (NVR). The test was examined by the CNE for suitability in the Togolese context, judged generally acceptable, and was administered to fourth-grade pupils.22 The reason for the choice of grade four pupils is threefold. First, there is no standardized national or international evaluation of this level, although PASEC evaluates grades two and five. Second, the sample of children in school becomes more and more self-selective as one goes higher up due to drop-out rates. Finally, there is growing evidence that cognitive ability is most malleable at younger ages. It is therefore especially important to get a snapshot of pupil’s learning and the quality of teaching provided at younger ages. The test was designed as a one-on-one test with enumerators reading out instructions to pupils in their mother tongue. This was done so as to build up a differentiated picture of pupils’ cognitive skills, using oral one-to-one testing allows us to evaluate whether a child can solve a mathematics problem even when her reading ability is so low that she would not be able to attempt the problem independently. The French test consisted of a number of different tasks including knowledge of the alphabet, word recognition, reading out loud, and reading comprehension. The mathematics test contained tasks such as identifying and sequencing numbers, addition of one- to three-digit numbers, one- and two-digit subtraction, and single digit multiplication and divisions. The non-verbal reasoning section consisted of four questions. Pupils in Togo achieved an average score of 45.1 percent on the assessment (Table 15). This masks very large variations in the ownership and location dimensions that are almost always significant at the one percent level. These differences are correlations, but cannot be read as causal statements, since there are both omitted variables and historical effects for which there is no accounting. Private school pupils scored 64 percent or fully 26 percentage points more than their public school counterparts. 23 Likewise, urban public pupils score 30 percent, or 11 percentage points higher than their rural peers. This pattern is matched by urban schoolchildren, who score 50 percent, or 20 percentage points, more points than their rural peers (Table 15). 22 The curriculum guidelines give the following time table for division: conceptual introduction in February, single-digit division in March, and continuation in April and May. Division is introduced based upon the multiplication table. Multiplication is initially taught in January. 23 It stands to reason that pupil absences hinder knowledge acquisition. However, the school-level absence data do not provide a conclusive answer, whether using the data from the fourth-grade class observation or the data for all grades in the school. Generally, private school children are less absent than their public school counterparts (Table 29). 23 Table 15. Student performance results (percent) % Diff. % Diff. Urban Rural All Public Private (Public- (Rural Public- Public Public Private) Urban Public) Pupil overall test score 45.1 38.4 63.8 46.9 36.0 -66.1*** -30.4*** Pupil literacy score 44.9 37.3 66.3 47.2 34.5 -77.7*** -37.0*** Pupil numeracy score 43.9 41.0 52.1 43.1 40.4 -27.1*** -6.6 Pupil nonverbal reasoning 53.2 52.5 55.1 56.0 51.5 -5.0* -8.7*** score Pupil can read letter 76.7 71.4 91.5 76.8 69.9 -28.2*** -9.9 Pupil can identify basic words 63.7 56.7 83.3 68.9 53.3 -46.9*** -29.2*** Pupil can read simple 25.6 16.1 52.2 27.1 13.0 -224.2*** -109.2*** sentence Pupil can read paragraph 16.4 8.4 38.6 16.6 6.1 -359.5*** -171.7*** Pupil comprehension score 16.9 9.6 37.3 17.3 7.4 -288.5*** -133.6*** Can do single digit addition 76.5 73.6 84.7 74.3 73.3 -15.1*** -1.3 Can do double digit addition 64.6 60.8 75.4 63.3 60.1 -24.0*** -5.4 Can do single digit subtraction 64.5 59.9 77.3 62.9 59.0 -29.0*** -6.6 Can do double digit 20.8 16.1 34.0 22.2 14.4 -111.2*** -54.7** subtraction Can multiply single digits 10.5 10.1 11.7 13.4 9.2 -15.8 -45.7 Can multiply double digits 5.3 3.1 11.4 5.0 2.6 -267.7*** -92.0 Can do single digit division 35.1 29.1 51.7 32.3 28.2 -77.7*** -14.6 Can do double-digit division 11.1 6.6 23.8 7.5 6.3 -77.2*** -18.1 Note: Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Results based on 1, 938 pupils in 195 schools (weighted results). Differences are relative to public and rural public, respectively. The regressions reported in Table 31 (French) and Table 32 (math) and shown graphically in Figure 8 provide some broadly suggestive results. Female pupils do at least three (four) standard deviations worse than their male peers in French (math) and the results are at least significant at the five percent level in all models. Rural students do worse in French (generally significant at p<0.10), but not in math. 24 Figure 8. Student learning correlates (augmented models without poverty) Language Math Student Student's age Girl School Maritime region Plateaux region Central region Kara region Savanes region Rural Private school Exercise book (%students) Textbook (%students) Teacher Time spent Teaching Grade 4 teacher's language score Grade 4 teacher's pedagogy score Grade 4 teacher's math score -40 -20 0 20 -40 -20 0 20 Geographically speaking, the simple models suggest that as the school’s region is increasingly far from the capital, the negative association with student performance becomes stronger (with the exception of the Centrale region). Interestingly, after controlling for canton-level poverty, the regional and rural dummies are no longer significant, perhaps reflecting the results from the last poverty survey, which showed 29% percent poverty in the capital area, 41% percent in other towns, 67% percent in the rural south, and 86% percent in the rural north. Table 6 shows a clear pattern in schooling types: in the poorest quintile eight percent of children attend public schools and 10 percent attend private (faith-based or NGO) schools; in the richest quintile 41 percent attend public schools and 57 percent attend private schools. The relative shares of public and private decline and increase, respectively, in a linear fashion from the lowest to the highest quintile. Those children of the lowest quintile who attend private school overwhelmingly attend faith-based schools. As a family’s income rises, children attending private school are increasingly likely to attend non-faith based schools. This pattern of inequality is clearly highlighted in the comparative performance of pupils in public and private schools (Figure 9) and in the pupil performance regressions. 25 Figure 9. Pupil evaluation distribution by section and school ownership Overall French .025 .02 .02 .015 .015 .01 .01 .005 .005 0 0 20 40 60 80 100 0 20 40 60 80 100 Mathematics Non-verbal reasoning .025 56.4 .02 44.2 .015 30.932.0 .01 15.9 .005 7.5 3.4 5.1 1.8 2.8 0 0 25 50 75 100 0 20 40 60 80 100 Private Public Note: private in blue, public in red; vertical: density, horizontal: score. Table 16 presents breakdowns across gender and school location, while Table 17 breaks down the data across gender and school ownership. The results in language show strong differences. As shown in Figure 9, private school pupils score 29 percentage points, or 78 percent, better higher than their public school counterparts (significant at the one percent level). Among public schools, urban pupils score 12.8 percentage points higher than rural ones (p<0.01). In general, rural pupils score 22.7 points lower than their urban counterparts (p<0.01). Consistently, the scoring gap rises across basic skills and peaks at the capacity to read a simple sentence, while remaining nearly as high for the comprehension elements. At this peak difference, private school pupils perform 4.6 times better than their public counterparts, urban public pupils 2.7 times better than rural public pupils, and urban pupils 3.3 times better than rural ones. This is borne out in Figure 9. A similar pattern exists between rural and urban schools (Figure 11). In mathematics, the differences are less pronounced in percentage terms, but private school pupils still score 11.1 percentage points higher than public school pupils (p<0.01) and urban public pupils perform 2.7 percentage points better than rural ones. Generally, private school pupils perform at least 1.2 and 3.6 times as well as their public school counterparts (generally significant p<0.01). When comparing across genders and school ownership, the major difference is public/private rather than boy/girl, as Table 17 shows. Private school girls do better in all areas than their male classmates. However, private school boys do significantly better than public school boys and girls, generally at the one percent level. Interestingly, public school girls do not do better than public school boys. 26 Table 16. Pupil evaluation: gender and location breakdowns (percent) % Diff. % Diff. % Diff. % Diff. (Urban (Urban (Urban Urban Rural Urban Rural (Urban All boys- boys- boys- boys boys girls girls girls-Rural Rural Urban Rural girls) boys) girls) girls) Overall 45.1 62.1 41.0 56.2 36.9 34.0*** 9.5*** 40.6*** 34.3*** French 44.9 64.3 40.1 58.1 35.6 37.6*** 9.6*** 44.6*** 38.7*** Mathematics 43.9 51.1 43.3 46.7 40.4 15.3*** 8.6*** 20.9*** 13.5*** Non-verbal 53.2 58.5 52.0 54.6 51.7 11.1*** 6.7** 11.6*** 5.3* reasoning Note: Weighted means using sampling weight and the sample design. Results based on 1,928 pupils in 194 schools. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Gender inequality (Table 16) is both marked across public/private and across locations. However, the larger differentials are clearly across locations as rural pupils’ score roughly one-third lower than their urban peers. Overall levels are low and there is no significant gender inequality in these results, although boys generally score two to three points higher in all sections (except non-verbal reasoning) than girls in a given location. However, comparing genders across locations, there are strong differences in all areas. The same is true when using public/private instead of rural/urban (Table 17), except in private schools, where girls generally do better, if not statistically significantly, than boys. Table 17. Pupil evaluation: gender and school ownership breakdowns (percent) % Diff. % Diff. % Diff. % Diff. (Private (Private (Private (Private Priva Publ Priva boys- boys- Boys- Girls- Public te ic te Public Public Private Public All boys boys girls girls boys) girls) Girls) Girls) Overall 45.1 41.3 62.6 35.6 64.7 51.6*** 43.1*** -3.4 45.0*** French 44.9 40.4 64.9 34.2 67.5 60.6*** 47.3*** -4.0 49.3*** Mathematics 43.9 43.2 51.8 38.8 52.4 19.9*** 25.1*** -1.2 26.0*** Non-verbal reasoning 53.2 52.8 56.5 52.2 53.8 7.0* 7.6** 4.8 3.0 Note: Weighted means using sampling weight and the sample design. Results based on 1,928 pupils in 194 schools. Differences are relative to private-school boys, except for the final difference which compares private to public for girls. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Language factors may play an important role in overall performance. While gender inequality is not very marked across-gender and within-location, it is quite different within-gender and across locations. This is suggested by the difference in the distributions in mathematics and language. Although there are differences in mathematics, they are far smaller than those in language. What does this mean for Togo? It is useful to consider the Togolese context as the Education Sector Plan for 2010-2020 was being prepared. In 2006, slightly more than three-quarters (78 percent) of all boys completed primary school as compared to 61 percent of girls. Rural literacy rates were 48 percent female and 74 percent male, and more than one adult in four (28 percent) who had completed primary school did not remain literate as an adult. The results of the 2010 PASEC evaluation, particularly when compared to the 2000 evaluation, showed a system that was regressing. The Education Sector Plan (2010-2020) set as its priority objective to achieve universal quality primary education by 2020. It 27 proposed a balanced development of the education and training system as a whole with emphasis on improving quality and external efficiency of post-basic education for contributing better to economic growth. The commitment to improvement was in a positive context as Togo reached the HIPC Completion Point in 2010. According to the Togo BOOST data, salaries and investments are the two largest budgeted items for the ministry.24 However, the execution rates make investments the worst performer for the ministry. Although the ministry receives above-average allocations, its execution rates for personnel (average) and investments (28 percent percentile or below) limit its ability to deliver services. The Creditor Reporting System of the OECD/DAC allows the disaggregation of funds by sector; in the case of primary education in Togo, it shows limited resources. External financing, remained quite limited relative to the overall aid envelope (average value USD 3.6 million in constant dollars per year for 2004-2012, or one percent of the overall aid budget), albeit with lower volatility (the standard deviation is 50 percent of the median as compared to 148 percent of the median). It was also limited relative to government’s own budgetary execution for the sector (even after excluding investments that might be externally-financed). From the financing perspective, this would have created a context with little room for improvement, limited funds for inputs (functioning stayed flat in the face of the effect of free primary education), and limited physical capital improvements. These budgetary pressures in the face of growing pupil populations and insufficient teachers led to a strategy of lower-cost teachers, which may have lowered the quality of teachers. (Vargas and De Laat, 2003) In addition, there is an important element of volunteer teachers, who did not achieve entry to teachers’ colleges, but attempt to join the civil service ranks through years of teaching experience. PASEC evaluations conducted in the 2000/1 and 2010/1 school years showed a marked decline in the level of both second and fifth-grade teachers, suggesting considerable challenges in the primary education system. (PASEC and MEPSA, 2012). The results of these and other factors is that educational system in Togo shows clear divisions in the dimensions of provider ownership (public/private) and location (rural/urban). For inputs, private is superior to public in both infrastructure availability (31 percent versus 27 percent) and teaching equipment availability (39 percent vs. 15 percent; significant at the one percent level) while urban public is better than rural public for infrastructure (20 percent vs. 14 percent), but not for equipment (nine percent vs 31 percent, significant at the one percent level). Although Togo is still primarily rural (62.3 percent; 2011 census), there is greater pressure in urban public classrooms, where the pupil-teacher ratio is 10 pupils higher than in rural classrooms (significant at the one percent level). Likewise, private school pupils have nearly seven less classmates than their public peers (significant at the five percent level). These factors are compounded by generally weak levels of minimum knowledge; less than two percent of Togolese teachers score 80 percent on an evaluation of their math and French knowledge at the lower primary level. Private school teachers do three times better than their public school peers, but still score only three percent. Urban public school teachers are five times more likely to be proficient by this metric, but also score three percent. The absence rate statistics are biased upwards affected by the strikes, but the general pattern is that teachers are absent from school and absent from the classroom in roughly similar proportions across all breakdowns. In the end, the production function is designed to equip pupils with knowledge and skills to either 24 From http://isdatabank.info/boost_togo/, accessed on 9 September 2014. Transfers were larger than investments in 2009. 28 continue with their education or to join the workforce. The situation depicted in Figure 9, Figure 11, Table 16, and Table 17 show strong inequalities of pupil learning outcomes across school ownership, school location, and pupil gender. These remain the ultimate challenge for the system. Comparing Togo with other countries that have done SDI In the area of teacher effort, Togo has a school absence rate that is similar to most countries, but the classroom absence rate is lower, despite the teacher strikes. Teaching time is likewise relatively similar to other countries that have done SDI surveys, even across languages, with the notable exceptions of Nigeria and Kenya that have roughly three-quarters of an hour of additional teaching time. However, minimum knowledge among teachers is particularly low in Togo—this is the most significant negative deviation among all of the indicators, although they are positively correlated with pupil performance in the evaluated subjects (Table 31 and Table 32). Compared to the other countries surveyed, Togolese schoolchildren have a higher share of pupils with textbooks than almost any other country. This was positively correlated with pupil performance in both mathematics and French in the regression analysis (Figure 8). Togo lags behind countries like Kenya and Uganda in infrastructure availability and is significantly worse on teaching equipment availability. The former is largely due to problems with electricity and latrines, while the latter is imputable primarily to the relatively few schools that have all required elements simultaneously (even allowing for 90 percent availability of pencils/pens and exercise books). 29 Annex 1. Sampling The SDI survey was done jointly with the PASEC survey undertaken by the Ministry of Education’s CNE and therefore used its sample. What follows is a synthesis of the sampling approach for that survey; the detailed information is available in the survey report. The population of interest was defined as the universe of second, fourth, and fifth graders enrolled in Togolese primary schools that adhered to the national curriculum.25 Only formally-recognized primary schools are considered, excluding community schools and special-needs schools (e.g. handicapped). A community schools stratum was considered and abandoned, since government routinely adds them to the list of official schools and then provides normal support even during the school year. The sample frame was drawn from the 2011/2 school statistical yearbook produced by the statistical department (DPEE) of the Ministry of Education. The yearbook contains a list of all recognized schools that have a primary cycle. For sampling purposes, the yearbook contains the universe of schools, the size of each grade in each school, the administrative geography, and the physical geography (rural/urban). Inferences about the expected population of pupils in grades two, four, and five pupils in 2012/3 were based upon the 2011/2 information for pupils in grades one, three, and four and assumptions regarding progression rates. The sample frame was stratified along ownership and regional lines to maximize within-group homogeneity. There are eight strata that may be broken down into public and non-public and further within each group. Public schools are broken down by education regions (six in total) and non-public schools are broken down into faith-based and non-denominational institutions. Depending upon the concepts of interest, the sampling procedure is done in two (absence rate, classroom observation) or three stages (pupils). In all cases, the first stage is the drawing of schools independently and without replacement from the different strata. The draw was done with probability proportional to the size of the school, defined as the sum of first, third, and fourth graders in 2011/2. This is the first weight. The different approaches are explained below. To calculate the absence rate, a secondary sampling frame of all teachers who work at the school is prepared (Module 2A). The methodology requires ten teachers, or all those in the school if it has less than ten teachers. Volunteer teachers are excluded from this count, since they have no contractual obligation to be present at any given time. If a school has more than ten teachers, a random selection without replacement is undertaken where each teacher has equal probability of being selected. This gives the inflation factor, or weight, for the teacher absence rate, defined as the inverse of the product of the probability of selecting the school and the probability of selecting a given teacher in the school. For classroom observation, the secondary sampling frame is the number of fourth-grade classrooms in use in the school (Module 1). A simple random selection with equal probability of selection was undertaken if there was more than one relevant classroom in the school, giving a probability weight for classrooms. This secondary sampling unit was used for the selection of pupils, the third-stage sampling unit. From the classroom selected for observation, the procedure is to draw up to ten pupils for the 25 This excludes, among other cases, all schools adhering to other curricula, e.g. “American” or “English” schools. 30 evaluation of their learning outcomes (Module 5). The sample frame for the selection of students for this was the teacher’s pupil roster updated to remove pupils not present on the day of the first visit. A random selection without replacement and with equal probability of selection was undertaken if there were more than ten pupils in the classroom. A risk of pupils not returning after lunch was identified during pre-testing and mitigated during piloting through two steps. The field teams were instructed to learn from the teacher which sampled pupils might not return after lunch in order to minimize the risk of truncating the lower tail of the performance distribution. In the same sample procedure, the teams also drew three extra pupils that were kept in reserve in case one of the first ten pupils was not available. A probability weight is calculated for these pupils as the inverse of the product of the probabilities of selecting the school, the classroom, and the student and is used to estimate pupil-related indicators. The size of the sample was determined by the experience of the PASEC surveys and the tradeoff inherent in the clustered sampling of pupils rather than simple random sampling. PASEC has estimated the intra-cluster correlation to be 0.4. Based upon the research done by Ross and Postlethwaite (1988) for the IEA, a sample slightly larger than the size required to provide the equivalent level precision for the population mean of a simple random sample of 400 pupils for clusters of 15 pupils was drawn. This sample is also expected to provide five percent precision for the estimation of proportions. With an (assumed) intra-cluster correlation of 0.4 and a cluster size of 15 pupils (PASEC), a sample of 176 schools was required. To leave a margin for error, 200 schools were selected. In the case of Togo, the observed intra-cluster correlation from the sample is 0.44 for the overall score, 0.42 for the language, 0.37 for the math, and 0.12 for the non-verbal reasoning. This is both consistent with what is observed in other SDI samples (range: 0.33 to 0.53) and, more generally, in the field. Grading of teacher evaluations was done by the Laboratoire des Sciences de l’Education et de la Formation of the University of Lomé. To ensure inter-rater reliability, 15 percent of the sample was doubled-graded blindly. These were monitored for quality using both pair-wise correlations and Cohen’s Kappa as measures of reliability. Initial correlations were above 95 percent on the language and mathematics sections and the Kappa scores were all significant. Problems identified by across graders were corrected and used to identify broader potential problems that were also corrected. 31 Annex 2. Definition of the Service Delivery Indicators in Education School absence rate Share of a maximum During the first announced visit, a maximum of ten teachers are randomly selected of ten randomly from the list of all teachers who are on the school roster. The whereabouts of these selected teachers ten teachers are then verified in the second, unannounced visit. Teachers found absent from school anywhere on the school premises are marked as present. during an unannounced visit Classroom absence rate Share of teachers The indicator is constructed in the same way as the school absence rate indicator, who are present in with the exception that now the numerator is the number of teachers who are either the classroom absent from school, or present at school but absent from the classroom. A small during scheduled number of teachers may be found teaching outside, and these are marked as present teaching hours as for the purposes of the indicator. observed during an unannounced visit Time spent teaching per day (also known as Time on Task) Amount of time a This indicator combines data from the Staff Roster Module (used to measure absence teacher spends rate), the Classroom Observation Module, and reported teaching hours. The teaching teaching during a time is adjusted for the time teachers are absent from the classroom, on average, and school day for the time the teacher teaches while in classrooms based on classroom observations While inside the classroom distinction is made between teaching and non-teaching activities. Teaching is defined very broadly, including actively interacting with pupils, correcting or grading pupil's work, asking questions, testing, using the black board, or having pupils working on a specific task, drilling, or memorization. Non-teaching activities is defined as work that is not related to teaching, including working on private matters, maintaining discipline in class, or doing nothing and thus leaving pupils not paying attention. Minimum knowledge among teachers Share of teachers This indicator measures teacher's knowledge and is based on mathematics and with minimum language tests covering the primary curriculum administered at the school level to knowledge all mathematics or language teachers that taught grade three in the previous year or grade four in the year the survey was conducted. It is calculated as the percentage of teachers who score more than 80 percent on the language and mathematics portion of the test. The indicator is representative of the average teacher in the universe of teachers in a given country rather than the average teacher at the average school. Test score: This indicator measures teacher’s knowledge and it is calculated as the overall score of a mathematics, language, and pedagogy tests covering the primary curriculum administered at the school level to all mathematics and language teachers that taught grade three in the previous year or grade four in the year the survey was conducted. Infrastructure availability Unweighted average Minimum infrastructure resources is a binary variable capturing availability of: of the proportion of (i) functioning toilets operationalized as being clean, private, and accessible; and schools with the (ii) sufficient light to read the blackboard from the back of the classroom. following available: functioning Functioning toilets: Whether the toilets were functioning was verified by the electricity and enumerators as being accessible, clean, and private (enclosed and with gender sanitation separation). 32 Electricity: Functional availability of electricity is assessed by checking whether the light in the classroom works and gives minimum light quality. The enumerator places a printout on the board and checks (assisted by a mobile light meter) whether it was possible to read the printout from the back of the classroom. Teaching Equipment availability Unweighted average Equipment availability is a binary variable equal to one if (i) the randomly of the proportion of selected grade four classroom has a functioning blackboard and chalk, (ii) the schools with the share of pupils with pens is equal to or above 90 percent, and (iii) the share of following available: pupils with notebooks in that classroom is equal to or above 90 percent. functioning blackboard with Functioning blackboard and chalk: The enumerator assesses if there was a chalk, pencils, and functioning blackboard in the classroom, measured as whether a text written on notebooks the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. Pencils/pens and notebooks: The enumerator counts the number of pupils with pencils or pens and notebooks, respectively. By dividing each count by the number of pupils in the classroom, one can then estimate the share of pupils with pencils or pens and the share of pupils with notebooks. Share of pupils with textbooks Number of The indicator measures in one randomly selected grade four class the number of mathematics and pupils with the relevant textbooks (mathematic or language conditional on which language books used randomly selected class is observed), and divided by the number of pupils in the in a grade four classroom. classroom divided by the number of pupils present in the classroom Pupil- teacher ratio Average number of The indicator of teachers’ availability is measured as the number of pupils per grade four pupils teacher in one randomly selected grade four class at the school based on the per grade four classroom observation module. teacher 33 Annex 3. Additional Results At-a-glance tables for schools: non-striking and all Table 18. Education at a glance for non-striking schools and teachers Urban Rural All Public Private Public Public School absence rate 18.4 19.7 15.3 12.8 21.0 Classroom absence rate 33.6 34.2 32.1 30.4 34.9 Classroom teaching time 2h 55 min 2h 41 min 3h 18 min 3h 10 min 2h 31 min Minimum knowledge among teachers 2.5 1.4 4.8 4.5 0.8 Pupil‐teacher ratio 42.8 44.8 38.0 53.3 43.1 Pupils with textbooks (%) 68.5 76.0 52.6 76.8 75.9 Teaching Equipment availability 28.3 27.1 30.8 9.4 30.8 Infrastructure availability 22.8 14.9 39.2 20.3 13.8 Note: Volunteer teachers are excluded and only non-striking schools are considered in the absence rates and teaching time indicators. The information here repeats Table 2 for convenience. Table 19. Education at a glance for all schools and teachers Urban Rural All Public Private Public Public School absence rate 17.3 18.9 13.5 16.7 19.4 Classroom absence rate 31.5 34.0 25.6 28.4 35.1 Classroom teaching time 3h 5 min 2h 53 min 3h 23 min 3h 22 min 2h 54 min Minimum knowledge among teachers 2.5 1.4 4.8 4.5 0.8 Pupil‐teacher ratio 42.8 44.8 38.0 53.3 43.1 Pupils with textbooks (%) 68.5 76.0 52.6 76.8 75.9 Teaching Equipment availability 14.6 12.8 18.4 3.0 14.8 Infrastructure availability 22.8 14.9 39.2 20.3 13.8 Note: All teachers are included; definitions are the same as for Table 18. 34 School breakdowns Figure 10. Education infrastructure co-availability Components of education infrastructure N = 6215 (96 %) Board functions (84 %) Chalk available 260 4% 2461 152 40 % 2% 1432 421 81 23 % 7% 1% 934 15 % 106 197 2% 3% Students with exercise book 96 (28 %) Students with pen 2% (47 %) 75 (1 %) 26 Aug 2014 % of total File: ../data/sdi/education/ip/sdi_educ_poids.dta (26 Aug 2014 ) Note: Ellipses are drawn for the case where the sub-indicator is available to all. The numbers in parentheses are the population percentage of schools with a score of 100% for the given sub-indicator. Data are weighted to account for the survey design. Unless otherwise stated, all tables are weighted. 35 Individual breakdowns Table 20. Teacher effort in all schools with all teachers % Diff. % Diff. (Urban Urban Rural All Public Private (Public- Public- Public Public Private) Rural Public) Absence from school 21.6 23.0 18.5 17.7 24.0 19.6 -35.6 Absence from class 37.2 38.9 33.3 33.5 40.0 14.4 -19.4 2h 40 2h 33 Time spent teaching 2h 38 min 2h 44 min 3h 8 min -3.8 18.6 min min Scheduled lesson time 5h 29 5h 28 5h 28 5h 28 min 5h 33 min -1.5 0.0 per day min min min Note: Results based on observations from 1,007 teachers in 195 schools. Data collapsed at the school level. Differences are in percentage points or units. Table 21. Teacher effort excluding schools with striking teachers and volunteer teacher s % Diff. % Diff. Urban Rural (Urban All Public Private (Public- Public Public Public- Private) Rural Public) Absence from school 17.3 18.9 13.5 16.7 19.4 28.6 -16.2 Absence from class 31.5 34.0 25.6 28.4 35.1 24.7 -23.6 2h 50 3h 22 2h 44 Time spent teaching 2h 55 min 3h 6 min -9.4 18.8 min min min Scheduled lesson time 5h 28 5h 32 5h 28 5h 29 min 5h 32 min -1.2 1.2 per day min min min Note: Results based on observations from 716 teachers in 173 schools (291 volunteer teachers are excluded). Data collapsed at the school level. Differences are in percentage points or units. 36 Table 22. Teacher absence rates, by teacher status (percent) % Diff. % Diff. % Diff. Contractual Civil Servant Auxiliary Volunteer (Civil Indicator (%) All (Auxiliary- (Volunteer- Estimate Estimate Estimate Estimate servant- Contractual) Contractual) Contractual) Absent from school 21.1 11 22.3 16.7 27.5 50.7* 34.1 60.0*** Absent from class, at 15.4 8.9 16.6 17.1 16.9 46.4* 48.0* 47.3* school Absent from class 36.3 19.7 38.9 33.7 44.4 49.4*** 41.5** 55.6*** Note: Estimates based upon individual teachers and not collapsed at the school level first. Differences are relative to contractual (private school) teachers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Differences are in the units of the indicator. Table 23. Public school teacher absence rates, by region (percent) % Diff. % Diff. % Diff. % Diff. % Diff. Lomé (Maritim (Savane Natio Maritim Plateau Central Kar (Plateau (Centrale (Kara- Indicator (%) / Savanes e- s- nal e x e a -Lomé/ - Lomé/ Lomé/ Golfe Lomé/ Lomé/ Golfe) Golfe) Golfe) Golfe) Golfe) Absent from 13. 23.6 27.4 30.8 13.2 21.7 41.4 11.0 -107.6* -26.3 -100.0 33.8 school 7 Absent from 16.8 12.7 17.3 26.2 23.5 1.6 13.0 26.6 51.5 46.0 -693.8** 2.3 class, at school Absent from 15. - 40.3 40.0 47.9 39.5 45.2 54.3 16.5 -1.3 11.5 26.3 class 4 159.7*** Number of 811 132 149 178 118 101 133 11.4 25.8 -11.9 teachers Note: Estimates based upon individual teachers and not collapsed at the school level first. Differences are relative to the Lomé/Golfe region teachers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Differences are in the units of the indicator. 37 Table 24. Teacher effort and knowledge for all schools and non-volunteer teachers % Diff. % Diff. (Urban Urban Rural All Public Private (Public- Public- Public Public Private) Rural Public) Absent from school (%) 18.4 19.7 15.3 12.8 21.0 22.3 -64.1 Absent from class (%) 33.6 34.2 32.1 30.4 34.9 6.1 -14.8 Time on task 2h 55 min 2h 41 min 3h 18 min 3h 10 min 2h 31 min -23.0** 20.5 Lesson taught (%) 52.6 49.0 58.1 57.4 46.3 -18.6* 19.3 Scheduled teaching day 5h 29 min 5h 28 min 5h 33 min 5h 28 min 5h 28 min -1.5 0.0 Teachers with minimum 3.8 2.2 6.7 5.2 1.3 -204.5 75.0 knowledge (%) Teacher average score (%) 46.5 43.6 52.0 52.7 40.9 -19.3** 22.4*** Note: Data are collapsed at the school level. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Table 25. Teacher effort and knowledge for all schools and teachers % Diff. % Diff. (Urban Urban Rural (Public- All Public Private Public- Public Public Private Rural ) Public) Absent from school (%) 21.6 23.0 18.5 17.7 24.0 19.6 -35.6 Absent from class (%) 37.2 38.9 33.3 33.5 40.0 14.4 -19.4 2h 39 2h 37 2h 44 2h 31 Time on task min min min 3h 8 min min -4.5 19.7* Lesson taught (%) 48.6 48.4 49.0 57.6 46.6 -1.2 19.1* Teachers with minimum 3.8 2.2 6.7 5.2 1.3 -204.5 75.0 knowledge (%) Teacher average score 46.5 43.6 52.0 52.7 41.0 -19.3** 22.2*** (%) Note: Data are collapsed at the school level. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. . 38 Table 26. Teacher evaluation breakdowns Panel A: Teacher evaluation: ownership and urban-rural within public breakdown (percent) % Diff. % Diff. Publi Privat Urban Rural (Urban All (Public- c e Public Public Public- Rural Private) Public) Math and French 45.1 43.3 48.7 49.6 41.9 -12.5*** 15.5*** French 50.9 49.6 53.7 54.4 48.5 -8.3** 10.8*** Grammar 76.2 74.6 79.7 80.5 73.2 -6.8*** 9.1*** Cloze task 30.2 29.1 32.6 33.0 28.2 -12.0 14.5 Composition 25.2 25.2 25.3 29.5 24.2 -0.4 18.0*** Math 33.4 31.1 38.1 38.6 29.5 -22.5** 23.6** Basic math 39.3 37.1 43.8 44.7 35.4 -18.1** 20.8** Advanced math 22.5 20.0 27.6 27.2 18.3 -38.0** 32.7** Fractions 14.7 12.9 18.4 19.2 11.5 -42.6* 40.1** Venn diagrams 22.0 19.7 26.8 28.3 17.8 -36.0* 37.1** Graphs 14.1 12.1 18.2 17.6 10.8 -50.4** 38.6** Pedagogy 19.6 19.3 20.2 20.1 19.1 -4.7 5.0 Lesson 28.1 28.1 28.0 28.6 28.0 0.4 2.1 preparation Pupil 0.3 0.3 0.3 0.3 0.3 0.0 0.0 comparisons Pupil evaluations 6.5 6.0 7.5 6.2 6.0 -25.0 3.2 Teachers 831 609 222 457 152 Note: These results are also shown in Tables 8-10 as subject-specific tables. Bold-faced items are the major indicators and the primary breakdowns. Items not in boldface are further details of their bold-faced headers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 39 Panel B. Teacher evaluation: contractual status breakdown (percent) % Diff % Diff. % Diff. (Private Civil Private (Auxiliary- (Volunteer- Indicator (%) All Auxiliary Volunteer School- servant school Civil Civil Civil Servant) Servant) Servant) Average score 45.1 44.8 50.0 40.3 49.1 10.4*** -11.2** 8.8* Minimum 1.8 0.4 4.3 1.0 3.0 90.7* 60.0 86.7 knowledge French 50.9 51.4 53.3 48.6 52.0 3.6 -5.8 1.2 Grammar 76.2 76.5 80.7 72.0 79.1 5.2** -6.3** 3.3 Cloze task 30.2 30.8 30.4 29.9 29.8 -1.3 -3.0 -3.4 Composition 25.2 27.5 32.0 20.6 24.3 14.1** -33.5*** -13.2 Math 33.4 32.7 41.1 26.0 40.0 20.4*** -25.8** 18.3* Basic math 39.3 38.2 46.7 32.0 46.2 18.2*** -19.4** 17.3** Advanced math 22.5 22.3 30.6 14.8 28.2 27.1** -50.7*** 20.9 Fractions 14.7 12.8 22.4 10.5 17.9 42.9*** -21.9 28.5 Venn diagram 22.0 21.6 28.9 15.5 26.8 25.3 -39.4 19.4 Graphs 14.1 12.9 23.3 6.9 19.9 44.6*** -87.0*** 35.2* Pedagogy 19.6 21.6 23.7 15.1 20.7 8.9 -43.0*** -4.3 Lesson 16.3 18.4 21.0 11.5 17.3 12.4 -60.0*** -6.4 preparation Pupil 25.2 27.1 28.3 21.4 26.6 4.2 -26.6*** -1.9 comparisons Pupil evaluations 28.1 30.4 32.6 24.5 27.1 6.7 -24.1** -12.2 Teachers 0.3 0.4 161 254 127 Note: Bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold-faced headers. Differences are relative to civil servants. A community teacher is excluded. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 40 Panel C. Teacher evaluations: academic training breakdowns (percent) % Diff. % Diff. (Lower (High School Lower High school Indicator (%) All Baccalaureate Secondary- incomplete- secondary incomplete Baccalaureat Baccalaureate e) ) Average score 45.1 52.8 37.7 45.0 -40.1*** -17.3*** Minimum knowledge 1.8 3.7 2.4 0.7 -54.2 -428.6 French 50.9 56.0 45.4 51.2 -23.3*** -9.6*** Grammar 76.2 82.7 69.0 76.4 -19.9*** -8.2*** Cloze task 30.2 33.7 26.6 30.7 -26.7*** -9.8 Composition 25.2 34.2 16.9 24.7 -102.4*** -38.5*** Math 33.4 43.7 24.2 32.7 -80.6*** -33.6*** Basic math 39.3 49.6 29.9 38.7 -65.9*** -28.2*** Advanced math 22.5 32.8 13.4 21.6 -144.8*** -51.9*** Fractions 14.7 24.4 8.5 12.6 -187.1*** -93.7*** Venn diagram 22.0 31.2 13.7 21.2 -127.7*** -47.2** Graphs 14.1 24.2 5.6 12.7 -332.1*** -90.6*** Pedagogy 19.6 27.3 13.0 18.9 -110.0*** -44.4*** Lesson preparation 28.1 36.2 20.7 27.5 -74.9*** -31.6*** Pupil comparisons 0.3 0.5 0.2 0.3 -150.0*** -66.7*** Pupil evaluations 6.5 9.3 2.8 6.7 -232.1*** -38.8* Teachers 831 177 172 460 Note: Bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold-faced headers. Differences are relative to Baccalaureate. Teachers in “lower secondary” completed primary and went up to and including the BEPC. There are t hree teachers with only primary education and 19 with a university degree of any kind. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 41 Panel D. Teacher evaluations: grade taught breakdowns (percent) % Diff. % Diff. % Diff. % Diff. % Diff. Indicator (%) All Gr.4 Gr.1 Gr.2 Gr. 3 Gr.5 Gr.6 (Gr.5- (Gr.6- (Gr.1- Gr.4) (Gr.2-Gr. 4) (Gr.3-Gr.4) Gr4) Gr.4) Average score 45.1 46.5 41.1 38.5 43.3 48.2 53.2 -13.1 -20.8*** -7.4 3.5 12.6*** Minimum knowledge 1.8 3.8 0.0 0.6 1.2 1.8 3.0 -533.3 -216.7 -111.1 -26.7 French 50.9 52.6 48.7 47.3 49.0 51.9 57.2 -8.0 -11.2*** -7.3* -1.3 8.0*** Grammar 76.2 77.1 69.9 72.3 74.2 78.8 83.0 -10.3 -6.6** -3.9 2.2 7.1*** Cloze task 30.2 32.7 32.1 27.1 28.6 29.6 36.0 -1.9 -20.7** -14.3 -10.5 9.2 Composition 25.2 27.3 19.9 19.5 22.5 28.6 32.3 -37.2** -40.0*** -21.3** 4.5 15.5** Math 33.4 34.1 27.4 24.5 31.6 39.0 42.2 -24.5 -39.2*** -7.9 12.6* 19.2*** Basic math 39.3 40.4 33.6 29.6 37.5 44.5 49.1 -20.2 -36.5*** -7.7 9.2 17.7*** Advanced math 22.5 22.1 15.8 15.0 20.4 28.6 29.4 -39.9 -47.3** -8,3 22.7** 24.8** Fractions 14.7 16.5 4.5 5.7 12.8 20.5 23.5 -266.7*** -189.5*** -28.9 19.5 29.8 Venn diagram 22.0 21.8 22.6 15.8 19.9 27.5 25.1 3.5 -38.0 -9.5 20.7 13.1 Graphs 14.1 16.3 2.5 8.5 11.2 21.1 16.7 -552.0*** -91.8** -45.5 22.7 2.4 Pedagogy 19.6 20.1 12.3 15.0 19.5 22.3 23.9 -63.4*** -34.0*** -3.1 9.9 15.9** Lesson preparation 28.1 27.6 20.8 22.5 29.1 30.5 33.9 -32.7 -22.7** 5.2 9.5 18.6** Pupil comparisons 0.3 0.4 0.2 0.2 0.3 0.4 0.4 -100.0*** -100.0** -33.3 0.0 0.0 Pupil evaluations 6.5 7.2 2.3 4.5 5.1 7.7 11.0 -213.0*** -60.0** -41.2* 6.5 34.5** Teachers 931 149 37 163 179 188 115 Note: Bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold-faced headers. Differences are relative to Grade 4 teachers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 42 Panel E. Teacher evaluations: teacher training breakdowns (percent) % Diff. (CAP- % Diff. (CEAP- % Diff. (None- Indicator (%) All CAPE CAP CEAP None CAPE) CAPE) CAPE) Average score 45.1 51.5 45.3 41.6 45.9 -13.7*** -23.8*** -12.2* Minimum knowledge 1.8 2.0 0.8 1.5 4.9 -150.0 -33.3 59.2 French 50.9 54.8 51.1 48.4 51.5 -7.2* -13.2** -6.4 Grammar 76.2 83.5 76.6 71.3 78.0 -9.0*** -17.1*** -7.1** Cloze task 30.2 30.6 30.0 30.1 30.1 -2.0 -1.7 -1.7 Composition 25.2 35.9 27.3 19.5 22.2 -31.5*** -84.1*** -61.7*** Math 33.4 42.2 33.6 28.7 35.0 -25.6** -47.0*** -20.6 Basic math 39.3 48.1 39.6 34.9 40.3 -21.5** -37.8** -19.4 Advanced math 22.5 31.2 22.3 17.0 25.0 -39.9* -83.5** -24.8 Fractions 14.7 24.2 14.1 10.6 18.5 -71.6* -128.3** -30.8 Venn diagram 22.0 30.9 21.4 18.5 23.5 -44.4 -67.0 -31.5 Graphs 14.1 28.2 12.5 8.5 19.0 -125.6** -231.8*** -48.4 Pedagogy 19.6 26.1 21.0 12.7 19.0 -24.3** -105.5*** -37.4*** Lesson preparation 28.1 34.3 30.0 20.6 26.3 -14.3 -66.5*** -30.4** Pupil comparisons 0.3 0.5 0.3 0.2 0.3 -66.7 -150.0*** -66.7 Pupil evaluations 6.5 12.4 7.2 4.6 4.7 -72.2** -169.6*** -163.8*** Teachers 831 54 502 106 141 Note: Bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold-faced headers. Differences are relative to teachers with a CAPE. Abbreviations are “CEAP” for the lower-level teacher certification, “CAP” for the higher-level certification, and “CAPE” for the higher certification obtained at a teacher training college. Sixteen teachers have a CEAP from teacher training colleges and 12 have a monitor’s diploma (does not qualify to teach). Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 43 Panel F. Teacher evaluations: teacher training college degree breakdowns (percent) % Diff. (Did not Indicator (%) All Attended Did not attend attend – Attended) Average score 45.1 50.6 44.7 -13.2*** Minimum knowledge 1.8 1.5 1.8 16.7 French 50.9 54.3 50.7 -7.1* Grammar 76.2 81.4 75.8 -7.4*** Cloze task 30.2 31.8 30.1 -5.6 Composition 25.2 31.6 24.7 -27.9*** Math 33.4 41.2 32.9 -25.2** Basic math 39.3 46.8 38.7 -20.9** Advanced math 22.5 30.6 21.8 -40.4* Fractions 14.7 19.7 14.3 -37.8 Venn diagram 22.0 31.7 21.3 -48.8* Graphs 14.1 27.2 13.1 -107.6** Pedagogy 19.6 25.1 19.2 -30.7*** Lesson preparation 28.1 34.7 27.6 -25.7** Pupil comparisons 0.3 0.4 0.3 -33.3 Pupil evaluations 6.5 11.2 6.2 -80.6** Teachers 831 70 761 Note: differences are relative to teachers who attended teacher training college (irrespective of the degree achieved). Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 44 Table 27. Teacher characteristics (absenteeism sample) % Diff. All Public Private (Public-Private) Public school 68 100 0 100 Civil servant 31.3 38.9 15.6 60*** Auxiliary teacher 15.8 20.3 6.4 69*** Volunteer teacher 33.7 40.6 19.3 53*** Private-sector contractual teacher 19.1 0.2 58.4 -29,100*** Rural 31 19 55 -187*** Teacher's college degree 7 9 1 88*** Strike during second visit 11 11 10 15 Days since last inspection (number) 341 333 358 -8 Inspection visits (number) 4 4 4 -13 Teacher teaches multiple grades 20 21 19 13 School director absent in 2nd visit 20 25 9 64*** Poverty headcount (%) 65 72 49 32*** Share of students with paper (%) 75 73 81 -12* Functional toilet 29 22 44 -102*** Note: Weighted means using sampling weights for teachers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 45 Table 28. Correlates of teacher effort Absence from school Absence from classroom Estimate Std. Err. Estimate Std. Err. Public school teacher -0.223 (0.208) 0.0398 (0.210) Auxiliary teacher -0.304* (0.168) -0.0856 (0.142) Volunteer teacher 0.212 (0.160) 0.166 (0.129) Private-sector contractual teacher -0.473 (0.319) -0.228 (0.356) Rural 0.0102 (0.162) -0.0284 (0.167) Teacher's college degree -0.101 (0.268) -0.124 (0.212) Maritime 0.943*** (0.348) Plateaux 1.030* (0.578) Centrale 1.444** (0.621) Kara 1.193** (0.563) Savanes 1.791** (0.697) Strike during second visit 0.722*** (0.250) 0.866*** (0.229) Days since last inspection (number) -0.000300 (0.000554) Inspection visits (number) -0.0341 (0.0212) Teacher teaches multiple grades 0.248 (0.170) School director absent in 2nd visit 1.417*** (0.148) 1.138*** (0.171) Poverty headcount (%) -0.0212** (0.00815) -0.00482 (0.00363) Share of students with paper (%) -0.106 (0.291) Functional toilet -0.321* (0.184) Constant -0.576 (0.385) -0.226 (0.344) Observations 914 848 F 8.903 7.911 df_m 16 12 df_r 153 142 Notes: 1. Variables are binary unless indicated otherwise. The results are from survey-weighted probit models with linearized standard errors. 2. Community teachers are excluded. 3. Superscript (*) indicates significance at the 1 percent (***), five percent (**) or ten percent (*) levels. 4. Indicator definitions and summary statistics may be found in Table 27. Teacher characteristics (absenteeism sample) 46 Table 29. Pupil absence rates, by school type Publi Publi Publi Priva Diff Urba c Indicator (%) All Rural Diff. c Diff. c te . n Urba Rural n 14. 4.7 All schools 15.7 11.0 14.3 13.9 0.4 15.6 16.1 -0.5 2 * Schools without 14. 15.5 11.8 3.8 14.3 14.4 -0.0 15.2 17.1 -1.9 strikers 3 Note: *** p<0.01, ** p<0.05, * p<0.1. Scores and their differences are in percentage points. Data are for the fourth-grade classrooms from which pupils were selected. Figure 11. Pupil evaluation distribution by section and school location Location and Ownership French French .02 .02 .015 .015 .01 .01 .005 .005 0 0 0 20 40 60 80 100 0 20 40 60 80 100 Mathematics Mathematics .025 .025 .02 .02 .015 .015 .01 .01 .005 .005 0 0 0 20 40 60 80 100 0 20 40 60 80 100 Note: urban in blue, rural in red; private in blue, public in red. 47 Table 30. Pupil performance details (percent) % Diff. Urban Rural % Diff. (Rural Public- All Public Private Public Public (Public- Private) Urban Public) Pupil overall test score 45.1 38.4 63.8 58.7 39.0 -66.1*** -50.5*** Pupil literacy score 44.9 37.3 66.3 60.7 38.0 -77.7*** -59.7*** Pupil numeracy score 43.9 41.0 52.1 48.6 41.9 -27.1*** -16.0*** Pupil nonverbal reasoning score 53.2 52.5 55.1 56.2 51.9 -5.0* -8.3*** Pupil can read letter 76.7 71.4 91.5 86.0 72.6 -28.2*** -18.5*** Pupil can identify basic words 63.7 56.7 83.3 79.5 56.7 -46.9*** -40.2*** Pupil can read simple sentence 25.6 16.1 52.2 44.5 17.2 -224.2*** -158.7*** Pupil can read paragraph 16.4 8.4 38.6 31.5 9.7 -359.5*** -224.7*** Pupil comprehensi on score 16.9 9.6 37.3 31.7 10.3 -288.5*** -207.8*** Can do single digit addition 76.5 73.6 84.7 79.7 75.1 -15.1*** -6.1* Can do double digit addition 64.6 60.8 75.4 70.4 62.1 -24.0*** -13.4** Can do single digit subtraction 64.5 59.9 77.3 71.3 61.5 -29.0*** -15.9*** Can do double digit subtraction 20.8 16.1 34.0 28.3 17.4 -111.2*** -62.6*** Can multiply single digits 10.5 10.1 11.7 13.3 9.3 -15.8 -43.0* Can multiply double digits 5.3 3.1 11.4 9.7 3.4 -267.7*** -185.3*** Can do single digit division 35.1 29.1 51.7 43.0 31.6 -77.7*** -36.1*** Note: Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Scores and their differences are in percentage points. 48 Table 31. Correlates of pupil performance in language Standard Standard Poverty- Standard Variables Basic Error Augmented Error augmented Error Rural (d) -5.130 (3.134) -6.135* (3.242) -6.141* (3.270) Private school (d) 8.187** (3.610) 6.144 (4.341) 6.323 (4.350) Pupil’s age -0.618 (0.986) -0.465 (0.847) -0.465 (0.855) Female pupil (d) -3.147** (1.332) -3.292** (1.388) -3.478** (1.396) Maritime -9.972** (3.919) -10.94** (4.269) -8.254 (5.078) Plateaux -16.00*** (4.346) -13.46*** (4.918) -8.874 (7.279) Centrale -15.85*** (4.077) -13.94*** (4.960) -7.978 (7.845) Kara -16.77*** (4.263) -15.35*** (4.989) -10.00 (7.478) Savanes -24.61*** (3.504) -22.74*** (4.118) -14.64 (9.105) Time spent Teaching -0.000940 (0.00996) -0.000463 (0.00986) Grade 4 teacher's language score 3.862 (6.155) 4.449 (6.381) Grade 4 teacher's pedagogy score 12.86* (7.202) 12.25 (8.557) Exercise book (% pupils) 7.952** (3.630) 8.463** (3.779) Textbook (% pupils) 4.364 (3.943) 4.300 (3.907) Poverty headcount (%) -0.150 (0.154) Constant 49.23*** (10.80) 34.01*** (12.32) 39.72*** (13.50) Observations 1,128 1,043 1,043 R-squared 0.219 0.226 0.230 F(n,d) 14.62 9.246 7.809 n 9 14 15 d 132 121 121 Pr>F 0 0 0 Note: Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. The survey-weighted model is estimated in standard errors of the dependent variable. One standard deviation is 1.68 points. Model standard errors are jackknife estimates and account for clustering at the school level. Poverty headcounts are from the poverty mapping results in Coulombe and Malé (2012) and are for the canton in which the school is located. 49 Table 32. Correlates of pupil performance in mathematics Standard Standard Poverty- Standard VARIABLES Basic Error Augmented Error augmented Error Rural (d) -2.484 (3.027) -2.131 (3.176) -2.371 (3.217) Private school (d) 6.457* (3.321) 3.650 (3.819) 3.843 (3.852) Pupil’s age -0.268 (0.236) -0.167 (0.145) -0.172 (0.157) Female pupil (d) -4.054*** (1.342) -4.916*** (1.404) -5.133*** (1.382) Maritime -4.711 (4.212) -7.646* (4.539) -4.157 (5.549) Plateaux -15.51*** (3.846) -14.58*** (4.321) -8.914 (6.409) Centrale -13.08*** (3.964) -13.05*** (4.582) -5.708 (7.070) Kara -18.56*** (4.155) -18.45*** (5.311) -12.04 (7.444) Savanes -20.82*** (3.731) -20.28*** (3.896) -10.24 (8.321) Time spent Teaching -0.00222 (0.00968) -0.00241 (0.00957) Grade 4 teacher's math score 1.746 (4.771) 0.465 (4.811) Grade 4 teacher's pedagogy score 3.822 (8.490) 3.891 (8.385) Exercise book (% pupils) 7.509** (3.624) 8.145** (3.712) Textbook (% pupils) 2.335 (3.446) 2.200 (3.383) Poverty headcount (%) -0.184 (0.120) Constant 69.70*** (4.616) 60.98*** (7.139) 68.92*** (8.372) Observations 1,128 1,043 1,043 R-squared 0.189 0.183 0.189 F(n,d) 9.897 5.878 5.350 n 9 14 15 d 132 121 121 Pr>F 0 1.98e-08 6.75e-08 Note: Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. The survey-weighted model is estimated in standard errors of the dependent variable. One standard deviation is 1.68 points. Model standard errors are jackknife estimates and account for clustering at the school level. Poverty headcounts are from the poverty mapping results in Coulombe and Malé (2012) and are for the canton in which the school is located. 50 REFERENCES CONFEMEN and MEPSA (2012), “ Améliorer la qualité de l’éducation au Togo : les facteurs de réussite », Conférence des ministres de l’Education des pays ayant le français en partage. Coulombe, H. and C. Malé (2012), “Togo: Profil de pauvreté 2006-2011”, UNDP and DGSCN. DGSCN (2011), « Recensement général de la population et de l’habitat : Résultats définitifs », Direction générale de la statistique et de la comptabilité nationale. Johnson, Cunningham, and Dowling (2012) “Draft Final Report, Teaching Standards and Curriculum Review”. Ross, Kenneth N. and T. Neville Postlethwaite (1988) Sample Design Procedures for the IEA International Study of Reading Literacy, IEA. Vargas, E. and J. De Laat (2003), “Do differences in teacher contracts affect pupil performance? Evidence from Togo”, World Bank Working Paper Number 26955. World Bank (2004), World Development Report: Making Services Work for Poor People. Document of the World Bank