Report No: . Republic of Niger Niger Service Delivery Indicators Education 2015 . June 26, 2017 . GGHCE with GEDDR and GGODR AFRICA . i . . Standard Disclaimer: This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Copyright Statement: . The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. ii ACKNOWLEDGEMENTS................................................................................................................................................................... vi EXECUTIVE SUMMARY ...................................................................................................................................................................vii INTRODUCTION ................................................................................................................................................................................... 3 Methodology and Implementation .............................................................................................................................................. 8 RESULTS ................................................................................................................................................................................................10 A. Teacher effort .................................................................................................................................................................................10 Absence from school ........................................................................................................................................................................10 Absence from class ...........................................................................................................................................................................12 Time spent teaching per day ........................................................................................................................................................15 B. Teachers with minimum knowledge ....................................................................................................................................17 French .....................................................................................................................................................................................................19 Mathematics.........................................................................................................................................................................................19 Pedagogy ...............................................................................................................................................................................................20 A broader look at teacher knowledge using all teachers evaluated by the survey ..............................................20 C. Availability of inputs at the school ........................................................................................................................................23 Functioning school infrastructure .............................................................................................................................................24 Availability of teaching resources ..............................................................................................................................................25 Availability of textbooks .................................................................................................................................................................25 Pupil-teacher ratio ............................................................................................................................................................................26 D. Assessment of pupil learning ...................................................................................................................................................27 E. Incentives, leadership, and management ...........................................................................................................................33 Leadership and management .......................................................................................................................................................33 Supervision...........................................................................................................................................................................................35 Community engagement ................................................................................................................................................................36 What does this mean for Niger? ..................................................................................................................................................37 Comparing Niger with other countries that have done SDI .................................................................................................39 Annex 1. Sampling.........................................................................................................................................................................................40 Annex 2. Definition of the Service Delivery Indicators in Education .....................................................................................42 Annex 3. Additional Results ......................................................................................................................................................................44 A. School breakdowns ......................................................................................................................................................................44 B. Individual breakdowns...............................................................................................................................................................48 REFERENCES .......................................................................................................................................................................................69 iii Tables and Figures Table 1. Key recommendations................................................................................................................................................................ ix Table 2. Service Delivery Indicators at-a-glance ................................................................................................................................x Table 3. Comparison of SDI results across countries (all schools) .............................................................................................i Table 4. Comparison of SDI results across countries (public schools only) ......................................................................... ii Figure 1. Relationships of accountability between citizens, service providers, and policymakers ........................... 6 Table 5. Education Indicators .................................................................................................................................................................... 7 Table 6. Survey modules .............................................................................................................................................................................. 9 Table 7. SDI Education sample ................................................................................................................................................................10 Figure 2. Teacher activities during the school day (percent) ....................................................................................................11 Table 8. Teacher Effort................................................................................................................................................................................11 Figure 3. Reasons why teachers are absent from school (percent) ........................................................................................12 Figure 4. Teacher absence rate by second visit date (%) ............................................................................................................13 Figure 5. Teacher absence correlates ...................................................................................................................................................15 Figure 6. Distributions of teacher effort indicators .......................................................................................................................17 Table 9. Teacher assessment....................................................................................................................................................................19 Table 10. Teacher French scores (percent) .......................................................................................................................................19 Table 11. Teacher mathematics scores (percent) ..........................................................................................................................20 Table 12. Teacher pedagogy scores (percent) .................................................................................................................................20 Figure 7. Teacher evaluation cumulative distribution (all teachers).....................................................................................21 Table 13. School input indicators...........................................................................................................................................................24 Table 14. School textbook indicators (percent) ..............................................................................................................................26 Figure 8. Availability of inputs ................................................................................................................................................................27 Table 15. Student performance results (grade 4; percent) ........................................................................................................29 Table 16. Pupil evaluation comparison between rising fourth and fifth graders.............................................................30 Figure 9. Student learning correlates ...................................................................................................................................................31 Figure 10. Pupil evaluation distribution by module and location among public schools .............................................32 Table 17. Pupil evaluation: gender and location breakdowns for grade 4 (percent) .....................................................32 Table 18. Pupil evaluation: gender and school ownership breakdowns for grade 4 (percent) .................................33 Table 19. Constraints to service delivery (percent) ......................................................................................................................34 Table 20. Teacher declaration of director’s supervision frequency (percent) ..................................................................35 Table 21. Responses to common challenges .....................................................................................................................................36 Table 22. Community engagement (percent) ...................................................................................................................................37 Table 23. School attendance by welfare quintile (percent) .......................................................................................................39 Table 24. Pupil evaluation: gender and location breakdowns for grade 4 public (percent) .......................................39 Table 25. Distribution of fourth grade students by region, curriculum, and location (percent) ...............................40 Table 26. School input indicators, detailed........................................................................................................................................47 Table 27. Teacher absence rates, by status (percent) ..................................................................................................................48 Table 28. Teacher absence rates, by region (percent)..................................................................................................................48 Table 29. Teacher evaluation breakdowns ........................................................................................................................................49 Panel A: Teacher evaluation: ownership, urban-rural within public, and rural-urban breakdowns (percent) .49 Panel B. Teacher evaluation: contractual status breakdown (percent) ...............................................................................50 Panel C. Teacher evaluations: academic training breakdowns (percent)............................................................................51 Panel D. Teacher evaluations: grade taught breakdowns (percent) ......................................................................................52 Panel E. Teacher evaluations: teacher training breakdowns (percent)................................................................................53 Panel F. Teacher evaluations: teacher training college degree breakdowns (percent).................................................54 Panel G. Teacher evaluations: breakdowns by career start year (percent) ........................................................................55 Panel H. Teacher evaluations: breakdowns by position (percent) .........................................................................................56 Figure 14. Teacher evaluation performance progression by year of hire ...........................................................................57 Table 30. Teacher characteristics (absenteeism sample) ...........................................................................................................58 Table 31. Correlates of teacher effort...................................................................................................................................................59 Figure 15. Pupil evaluation distribution by section and school ownership, grade 4 ......................................................60 Table 32. Pupil performance details, grade 4 (percent) ..............................................................................................................61 Table 33. Pupil performance details, grade 5 (percent) ..............................................................................................................62 Table 34. Pupil performance details by region, grade 4 (percent) .........................................................................................63 iv Table 35. Pupil performance details by region, grade 5 (percent) .........................................................................................64 Table 36. Correlates of pupil performance in language ...............................................................................................................65 Table 37. Correlates of pupil performance in mathematics .......................................................................................................66 Table 38. Teacher performance evaluation elements...................................................................................................................67 Table 39. Supervision content .................................................................................................................................................................68 v ACKNOWLEDGEMENTS This report was prepared jointly between the Ministry of Primary Education (Ministère de l’Enseignement Primaire, de l’Alphabétisation, de la Promotion des Langues Nationales et de l’Education Civique), the Public Sector Capacity and Performance for Service Delivery Project, and the World Bank. The Ministry of Education team was led by Abdou Lawan Marouma (Director of Studies and Planning in the ministry) and included Abdoulaye Erambel (Regional Education Director for Niamey), Fatimé-Lara Ibrah (national PASEC team), Mahaman Djibo (Director of Statistics in the ministry), and Alio Boukari (member of the Statistics Directorate). Adama Azizou (team lead for education, National Economic and Social Development Plan) also contributed on the governance module. Claire Ledru Hanounou coordinated the Public Sector Capacity and Performance for Service Delivery project which included Sani Kanta, that managed the contract with the data collection firm. The Institut National de la Statistique, led by its Director General, was responsible for data collection and entry with support from World Bank and ministry counterparts. The Laboratoire d'études et de recherches sur l'émergence économique of Université Abdou Moumouni (Niamey, Niger) and led by Professor Malam Maman Nafiou corrected the teacher evaluations. The World Bank Niger SDI team was led by Christophe Rockmore, with important contributions from Nestor Coffi (Country Manager), Siaka Coulibaly (Country Manager), Seydou Garba Hamidou (education SDI advisor), Helene Grandvoinnet (peer reviewer), Emanuela di Gropello (Human Development Program Leader), Harouna Hamani (field coordinator), Moustapha Lo (peer reviewer), Kirsten Majgaard (Task Team Leader for the Global Partnership for Education Support to Quality Education Project), Michel Maellberg (Task Team Leader, Public Sector Capacity and Performance for Service Delivery Project), Lucia Nhampossa (peer reviewer), Adama Ouédraogo (co-Task Team Leader for the Global Partnership for Education Support to Quality Education Project), Owen Ozier (sampling), and Waly Wane (SDI Program Manager). Guidance was provided by Nestor Coffi, Siaka Bakayoko, and Meskerem Mulatu. We thank the teachers and students for the time and effort which they gave us for this project. Funding for this activity was provided by the Government of the Republic of Niger through the Public Sector Capacity and Performance for Service Delivery Project coordinated by the Ministry of Planning, the William and Flora Hewlett Foundation, and the World Bank. vi EXECUTIVE SUMMARY This report presents the results of the Service Delivery Indicators in the education sector in Niger in 2015. Survey implementation was preceded by extensive consultation with Government and key stakeholders on survey design, sampling, and adaptation of survey instruments. Pre-testing of the survey instruments took place in 2013, while training of field staff and field work took place in 2015, and data entry, cleaning, and analysis took place in 2016 and 2017. Information was collected from 256 primary schools, 1,748 teachers, and 3,661 grade four and five pupils in Niger. The results provide a representative snapshot of primary education service delivery in Niger. The survey provides information on four elements of service delivery: measures of (i) teacher effort; (ii) teacher knowledge and ability; (iii) the availability of key inputs and infrastructure; and (iv) management, supervision, and community engagement. What providers know Pupils cannot learn more from their teachers than what the teachers know. On average, primary school teachers who taught fourth grade in 2015/16 or third grade the previous school year mastered 41 percent of an exam set at the lower primary level and zero percent of the current-year fourth grade or previous-year third grade scored 80 percent or higher. For all teachers and directors, 0.4 percent reached the threshold. Teachers in private and rural public schools did better than their public or urban public peers. What providers do (teacher effort) During an unannounced visit, 16.6 percent of teachers were not at their school during and a further 10.4 percent were at their school, but not in their classroom. However, once in the classroom, teachers taught the majority of the time, meaning that pupils receive 77.3 percent of the scheduled teaching time. This is the highest rate among countries that have done an SDI. What providers can use (availability of key inputs) The observed pupil-teacher ratio in grade 4 averaged 38 pupils per teacher, slightly below than the 40 pupils that is the norm in Niger. There are important input deficiencies that make teaching more difficult. Approximately one in five schools (19.7 percent) had the minimum infrastructure, primarily because one school in four (24.3 percent) has functional, private, and accessible latrines. Similarly, 24.7 percent of schools have the minimum teaching equipment, primarily because pupils lack exercise books (8.9 percent availability). These are often cited as primary constraints to performance by public school directors. Ownership matters Private schools offered better service delivery than public schools in all areas except teachers meeting the minimum knowledge threshold (all had zero percent). Differentials were highest in pupil outcomes, followed by teacher absence, and then input availability. The private sector, which is small and concentrated in urban areas, offers a different vii education than the public sector. Despite a disadvantage of being measured at the start of fourth grade rather than later in the year, private school children perform on par with those from Kenya, a middle-income country. Kenya has the highest average student performance observed among SDI countries. However, public school children are worst among those evaluated. Comparing the rising fifth graders to their fourth-grade peers from other countries, they only surpass Nigeria and Mozambique. Geographic and gender differences among public schools Except for class size, rural pupils are at a disadvantage compared to their urban peers in all areas. Combined with other factors, this translates into significantly lower scores on the learning evaluation. Girls do worse than boys in the tests, and the effects are cumulative: rural boys do worse than urban ones and rural girls do the worst. Among regions, students in Niamey generally do the best on almost all French items and some in mathematics in grade four. In French, Maradi region does the worst, while it is Dosso in mathematics. For grade 5, the situation is somewhat different, but the same regions do worst in the same areas. Leadership and management School directors are the ministry’s first supervision agents and are most directly able to monitor teaching. However, there are important variations in supervision across private, urban public, and rural public schools. Directors’ evaluation of teacher performance was more likely to include direct teacher observation, teacher absence rates, and student learning in private than in public schools. Within public schools, urban school directors pay more attention to these aspects than rural school directors. When asked, teachers report similar patterns of comments from their directors’ supervision. Supervision External supervision is similar to leadership and management in that the private and public schools differ. In private, supervisors are more likely to use templates and to observe teaching. However, public school supervision is more likely to meet with the community, to check inventory, and to review the latest school management committee annual report. The pattern across urban and rural public schools mirrors that of private and public schools. Private schools were the most supervised in the 2014/15 school year and urban public schools had nearly twice the supervision of rural public schools. Community engagement Almost all primary schools (96.9 percent) have a school management committee although not all are functional. School management committees are 1.5 times as likely to have an action plan and 1.7 times as likely to have evaluated the action plan in private relative to public schools. Urban public schools are more likely to have a school management committee, an action plan, and an annual report than rural public schools. viii Recommendations from the survey and its analysis Table 1, below, presents key results from the survey, summarized above, and possible actions. To make them operational, those responsible and the timelines are included. Table 1. Key recommendations SDI finding Action Responsible Timeframe Shortage of textbooks Purchase and distribute Ministry with project Next school year textbooks based upon needs and ongoing (students and availability) Insufficient focus on key Explore reasons (qualitative Ministry with Next school year performance elements in work) technical assistance directors’ oversight of Develop training content teacher (particularly in Deliver training content rural areas) Teachers have weak Develop training content Ministry (content and Summer and next pedagogical and basic Organize training sessions oversight/evaluation) school year content skills and evaluate learning and regional directorates (practice) Further strengthen the role Develop basic tools Central ministry to Next school year and capacity of school Prepare training develop tools (norms- and ongoing management committees Deliver it through school setting) inspection visits Inspectors to ensure follow-up and to report on progress How does Niger compare with other countries? Table 3, below, compares Niger with other countries for all schools and Table 4 compares for the public sector. In terms of teacher effort, Niger has a relatively lower absence rate than the average of the other countries. This, combined with teacher behavior in class, means that Niger ranks highest in terms of instructional time per day. However, teacher knowledge is particularly low; it is the greatest differential among all the indicators. Schools in Niger have the fewest teachers who reach the minimum knowledge threshold, low absence rates (fourth-best), the second-lowest classroom absence rate (behind Nigeria), the best teaching time per day, the second-lowest textbook availability (above Uganda), and the lowest availability of infrastructure and teaching equipment. As noted above, generally, students in Niger have among of the worst learning outcomes of the SDI surveys. ix Table 2. Service Delivery Indicators at-a-glance Urban Rural Niger Public Private Diff (%) public public Diff. (%) Teacher Ability Minimum knowledge 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Test score (all areas) 33.3 33.3 37.5 4.2* 32.6 34.5 -1.9 Teacher Effort School absence rate 16.6 17.2 1.9 -89.2*** 17.3 17.1 1 Classroom absence rate 27.0 27.6 11.9 -57.1*** 31.2 25.7 21.5 Scheduled teaching time 5h 40m 5h 41m 5h 17m -25.6*** 5h 41m 5h 42m -0.2 Time spent teaching per day 4h 23m 4h 13m 5h 32m 2.8 4h 6m 4h 15m 3.8 Availability of Inputs Observed pupil-teacher ratio 38.1 38.1 46.9 8.8*** 48.2 36 12.2*** Share of pupils with textbooks 8.7 8.7 33.6 25** 9.5 8.5 1 Minimum equipment 23.4 23.4 72.4 49*** 38.9 20.1 18.8*** Minimum infrastructure 19.7 19.7 80.7 61.1*** 28.3 17.9 10.4 Pupil Learning Test score in language and math (%) 21.3 21.3 65.5 208.1*** 32.6 18 80.9*** Language test score 21.7 21.7 73.1 236.6*** 34.8 17.9 93.8*** Mathematics test score 11.5 11.5 24.7 116.1*** 15 10.4 44.6*** Notes: definitions of indicators are in Annex B. Significance levels: *** p<0.01; ** p<0.05; * p<0.1. Differences are relative to public and rural public, respectively. x Table 3. Comparison of SDI results across countries (all schools) Niger Average Madagascar Tanzania Kenya Mozambique Nigeria Senegal Tanzania Togo Uganda Element 2015 SDI 2016 2014 2012 2014 * 2013 2011 2011 2013 2013 Teacher Ability Minimum knowledge 0.0 14.6 0.1 21.5 40.4 0.3 3.7 n/c n/c 1.6 19.5 Test score (all areas) 33.3 43.0 33.2 48.3 57.1 26.9 32.9 n/c n/c 35.6 45.3 Teacher Effort School absence rate 16.6 18.6 30.6 14.4 14.1 44.8 13.7 18.0 23.0 20.5 26.0 Classroom absence rate 27.0 39.8 37.8 46.7 42.1 56.2 19.1 29.0 53.0 35.8 52.8 Scheduled teaching time 5h 40m 5h 34m 5h 12 m 5h 54m 5h 37m 4h 17m 4h 53m 4h 36m 5h 12m 5h 29m 7h 18m Time spent teaching per day 4h 23m 3h 02m 3h 09m 2h 46m 2h 49m 1h 41 m 3h 26m 3h 15m 2h 04m 3h 29m 3h 18m Availability of Inputs Observed pupil-teacher ratio 38.1 40.4 17.6 43.5 35.2 21.4 21.6 27.2 52.0 29.7 47.9 Share of pupils with textbooks 8.7 37.1 10.3 25.3 48.0 68.1 38.2 18.0 19.7 68.5 5.0 Minimum equipment 23.4 60.5 65 61.4 78.8 76.8 54.8 n/c n/c 26.4 80.6 Minimum infrastructure 19.7 38.1 20.2 40.4 59.5 29.1 18.5 n/c n/c 22.3 53.7 Pupil Learning Test Score language, and mathematics; % 21.3 49.6 50.6 40.1+* 72.0 20.8 32.2 n/c n/c 45.7 48.6 Language test score 21.7 49.5 44.5 36.5+* 75.4 18.7 31.4 n/c n/c 45.5 47.1 Mathematics test score 11.5 47.3 56.8 58.2 59.0 25.1 31.9 n/c n/c 44.6 43.4 Notes: values for Nigeria are the weighted average of the four states surveyed: Anambra, Bauchi, Ekiti, and Niger. These statistics reflect the updated SDI methodology. More detailed information on the methodology is available at www.SDIndicators.org. Complete indicator definitions are in Annex 2. Data for Mozambique are for the public sector. Data for Tanzania have been revised to reflect the new methodology and may differ from earlier published reports. The abbreviation “n/c” means not comparable. i Table 4. Comparison of SDI results across countries (public schools only) Niger Average Madagascar Tanzania Kenya Mozambique Nigeria Senegal Tanzani Togo Uganda Element 2015 SDI 2016 2014 2012 2014 * 2013 2011 a 2011 2013 2013 Teacher Ability Minimum knowledge 0.0 12.7 0.0 21.5 34.8 0.3 2.4 n/c n/c 0.9 19.4 Test score (all areas) 33.3 42.0 32.1 48.3 55.6 26.9 30.5 n/c n/c 33.9 45.5 Teacher Effort School absence rate 17.2 20.1 35.9 14.4 15.2 44.8 16.9 18.0 23.0 22.6 29.9 Classroom absence rate 27.6 42.1 42.2 46.7 47.3 56.2 22.8 29.0 53.0 39.3 56.9 Scheduled teaching time 5h 41m 5h 31m 5h 03m 5h 54m 5h 31m 4h 17m 4h 44m 4h 36m 5h 12m 5h 28m 7h 13m Time spent teaching per day 4h 13m 2h 53m 2h 56m 2h 46m 2h 30m 1h 41 m 3h 10m 3h 15m 2h 04m 3h 15m 2h 56m Availability of Inputs Observed pupil-teacher ratio 38.1 42.1 16.9 43.5 39.3 21.4 21.5 27.2 52.0 31.4 53.9 Share of pupils with textbooks 8.7 37.2 6.8 25.3 44.5 68.1 33.7 18.0 19.7 76.0 6.0 Minimum equipment 23.4 57.8 65.1 61.4 74.3 76.8 48.2 n/c n/c 24.3 79.5 Minimum infrastructure 19.7 36.2 16 40.4 60.2 29.1 13.4 n/c n/c 14.4 57.2 Pupil Learning Test Score language, and mathematics; % 21.3 45.4 46.6 40.1+* 69.4 20.8 25.1 n/c n/c 38.1 45.3 Language test score 21.7 44.8 39.7 36.5+* 72.5 18.7 23.3 n/c n/c 36.9 43.4 Mathematics test score 11.5 45.2 53.5 58.2 57.4 25.1 28.2 n/c n/c 41.3 41.7 Notes: values for Nigeria are the weighted average of the four states surveyed: Anambra, Bauchi, Ekiti, and Niger. These statistics reflect the updated SDI methodology. More detailed information on the methodology is available at www.SDIndicators.org. Complete indicator definitions are in Annex 2. Data for Tanzania have been revised to reflect the new methodology and may differ from earlier published reports. The abbreviation “n/c” means not comparable. ii INTRODUCTION Niger is a large landlocked country with a territory of 1.27 million km2 of which more than three-quarters is covered by desert. Niger is situated in the West-Central part of the African continent, and located between Mali, Algeria, Libya, Chad, Nigeria, Benin, and Burkina Faso. The country has a population of about 17.1 million (2012 census), which is growing rapidly at 3.5 % per annum. Child mortality in Niger has started to decline, but the fertility rate is still very high leading to significant population growth. The United Nations projects that the population in Niger will increase to approximately 22 million in 2020. Most of the country has low population density, and the majority of the population is concentrated in a third of the territory, mainly in the southern and western regions of the country. The majority of the population lives in rural areas (82%). Women make up 49.7% of the total population, and almost half of the population (49%) is under the age of 15. According to official projections (National Institute of Statistics, INS, 2010), the number of children aged 7 to 12 rose from 1.85 million in 2001 to 2.35 million in 2008, and is expected to reach 3.38 million in 2020. The number of children of primary school age is thus expected to increase by 43% between 2008 and 2020 leading to a large increase in the demand for education services, which is expected to put a significant pressure on the Niger educational system by 2020. Niger is a poor country. In the latest available UNDP Human Development Index (2014), it was ranked last, with a Gross Domestic Product (GDP) per capita of US$897 in 2015 in Parity Purchasing Power ($2011) terms- one of the lowest in the world. The country remains relatively fragile and continues to be affected by several adverse weather conditions 1 and other factors, which represent real obstacles to its harmonious development (59.5% of its population lives below the poverty threshold): (a) adverse weather conditions lead to recurrent drought in the northwestern and western areas where for several years there have been successive famines and an increase in vulnerability. This insecurity affects both the supply and demand for schooling and limits potential learning achievements; (b) instability in countries of the sub-region and in the Sudano-Sahelian strip affects the border areas (exposure to violent incidents: conflicts and terrorism) and undermines the government development efforts. Nonetheless, the exploitation of Niger‘s significant mineral and oil resources could provide important economic opportunities, if well managed. Economic growth is highly volatile due to the dependence on rain fed agriculture. Agriculture and livestock, which contribute respectively to about 28% and 11% of GDP, provide livelihood for the majority of the population. The high dependence on agriculture contributes to fluctuations in Niger’s economic growth. Over the last five years the annual growth of Niger’s GDP has been as low as 2.3 percent in 2011 and reached a high of 11.8 percent in 2012, with an average annual growth rate of 6 percent. Despite an average economic growth above the population growth and thus overall economic growth per capita, the distribution of income is likely to have resulted in a smaller effect on poverty reduction than average growth would have justified. This is due to the uneven distribution 1 About 80 % of Niger’s population depends on rain-fed agriculture and livestock and only about 12 % of all its land is arable. 3 of income, as the majority of the population relies on the agricultural sector, which has experienced a more modest growth. Furthermore, population grows at a faster rate in rural areas leading to significant lower income growth per capita in rural areas. The education system in Niger consists of three years of pre-primary school, followed by six years of primary education, then four years of lower secondary education, and three years of upper secondary education, which ends with a baccalaureate. Graduates from lower secondary school can chose to continue with technical and professional education, which is divided into upper technical secondary education of three years and professional training of two or three years. Graduates from upper secondary can continue to higher education including university, where the number of years of education depends on the program. Primary education is mandatory, and the school enrolment age is seven years. Gross primary enrollment rates in Niger were at 72.5 percent in 2015, with boys (77.9 percent) fully 11.1 percentage points more likely to be enrolled than girls. 2 In its policy framework, the Government considers the six years of primary education and the four years of lower secondary education as basic education. The objective is to provide universal access to this level and ensure that graduates from basic education have the required skills needed to function as productive members of society. Education is, for the most part, provided by the public sector. As is the case for most Sub- Saharan African countries, the public sector provides most of the education in Niger. Only 3.3% of primary enrollment was in private schools in 2014. The Ministry of National Education (Ministère de l’Education Nationale, de l’Alphabétisation et de la Promotion des Langues Nationales - MEN) is responsible for preschool, primary education (public and private), and literacy and non-formal education (literacy, Koranic schools), while the Ministry of Professional Training and Employment (Ministère de la Formation Professionnelle et de l’Emploi – MFPE) manages technical and vocational (public and private) education, and the Ministry of Secondary and Tertiary Education, Research and Technology (Ministère des Enseignements Moyens et Supérieur et de la Recherche Scientifique – MESSRT) manages general secondary, university education, as well as national and international training and research centers located in Niger. Each of the three ministries has a regional departmental headquarter located in each of the eight regions of the country managing issues related to the sector under their jurisdiction. The education system in Niger faces a number of challenges. A high population growth rate, low initial enrollment rates, and lack of retention are among the factors preventing Niger from achieving universal primary education coverage and completion. In particular, Niger faces challenges to increase access and completion among vulnerable groups including girls in rural areas, children in nomadic areas, and children with disabilities. Other key related issues are the poor quality of learning and management of the education system. In addition to these structural issues, the country’s vulnerability to frequent weather shocks has impacts on the education sector. Despite significant increase in enrollment ratios, gross enrollment in primary education is 2 From http://uis.unesco.org/country/ne accessed on 19 March 2017. 4 still low in Niger compared to other Sub-Saharan African countries. Implementation of the previous ten-year strategy, i.e. the Programme Décennal de Développement du Secteur de l’Education (PDDE), formulated by the Government at the beginning of the last decade and covering 2003-2013, has resulted in increased enrollment at all levels of education. Over the last decade, the gross enrollment rate for primary education doubled in Niger going from 36% in 2001 to 76% in 2011. This significant growth has been supported by increased priority to primary education in the budget, extensive construction and rehabilitation of schools, and recruitment of teachers. However, despite the massive enhancement, enrollment rates are still quite low compared to other countries in the region, 3 due to a combination of low initial access for vulnerable populations, pointing to important inequities in the system, and low overall retention (overall and more particularly for vulnerable groups). The report is structured as follows: Box 1, below, presents the analytical underpinnings of the indicators and the way in which they are organized. Box 2 provides additional information on the SDI program and a detailed description of each indicator. Section 2 covers implementation and the SDI education methodology. Results are presented and analyzed in Section 3 and Section 4. The report concludes with analyses of implications for Niger and a comparison between the Niger results and those of other countries that have done SDI. 3 In 2010, gross enrollment in primary education exceeded 100% for 22 countries out of the 34 African countries for which data exist, Source: World Development Indicators. 5 Box 1. Analytical Underpinnings Service delivery outcomes are determined by the relationships of accountability between policymakers, service providers, and citizens (Figure 2, World Bank 2004). Human development outcomes are the result of the interaction between various actors in the multi‐step service delivery system, and depend on the characteristics and behavior of individuals and households. While delivery of quality education is contingent foremost on what happens in classrooms, a combination of several basic elements have to be present in order for quality services to be accessible and produced by teachers at the frontline, which depend on the overall service delivery system and supply chain. Adequate financing, infrastructure, human resources, material, and equipment need to be made available, while the institutions and governance structure provide incentives for the service providers to perform. Figure 1. Relationships of accountability between citizens, service providers, and policymakers Service Delivery Production Function Consider a service delivery production function, f, which maps physical inputs, x, the effort put in by the service provider, e, as well as his/her type (or knowledge), θ, to deliver quality services into individual level outcomes, y. The effort variable e could be thought of as multidimensional and thus include effort (broadly defined) of other actors in the service delivery system. We can think of type as the characteristic (knowledge) of the individuals who are selected for a specific task. Of course, as noted above, outcomes of this production process are not just affected by the service delivery unit, but also by the actions and behaviors of households, which we denote by ε. We can, therefore, write y = f(x,e,θ) +ε To assess the quality of services provided, one should ideally measure f(x,e,θ). Of course, it is notoriously difficult to measure all the arguments that enter the production, and would involve a huge data collection effort. A more feasible approach is therefore to focus instead on proxies of the arguments which, to a first‐order approximation, have the largest effects. Indicator Categories and the Selection Criteria There are a host of data sets available in education. To a large extent, these data sets measure inputs and outcomes/outputs in the service delivery process, mostly from a household perspective. While providing a wealth of information, existing data sources (like Living Standards Measurement Survey (LSMS), Welfare Monitoring Surveys (WMS), and Core Welfare Indicators Questionnaire Survey (CWIQ)) cover only a sub‐sample of countries and are, in many cases, outdated. 6 Box 1. Analytical Underpinnings (continued) The proposed choice of indicators takes its starting point from the recent literature on the economics of education and service delivery, more generally. Overall, this literature stresses the importance of provider behavior and competence in the delivery of education services (as opposed to water and sanitation services and housing that rely on very different service delivery models). Conditional on service providers exerting effort, there is also some evidence that the provision of physical resources and infrastructure has important effects on the quality of service delivery. The somewhat weak relationship between resources and outcomes documented in the literature has been associated with deficiencies in the incentive structure of school and education systems. Indeed, most service delivery systems in developing countries present frontline providers with a set of incentives that negate the impact of pure resource‐based policies. Therefore, while resources alone appear to have a limited impact on the quality of education in developing countries, it is possible inputs are complementary to changes in incentives, so coupling improvements in both may have large and significant impacts (see Hanushek, 2006). As noted by Duflo, Dupas, and Kremer (2011), the fact that budgets have not kept pace with enrollment, leading to large pupil‐teacher ratios, overstretched physical infrastructure, and insufficient number of textbooks, etc., is problematic. However, simply increasing the level of resources might not address the quality deficit in education without also taking providers’ incentives into account. SDI proposes three sets of indicators: (i) provider effort; (ii) knowledge of service providers and (iii) availability of key infrastructure and inputs at the frontline service provider level. Providing countries with detailed and comparable data on these important dimensions of service delivery is one of the main innovations of the Service Delivery Indicators. Additional considerations in the selection of indicators are (i) quantitative (to avoid problems of perception biases that limit both cross‐country and longitudinal comparisons), (ii) ordinal in nature (to allow within and cross‐country comparisons); (iii) robust (in the sense that the methodology used to construct the indicators can be verified and replicated); (iv) actionable; and (v) cost effective to collect. Table 5. Education Indicators Teacher Effort School absence rate Classroom absence rate Time spent teaching per day Teacher Knowledge and Ability Minimum knowledge in mathematics Minimum knowledge in English Minimum knowledge in pedagogy Availability of Inputs Minimum infrastructure availability Minimum equipment availability Share of pupils with textbooks Observed pupil-teacher ratio 7 Box 2. The Service Delivery Indicators (SDI) Program A significant share of public spending on education is transformed to produce good outcomes at schools. Understanding what takes place at these frontline service provision centers is the starting point in establishing where the relationship between public expenditure and outcomes is weak within the service delivery chain. Knowing whether spending is translating into inputs that teachers have to work with (e.g. textbooks in schools), or how much work effort is exerted by teachers (e.g. how likely are they to come to work), and their competency would reveal the weak links in the service delivery chain. Reliable and complete information on these measures is lacking, in general. To date, there is no robust, standardized set of indicators to measure the quality of services as experienced by the citizen in Africa. Existing indicators tend to be fragmented and focus either on final outcomes or inputs, rather than on the underlying systems that help generate the outcomes or make use of the inputs. In fact, no set of indicators is available for measuring constraints associated with service delivery and the behavior of frontline providers, both of which have a direct impact on the quality of services that citizens are able to access. Without consistent and accurate information on the quality of services, it is difficult for citizens or politicians (the principal) to assess how service providers (the agent) are performing and to take corrective action. The SDI provides a set of metrics to benchmark the performance of schools in Africa. The Indicators can be used to track progress within and across countries over time, and aim to enhance active monitoring of service delivery to increase public accountability and good governance. Ultimately, the goal of this effort is to help policymakers, citizens, service providers, donors, and other stakeholders enhance the quality of services and improve development outcomes. The perspective adopted by the Indicators is that of citizens accessing a service. The Indicators can thus be viewed as a service delivery report card on education. However, instead of using citizens’ perceptions to assess performance, the Indicators assemble objective and quantitative information from a survey of frontline service delivery units, using modules from the Public Expenditure Tracking Survey (PETS), Quantitative Service Delivery Survey (QSDS), and Staff Absence Survey (SAS). The literature points to the importance of the functioning of schools and more generally, the quality of service delivery. The service delivery literature is, however, clear that, conditional on providers being appropriately skilled and exerting the necessary effort, increased resource flows for health can indeed have beneficial education outcomes. The SDI initiative is a partnership of the World Bank, the African Economic Research Consortium (AERC), and the African Development Bank to develop and institutionalize the collection of a set of indicators that would gauge the quality of service delivery within and across countries and over time. The ultimate goal is to sharply increase accountability for service delivery across Africa, by offering important advocacy tools for citizens, governments, and donors alike; to work toward the end goal of achieving rapid improvements in the responsiveness and effectiveness of service delivery. More information on the SDI survey instruments and data, and more generally on the SDI initiative can be found at: www.SDIndicators.org and www.worldbank.org/SDI, or by contacting SDI@worldbank.org. Methodology and Implementation The SDI indicators draw information from a stratified random sample of 256 schools, comprised of 223 public and 33 private schools. This sample provides a representative snapshot of the learning environment in both public and private schools. The standard SDI 8 survey instruments were adapted to the Nigerien context through a participatory process involving technical discussions, training, and piloting with the Ministry of Education and the National Statistical Institute (Institut National de la Statistique; INS). The survey was implemented by INS with support and supervision from the World Bank’s Service Delivery Indicators (SDI) team. The modules of the survey instrument are shown in Table 6 below. Table 6. Survey modules Module Description 1. School information Administered to the director to collect information about school type, facilities, school governance, number of pupils, and school hours. Includes direct observations of school infrastructure by enumerators. 2. Teacher information and Administered to director and individual teachers to obtain a list of absence all school teachers, to measure teacher absence and to collect information about teacher characteristics. Includes an unannounced visit (Module 2B) to evaluate absence using the best known method from research on the subject. 3. Governance, Administered to the director to collect information about school management, and functioning and the role of the director. leadership 4. Classroom observation Information on teaching activities, classroom conditions; collected through direct observation in the classroom. 5. Pupil assessment Test of randomly selected grade-four pupils to measure their learning outcomes in mathematics and language. 6. Teacher assessment Evaluation of teachers’ subject knowledge (mathematics and language) and teaching skills. In 2014, parents could send their students to public (96.4 percent of primary students), private (3.3 percent), or community (0.2 percent) schools, and could choose between the traditional (86.6 percent of students) and Koranic (médersa; 12 percent) curricula, with very limited experimental (1.2 percent) or special-needs (0.2 percent) places. For reasons of comparability across countries, the survey uses the schools teaching traditional curriculum, which represents nearly 87 percent of students. The region of Diffa was declared to be in a state of emergency by the Government of Niger, and was therefore removed from the sample frame as were Tesker, Tassara, and Tillia (localized insecurity) and Bilma (few schools, massive distances). The final sample frame contained all public and private primary schools in Niger which taught a traditional curriculum and were neither experimental nor special- needs. Further details on the sampling procedure are in Annex 1. Table 7 provides details of the sample for the Education Service Delivery Indicators. The survey involved 256 primary schools, of which 87 percent were public schools and the remaining 13 percent either private for-profit or private not-for-profit schools. The survey assessed the knowledge of 1,604 primary school teachers, surveyed 1,748 teachers as part of the study of the absence rate, and observed 255 grade four lessons. In addition, learning outcomes were measured for 1,938 grade four pupils. 9 Table 7. SDI Education sample Sample Weighted Variable Total Percent Distribution Ownership 256 100 100 Public 223 87 97 Private 33 13 3 Location Rural 169 66 80 Urban 87 34 20 Urban public 55 25 18 Rural public 169 75 82 Teachers Public 1435 82 96 Private 313 18 4 Pupils 2,089 100 100 Note: subtotals may not add to the totals due to rounding. RESULTS A. Teacher effort Absence from school During the first announced visit, a maximum of ten teachers are randomly selected from the list of all teachers who are on the school roster. The whereabouts of these ten teachers are then verified in the second, unannounced, visit. Teachers found anywhere on the school premises are marked as present. The statistics are weighted, with the interpretation of the indicator being the percentage of teachers who are absent during an unannounced visit. The indicators relating to teacher effort (Absence from school, Absence from class, and Time spent teaching) and the differences in outcomes between public and private, and urban and rural schools, respectively, are presented in Table 8. They are summarized in Figure 2, which shows that out of 100 teachers approximately 64 percent of teachers were in class teaching on a given day. Roughly one teacher in six (17 percent) is absent from school and a further 10 percent were at school, but not in the classroom. 10 Figure 2. Teacher activities during the school day (percent) Table 8 shows that one teacher in six was absent on any given day in Niger. Private school teachers are nearly 90 percent less likely to be absent from school and 57 percent less likely to be absent from class than their public sector counterparts. Among public school teachers, there is no notable difference between those in rural and urban areas. When the head is absent, rural public school teachers are 6.68 times as likely to be absent, which is similar to urban public school teachers (6.0 times). The finding that the head’s absence is positively correlated higher teacher absence is consistent across countries that have done SDI. When looking at absence rates, 54 percent of schools have all teachers present during the unannounced visit. Table 8. Teacher Effort Diff. Urban Rural Diff. Indicator All Public Private (%) Public Public (%) Absence From school 16,6 17,2 1,9*** -89,2 17,3 17,1 1,0 From class, but at school 27,0 27,6 11,9*** -57,1 31,2 25,7 21,5 Teaching Time spent teaching 4h 23m 4h 13m 5h 17m*** 25,6 4h 06m 4h 15m 3,8 Daily scheduled lesson time 5h 40m 5h 41m 5h 32m -2,8 5h 41m 5h 42m -0,2 Note: significance levels: *** p<0.01; ** p<0.05; * p<0.1. Differences are relative to public and rural public, respectively. Figure 3 summarizes the many reasons given for teachers’ absence; the major ones are 11 strike (24 percent) and gone to collect salary (23 percent). Niger’s high fertility rate is also reflected in teacher absences: 11 percent of those absent are on maternity leave. Cultural considerations are observed as well; 3.8 percent of absent teachers have gone to funerals. Figure 3. Reasons why teachers are absent from school (percent) Absence from class The indicator is constructed in the same way as the school absence rate indicator, with the exception that now the numerator is the number of teachers who are either absent from school, or present at school but absent from the classroom. A small number of teachers may be found teaching outside, and these are marked as present for the purposes of the indicator. The interpretation of the indicator is the percentage of teachers who are absent during an unannounced visit. On any given day, 27 percent of teachers are not in the classroom, with urban public school teachers five percentage points more likely to be absent than their rural counterparts (Table 8). Public school teachers are far more likely to be absent from school than their private counterparts (p<0.01), but are relatively less likely to be at school and not teaching (p<0.01 still). As Table 27 shows, compared to private school teachers, civil servants are 8.6 times more likely (p<0.01) to be absent from school and contractual teachers are 13.8 times more likely (p<0.01). Also, civil servants and contractual staff are more likely to be absent from class than private school teachers (p<0.01). Niamey has the highest levels of absence, but statistically only Agadez and Tahoua show differences for absent from class and absent from class, at school (p<0.10 in all cases; Table 28). The comparison of absence from school and absence from class shows that nearly two-thirds 12 of the absence rate of teachers from the classroom is driven by absence from the school. Estimates for absence from school, absence from class, and absence from class, at school, broken down by teacher type are in Table 27. Unsurprisingly, private school teachers are statistically significantly more likely to be present than any other teacher category, however their absence from the classroom is primarily due to teachers who are at school, rather than those who do not show. 4 The primary education system in Niger creates teacher absence because they are not paid into bank accounts, but rather at centralized locations. Once extrapolated to the population, teachers in Niger require a day to collect their salaries, based upon their declarations. While there should be a definite end-of-month pattern, the survey time period only gives one end of month (November) and the pattern is not so clear in the data (Figure 4). Likewise, a Wald test of the significance of the day dummies in the regressions reported below shows they are jointly insignificant. Even the 27 November dummy has a p-value of 0.439. Figure 4. Teacher absence rate by second visit date (%) Note: data are weighted. Table 31 reports regression models of absence from school and the classroom for teachers in all schools. The information is shown graphically in Figure 5. These models demonstrate correlations, rather than causations, but still offer information to be considered. There are four broad categories of variables in the models: (i) teacher-specific variables, (ii) school-specific variables, (iii) ministry supervision variables, and (iv) school 4 For more information on the characteristics and absence rates of teachers, see Table 30. 13 management committee variables. Looking at the results, items related to oversight and personal incentives seem to be the most important aspects. Frequent pay delays do not motivate someone to work and the regression results show that an additional standard deviation in delays (1 month) is associated with seven percent of an additional standard deviation of absence (1.06 percentage points; p<0.05). Other personal variables, such as gender, contractual status, and school headship are not statistically significantly related. Among school-level variables, having an absent head means a teacher is 25 units more likely to be absent from school (p<0.10) and 28 units more likely to be absent from the classroom (p<0.01). Having teaching resources, in the SDI sense, is associated with a 4.8- unit decrease in absence from school (p<0.10). Poverty, rural/urban breakdowns, school ownership, and peers do not have a statistically-significant correlation in these models. Supervision by the ministry has significant correlations with absence. Use of a supervision worksheet, is associated with a five-unit drop in absence from school (p<0.05) and a nine- unit drop in absence from class (p<0.01). When supervisors observed class, the correlation with absence is positive, both for school (7 units, p<0.01) and class (8 units; p<0.01). While this may seem paradoxical, if supervisors do not observe each time, teachers may expect lower intensity in later visits and therefore take advantage. Finally, the involvement of the community matters. Relative to an inactive committee, a moderately active one, as defined by the principal, is associated with a 13.6-unit decrease in absence from school (p<0.01) and a 9.8-unit decrease in absence from the classroom (p<0.10). For the latter, a highly-active committee is also statistically significantly correlated with a decrease in absence, by 7.9 units (p<0.10). 14 Figure 5. Teacher absence correlates Note: estimates from weighted models are expressed in standard deviations of the dependent variable. The dot represents the point estimate and the line represents the 95 percent confidence interval. Variable definitions are in Table 30. Time spent teaching per day This indicator combines data from the staff roster module (used to measure absence rate), the classroom observation module, and reported teaching hours. The teaching time is adjusted for the time teachers are absent from the classroom, on average, and for the time the teacher teaches while in classrooms based on classroom observations. While inside the classroom distinction is made between teaching and non-teaching activities. Teaching is defined very broadly, including actively interacting with pupils, correcting or grading pupil's work, asking questions, testing, using the blackboard, or having pupils working on a specific task, drilling, or memorization. Non-teaching activities is defined as work that is not related to teaching, including working on private matters, maintaining discipline in class, or doing nothing and thus leaving pupils not paying attention. The interpretation of the weighted indicator is the percentage of time taught in fourth-grade classrooms in an average day when accounting for absence from the classroom. This indicator measures the amount of time a teacher spends teaching in a school during a normal day. It is calculated by recording the reported scheduled time of a teaching day from school records, i.e., five hours and 40 minutes after break times (Table 8). This 15 number is multiplied by the proportion of teachers absent from the classroom. The idea being that if 10 teachers are supposed to teach five hours and 41 minutes per day, but 2.7 of them are absent from either the school or the class at any one time, then the scheduled teaching time is reduced to four hours and 23 minutes (five hours and 40 minutes x 0.775). On average in Niger, a teacher will teach for four hours and 23 minutes (Table 8). That is, teachers teach approximately 77.5 percent of the scheduled time (the reported scheduled time for grade four pupils is five hours and 40 minutes after break times). Table 8 reports some intermediate inputs used in the calculation of the indicator. Even when in the class, however, teachers may not necessarily be teaching. The percentage of the lesson lost to non-teaching activities is measured through observation of a grade four lesson. 5 The distributions of the underlying variables are shown in Figure 6. Taken independently, it appears that there is a group of teachers who are frequently absent from the classroom, but that most teachers provide instruction the majority of time they are in the classroom. Table 8 also shows significant variation between public and private schools in teacher effort, whether measured in terms of classroom time taught or net teaching time for students. Private school teachers teach 95.3 percent of the time they are in class, which is fully 28.7 percent (p<0.01) more than their public school counterparts. Accounting for teacher absence and use of classroom time, private school students receive, on average, 25.6 percent more instruction per day than public school students (p<0.01), which represents an additional one hour and four minutes per day. There is no significant variation in the use of classroom time or overall teaching time between public schools in rural and urban areas. The phenomenon of classrooms without teachers, called “orphaned classrooms”, 6 is more common in public than in private and in urban public than in rural public schools, but the differences are not statistically significant at conventional levels except in the public/private space. Public schools, with an 8.6 percent rate, are nearly four times as likely to have orphaned classrooms as private schools (p<0.01), which have a 2.2 percent rate. 5 This is most likely an upper bound on the time devoted to teaching during a lesson; since presumably a teacher is more likely to teach when under direct observation (i.e. Hawthorne effects will bias the estimate upward). During the observation, enumerators first had to judge whether the teacher was teaching or not. If they judged the teacher to be teaching, they were supposed to indicate how much time the teacher spent on any of the following teaching activities: teacher interacts with all children as a group; teacher interacts with small group of children; teacher interacts with children one on one; teacher reads or lectures to the pupils; teacher supervises pupil(s) writing on the board; teacher leads kinesthetic group learning activity; teacher writing on blackboard; teacher listening to pupils recite/read; teacher waiting for pupils to complete task; teacher testing pupils in class; teacher maintaining discipline in class; teacher doing paperwork. 6 This is measured by inspecting the school premises, counting the number of classrooms with pupils, and recording whether a teacher is present in the classroom or not. The share of orphaned classrooms is then calculated by dividing the number of classrooms with pupils but no teacher by the total number of classrooms that contained pupils. 16 Figure 6. Distributions of teacher effort indicators Box 3. Assessment of knowledge of teachers This indicator measures teacher's knowledge and is based on mathematics and language tests covering the primary curriculum administered at the school level to all mathematics or language teachers that taught grade 3 in the previous year or grade 4 in the year the survey was conducted. It is calculated as the percentage of teachers who score more than 80 percent on the language and mathematics portion of the test. Efforts were made to include all possible teachers in the evaluation and those results are reported in the last part of this section. The indicator is representative of the average teacher in the universe of teachers in the country rather than the average teacher at the average school. B. Teachers with minimum knowledge The objective of the teacher test is to both evaluate mastery of basic and more advanced reading, writing, and mathematical skills and to evaluate mastery of pedagogical skills. Taken together, these are critical elements in the pupils’ acquisition of subject knowledge. The basic reading, writing, and arithmetic skills that lower primary pupils need to have in order to progress further with their education are interpreted as the minimum knowledge required for the teacher to be effective and serve as the basis for the "Share of teachers with minimum knowledge” indicator. While in some Anglophone countries, primary school teachers were specialized in language or mathematics, in Niger, teachers taught all subjects in primary school. The test was validated against the primary school curriculum of Niger and 12 other Sub-Saharan 17 primary school curricula. 7 The minimum knowledge indicator was calculated as the percentage of teachers who scored more than 80 percent on the lower primary part of the language and mathematics test (Table 9). The test also contained more advanced questions in both subjects, as well as a pedagogy section. Satisfactory mastery of content knowledge among teachers in Niger was among the lowest recorded across all SDI countries as zero percent of teachers of grade 4 (2015/16 school year) and grade 3 (2014/15 school year) achieved a score of 80 percent or higher in the combined French and mathematics sections. Private school teachers are not more proficient than their public school counterparts at the levels deemed sufficient for proficiency or higher and only outperform when the standard is set at 70 percent of the possible points. The results are generally similar when all evaluated teachers are included, even though the average score rises to 35.4 percent from 33.5 percent, and 0.4 percent of teachers reach the proficiency level. As the cumulative density plots in A broader look at teacher knowledge using all teachers evaluated by the survey The minimum knowledge indicator was calculated as the percentage of teachers who scored more than 80 percent on the lower primary part of the language and mathematics test (Table 9). The test also contained more advanced questions in both subjects, as well as a pedagogy section. Satisfactory mastery of content knowledge among teachers in Niger was among the lowest recorded across all SDI countries as zero percent of teachers of grade 4 (2015/16 school year) and grade 3 (2014/15 school year) achieved a score of 80 percent or higher in the combined French and mathematics sections. Private school teachers are not more proficient than their public school counterparts at the levels deemed sufficient for proficiency or higher and only outperform when the standard is set at 70 percent of the possible points. The results are generally similar when all evaluated teachers are included, even though the average score rises to 35.4 percent from 33.5 percent, and 0.4 percent of teachers reach the proficiency level. 7 See “Teaching Standards and Curriculum Review,” prepared as background document for the SDI Program by David Johnson, Andrew Cunningham, and Rachel Dowling. 18 Table 9. Teacher assessment Diff. Urban Rural Diff. Percent Niger Public Private (%) Public Public (%) Teachers with minimum knowledge 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Average score French, math, and pedagogy 33.5 33.3 37.5* 12.6 32.6 34.5 5.8 French and math 40.9 40.7 44.1 8.4 39.8 42.3 6.3 Sensitivity analysis (French and math) Minimum knowledge: 100% 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Minimum knowledge: 90% or higher 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Minimum knowledge: 80% or higher 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Minimum knowledge: 70% or higher 0.8 0.6 4.0 566.7 1.0 0.0 -100 Note: Results based upon evaluations of 386 teachers (307 public and 79 private) in 203 schools. Breakdowns by grade taught are in Table 29 for all teachers evaluated. Differences are relative to public and rural public, respectively. French Table 10 shows that teachers successfully completed roughly one-half of the French content on the evaluation. Overall, there is no significant different along ownership or rural/urban within public lines. Private school teachers did better in grammar and composition than public school teachers and urban public teachers did worse than rural public teachers in composition. Table 10. Teacher French scores (percent) Diff. Urban Rural Diff. Diff. Niger Public Private (%) Public Public (%) Rural Urban (%) French 47.7 47.7 47.1 -1.3 47.7 47.7 0.0 47.7 47.6 -0.2 Grammar 69,7 69,4 74,8*** 7,8 68,5 71,2 3,9 68,5 71,7* 4,7 Cloze task 30,9 31,2 24,5* -21,5 32,2 29,3 -9,0 32,2 28,7 -10,9 Composition 12,0 11,6 19,6*** 69,0 10,0 14,4*** 44,0 10,0 15,0*** 50,0 Note: these results are for teachers who taught grade 3 in the year prior to the survey and/or grade 4 the year of the survey. Differences are relative to public, rural public, and rural, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Mathematics Teachers who taught grade 3 in the year prior to the survey or grade 4 in the year of the survey, correctly completed 26.7 percent of the basic mathematics examination (Table 11). Private school teachers performed better than their public school counterparts in all evaluation areas (p<0.1 or better, except for Venn diagrams), particularly in “advanced” mathematics, which tested concepts beyond the primary school level. Unlike French, where urban public school teachers generally outperformed their rural counterparts, there is no statistical difference in mathematics. Urban teachers also generally outperform their rural counterparts, although more in basic than advanced math. 19 Table 11. Teacher mathematics scores (percent) Diff. Urban Rural Diff. Diff. Niger Public Private (%) Public Public (%) Rural Urban (%) Math 26.7 26.3 34.1* 29.7 24.8 28.9 16.5 24.8 29.6* 19.4 Basic 34.1 33.7 41.2* 22.3 32.0 36.9 15.3 32.0 37.4* 16.9 Advanced 12.8 12.4 21.0** 69.4 11.4 14.0 22.8 11.4 14.9 30.7 Fractions 8.8 8.5 15.1* 77.6 7.2 10.8 50.0 7.2 11.4 58.3 Venn diag. 16.1 15.8 21.9 38.6 14.4 18.5 28.5 14.4 18.9 31.3 Graphs 6.0 5.6 14.0* 150.0 5.1 6.5 27.5 5.1 7.4 45.1 Notes: these results are for teachers who taught grade 3 in the year prior to the survey and/or grade 4 the year of the survey. The abbreviations “basic” and “advanced” refer to primary and supra-primary evaluation items, respectively, while “Venn diag.” means “Venn diagrams”. Differences are relative to public, rural public, and rural, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Pedagogy Although this component is not included in the minimum knowledge variable, it does provide some insight into teachers’ ability to prepare and deliver lessons and evaluate students. As with the other evaluation sections, private-school teachers perform better than those in public school by 5.6 percent (p<0.05), and particularly in more advanced pedagogical skills and class preparation (p<0.01 in both cases). There are no significant differences among public school teachers by school location or between rural and urban teachers in general. Table 12. Teacher pedagogy scores (percent) Diff. Urban Rural Diff. Diff. Niger Public Private (%) Public Public (%) Rural Urban (%) Pedagogy 21.0 20.8 26.1*** 25.4 21.8 20.3 7.6 20.3 22.3* 9.9 Basic 17.8 17.6 21.9*** 24.5 18.4 17.2 7.1 17.2 18.8 9.3 Advanced 26.6 26.3 33.3*** 26.3 27.7 25.6 8.2 25.6 28.3** 10.5 Prepare 26.8 26.6 31.0*** 16.3 28.1 25.9 8.3 25.9 28.4* 9.5 Compare 0.4 0.4 0.6*** 35.7 0.4 0.4 7.3 0.4 0.5 9.8 Evaluate 5.4 5.3 7.2** 36.1 5.5 5.2 4.8 5.2 5.6 8.5 Notes: these results are for teachers who taught grade 3 in the year prior to the survey and/or grade 4 the year of the survey. The abbreviation “adv. math” refers to “advanced math” and “Venn diag.” to “Venn diagrams”. Differences are relative to public, rural public, and rural, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. A broader look at teacher knowledge using all teachers evaluated by the survey This section uses the information on all teachers and principals evaluated, which differs from the SDI indicator, which focuses on those who taught fourth grade the year of the survey or third grade the year preceding the survey. As Figure 7 shows, the majority of teachers in Niger scored between 20 and 60 percent on the evaluation, which is still relatively far from the 80 percent minimum level to be deemed to have a sufficient level of the primary curriculum. 20 Figure 7. Teacher evaluation cumulative distribution (all teachers) Table 29 presents various breakdowns of teacher performance for all teachers evaluated. Panel A provides the same comparisons as discussed above. There are few deviations from the conclusions above. Private school teachers score 11.3 percent higher than public school teachers on the evaluation (p<0.01), but statistically their level of mastery is equivalent and closer to zero than any positive number. On average, private-school teachers perform better in French (7.8 percent; p<0.1), math (31.7 percent; p<0.05), and pedagogy (142.7 percent; p<0.01). However, in a pattern that is repeated throughout Panel A, the public (rural public, rural) teachers perform better on composition (27.2 percent; p<0.01 for public). This is the only statistically-significant exception to the rule that private-school teachers perform better, on average, at any task than their public-school counterparts. The major difference with the smaller sample discussed above is that the differences among all teachers are statistically more significant among public school teachers or across rural and urban when all information is used. Among public school teachers, 0.5 percent of those in rural reach the proficiency threshold compared to zero percent in urban (p<0.05), but this is still very low. However, urban public performs statistically better in pedagogy (69.4 percent, p<0.05), math (35.6 percent; p<0.01), and French (9.3 percent; p<0.01). The same pattern is observed when comparing rural and urban; while rural teachers are more likely to reach the minimum knowledge (0.5 percent; p<0.1), urban teachers perform statistically better in pedagogy (83.7 percent, p<0.01), math (37.0 percent; p<0.01), and 21 French (9.5 percent; p<0.01). Panel B compares the teachers’ performance by contractual status and shows a cost of the heavy reliance on contractual teachers in Niger (69.8 percent of teachers) as they perform worst among the three categories. Civil servants uniformly outperform the contractual teachers including by 13.3 percent in French (p<0.01), 32.1 percent in math (p<0.01). and 46.5 percent in pedagogy (p<0.01). Private school teachers are weaker than civil servants in French (11.3 percent; p<0.05) and pedagogy (17.2 percent; p<0.1), but have a nearly identical mathematics sub-aggregate. Panel C breaks down teachers by their level of academic achievement. Teachers who completed more education score better, but the change from no high school to some high school seems more important than that of some high school to completed high school. Looking across the sub-aggregates, teachers with a high school degree performed between 13 percent (French; p<0.01) and 40 percent (pedagogy; p<0.01) better than teachers who had not reached high school. Teachers who had at least some high school education perform better, but are still worse than teachers who completed high school in nearly all areas. In French, the difference is 7.1 percent (p<0.05) and in pedagogy it is 18.8 percent (p<0.01). Panel D shows that teacher performance is generally positively correlated with the grade taught (aside from first grade). The most notable impacts are felt in mathematics and pedagogy. The effect appears to be nonlinear and asymmetric: comparing grades three and five, the effect is larger for grade three; comparing grades two and six, the effect is larger for grade six. Panels E and F present comparisons of the effect of attending teacher training college and the level of professional training attained. As with academic progression, higher teacher certifications are positively correlated with higher teacher performance. Interestingly, among levels of certification, it is not clear that attending teacher training college has a significantly positive impact on the teacher’s performance. The best example in Panel E is the comparison between the “ENII” and “CAP” columns, both of which report on teachers who have achieved the highest certificate for teaching primary school. Those who did not attend ENI (“CAP”) generally do as well as those who did, and in a few cases (Grammar, Venn diagrams, pedagogy, and basic pedagogy) actually do better. Panel F compares across those who attended and those who did not and also shows that there is generally not a statistically-significant difference among the two populations. Panel G shows that recruitment, over time, has diluted the level of knowledge of teachers. Teachers hired before 2000 score higher in all evaluation areas, particularly mathematics and pedagogy (p<0.01 for all items but the Cloze task). Likewise, teachers hired before 2008 score better than those hired after on all items but the Cloze task with highly significant differences (p<0.01). Figure 14 shows this graphically, with a globally negative evaluation performance trend for teachers hired after 2000 until those hired in 2015. When looking at the data, teachers hired before 2000 were more likely to have completed high school (14 percent vs. six percent) or further education (17 percent vs. four percent; 22 p<0.01 for the cross-tabulation). This likely also reflects the rapid expansion of the education system in a context of insufficient human resources, which then repeats itself as new generations of teachers are recruited from the system. C. Availability of inputs at the school Functioning school infrastructure: minimum infrastructure resources is a binary 0-1 capturing availability of: (i) functioning toilets operationalized as being clean, private, and accessible; and (ii) sufficient light to read the blackboard from the back of the classroom. Functioning toilets: whether the toilets were functioning was verified by the enumerators as being accessible, clean, and private (enclosed and with gender separation). Electricity: functional availability of electricity is assessed by checking whether the light in the classroom works and gives minimum light quality. The enumerator places a printout on the board and checks (assisted by a mobile light meter) whether it was possible to read the printout from the back of the classroom. Availability of teaching resources: equipment availability is a binary variable equal to 1 if (i) the randomly selected grade 4 classroom has a functioning blackboard and chalk, (ii) the share of pupils with pens is equal to or above 90 percent, and (iii) the share of pupils with notebooks in that classroom is equal to or above 90 percent. Functioning blackboard and chalk: The enumerator assesses if there was a functioning blackboard in the classroom, measured as whether a text written on the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. Pencils/pens and notebooks: The enumerator counts the number of pupils with pencils or pens and notebooks, respectively. By dividing each count by the number of pupils in the classroom, one can then estimate the share of pupils with pencils or pens and the share of pupils with notebooks. Availability of textbooks: the indicator measures in one randomly selected grade 4 class the number of pupils with the relevant textbooks (mathematic or language conditional on which randomly selected class is observed), and divided by the number of pupils in the classroom. Pupil-teacher ratio: the indicator of teachers’ availability is measured as the number of pupils per teacher in one randomly selected grade 4 class at the school based on the Classroom Observation Module. The indicators Availability of teaching resources, Functioning school infrastructure, Pupil- teacher ratio, and Pupils per textbook are all constructed using data collected through visual inspections of a grade four classroom and the school premises in each primary school surveyed. Table 13 summarizes the findings. 23 Table 13. School input indicators Indicator (% unless otherwise Diff. Urban Rural Diff. noted) Niger Public Private (%) Public Public (%) Minimum teaching equipment 24,7 23,4 72,4 209,4 38,9 20,1 -48,3 Pupils with pencils 91,1 90,9 97,6*** 7,4 97,7*** 89,5 -8,4 Pupils with exercise books 45,5 44,4 89,8*** 102,3 69,8*** 39 -44,1 Classroom with board 100 100 100 0,0 100 100 0 Classroom with chalk 95,8 95,8 97,2 1,5 98,2 95,2 -3,1 Minimum infrastructure 21,3 19,7 80,7*** 309,6 28,3 17,9 -36,7 Contrast to read the board 32,1 34,1 28 -17,9 10,5*** 38,8 269,5 Minimum visibility by enumerator 86,4 86,2 94,5 9,6 100 83,3 -16,7 Toilet functioning and available 24,3 22,6 83,5*** 269,5 28,3 21,4 -24,4 Toilet clean 42,7 41,1 100 143,3 70,4 34,9 -50,4 Toilet private 27,7 26,2 83,5 218,7 35,2 24,3 -31 Toilet available 42,2 40,8 94,5 131,6 74,1 33,7 -54,5 Observed pupil-teacher ratio 38,3 38,1 46,9 23,1 48,2 36 -25,3 Textbook availability 9,3 8,7 33,6 286,2 9,5 8,5 -10,5 Pupils with math textbook 7,1 5,5 48,9 789,1 7,9 5 -36,7 Pupils with French textbook 10,7 10,3 27 162,1 10,6 10,3 -2,8 Note: Results based on observations in 256 schools. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Information on rural/urban breakdowns for the same set of indicators is in Table 26. Functioning school infrastructure Minimum infrastructure resources is a binary variable capturing availability of: (i) functioning toilets operationalized as being clean, private, and accessible; and (ii) sufficient light to read the blackboard from the back of the classroom. Functioning toilets: Whether the toilets were functioning was verified by the enumerators as being accessible, clean, and private (enclosed and with gender separation). Electricity: Functional availability of electricity is assessed by checking whether the light in the classroom works and gives minimum light quality. The enumerator places a printout on the board and checks (assisted by a mobile light meter) whether it was possible to read the printout from the back of the classroom. The minimum infrastructure indicator is four times more likely to be met in private than in public schools (p<0.01). Despite this, only four in five private schools meet the requirements for minimum infrastructure. This is driven primarily by the toilet, which is not always private or functioning in public or private. However, private school pupils are two to three times more likely to have a toilet that meets any condition (p<0.01). They are also more likely to be able to read the board, whether measured by a lux meter or by the interviewer’s perception. The differences between rural and urban public schools is less pronounced for infrastructure, although urban schools are 16.7 percent more likely to have minimum infrastructure than rural ones (p<0.01), primarily because they have more toilets that are available (40.3 percent; p<0.01) or toilets that are clean (35.5 percent; p<0.01). Even considering those factors, urban public schools are still 2.9 times less likely to meet the minimum infrastructure indicator’s requirements than private schools. 24 Equipment availability is a binary variable equal to 1 if (i) the randomly selected grade four classroom has a functioning blackboard and chalk, (ii) the share of pupils with pens is equal to or above 90 percent, and (iii) the share of pupils with notebooks in that classroom is equal to or above 90 percent. Functioning blackboard and chalk: The enumerator assesses if there was a functioning blackboard in the classroom, measured as whether a text written on the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. Pencils/pens and notebooks: The enumerator counts the number of pupils with pencils or pens and notebooks, respectively. By dividing each count by the number of pupils in the classroom, one can then estimate the share of pupils with pencils or pens and the share of pupils with notebooks. Availability of teaching resources SDI measures whether or not all elements of the minimum teaching resources are simultaneously present in a given school rather than individual or average presence of the elements. Minimum teaching resources is a pressing constraint. A visual presentation of the co-availability of the inputs is in Figure 11. The minimum teaching equipment indicator shows that private school pupils are 3.1 times more likely to be in a school with all necessary elements (p<0.01), both because they are 10 percent more likely to have a pencil (p<0.01) and twice as likely to have an exercise book (p<0.01), but also because the requirement to have those and a functional board simultaneously are hard to meet in the public schools. Among public schools, there are differences driven by the relative lack of pencils (-8.3 percent; p<0.01) and exercise books (-30.8 percent; p<0.01) in rural areas. Interestingly, rural public classrooms were 28.3 percent more likely to have blackboards readable from the back of the class than urban public classrooms (p<0.01). Availability of textbooks The indicator measures in one randomly selected grade 4 class the number of pupils with the relevant textbooks (mathematic or language conditional on which randomly selected class is observed), and divided by the number of pupils in the classroom. At the end of the observation in French and mathematics classes, textbook availability was directly measured by asking pupils to raise the textbook for the subject observed by the field teams. In Niger, less than one in eleven pupils had the textbook for the class observed (8.9 percent). Private school students fared better than public school students for both subjects (p<0.05), but even they had a textbook only one-quarter of the time. Textbooks for French were more available those for math in private and public irrespective of location. Despite this, on average there are at least three children per book, even in the best case (French in private schools). The general level of textbook availability is not significantly different across public schools in rural and urban areas. 25 Table 14. School textbook indicators (percent) Diff. Urban Rural Diff. Niger Public Private (%) Public Public (%) Pupils with textbook 8,9 8,4 24,0** 185.7 8,6 7,7 -10.5 Male with textbook 9,3 8,7 27,5** 216.1 8,9 8,0 -10.1 Female with textbook 8,6 7,9 29,9*** 278.5 8,4 7,2 -14.3 Pupils with math textbook 7,6 6,9 24,3* 252.2 8,1 4,6 -43.2 Male with math textbook 7,9 7,1 26,8* 277.5 9,2 4,6 -50.0 Female with math textbook 7,5 6,7 26,7* 298.5 7,6 4,6 -39.5 Pupils with French textbook 9,8 9,2 28,0* 204.3 8,9 9,3 4.5 Male with French textbook 9,6 9,1 28,1* 208.8 8,7 9,2 5.7 Female with French textbook 9,2 8,6 32,9** 282.6 8,9 8,5 -4.5 Note: Results based on observations in 256 schools. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Information on rural/urban breakdowns for the same set of indicators is in Table 26. Pupil-teacher ratio The indicator of teachers’ availability is measured as the number of pupils per teacher in one randomly selected grade four class at the school based on the classroom observation module. Fourth grade pupils in Niger have, on average and based upon direct observation, 37 classmates. Private school pupils are in larger classes than public school pupils (8.8 percent; p<0.01) as are urban public school pupils compared to their rural peers (12.2 percent; p<0.01). As Figure 8 shows, there is some variation in class size, but most schools do not have the teaching resources and almost all schools do not have textbooks for their pupils. 26 Figure 8. Availability of inputs D. Assessment of pupil learning It is instructive to think of the Service Delivery Indicators as measuring key inputs, with a focus on what teachers do and know, in an education production function. These inputs are actionable and are collected using objective and observational methods at the school level. The outcome in such an education production function is pupil learning achievement. While learning outcomes capture both school-specific inputs (for instance, the quality and effort exerted by the teachers) and various child-specific factors (for instance, innate ability) and household- specific factors (e.g. household welfare), and thus provide, at best, reduced form evidence on service provision, it is still an important measure to identify gaps and to track progress in the sector. Moreover, while the Service Delivery Indicators measure inputs (learning outcomes are not part of the Indicators), in the final instance we should be interested in inputs not in and of themselves, but only in as far as they deliver the outcomes we care about. Therefore, as part of the collection of the Service Delivery Indicators in Niger, learning outcomes were measured for grade four pupils. This section reports on the findings. The objective of the pupil assessment was to assess basic reading, writing, and arithmetic skills. The test was designed by experts in international pedagogy and based on a review of primary curriculum materials from 13 African countries. 8 The pupil assessment also measured nonverbal reasoning skills on the basis of Raven’s matrices, a standard IQ measure that is designed to be valid across different cultures. This measure complements the pupil test scores in French and mathematics and can be used as a rough measure to control for innate pupil ability when comparing outcomes across different schools. Thus, 8 For details on the design of the test, see Johnson, Cunningham, and Dowling (2012) “Draft Final Report, Teaching Standards and Curriculum Review”. 27 the pupil assessment consisted of three parts: mathematics, French, and non-verbal reasoning (NVR). The test was examined by the Ministry of Education team for suitability in the local context, judged acceptable after some minor modifications to maintain comparable difficulty, and was administered to fourth- and fifth-grade pupils. The reason for the choice of grade four pupils is threefold. First, there is no standardized national or international evaluation of this level, although PASEC evaluates grades two and six (previously: five). Second, the sample of children in school becomes more and more self- selective as one goes higher up due to drop-out rates. Finally, there is growing evidence that cognitive ability is most malleable at younger ages. It is therefore especially important to get a snapshot of pupil’s learning and the quality of teaching provided at younger ages. The test was designed as a one-on-one test with enumerators reading out instructions to pupils in their mother tongue. This was done so as to build up a differentiated picture of pupils’ cognitive skills, using oral one-to-one testing allows us to evaluate whether a child can solve a mathematics problem even when her reading ability is so low that she would not be able to attempt the problem independently. The French test consisted of a number of different tasks including knowledge of the alphabet, word recognition, reading out loud, and reading comprehension. The mathematics test contained tasks such as identifying and sequencing numbers, addition of one- to three-digit numbers, one- and two-digit subtraction, and single digit multiplication and divisions. The non-verbal reasoning section consisted of four questions. Fourth-grade pupils in Niger achieved an average score of 22.6 percent on the assessment (Table 15), which means that children mastered, on average, one-fifth of the fourth grade curriculum at the start of the year. While this may seem encouraging, the details are sobering; one child in nine (11.2 percent) can read a simple sentence, less than one child in two can do single-digit addition, two children out of five can do single-digit subtraction. However, there are large differences between private and public school students and between urban public and rural public school students. Generally speaking, private school students do between twice and seven times as well as public school students in language and mathematics until division (p<0.01 always). Likewise, urban public school students do one-third to two-thirds better than rural public school students (p<0.01 for all except double-digit subtraction and single-digit multiplication with p<0.05) through single-digit multiplication. In terms of performance, there are three school systems: private, urban public, and rural public. When comparing across regions (Table 34), all regions do worse than Niamey (p<0.01 except Agadez, p<0.10). 9 All regions perform worse than Niamey in both language (p<0.01 uniformly) and mathematics (p<0.01 except for Tahoua). The differences are often large and suggest that children in other regions are at a serious learning disadvantage compared with those in Niamey. 9 For the rest of the section, Agadez will be ignored, since it has very few observations, which also corresponds to its proportionally low student population. The regions were not strata for the sample, so there was no minimum count to ensure the possibility of analyzing the data in this way. 28 Table 15. Student performance results (grade 4; percent) Diff. Urban Rural Diff. Niger Public Private ( %) Public Public (%) Overall 22,6 21,3 65,5*** 207,5 32,6 18,0*** -44,8 Literacy 23,2 21,7 73,1*** 236,9 34,8 17,9*** -48,6 Letter recognition 45,5 44,0 95,5*** 117,0 61,3 39,0*** -36,4 Read basic words 29,7 28,0 84,7*** 202,5 46,8 22,6*** -51,7 Read simple sentence 12,2 10,5 66,5*** 533,3 19,5 7,9*** -59,5 Read paragraph 9,1 7,5 59,1*** 688,0 16,4 5,0*** -69,5 Reading comprehension 6,5 5,5 41,9*** 661,8 11,4 3,7*** -67,5 Numeracy 11,8 11,5 24,7*** 114,8 15,0 10,4*** -30,7 Add: 1-digit 45,4 44,3 82,1*** 85,3 59,3 39,9*** -32,7 Add: 2-digit 26,9 25,8 64,5*** 150,0 38,7 22,0*** -43,2 Subtract: 1-digit 40,6 39,3 83,5*** 112,5 51,8 35,7*** -31,1 Subtract: 2-digit 8,5 7,9 26,5*** 235,4 13,2 6,4** -51,5 Multiply: 1-digit 9,6 9,0 27,9*** 210,0 14,6 7,4** -49,3 Multiply: 2-digit 0,7 0,6 4,4** 633,3 1,3 0,4 -69,2 Divide: 1-digit 8,3 8,1 12,6 55,6 9,5 7,7 -18,9 Divide: 2-digit 1,6 1,6 3,1 93,8 2,0 1,4 -30,0 Nonverbal reasoning 53,1 52,7 66,7*** 26,6 56,9 51,5** -9,5 Note: Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Results are based upon 1,651 pupils in 256 schools (weighted results). Differences are relative to public and rural public, respectively. Since the survey was done in the first few months of the 2015-16 school year, grade 5 students (CM1) were also evaluated with the same questionnaire. This allowed for a comparison between new fourth graders and new fifth graders, who presumably had a fourth-grade level. When comparisons are made, there are important gains from the fourth grade year (pooling grades) as shown in Table 16. These gains are robust the way in which the students are treated for analysis, indicating increased mastery of the material by students a grade above their peers on average. However, even fifth graders do not perform well on the evaluation, as shown in Table 33. After fourth grade, three children in five can recognize a letter (61.1 percent), one in five (24.1 percent) can read a simple sentence, three in five can do single-digit addition (57.1 percent), and one out of two can do single- digit subtraction (49.8 percent). When comparing the results in Table 32 and Table 33, the difference between public and private and rural and urban within public decreases over time for language skills, but rises for mathematics which means that an extra grade of instruction serves to reduce the language skills gap, but widens the mathematics skill gap. Regional disparities follow similar patterns as for fourth grade (Table 35), but Tahoua and Tillabéri are less different from Niamey in mathematics, rather than just Tahoua as was the case in fourth grade. 29 Table 16. Pupil evaluation comparison between rising fourth and fifth graders CE2 CM1 Gains from fourth grade Split Split Pooled Module Mean SD Mean SD Z-score (%) Z-score Combined 0.226 0.011 0.336 0.016 10.0 48.7 8.7 Math 0.118 0.006 0.182 0.008 11.2 53.8 10.3 French 0.232 0.013 0.354 0.019 9.7 52.5 8.4 Non-verbal reasoning 0.531 0.011 0.566 0.011 3.3 6.6 3.2 Note: based upon 1,651 fourth-graders and 1,230 fifth-graders. The notation “SD” refers to a standard deviation and “Z- score” is the standardized value, defined as (actual-mean)/SD. The gains from fourth grade compare the CM1 students (new fifth graders) to the CE2 students (new fourth graders). The “split” columns treat each grade as a distinct population and the Z-score is standardized with the grade 4 data. The “pooled” column treats all students as coming from the same population and the standardization is relative to the pooled data. 10 The regressions reported in Table 36 (French) and Table 37 (math) and shown graphically in Figure 9 provide some broadly suggestive results. Female pupils perform two (2.6) standard deviations worse than their male peers in French (math) and the results are at significant in math in all models (p<0,05). Completing grade 4 is associated with a performance increase between eight and nine standard deviations in math and French (p<0.01 in all models). For both subjects, a one-standard deviation increase in time on task represents an eight-tenths of a standard deviation increase in learning (p<0.1). In French, a one percent increase in poverty results in a three-tenths of a standard deviation decrease in performance (p<0.05). In math, attending a private school is associated with a 36 standard deviation increase in the score (p<0.05). That is equivalent to 152 percent of the mean student performance. The effect is language is also of a large magnitude, but is not statistically significant. 10 When pooling, the variance estimates are generated using clusters at the school-grade level, which is accurate for the survey design so long as there is only one class for each grade in the school. That is the case in 168 of the fourth grades and 101 of the sixth grades. Not including the additional detail will likely overestimate the variance. 30 Figure 9. Student learning correlates Note: the models are survey-weighted and the standard errors account for clustering by school and grade within the school. Variables with an “(s)” are standardized; those with “(d)” are dummy (binary). The dummy for CE2 is relative to CM1. The models are estimated in standard deviations of the dependent variables, language (1.24 points) and math (0.54 points). Poverty headcounts are from the poverty mapping results undertaken by the Institut National de la Statistique and the World Bank (unpublished). Table 17 presents breakdowns across gender and school location, while Table 18 breaks down the data across gender and school ownership. Rural pupils, irrespective of gender, perform significantly worse than urban boys (p<0.01 uniformly) and even urban girls perform somewhat worse than their male classmates (p<0.1 for French). However, the gender gap is smaller in rural than in urban. As highlighted in Figure 10, urban public pupils have higher French and math skills than their rural counterparts. Comparing across the various sub-items of the modules, urban public pupils score 50 percent better (average: 52 percent, median: 54 percent) on every evaluation sub-domain in French and mathematics (p<0.01) except division, where the pupils are, on average, similar. Controlling for school ownership, this represents a serious gap in learning outcomes for rural pupils compared to urban ones. 31 Figure 10. Pupil evaluation distribution by module and location among public schools Table 17. Pupil evaluation: gender and location breakdowns for grade 4 (percent) Estimates for sub-populations Diff. (%) Urban Rural Urban Rural Rural Urban Rural Module Niger boys boys girls girls boys girls girls Overall 22,6 39,6 18,8*** 33,4 16,9*** -52,5 -33,0 -68,0 French 23,2 43,0 18,9*** 35,7 16,7*** -60,9 -38,8 -78,7 Mathematics 11,8 17,3 10,6*** 15,1 10,2*** -16,9 -11,7 -21,3 Non-verbal reasoning 53,1 56,0 51,6* 60,0 51,4 -11,1 21,3 -13,8 Note: weighted means using sampling weight and the sample design. Results based on 1,651 pupils in 256 schools. Differences are relative to urban boys. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. When comparing public with private, the overall gaps are more pronounced, with private school children scoring nearly three times more on any given learning evaluation module. Looking at the ratio of private to public scores for a module, it is higher for girls in fourth grade and for boys in fifth grade. 32 Table 18. Pupil evaluation: gender and school ownership breakdowns for grade 4 (percent) Estimates for sub-populations Diff. (%) Public Private Public Private Public Private Public Module Niger boys boys girls girls boys girls girls Overall 22,6 22,2*** 64,9 20,2*** 66,4 -65.8 -68.9 2.3 French 23,2 22,8*** 72,4 20,4*** 74,0 -68.5 -71.8 2.2 Mathematics 11,8 11,7*** 25,2 11,2*** 24,2 -53.6 -55.6 -4.0 Non-verbal reasoning 53,1 52,3*** 62,4 53,3*** 72,1 -16.2 -14.6 15.5 Note: Weighted means using sampling weight and the sample design. Results based on 1,651 pupils in 256 schools. Differences are relative to private-school boys. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. E. Incentives, leadership, and management The Niger SDI survey tested a module on incentives, leadership, and management. The goal of the module was to provide additional information on observed service delivery strengths and weaknesses in the facilities. Work in education in Mozambique had showed that directors’ knowledge of teacher absenteeism seemed unrelated to what was observed in practice. However, work in a number of SDI surveys showed that the most significant correlate of absenteeism, whether correlation or size, was the absence of the head of the school or health facility. Therefore, this module was designed to gain additional knowledge on the interplay of the formal institutional rules and the realities of the service delivery units. Leadership and management School directors were asked, among other topics, about their management training, major constraints, and teacher management. Nearly half of directors (42.7 percent) report having received training in management, although nearly as many received it from teacher training college (29.5 percent) as from peers and school inspectors (26.8 percent). Nearly one in ten directors received such training during the 30 to 45-day training session for teachers who did not go through the teacher training college. Private school directors (75 percent; p<0.01) were 1.8 times as likely as their public counterparts to have received training although they were also 2.3 times as likely (58.3 percent; p<0.01) to have received it from peers or education supervision missions than public school directors. Rural public directors were 2.3 times more likely (40.3 percent; p<0.05) to have received their management training from “other” than urban public directors. Table 19 shows that there are clear differences in constraints across public and private. Public schools cited infrastructure (p<0.01), teaching materials (p<0.01), and teachers; private schools cited none (p<0.01), other, and autonomy (p<0.05). Within public schools, the significant differences are equipment, which rural schools expressed more than three times more than urban ones (p<0.10), and other which was more common for urban public schools (p<0.01). 33 Table 19. Constraints to service delivery (percent) Primary Diff. Rural Urban Diff. constraint Niger Public Private (%) Public Public (%) Teaching materials 14,2 14,4 3,1*** -78,5 12,7 14,8 16,5 Teachers 9,7 9,7 9,4 -3,1 10,9 9,5 -12,8 Infrastructure 32,4 33,1 3,1*** -90,6 32,7 33,1 1,2 Equipment 5,6 5,7 3,1 -45,6 12,7 4,1* -67,7 Leadership ,, ,, ,, ,, ,, ,, ,, Autonomy 3,1 2,8 18,8** 571,4 7,3 1,8 -75,3 None 3,7 2,9 37,5*** 1193,1 5,5 2,4 -56,4 Other 31,3 31,5 25,0 -20,6 18,2 34,3** 88,5 Notes: comparisons are relative to public and rural public, respectively. Levels of significance: *** p<0.01, ** p<0.05, * p<0.1. No director considered leadership to be a primary constraint. The school director and ministry are both responsible for the development of teachers. As reported in Table 38, nationally, 33 percent of directors had individual performance evaluation meetings with teachers and 22 percent of directors reported that supervision missions had such meetings. Both for meetings organized by the director and those organized by the external supervision, private schools emphasized more direct teacher observation (p<0.05 director, p<0.01 external), teacher absence rate (p<0.01 director), parent satisfaction (p<0.01 and p<0.05, respectively), student learning (p<0.01 and p<0.05, respectively), examination performance (p<0.01 and p<0.01, respectively), and teacher willingness to improve (p<0.01 for directors). Among public schools, urban directors pay more attention to student performance (p<0.05), but rural external supervisions pay more attention to parent satisfaction (p<0.10) than urban ones. This is corroborated by what the teachers say about the content of supervision by their directors: private school teachers report higher comment rates on discipline in class (33.6 percent vs 9.9 percent; p<0.01), on student evaluation (35.2 percent vs 7.8 percent; p<0.01), and on student attendance (22.7 percent vs. 6.5 percent; p<0.01). Among public schools, urban public school teachers report higher frequencies of comments on discipline (14.8 percent vs 7.3 percent; p<0.01) and on student evaluation (17.2 percent vs. 2.8 percent; p<0.01) than their rural counterparts. The work done by school directors to supervise their teachers varies across ownership, but not across location for public schools. As shown in Table 20, 30.9 percent of public school teachers say they are supervised annually or less as compared to 10.1 percent of private school teachers (p<0.01). At the other extreme, 64.4 percent of private school teachers say they are supervised multiple times per week compared to 23.2 percent in public schools (p<0.01). Interestingly, whether or not a director has teaching responsibilities does not appear to influence the supervision outcomes statistically-speaking. 34 Table 20. Teacher declaration of director’s supervision frequency (percent) Director Diff. Urban Rural Diff. does not Director Diff. Frequency Niger Public Private (%) Public Public (%) teach teaches (%) Never 30.0 30.9 10.1*** -67.3 31.2 30.7 1.6 28.4 31.6 11.3 Monthly or quarterly 8.0 8.3 1.0*** -88.0 8.6 8.2 4.9 9.2 6.8 -26.1 Monthly 23.6 24.0 14.2** -40.8 23.0 24.6 -6.5 22.1 25.1 13.6 Weekly 13.5 13.7 10.3 -24.8 13.7 13.6 0.7 12.5 14.6 16.8 More than weekly 24.9 23.2 64.4*** 177.6 23.5 22.9 2.6 27.8 21.9 -21.2 Note: comparisons are relative to public, rural public, and a director who does not teach. Levels of significance: *** p<0.01, ** p<0.05, * p<0.1. Supervision These results are confirmed by the content of supervision visits, shown in Table 39. Private schools were 33 percent more likely to have been supervised in 2014/15 (p<0.10) and urban public schools had twice the number of supervision visits as rural public schools (p<0.01). Compared to public schools, supervision in private schools is more likely to use a template (46.4 percent; p<0.01), to observe teaching (80.6 percent; p<0.01), but less likely to meet with the community (-89.1 percent; p<0.01), to check the inventory (-51 percent; p<0.01), or to review the school management committee’s (SMC) latest annual report (-38.3 percent; p<0.10). Among public schools, those in urban areas report higher template usage (43.7 percent; p<0.01) and classroom observation (30.3 percent; p<0.10), but lower rates of community meetings (-75.2 percent; p<0.01), verification of the SMC action plan (- 34.8 percent; p<0.05), reviewing the SMC’s latest annual report (-46.8 percent; p<0.01), or debriefing the teachers (-18.3 percent; p<0.10). The different training levels and constraints are also visible in how directors manage challenges. Four questions were asked, each with four possible answers from which directors had to select the one closest to what they would do. Table 21 shows the results, which suggest that private school directors have more ability to take action and greater resources than their public counterparts. Rural public directors face the greatest challenges, but are firm when dealing with contractual teachers. Private school directors are far less tolerant of school absences than their public school counterparts, whether it be helping the teacher come to school (p<0.10) or tolerating it entirely (p<0.01). They are also 27 percent more likely to request a transfer for a contractual teacher (p<0.10) and will again not tolerate the absence outright (p<0.01). They are also more likely to pay, or to have teachers pay, for chalk or pens that the school requires (p<0.01) than to ask parents (p<0.01). Among public schools, directors in rural areas are more likely to provide support (p<0.01) or to tolerate the absence (p<0.01) of civil servants than their urban counterparts. However, when dealing with school-contracted teachers, rural public schools are far less tolerant than their urban counterparts (p<0.05). Finally, in urban public schools, directors are 81.9 percent more likely to ask teachers to finance basic inputs than in rural areas (p<0.01). 35 Table 21. Responses to common challenges Situation and Diff. Rural Urban Diff. response Niger Public Private (%) Public Public (%) A. Civil servant teacher habitually late on Mondays, because he lives elsewhere Transfer 69,9 70,0 65,6 -6,3 70,9 69,8 -1,6 Accept, find a 5,3 5,0 15,6* 212,0 14,6** 3,0 -79,5 replacement Provide transport 19,1 19,1 18,8 -1,6 14,6 20,1 37,7 assistance Accept the absence 5,7 5,8 0,0** -100,0 0,0*** 7,1 ,, B. Contractual teacher habitually late on Mondays, because he lives elsewhere Transfer 51,9 51,5 65,6* 27,4 54,6 50,9 -6,8 Accept, find a replacement 17,2 17,3 12,5 -27,7 18,2 17,2 -5,5 Provide transport assistance 21,2 21,2 21,9 3,3 23,6 20,7 -12,3 Accept the absence 9,7 9,9 0,0*** -100,0 3,6** 11,2 211,1 C. Supervision visit finds a teacher is not following the curriculum or not teaching properly Retrain the teacher 86,3 86,4 84,4 -2,3 89,1 85,8 -3,7 Demand improvement 12,6 12,5 15,6 24,8 7,3 13,6 86,3 Transfer 0,8 0,8 0,0 -100,0 1,8 0,6 -66,7 Ignore the problem 0,3 0,3 0,0 -100,0 1,8 0,0 -100,0 D. School is missing chalk or pens; how do you respond? Director finances 1,2 0,8 18,8*** 2 250,0 1,8 0,6 -66,7 Teachers finance 9,1 8,6 28,1*** 226,7 1,8*** 10,1 461,1 Administrative request 25,7 25,5 37,5 47,1 25,5 25,4 -0,4 Ask parents to finance 64,0 65,2 15,6*** -76,1 70,9 63,9 -9,9 Notes: comparisons are relative to public and rural public, respectively. Levels of significance: *** p<0.01, ** p<0.05, * p<0.1. No director considered leadership to be a primary constraint. Community engagement Schools exist to serve students and their parents, who are generally organized in communities. Part of the governance mechanism of the education sector is the engagement of the schools with their communities through various planning and oversight bodies. Table 22 presents some elements of community engagement and oversight. Private schools are half as likely to have a school management committee (SMC) than public ones (p<0.01), but are 1.5 times as likely to have an action plan and 1.7 times as likely to have an annual evaluation (p<0.01 for both). Among public schools, urban ones are more likely to have a CGDES (p<0.01) with an action plan (p<0.01), and an annual report (p<0.01). The urban schools are also more 3.8 times likely to have CDGES members who have attended higher education (p<0.01), 1.2 times more likely to have members with primary education only (p<0.10), and have 1.4 times as many female member (p<0.10). They are slightly more likely to have a parent-teachers’ association (p<0.05), but average one less meeting per year (p<0.01). Urban public schools are also more likely to share information about their finances and purchases with their communities (p<0.01). Likewise, SMC members in the 36 private sector are more likely to have greater educational attainment (p<0.05 for primary and p<0.01 for higher education, respectively). Table 22. Community engagement (percent) Diff. Urban Rural Diff. Item Niger Public Private (%) Public Public (%) CGDES in 2014 96,9 98,1 46,9*** -52,2 100,0 97,6*** -2,4 CGDES had an action plan 58,8 58,5 85,7*** 46,5 66,7 56,7*** -15,0 CGDES had an annual report 49,9 49,5 84,6*** 70,9 54,9 48,3*** -12,0 CGDES size 6,2 6,2 6,7 8,1 6,5 6,2 -4,6 CGDES members’ education Primary (number) 1,4 1,4 0,7** -50,0 1,7 1,4* -17,6 Secondary (number) 1,3 1,3 1,9 46,2 1,6 1,2 -25,0 Higher (number) 0,5 0,5 2,4*** 380,0 1,3 0,3*** -76,9 CGDES female members (number) 1,6 1,6 2,1 31,3 2,1 1,5* -28,6 Parent-teachers’ association last year 88,1 88,5 71,9** -18,8 92,7 87,6** -5,5 PTA meetings (N) 3,1 3,2 2,3*** -28,1 2,4 3,3*** 37,5 Student government last year 29,2 29,8 3,1*** -89,6 36,4 28,4*** -22,0 Student government meetings (N) 2,6 2,6 2,0** -23,1 2,4 2,7* 12,5 School shares financials 77,3 78,3 34,4*** -56,1 76,4 78,7*** 3,0 School shares receptions 84,4 85,6 34,4*** -59,8 81,8 86,4*** 5,6 School seeks user feedback 48,2 48,0 56,3 17,3 50,9 47,3 -7,1 Formal feedback to teachers 88,7 88,9 83,3 -6,3 89,3 88,8 -0,6 Feedback had an effect 62,4 61,5 94,4*** 53,5 67,9 60,0*** -11,6 Notes: comparisons within facility type are relative to public and rural public; comparisons across facility types are relative to hospitals. Levels of significance: *** p<0.01, ** p<0.05, * p<0.1. The COSAN meeting frequency is the number of months between meetings. What does this mean for Niger? The 2014-2024 Sector Strategy for Education and Training was designed as a successor to the previous decennial strategy and had three objectives for primary education, namely to: (i) improve access to basic education through increased supply of educational services yielding greater geographic coverage of educational services; (ii) improve quality of teaching; and (iii) improve sub-sector governance. At the same time, beyond improved access, the ambition was to advance as far as possible towards universal completion by 2024, which would require both an increase in access to education and an improvement of the retention of students. The system was, and remains, largely public with 2.35 percent of schools in the sample frame identified as being private. Progress in the previous strategy period was significant, with gross primary enrollment more than doubling to 76 percent (2011) from 36 percent (2001), with corresponding levels of funding for education through the budget for teachers and the construction or rehabilitation of schools. The expansion of the primary education system and significant improvement in enrollment has resulted in higher enrollment of girls in primary education. However, disparities still exist, and in 2011 the ratio of girls-to-boys enrollment in primary education was 83.7% and dropout rates were higher for girls. Gender inequality 37 in access to education is more pronounced in rural areas than in urban areas, and the problem is particularly serious among poor girls living in rural areas. However, the overall level of enrollment was still low compared to other countries in the region, both because of low initial enrollment for certain groups and poor retention for all students. Niger ranks last both at entry and exit from primary among the countries in the latest (2014) round of the Programme d’Analyse des Systèmes Educatifs de la CONFEMEN (PASEC). This suggests that beyond increasing resources and enrollment, there are quality dimensions that need to be addressed. Among the dimensions of quality are access to materials, such as textbooks and teacher guides, which are low in Niger. Other factors include the pupil-teacher ratios, which at 39 in 2011 was below the Global Partnership for Education norm of 40. However, there is variation across classrooms and the 2014/15 data show an average fourth grade size of 48 pupils., but ranging from 34.7 in Zinder to 81 in Niamey. Teachers are an important factor and the 2010 Education Sector Report suggested that teacher absenteeism costs students more than 18 days of schooling per year. 11 The heavy use of contractual teachers may not be a cost-saving measure if they still generate large budgetary pressures. Cost containment could lead to a strategy of lower-cost teachers, which is likely to have lowered the quality of teachers (Vargas and De Laat, 2003). The results of these and other factors is that educational system in Niger shows clear divisions in the dimensions of provider ownership (public/private) and location (rural/urban). For inputs, private is superior to public in both infrastructure availability (81 percent versus 20 percent, p<0.01) and teaching equipment availability (72 percent versus 23 percent; p<0.01) while urban public is better than rural public for both infrastructure (28 percent and 18 percent, respectively) and equipment (39 versus 20 percent, p<0.01). Although Niger is still primarily rural (83.7 percent; 2012 census), there is greater pressure in urban public classrooms, where the pupil-teacher ratio is 12 pupils higher than in rural classrooms (p<0.01). However, private school pupils have nearly nine more classmates than their public peers (p<0.01). These factors are compounded by generally weak levels of minimum knowledge; less than one percent of teachers in Niger score 80 percent on an evaluation of their math and French knowledge at the lower primary level. There is no difference between public and private school teachers. There are clear differences in absence rates, no matter how they are defined, primarily between public and private school teachers. Table 23 shows that access to education is correlated to level of welfare; as parents are able, they send their children to private schools. The survey does to allow for the calculation of intra-generational effects, but looking at the relative performance of pupils by school attendance, it is likely that those attending private schools will have higher levels of human capital in the future and be better-equipped for life. 11 Rapport d’Etat du Système Educatif Nigérien (2010). 38 Table 23. School attendance by welfare quintile (percent) Poorest Second Third Fourth Richest Total Public 80.8 69.4 65.4 39.8 26.5 42.0 Private 19.2 30.1 34.4 60.2 73.5 58.0 Other 0.4 0.1 0.0 0.1 Note: survey-weighted estimates from the 2014 poverty survey. In the end, the education production function is designed to equip pupils with knowledge and skills to either continue their education or to join the workforce. The situation depicted in this report suggests that the overall weak levels and high inequality of pupil outcomes remain the ultimate challenge for the system. Table 24 shows that the observed differences between rural and urban remain even after removing private school pupils, whose performance is much higher than their public school peers. The differences decrease by approximately five percent in the average scores, but the same patterns remain, suggesting that there are effectively three school systems in Niger. There is a very small private sector, an urban public sector, and finally a rural sector. The three have very different performances and inputs. Table 24. Pupil evaluation: gender and location breakdowns for grade 4 public (percent) Estimates for public school pupils Diff. (%) Urban Rural Urban Rural Rural Urban Rural Module Niger boys boys girls girls boys girls girls Overall 21,3 35,7 18,8*** 29,5 16,9*** -47.3 -17.4 -52.7 French 21,7 38,5 18,9*** 31,2 16,7*** -50.9 -19.0 -56.6 Mathematics 11,5 16,1 10,6*** 14,0 10,2*** -34.2 -13.0 -36.6 Non-verbal reasoning 52,7 55,1 51,6 58,6 51,4 -6.4 6.4 -6.7 Note: weighted means using sampling weight and the sample design. Results based on 1,651 pupils in 256 schools. Differences are relative to urban boys in public school. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Comparing Niger with other countries that have done SDI Table 3, above, compares Niger with other countries for all schools and Table 4 compares for the public sector. In terms of teacher effort, Niger has a relatively lower absence rate than the average of the other countries. This, combined with teacher behavior in class, means that Niger ranks first on instruction time per day. However, teacher knowledge is particularly low; it is the greatest differential among all the indicators. Schools in Niger have the fewest teachers who reach the minimum knowledge threshold, low absence rates (fourth-best), the second-lowest classroom absence rate (behind Nigeria), the best teaching time per day, the second-lowest textbook availability (above Uganda), and the lowest availability of infrastructure and teaching equipment. As noted above, generally, students in Niger have among of the worst learning outcomes of the SDI surveys. 39 Annex 1. Sampling The sample frame was provided by the ministry and comprised all schools that were expected to be functional for the 2015/16 school year (see Table 25). Given the nature of instruction, all Koranic schools were excluded from the sample frame (the écoles franco-arabes). This left 7,232 primary schools. The region of Diffa, which was in a state of emergency, was also excluded from the sample; this removed 1 percent of the student population from the frame as shown in Table 25. The population of interest was defined as the universe of fourth graders enrolled in school. Only formally-recognized primary schools are considered, excluding community schools and special-needs schools (e.g. handicapped). Based upon the 2014/15 school year, the sample was drawn for rising fourth graders. For sampling purposes, the frame contained the size of each grade in each school, the administrative geography, and the physical geography (rural/urban). The sample frame was divided into three strata: public rural, public urban, and private urban to maximize within-group homogeneity. Eight private rural schools were excluded. Table 25. Distribution of fourth grade students by region, curriculum, and location (percent) Agadez Diffa Dosso Maradi Niamey Tahoua Tillabéri Zinder Niger Curriculum U R U R U R U R U U R U R U R U R T Traditional 1 1 0 1 1 7 2 9 6 2 7 1 8 2 38 16 70 86 Experimental 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 Medersa 0 0 0 0 0 1 1 1 1 0 1 0 1 0 5 3 9 13 Specialized 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Total 1 1 0 1 2 7 3 10 7 3 8 1 9 2 44 19 81 100 Notes: calculations based upon the 2014/15 school year for then-third graders (rising fourth graders for the 2015/16 school year). The “U” refers to urban, the “R” to rural, and the “T” to “total” populations. Depending upon the concepts of interest, the sampling procedure is done in two (absence rate, classroom observation) or three stages (pupils). In all cases, the first stage is the drawing of schools independently and without replacement from the different strata. To reflect the experience of the average student, a simple random sample was drawn in the strata, with probability proportional to the size of the school, defined as the number of third graders in 2014/5. This selected the school. Subsamples were drawn in the schools as described below. To calculate the absence rate, a secondary sampling frame of all teachers who work at the school is prepared (Module 2A). The methodology requires ten teachers, or all those in the school if it has less than ten teachers. If a school has more than ten teachers, a random selection without replacement is undertaken where each teacher has equal probability of being selected. This gives the inflation factor, or weight, for the teacher absence rate, defined as the inverse of the product of the probability of selecting the school and the probability of selecting a given teacher in the school. For classroom observation, the secondary sampling frame is the number of fourth-grade 40 classrooms in use in the school (Module 1). A simple random selection with equal probability of selection was undertaken if there was more than one relevant classroom in the school, giving a probability weight for classrooms. This secondary sampling unit was used for the selection of pupils, the third-stage sampling unit. From the classroom selected for observation, the procedure is to draw up to ten pupils for the evaluation of their learning outcomes (Module 5). The sample frame for the selection of students for this was the teacher’s pupil roster updated to remove pupils not present on the day of the first visit. A random selection without replacement and with equal probability of selection was undertaken if there were more than ten pupils in the classroom. A risk of pupils not returning after lunch was identified during pre-testing and mitigated during piloting through two steps. The field teams were instructed to learn from the teacher which sampled pupils might not return after lunch in order to minimize the risk of truncating the lower tail of the performance distribution. In the same sample procedure, the teams also drew three extra pupils that were kept in reserve in case one of the first ten pupils was not available. A probability weight is calculated for these pupils as the inverse of the product of the probabilities of selecting the school, the classroom, and the student and is used to estimate pupil-related indicators. The sampling strategy was designed to produce estimates that had 5 percent confidence intervals for national, rural, and urban with a minimum power of 80 percent and an approximate sample size of 250 facilities given budget constraints. The two-stage approach retained was expected to give allow the differentiation of absenteeism estimates differ by 3.04 percentage points at the national level, 3.32 percentage points at the rural level, 5.67 percentage points at the urban level, and 8.59 percentage points at the regional level. Given the very small size of the private sector, the confidence interval was expected to be larger. Grading of teacher evaluations was done by the Laboratoire d'études et de recherches sur l'émergence économique of the University Abdou Moumouni in Niamey. To ensure inter- rater reliability, 15 percent of the sample was doubled-graded blindly. These were monitored for quality using both pair-wise correlations and Cohen’s Kappa as measures of reliability. Initial correlations were above 95 percent on the language and mathematics sections and the Kappa scores were all significant. Problems identified by across graders were corrected and used to identify broader potential problems that were also corrected. 41 Annex 2. Definition of the Service Delivery Indicators in Education School absence rate Share of a maximum During the first announced visit, a maximum of ten teachers are randomly selected of ten randomly from the list of all teachers who are on the school roster. The whereabouts of these selected teachers ten teachers are then verified in the second, unannounced visit. Teachers found absent from school anywhere on the school premises are marked as present. during an unannounced visit Classroom absence rate Share of teachers The indicator is constructed in the same way as the school absence rate indicator, who are present in with the exception that now the numerator is the number of teachers who are either the classroom absent from school, or present at school but absent from the classroom. A small during scheduled number of teachers may be found teaching outside, and these are marked as present teaching hours as for the purposes of the indicator. observed during an unannounced visit Time spent teaching per day (also known as Time on Task) Amount of time a This indicator combines data from the Staff Roster Module (used to measure absence teacher spends rate), the Classroom Observation Module, and reported teaching hours. The teaching teaching during a time is adjusted for the time teachers are absent from the classroom, on average, and school day for the time the teacher teaches while in classrooms based on classroom observations While inside the classroom distinction is made between teaching and non-teaching activities. Teaching is defined very broadly, including actively interacting with pupils, correcting or grading pupil's work, asking questions, testing, using the black board, or having pupils working on a specific task, drilling, or memorization. Non-teaching activities is defined as work that is not related to teaching, including working on private matters, maintaining discipline in class, or doing nothing and thus leaving pupils not paying attention. Minimum knowledge among teachers Share of teachers This indicator measures teacher's knowledge and is based on mathematics and with minimum language tests covering the primary curriculum administered at the school level to knowledge all mathematics or language teachers that taught grade three in the previous year or grade four in the year the survey was conducted. It is calculated as the percentage of teachers who score more than 80 percent on the language and mathematics portion of the test. The indicator is representative of the average teacher in the universe of teachers in a given country rather than the average teacher at the average school. Test score: This indicator measures teacher’s knowledge and it is calculated as the overall score of a mathematics, language, and pedagogy tests covering the primary curriculum administered at the school level to all mathematics and language teachers that taught grade three in the previous year or grade four in the year the survey was conducted. Infrastructure availability Unweighted average Minimum infrastructure resources is a binary variable capturing availability of: (i) of the proportion of functioning toilets operationalized as being clean, private, and accessible; and (ii) schools with the sufficient light to read the blackboard from the back of the classroom. following available: functioning Functioning toilets: Whether the toilets were functioning was verified by the electricity and enumerators as being accessible, clean, and private (enclosed and with gender sanitation separation). 42 Electricity: Functional availability of electricity is assessed by checking whether the light in the classroom works and gives minimum light quality. The enumerator places a printout on the board and checks (assisted by a mobile light meter) whether it was possible to read the printout from the back of the classroom. Teaching Equipment availability Unweighted average Equipment availability is a binary variable equal to one if (i) the randomly selected of the proportion of grade four classroom has a functioning blackboard and chalk, (ii) the share of pupils schools with the with pens is equal to or above 90 percent, and (iii) the share of pupils with following available: notebooks in that classroom is equal to or above 90 percent. functioning blackboard with Functioning blackboard and chalk: The enumerator assesses if there was a chalk, pencils, and functioning blackboard in the classroom, measured as whether a text written on the notebooks blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. Pencils/pens and notebooks: The enumerator counts the number of pupils with pencils or pens and notebooks, respectively. By dividing each count by the number of pupils in the classroom, one can then estimate the share of pupils with pencils or pens and the share of pupils with notebooks. Share of pupils with textbooks Number of The indicator measures in one randomly selected grade four class the number of mathematics and pupils with the relevant textbooks (mathematic or language conditional on which language books used randomly selected class is observed), and divided by the number of pupils in the in a grade four classroom. classroom divided by the number of pupils present in the classroom Pupil- teacher ratio Average number of The indicator of teachers’ availability is measured as the number of pupils per grade four pupils teacher in one randomly selected grade four class at the school based on the per grade four classroom observation module. teacher 43 Annex 3. Additional Results A. School breakdowns Figure 11. Education infrastructure co-availability, all schools Note: Ellipses are drawn for the case where the sub-indicator is available to all. The numbers in parentheses are the population percentage of schools with a score of 100% for the given sub-indicator. Data are weighted to account for the survey design. 44 Figure 12. Education infrastructure co-availability, public schools Note: Ellipses are drawn for the case where the sub-indicator is available to all. The numbers in parentheses are the population percentage of schools with a score of 100% for the given sub-indicator. Data are weighted to account for the survey design. 45 Figure 13. Education infrastructure co-availability, private schools Note: Ellipses are drawn for the case where the sub-indicator is available to all. The numbers in parentheses are the population percentage of schools with a score of 100% for the given sub-indicator. Data are weighted to account for the survey design. Unless otherwise stated, all tables are weighted. 46 Table 26. School input indicators, detailed Indicator (% unless otherwise Diff. Urban Rural Diff. Diff. noted) Niger Public Private (%) Public Public (%) Rural Urban (%) Minimum teaching equipment 24,7 23,4 72,4*** 209,4 38,9*** 20,1*** 93,5 20,1 43,4*** 115,9 Pupils with pencils 91,1 90,9 97,6*** 7,4 97,7*** 89,5*** 9,2 89,5 97,7*** 9,2 Pupils with exercise books 45,5 44,4 89,8*** 102,3 69,8*** 39,0*** 79,0 39,0 72,3*** 85,4 Classroom with board 100,0 100,0 100,0 0,0 100,0 100,0 0,0 100,0 100,0 0,0 Classroom with chalk 95,8 95,8 97,2 1,5 98,2 95,2 3,2 95,2 98,0 2,9 Contrast to read the board 32,1 34,1 28,0 -17,9 10,5 38,8*** -72,9 37,8 17,7** -53,2 Minimum infrastructure 21,3 19,7 80,7*** 309,6 28,3 17,9 58,1 17,9 35,5*** 98,3 Minimum visibility by enumerator 86,4 86,2 94,5* 9,6 100,0*** 83,3*** 20,0 83,3 99,2*** 19,1 Toilet functioning and available 24,3 22,6 83,5*** 269,5 28,3 21,4 32,2 21,4 35,9** 67,8 Toilet clean 42,7 41,1 100,0*** 143,3 70,4*** 34,9*** 101,7 34,9 74,4*** 113,2 Toilet private 27,7 26,2 83,5*** 218,7 35,2 24,3 44,9 24,3 41,7*** 71,6 Toilet available 42,2 40,8 94,5*** 131,6 74,1*** 33,7*** 119,9 33,7 76,8*** 127,9 Observed pupil-teacher ratio 38,3 38,1 46,9*** 23,1 48,2*** 36,0*** 33,9 36,0 48,0*** 33,3 Textbook availability 8,9 8,4 24,0** 185,7 8,6 7,7 11,7 7,7 11,9 54,5 Pupils with math textbook 7,6 6,9 24,3* 252,2 8,1 4,6 76,1 4,6 13,8* 200,0 Pupils with French textbook 9,8 9,2 28,0* 204,3 8,9 9,3 -4,3 9,3 11,0 18,3 47 B. Individual breakdowns Table 27. Teacher absence rates, by status (percent) Average rates Diff. relative to private sector Private Indicator Niger sector Civil Servant Contractual Civil servant Contractual Absent from school 16,6 1,4 12,1*** 19,3*** 764,3 1278,6 Absent from class, at school 10,5 9,7 16,6** 8,0 71,1 -17,5 Absent from class 27,1 28,7 27,2*** 11,1*** -5,2 -61,3 Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Table 28. Teacher absence rates, by region (percent) Indicator National Agadez Dosso Maradi Tahoua Tillabéri Zinder Niamey Absent from school (1) 16.6 10.6 12.1 14.6 13.3 23.4 17.0 18.9 Absent from class, at school (2) 12.5 8.9* 10.7 14.7 8.1* 11.2 9.8 19.6 Absent from class (3) 27.0 18.6* 21.6 27.1 20.3* 32.0 25.2 34.8 Observations (1 and 3) 1.741 37 194 334 233 266 264 413 Observations (2) 1,472 34 172 285 202 196 223 360 Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Differences are relative to Niamey. There are fewer observations from “absent from class, at school” since teachers who did not come to school are not counted. 48 Table 29. Teacher evaluation breakdowns Panel A: Teacher evaluation: ownership, urban-rural within public, and rural-urban breakdowns (percent) Diff. Rural Urban Diff. Diff. Niger Public Private (%) public public (%) Rural Urban (%) Average score 35,4 35,3 39,3*** 11,3 34,2 37,3*** 9,1 34,2 37,5*** 9,6 Minimum knowledge 0,4 0,4 0,3 -25,0 0,5 0,0** -100,0 0,5 0,0* -100,0 French 42,7 42,5 45,9* 8,0 41,2 45,0*** 9,2 41,2 45,1*** 9,5 Grammar 49,6 49,7 48,0 -3,4 49,3 50,3 2,0 49,3 50,1 1,6 Cloze task 70,2 70,0 76,1*** 8,7 68,5 72,8*** 6,3 68,5 73,2*** 6,9 Composition 34,1 34,4 25,1*** -27,0 35,2 32,9 -6,5 35,2 32,1 -8,8 Math 14,2 14,0 18,4** 31,4 12,4 16,9*** 36,3 12,4 17,0*** 37,1 Basic math 28,2 27,9 37,3** 33,7 25,8 31,7*** 22,9 25,8 32,3*** 25,2 Advanced math 35,7 35,4 43,7** 23,4 33,1 39,7*** 19,9 33,1 40,1*** 21,1 Fractions 14,2 13,7 25,1*** 83,2 12,1 16,8** 38,8 12,1 17,7*** 46,3 Venn diagrams 10,7 10,2 22,7*** 122,5 9,6 11,5 19,8 9,6 12,7* 32,3 Graphs 15,9 15,5 26,6** 71,6 13,5 19,3* 43,0 13,5 20,1** 48,9 Pedagogy 7,1 6,7 16,3*** 143,3 5,4 9,2** 70,4 5,4 9,9*** 83,3 Basic pedagogy 21,0 20,8 26,1*** 25,5 20,3 21,8 7,4 20,3 22,3* 9,9 Advanced pedagogy 17,8 17,6 21,9*** 24,4 17,2 18,4 7,0 17,2 18,8 9,3 Lesson preparation 26,6 26,3 33,3*** 26,6 25,6 27,7 8,2 25,6 28,3** 10,5 Pupil comparisons 26,8 26,6 31,0*** 16,5 25,9 28,1 8,5 25,9 28,4* 9,7 Pupil evaluations 0,4 0,4 0,6*** 50,0 0,4 0,4 0,0 0,4 0,5 25,0 Teachers (N) 1 604 1 234 280 817 507 817 787 Note: all teachers evaluated are included in these results, which are also shown in Tables 10-12 as subject-specific tables. Bold-faced items are the major indicators and the primary breakdowns. Items not in boldface are further details of their bold-faced headers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Comparisons within schools are relative to public, rural public, and rural. 49 Panel B. Teacher evaluation: contractual status breakdown (percent) Teacher types Diff. (%) Private Private Niger Civil servant Contractual school Contractual school Average score 42.7 49.9 38.8*** 46.9 -21,0 -7,9 Minimum knowledge 0.4 1.1 0.0** 0.3 , , French 49.6 54.0 47.5*** 48.2*** -13,3 -11,2 Grammar 70.2 76.9 66.6*** 76.1 -12,8 -3,1 Cloze task 34.1 36.2 33.5* 25.4*** -12,9 -28,7 Composition 14.2 20.7 10.6*** 18.6 -44,8 9,3 Math 28.2 37.5 23.0*** 39.0 -32,0 0,8 Basic math 35.7 45.8 30.1*** 45.6 -30,0 -3,8 Advanced math 14.2 21.8 9.6*** 26.7 -41,4 21,5 Fractions 10.7 16.6 7.0*** 24.1 -54,9 5,9 Venn diagram 15.9 22.0 12.2*** 27.9 -24,0 20,4 Graphs 7.1 12.5 3.8*** 17.4 -65,8 25,0 Pedagogy 21.0 29.7 16.3*** 26.3** -46,3 -17,1 Basic pedagogy 17.8 26.7 13.0*** 22.2** -51,7 -23,2 Advanced pedagogy 26.6 35.0 22.0*** 33.4 -39,9 -9,2 Lesson preparation 26.8 34.6 22.7*** 31.1** -35,8 -11,8 Pupil comparisons 0.4 0.7 0.3*** 0.6 -57,1 -28,6 Pupil evaluations 5.4 8.9 3.5*** 7.2 -50,7 -40,6 Teachers (N) 1,604 447 895 261 Note: all teachers evaluated are included in these results, which are also shown in Tables 10-12 as subject-specific tables. Items not in boldface provide details for bold- faced headers. Differences are relative to civil servants. A community teacher is excluded from the teacher type breakdowns. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 50 Panel C. Teacher evaluations: academic training breakdowns (percent) Diff. (%) High At least Baccal- Lower school some Lower High school At least some Niger aureate secondary incomplete university secondary incomplete university Average score 42.7 49.7 39.6*** 46.2** 51.6 -20.3 -7.1 3.8 Minimum knowledge 0.4 0.0 0.2 0.8 1.7 . . . French 49.6 54.5 47.1*** 50.6** 52.6 -13.6 -7.1 -3.5 Grammar 70.2 80.8 68.0*** 72.4*** 79.3 -15.8 -10.4 -1.8 Cloze task 34.1 33.5 31.3 34.0 30.8 -6.7 1.4 -7.9 Composition 14.2 22.2 11.7*** 15.9*** 24.0 -47.3 -28.2 8.4 Math 28.2 38.3 24.7*** 33.8 43.3 -35.6 -11.7 13.1 Basic math 35.7 44.9 32.2*** 41.8 50.7 -28.4 -7.1 12.8 Advanced math 14.2 25.9 10.6*** 18.9* 29.5 -59.1 -26.8 14.0 Fractions 10.7 16.0 8.2** 14.0 20.6 -48.7 -12.0 28.8 Venn diagram 15.9 23.7 12.4** 22.6 29.4 -47.6 -4.8 23.9 Graphs 7.1 16.5 4.2*** 11.2 18.5 -74.6 -32.3 12.2 Pedagogy 21.0 29.7 17.8*** 24.1*** 32.9 -40.1 -18.8 10.7 Basic pedagogy 17.8 26.2 14.7*** 20.8** 30.2 -44.1 -20.7 15.0 Advanced pedagogy 26.6 35.8 23.2*** 29.9*** 37.6 -35.0 -16.3 5.1 Lesson preparation 26.8 35.9 23.9*** 29.7*** 38.1 -33.4 -17.4 6.0 Pupil comparisons 0.4 0.6 0.3*** 0.5 0.8 -45.2 -17.7 21.0 Pupil evaluations 5.4 10.5 4.0*** 6.2*** 8.5 -62.3 -41.3 -18.9 Teachers (N) 1,604 106 735 345 97 Note: all teachers evaluated are included in these results, which are also shown in Tables 10-12 as subject-specific tables. Items not in boldface provide details for bold- faced headers. Differences are relative to Baccalaureate. Teachers in “lower secondary” completed primary and went up to and including the BEPC. There are 290 teachers with no educational attainment specified. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 51 Panel D. Teacher evaluations: grade taught breakdowns (percent) Average for the country or the grade taught Diff. (%) Niger Fourth First Second Third Fifth Sixth First Second Third Fifth Sixth Average score 42.7 41.2 39.0 36.5*** 38.1** 42.9 49.9*** -5,3 -11,4 -7,5 4,1 21,1 Minimum knowledge 0.4 0.0 0.0 0.0 0.0 0.0 0.5 , , , , , French 49.6 47.5 45.8 44.9 46.5 48.9 54.0*** -3,6 -5,5 -2,1 2,9 13,7 Grammar 70.2 69.8 65.1** 65.9** 66.3* 70.5 78.0*** -6,7 -5,6 -5,0 1,0 11,7 Cloze task 34.1 30.4 31.4 29.0 31.9 32.4 35.3*** 3,3 -4,6 4,9 6,6 16,1 Composition 14.2 12.7 11.0 11.1 9.9** 13.9 19.6*** -13,4 -12,6 -22,0 9,4 54,3 Math 28.2 27.4 24.9 20.8*** 21.9*** 29.6 37.7*** -9,1 -24,1 -20,1 8,0 37,6 Basic math 35.7 35.0 32.2 28.0*** 29.7** 37.0 45.8*** -8,0 -20,0 -15,1 5,7 30,9 Advanced math 14.2 13.2 11.2 7.4*** 7.4*** 15.8 22.5*** -15,2 -43,9 -43,9 19,7 70,5 Fractions 10.7 8.8 9.0 6.6 6.4 8.9 17.8*** 2,3 -25,0 -27,3 1,1 102,3 Venn diagram 15.9 16.2 15.1 10.0* 7.0*** 19.2 22.7** -6,8 -38,3 -56,8 18,5 40,1 Graphs 7.1 6.9 5.6 3.2* 2.4** 6.3 13.8*** -18,8 -53,6 -65,2 -8,7 100,0 Pedagogy 21.0 19.5 15.7*** 15.6*** 15.2*** 20.9 30.0*** -19,5 -20,0 -22,1 7,2 53,8 Basic pedagogy 17.8 16.4 12.8** 11.5*** 11.6*** 17.5 27.1*** -22,0 -29,9 -29,3 6,7 65,2 Advanced pedagogy 26.6 24.8 20.7** 22.6 21.6* 26.9 35.1*** -16,5 -8,9 -12,9 8,5 41,5 Lesson preparation 26.8 24.9 21.0** 23.7 22.7 27.8* 34.4*** -15,7 -4,8 -8,8 11,6 38,2 Pupil comparisons 0.4 0.4 0.3** 0.2*** 0.2*** 0.4 0.7*** -25,0 -50,0 -50,0 0,0 75,0 Pupil evaluations 5.4 4.1 4.6 3.0 3.5 3.5 8.7*** 12,2 -26,8 -14,6 -14,6 112,2 Teachers (N) 1,604 265 152 165 171 209 258 Note: all teachers evaluated are shown in these results; some individuals evaluated were directors, who do not teach and others did not report the grade which they teach. Bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold-faced headers. Differences are relative to Grade 4 teachers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. For 384 individuals, the class taught was not available. 52 Panel E. Teacher evaluations: teacher training breakdowns (percent) Average score for the breakdown Diff. relative to ENII (%) Niger ENII None CAM CEAP ENIIA CAP None CAM CEAP ENIIA CAP Average score 42.7 49.2 37.0*** 42.5** 40.0*** 41.9*** 52.3 -25.9 -12.7 -17.8 -14.9 5.7 Minimum knowledge 0.4 0.0 0.0 0.0 0.0 0.0 4.6** . . . . . French 49.6 51.1 44.6*** 51.8 47.1 48.5 55.1** -16.0 -0.8 -10.0 -7.0 5.1 Grammar 70.2 76.2 65.5*** 71.1* 70.3** 69.6*** 81.4*** -14.8 -7.9 -9.6 -9.2 5.8 Cloze task 34.1 31.0 28.9 37.5 29.0 32.6 34.1 -16.2 14.7 -9.5 -0.9 3.5 Composition 14.2 21.1 8.1*** 15.5 13.5*** 13.3*** 24.5 -59.4 -22.7 -29.5 -35.3 17.9 Math 28.2 39.1 22.3*** 25.1*** 26.0*** 27.8*** 42.4 -42.6 -32.9 -32.1 -27.9 9.7 Basic math 35.7 47.3 29.4*** 33.2*** 33.0*** 35.3*** 49.4 -37.0 -26.5 -27.0 -24.1 6.3 Advanced math 14.2 23.6 9.1*** 10.1*** 12.9** 13.6*** 29.2 -64.0 -56.1 -51.8 -42.1 22.8 Fractions 10.7 17.1 7.5** 7.1** 4.4*** 9.0** 24.0 -59.3 -41.9 -24.6 -46.1 41.3 Venn diagram 15.9 21.9 9.5*** 13.0 14.0 16.2 32.9** -59.0 -43.9 -37.7 -25.5 42.0 Graphs 7.1 15.6 3.3*** 2.7*** 1.6*** 6.7*** 16.9 -83.3 -82.0 -92.0 -58.0 8.0 Pedagogy 21.0 31.0 13.2*** 18.9*** 17.9*** 20.3*** 34.7* -55.2 -34.3 -36.4 -32.0 16.2 Basic pedagogy 17.8 27.4 10.1*** 17.6** 13.8*** 16.7*** 33.1* -60.7 -29.0 -45.0 -35.9 24.8 Advanced pedagogy 26.6 37.2 18.5*** 21.1*** 25.0*** 26.4*** 37.5 -48.6 -41.3 -25.7 -26.8 5.3 Lesson preparation 26.8 37.2 19.4*** 22.4*** 25.7*** 26.5*** 38.2 -46.7 -38.6 -26.9 -26.4 5.3 Pupil comparisons 0.4 0.7 0.2*** 0.4* 0.3*** 0.4*** 0.8** -66.7 -16.7 -50.0 -33.3 33.3 Pupil evaluations 5.4 10.0 3.1*** 2.2*** 6.1 4.6*** 10.2 -68.0 -80.4 -18.6 -51.5 5.2 Teachers (N) 1,604 111 216 28 57 755 123 Note: all teachers evaluated are included in these results. bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold- faced headers. Differences are relative to teachers with an ENII. Abbreviations are “None” for no teacher training diploma, “CAM” for a monitor’s degree (does not qualify to teach), “CEAP” for the lower-level teacher certification, “CAP” for the higher-level certification, “ENIIA” for the lower certification obtained at a teacher training college and “ENII” for the higher certification obtained at a teacher training college. Excluded from the breakdowns are teachers whose level is “other” (78) or with missing information (246) although they are in the “Niger” aggregate. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 53 Panel F. Teacher evaluations: teacher training college degree breakdowns (percent) Niger Attended Did not attend Diff. (%) Average score 42.7 43.0 43.3 0.7 Minimum knowledge 0.4 0.0 1.5** . French 49.6 49.7 49.4 -0.6 Grammar 70.2 70.2 71.6 2.0 Cloze task 34.1 34.3 32.4 -5.5 Composition 14.2 14.2 15.4 8.5 Math 28.2 28.6 29.7 3.8 Basic math 35.7 36.2 37.1 2.5 Advanced math 14.2 14.4 15.9 10.4 Fractions 10.7 9.9 13.8** 39.4 Venn diagram 15.9 16.4 17.2 4.9 Graphs 7.1 7.3 7.3 0.0 Pedagogy 21.0 21.3 22.1 3.8 Basic 17.8 17.9 19.4 8.4 Advanced 26.6 27.3 26.6 -2.6 Lesson preparation 26.8 27.6 27.0 -2.2 Pupil comparisons 0.4 0.4 0.5 25.0 Pupil evaluations 5.4 5.3 6.2 17.0 Teachers (N) 1,604 1,039 424 Note: all teachers evaluated are included in these results. Bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold-faced headers. Differences are relative to teachers who attended teacher training college irrespective of the degree achieved. For 141 teachers, information on attendance is not available (no answer; 69 teachers) or unclear (“other diploma”; 72). Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 54 Panel G. Teacher evaluations: breakdowns by career start year (percent) Diff. 2008 or Diff. Niger Pre-2000 2000 or later (%) Pre-2008 later (%) Average score 42.7 53.3 41.7*** -21,8 47.5 39.9*** -16,0 Minimum knowledge 0.4 2.2 0.2 -90,9 51.6 48.4*** -100,0 French 49.6 55.9 49.0*** -12,3 75.0 67.5*** -6,2 Grammar 70.2 81.4 69.2*** -15,0 33.4 34.4 -10,0 Cloze task 34.1 35.4 33.9 -4,2 18.9 11.4*** 3,0 Composition 14.2 26.3 13.1*** -50,2 35.4 24.1*** -39,7 Math 28.2 43.1 26.9*** -37,6 43.4 31.3*** -31,9 Basic math 35.7 50.7 34.4*** -32,1 20.4 10.6*** -27,9 Advanced math 14.2 28.9 12.9*** -55,4 15.4 8.0*** -48,0 Fractions 10.7 22.1 9.7*** -56,1 21.5 12.8*** -48,1 Venn diagram 15.9 30.1 14.7*** -51,2 11.3 4.7*** -40,5 Graphs 7.1 13.9 6.5*** -53,2 27.1 17.5*** -58,4 Pedagogy 21.0 36.3 19.7*** -45,7 24.0 14.2*** -35,4 Basic 17.8 34.7 16.3*** -53,0 32.4 23.3*** -40,8 Advanced 26.6 39.1 25.5*** -34,8 32.8 23.4*** -28,1 Lesson preparation 26.8 38.9 25.8*** -33,7 0.6 0.3*** -28,7 Pupil comparisons 0.4 0.9 0.4*** -55,6 6.9 4.5*** -50,0 Pupil evaluations 5.4 12.6 4.7*** -62,7 47.5 39.9*** -34,8 Teachers (N) 1,604 161 1,443 609 995 Note: all teachers evaluated are included in these results. Bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold- faced headers. Differences are relative to teachers who started after 1999 or after 2007, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 55 Panel H. Teacher evaluations: breakdowns by position (percent) Average score for the breakdown Diff. relative to teacher (%) Director- Director- Niger Teacher Director teacher Substitute Director teacher Substitute Average score 42.7 41.4 56.5*** 49.5*** 37.6* 36.6 19.6 -9.2 Minimum knowledge 0.4 0.1 6.3* 0.0 0.0 5,654.5 -100.0 -100.0 French 49.6 48.9 56.7*** 53.7*** 46.9 16.0 10.0 -4.0 Grammar 70.2 69.2 80.9*** 75.1*** 67.1 16.9 8.5 -3.1 Cloze task 34.1 33.6 37.4 37.3** 31.8 11.3 11.0 -5.5 Composition 14.2 12.8 26.5*** 20.5*** 11.7 107.3 60.7 -8.2 Math 28.2 26.4 49.3*** 36.7*** 22.4 86.7 38.9 -15.3 Basic math 35.7 33.9 56.4*** 45.3*** 28.3** 66.2 33.5 -16.6 Advanced math 14.2 12.3 36.0*** 20.6*** 11.3 192.1 67.2 -8.6 Fractions 10.7 9.0 30.1*** 17.3*** 8.2 236.3 93.7 -8.5 Venn diagram 15.9 14.5 36.0*** 23.1*** 8.6 148.2 58.9 -40.9 Graphs 7.1 5.9 22.8*** 10.0** 6.0 284.8 68.2 2.0 Pedagogy 21.0 18.7 38.7*** 31.4*** 18.9 106.9 67.6 1.1 Basic pedagogy 17.8 15.2 37.0*** 29.0*** 15.8 143.0 90.6 3.9 Advanced pedagogy 26.6 24.7 41.6*** 35.4*** 24.3 68.4 43.0 -1.9 Lesson preparation 26.8 24.9 41.7*** 35.7*** 25.0 67.7 43.6 0.3 Pupil comparisons 0.4 0.4 0.9*** 0.7*** 0.4 150.0 102.8 0.0 Pupil evaluations 5.4 4.5 15.8*** 7.7*** 4.9 250.6 69.6 7.8 Teachers (N) 1,604 1,269 73 164 98 Note: all teachers evaluated are included in these results. Bold-faced items are major indicators and primary breakdowns. Items not in boldface provide details for bold- faced headers. The different categories are “teacher” for a regular teacher, “director” for a school head who does not teach, “director-teacher” for a school head who also teaches, and “substitute” for a substitute teacher. Differences are relative to normal teachers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 56 Figure 14. Teacher evaluation performance progression by year of hire Note: weighted data. The term “CI” refers to the confidence interval about the mean and the term “linear” refers to a regression model with year dummies. 57 Table 30. Teacher characteristics (absenteeism sample) Diff. Niger Public Private (%) Female (d) 64.8 65.6 46.8*** -28.7 Pay delays (s) 0.0 0.8 -19.4*** -2525.0 Head (d) 4.5 4.3 9.1*** 111.6 Head absent (d) 3.5 2.9 16.6* 472.4 Peers with CAP (s) 1.3 0.7 15.9*** 2171.4 Peers attended ENI (s) 2.6 2.7 0.3 -88.9 Poverty headcount (%) 38.0 39.2 10.4*** -73.5 Rural (d) 62.5 65.1 0.0*** -100.0 Public (d) 96.0 100.0 0.0 -100.0 Teaching resources available (d) 29.4 27.4 77.1*** 181.4 Used a template (d) 57.8 57.2 71.0 24.1 Duration (s) 0.1 0.1 -1.7 -1800.0 School shares finances (d) 70.2 70.3 67.4 -4.1 Met with staff (d) 54.2 52.7 87.2*** 65.5 Observed class (d) 79.0 80.9 34.6*** -57.2 School seeks user feedback (d) 52.6 52.4 56.0 6.9 CGDES parental contributions (s) 6.4 6.8 -1.7* -125.0 Note: weighted means using sampling weights for teachers. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Differences are relative to public schools. Variables with “(s)” are standardized and those with “(d)” are binary (dummy) variables. Definition of variables • Pay delays: number of months in the past year without being paid on time. • Peers with CAP: the CAP is the highest pedagogical certificate for primary school teachers. • Peers attended ENI: ENI is the teacher training college. • Poverty headcount: regional levels, except for major urban areas, where commune estimates are used. • Teaching resources available: refer to the SDI indicator. • Used a template: the supervision team followed a supervision worksheet. • Duration: time in hours of the last supervision visit. • School shares finances: the interview team saw information about school finances in public areas. • Met with staff: the supervision team met with staff to discuss the results of the visit. • Observed class: the supervision team observed one or more teachers providing instruction. • School seeks user feedback: the school has a formal method for seeking user feedback. • CGDES parental contributions: the amount contributed by parents at the school in the 2014/15 school year. 58 Table 31. Correlates of teacher effort Absence from school Absence from classroom Estimate Std. Err. Estimate Std. Err. Teacher characteristics relative to a civil servant Female (d) 0.578 (1.606) 1.434 (1.520) Pay delays (s) 0.0702** (0.0272) 0.0674** (0.0271) Head (d) 0.504 (1.548) 0.955 (1.519) Government contractual teacher -0.134 (1.042) -0.0210 (0.926) Private-sector teacher 14.49 (22.07) 17.78 (24.07) School characteristics relative to a school in Niamey Head absent (d) 25.38* (12.94) 28.14*** (8.344) Peers with CAP (s) -0.110 (0.0874) -0.120 (0.0859) Peers attended ENI (s) 0.108 (0.0852) 0.146 (0.0970) Poverty headcount (%) 0.174 (0.140) 0.0769 (0.139) Rural (d) -6.642 (5.142) -5.681 (5.177) Public (d) 24.70 (22.14) 31.58 (23.61) Teaching resources available (d) -4.774* (2.838) -2.536 (2.695) School shares finances (d) -3.994 (3.265) -1.131 (3.423) School seeks user feedback (d) -2.087 (2.345) -1.807 (2.320) Agadez 2.904 (6.844) 0.418 (5.086) Dosso -9.853 (6.879) -12.27** (6.095) Maradi -8.583 (6.855) -6.715 (6.702) Tahoua -7.211 (5.888) -8.136 (5.469) Tillabéri -4.812 (6.526) -5.721 (6.393) Zinder -5.663 (6.050) -5.764 (5.714) Supervision by the ministry Used a template (d) -5.319** (2.695) -9.134*** (2.564) Duration (s) -0.0657 (0.0520) -0.134 (0.130) Met with staff (d) -2.926 (3.301) -1.880 (2.859) Observed class (d) 7.180*** (2.445) 8.163*** (2.545) School management committee (SMC) relative to an inactive SMC CGDES parental contributions (s) 0.0349 (0.0924) 0.0736 (0.0867) CGDES activity Moderate -13.63*** (4.715) -9.760* (4.967) CGDES activity Substantial -1.277 (4.337) -3.703 (3.980) CGDES activity High -5.551 (4.903) -7.866* (4.541) Constant -11.08 (23.91) -15.46 (24.59) Observations 1,284 1,284 R-squared 0.418 0.486 F(28,175) 1.495 3.361 Pr>F 0.0665 9.80e-07 Notes: logit models with jackknifed standard errors based upon the survey design are reported. Two teachers, one community and one “other,” are excluded. Variable definitions are in Table 30. Superscript (*) indicates significance at the 1 percent (***), five percent (**) or ten percent (*) levels. 59 Figure 15. Pupil evaluation distribution by section and school ownership, grade 4 60 Table 32. Pupil performance details, grade 4 (percent) Diff. Urban Rural Diff. Diff. Pupil average score Niger Public Private (%) public Public (%) Urban Rural (%) Overall 22,6 21,3 65,5*** 207.5 32,6 18,0*** 81.1 36,5 18,0*** 102.8 Literacy 23,2 21,7 73,1*** 236.9 34,8 17,9*** 94.4 39,3 17,9*** 119.6 Letter recognition 45,5 44,0 95,5*** 117.0 61,3 39,0*** 57.2 65,3 39,0*** 67.4 Read basic words 29,7 28,0 84,7*** 202.5 46,8 22,6*** 107.1 51,3 22,6*** 127.0 Read simple sentence 12,2 10,5 66,5*** 533.3 19,5 7,9*** 146.8 25,1 7,9*** 217.7 Read paragraph 9,1 7,5 59,1*** 688.0 16,4 5,0*** 228.0 21,5 5,0*** 330.0 Reading comprehension 6,5 5,5 41,9*** 661.8 11,4 3,7*** 208.1 15,0 3,7*** 305.4 Numeracy 11,8 11,5 24,7*** 114.8 15,0 10,4*** 44.2 16,2 10,4*** 55.8 Add: 1-digit 45,4 44,3 82,1*** 85.3 59,3 39,9*** 48.6 62,0 39,9*** 55.4 Add: 2-digit 26,9 25,8 64,5*** 150.0 38,7 22,0*** 75.9 41,8 22,0*** 90.0 Subtract: 1-digit 40,6 39,3 83,5*** 112.5 51,8 35,7*** 45.1 55,5 35,7*** 55.5 Subtract: 2-digit 8,5 7,9 26,5*** 235.4 13,2 6,4** 106.3 14,8 6,4*** 131.3 Multiply: 1-digit 9,6 9,0 27,9*** 210.0 14,6 7,4** 97.3 16,2 7,4*** 118.9 Multiply: 2-digit 0,7 0,6 4,4** 633.3 1,3 0,4 225.0 1,7 0,4** 325.0 Divide: 1-digit 8,3 8,1 12,6 55.6 9,5 7,7 23.4 9,9 7,7 28.6 Divide: 2-digit 1,6 1,6 3,1 93.8 2,0 1,4 42.9 2,2 1,4 57.1 Non-verbal reasoning 53,1 52,7 66,7*** 26.6 56,9 51,5** 10.5 58,1 51,5*** 12.8 Note: based upon 1,646 fifth graders, data are weighted. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Scores and their differences are in percentage points. 61 Table 33. Pupil performance details, grade 5 (percent) Diff. Urban Rural Diff. Diff. Pupil average score Niger Public Private (%) public Public (%) Urban Rural (%) Overall 33,6 32,3 77,7*** 140,6 45,7 28,6*** 59,8 49,5 28,6*** 73,1 Literacy 35,4 34,0 85,6*** 151,8 49,6 29,7*** 67,0 53,8 29,7*** 81,1 Letter recognition 61,1 60,1 97,8*** 62,7 76,7 55,5*** 38,2 79,2 55,5*** 42,7 Read basic words 43,8 42,4 91,5*** 115,8 62,3 36,9*** 68,8 65,8 36,9*** 78,3 Read simple sentence 24,1 22,5 81,7*** 263,1 38,8 18,0*** 115,6 43,9 18,0*** 143,9 Read paragraph 17,4 15,7 75,5*** 380,9 30,0 11,7*** 156,4 35,4 11,7*** 202,6 Reading comprehension 14,1 12,7 62,3*** 390,6 21,8 10,1*** 115,8 26,6 10,1*** 163,4 Numeracy 18,2 17,7 37,1*** 109,6 22,2 16,4*** 35,4 24,0 16,4*** 46,3 Add: 1-digit 57,1 56,2 86,5*** 53,9 68,2 53,0*** 28,7 70,4 53,0*** 32,8 Add: 2-digit 42,2 41,0 82,1*** 100,2 54,6 37,3*** 46,4 57,9 37,3*** 55,2 Subtract: 1-digit 49,8 48,8 86,0*** 76,2 59,3 45,9*** 29,2 62,4 45,9*** 35,9 Subtract: 2-digit 15,1 14,2 47,3*** 233,1 20,2 12,5** 61,6 23,4 12,5*** 87,2 Multiply: 1-digit 15,0 14,0 50,9*** 263,6 17,6 13,0 35,4 21,5 13,0*** 65,4 Multiply: 2-digit 3,8 3,3 21,4*** 548,5 5,4 2,7* 100,0 7,3 2,7*** 170,4 Divide: 1-digit 25,9 25,1 56,4*** 124,7 35,6 22,2*** 60,4 38,1 22,2*** 71,6 Divide: 2-digit 6,9 6,3 28,1*** 346,0 10,1 5,2** 94,2 12,3 5,2*** 136,5 Non-verbal reasoning 56,6 56,2 70,8*** 26,0 58,4 55,6 5,0 59,9 55,6** 7,7 Note: based upon 1,230 fifth graders, data are weighted. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Scores and their differences are in percentage points. 62 Table 34. Pupil performance details by region, grade 4 (percent) Pupil average score National Agadez Dosso Maradi Niamey Tahoua Tillabéri Zinder Overall 22.6 36.4* 19.6*** 14.9*** 45.9 27.5*** 19.0*** 18.3*** Literacy 23.2 38.6** 20.1*** 14.4*** 50.8 28.5*** 19.5*** 17.7*** Letter recognition 45.5 67.3 42.3*** 29.8*** 77.6 57.5*** 41.3*** 39.7*** Read basic words 29.7 57.8 25.1*** 14.9*** 59.7 38.7*** 29.8*** 23.4*** Read simple sentence 12.2 19.2 11.8*** 6.0*** 33.7 13.7*** 9.0*** 8.8*** Read paragraph 9.1 10.7*** 8.0*** 4.7*** 30.0 9.8*** 4.1*** 7.2*** Reading comprehension 6.5 10.2 3.5*** 5.0*** 20.6 5.7*** 3.8*** 6.1*** Numeracy 11.8 19.2 8.6*** 9.9*** 16.6 15.5 9.6*** 12.2** Add: 1-digit 45.4 73.1 42.4*** 34.6*** 65.8 52.0* 45.0*** 41.2*** Add: 2-digit 26.9 64.5 21.0*** 20.7*** 46.1 29.9** 23.2*** 26.2*** Subtract: 1-digit 40.6 63.6 31.8*** 31.7*** 61.0 54.0 34.6*** 37.8*** Subtract: 2-digit 8.5 28.2** 4.0** 6.2* 12.4 13.7 5.7* 8.7 Multiply: 1-digit 9.6 13.1 4.7*** 6.5*** 18.3 15.8 6.3*** 8.9** Multiply: 2-digit 0.7 1.4 0.4* 0.3* 2.7 1.2 0.0** 0.4* Divide: 1-digit 8.3 16.3 4.8 8.2 7.5 17.5** 4.3 4.5 Divide: 2-digit 1.6 4.3 0.5 1.4 0.3 4.4** 0.4 1.6 Non-verbal reasoning 53.1 59.3 54.7 47.1*** 58.7 56.3 48.2*** 58.7 Pupils 1,651 29 260 350 205 267 315 225 Note: data are weighted. Comparisons are relative to Niamey. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Scores and their differences are in percentage points. Data with less than 50 observations should be used with caution. 63 Table 35. Pupil performance details by region, grade 5 (percent) Pupil average score National Agadez Dosso Maradi Niamey Tahoua Tillabéri Zinder Overall 33.6 60.3* 27.3*** 27.1*** 51.4 39.6** 35.7** 27.4*** Literacy 35.4 65.8 29.0*** 28.0*** 56.4 41.6** 38.1** 28.0*** Letter recognition 61.1 100.0*** 55.8*** 57.1*** 84.2 68.4*** 62.9*** 47.3*** Read basic words 43.8 78.0 35.3*** 36.8*** 72.3 49.5*** 50.0*** 30.5*** Read simple sentence 24.1 47.1 15.8*** 16.6*** 45.3 26.1** 31.2 18.3*** Read paragraph 17.4 35.0 14.1*** 9.7*** 37.8 19.0*** 21.9** 12.0*** Reading comprehension 14.1 29.5 9.9*** 6.6*** 28.1 22.3 14.6** 9.9*** Numeracy 18.2 29.7** 12.1*** 16.5*** 23.0 22.6 17.8** 17.0*** Add: 1-digit 57.1 79.4 46.5*** 51.5*** 72.2 65.3 60.5** 50.9*** Add: 2-digit 42.2 76.3*** 31.6*** 39.1*** 58.5 45.9* 43.8** 36.7*** Subtract: 1-digit 49.8 76.6** 42.0*** 47.7** 62.5 55.5 54.6 40.2*** Subtract: 2-digit 15.1 41.2*** 10.0*** 11.0*** 26.2 22.0 11.0*** 13.1*** Multiply: 1-digit 15.0 29.6** 5.4*** 13.2 20.0 25.7 11.5* 12.6* Multiply: 2-digit 3.8 6.2 0.7*** 2.1*** 7.9 6.5 3.2* 3.4* Divide: 1-digit 25.9 39.3 12.8*** 19.5*** 39.8 35.7 22.9** 27.1* Divide: 2-digit 6.9 23.1 2.2*** 4.1* 9.3 9.8 9.6 6.0 Nonverbal reasoning 56.6 64.8 52.4 51.1** 58.3 65.6** 56.3 57.1 Pupils 1,230 28 126 210 286 161 184 245 Note: data are weighted. Comparisons are relative to Niamey. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. Scores and their differences are in percentage points. Data with less than 50 observations should be used with caution. 64 Table 36. Correlates of pupil performance in language Regional Standard Inspectorate Standard Variables dummies Error dummies Error Pupil characteristics (relative to Grade 5) Age (s) 0.0592** (0.0293) 0.0753*** (0.0280) Grade 4 pupil (d) -9.641*** (1.995) -9.170*** (2.059) Girl -1.951* (1.136) -1.991* (1.085) School characteristics relative to Niamey (IV) Urban (d) 4.739 (4.389) 7.227 (9.421) Agadez 5.248 (7.217) Dosso -7.914 (5.738) Maradi -10.29* (5.978) Tahoua 2.475 (5.637) Tillabéri -7.375 (6.738) Zinder -10.73* (5.680) Private school 26.06 (24.11) 10.14 (28.12) Poverty headcount (%) -0.280** (0.119) - Pencils (s) -0.0313 (0.0860) -0.00577 (0.158) Exercise book (s) 0.0734 (0.0698) 0.00224 (0.0979) Textbook (s) -0.0191 (0.0725) -0.0225 (0.0845) Functional toilet (d) 1.205 (2.516) 0.679 (3.350) Teacher characteristics (current-year) Time on task (s) 0.805* (0.445) 0.174 (0.633) Pedagogy score (s) 0.0291 (0.0415) 0.00758 (0.0573) Language score (s) 0.0275 (0.0453) 0.00737 (0.0591) Absent from class (s) 0.758* (0.455) 0.199 (0.642) Teacher speaks local language -1.797 (5.046) -3.300 (6.386) Contextualizes with local information (s) 0.0509 (0.0757) 0.0306 (0.116) Individualized attention (s) -0.0451 (0.101) -0.116 (0.165) Gives tasks (s) 0.00281 (0.0803) -0.0163 (0.115) Strikes pupils (s) -0.123* (0.0672) -0.221** (0.105) Contractual -0.741 (3.281) -1.370 (3.876) Private school teacher -8.034 (24.19) 6.886 (26.00) Math score (s) Constant 27.67*** (7.767) 22.73** (10.07) Model statistics Observations 2,384 2,384 R2 0.226 0.326 F(n,d) 27, 231 13.22 82, 210 . Pr>F 0 . Test that the inspection dummies are cumulatively insignificant F(n,d) 1,210 3.07 Pr>F 0.0811 Note: the notation “(s)” refers to a standardized variable and “(d)” refers to a dummy (binary) variable. The dummy for CE2 is relative to CM1. The symbol (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. The models account for the survey design. One standard deviation is 1.24 points. Model standard errors account for clustering by school and grade within the school. Poverty headcounts are from the poverty mapping results undertaken by the Institut National de la Statistique and the World Bank (unpublished). 65 Table 37. Correlates of pupil performance in mathematics Regional Standard Inspectorate Standard Variables dummies Error dummies Error Pupil characteristics (relative to Grade 5) Age (s) 0.125*** (0.0278) 0.131*** (0.0267) Grade 4 pupil (d) -8.320*** (1.914) -8.702*** (2.035) Girl -2.599** (1.195) -2.776** (1.072) School characteristics relative to Niamey (IV) Urban (d) 5.201 (3.374) 5.625 (4.328) Agadez 27.55*** (9.938) Dosso -4.301 (4.552) Maradi 1.244 (4.816) Tahoua 14.84*** (5.385) Tillabéri -1.611 (5.422) Zinder 3.272 (5.001) Private school 36.21** (17.59) - Poverty headcount (%) -0.129 (0.0906) -0.0385 (0.0934) Pencils (s) -0.0685 (0.0531) -0.0179 (0.0815) Exercise book (s) 0.0487 (0.0638) -0.133 (0.0965) Textbook (s) -0.0525 (0.0762) -1.481 (2.953) Functional toilet (d) -2.265 (2.145) Teacher characteristics (current-year) Time on task (s) 0.841* (0.489) 0.0411 (0.587) Pedagogy score (s) 0.0166 (0.0328) -0.00423 (0.0449) Language score (s) Absent from class (s) 0.836* (0.497) 0.0608 (0.591) Teacher speaks local language -1.090 (4.639) 0.464 (6.175) Contextualizes with local information (s) 0.102 (0.0652) 0.0789 (0.0976) Individualized attention (s) 0.000482 (0.0650) 0.0259 (0.0959) Gives tasks (s) 0.109 (0.0716) 0.149 (0.104) Strikes pupils (s) -0.0947* (0.0570) -0.201*** (0.0702) Contractual -3.484 (3.264) -4.549 (3.708) Private school teacher -14.36 (18.13) -9.238 (25.50) Math score (s) -0.0877* (0.0478) -0.0971 (0.0589) Constant 13.28* (7.044) 12.24 (7.602) Model statistics Observations 2,384 2,384 R2 0.188 0.283 F(n,d) 27, 231 9.026 82, 210 . Pr>F 0 . Test that the inspection dummies are cumulatively insignificant F(n,d) 1,210 0.18 Pr>F 0.6703 Note: the notation “(s)” refers to a standardized variable and “(d)” refers to a dummy (binary) variable. The dummy for CE2 is relative to CM1. The symbol (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. The models account for the survey design. One standard deviation is 0.54 points. Model standard errors account for clustering at the school and grade within the school. Poverty headcounts are from the poverty mapping results undertaken by the Institut National de la Statistique and the World Bank (unpublished). 66 Table 38. Teacher performance evaluation elements Diff. Urban Rural Diff. Niger Public Private (%) public public (%) Director Individual performance evaluation 33,2 33,1 40,6 22,7 38,2 32,0 19,4 Based upon (multiple responses possible) Direct observation 60,0 59,3 84,6** 42,7 66,7 57,4 16,2 Absence rate 20,5 18,6 84,6*** 354,8 19,1 18,5 3,2 Class size 4,7 4,4 15,4 250,0 14,3 1,9 652,6 Parent satisfaction 17,7 16,7 53,9*** 222,8 23,8 14,8 60,8 Student learning 41,2 39,7 92,3*** 132,5 42,9 38,9 10,3 Exam performance 19,4 18,1 61,5*** 239,8 38,1 13,0** 193,1 Teacher behavior 54,8 53,9 84,6*** 57,0 61,9 51,9 19,3 Other 9,1 9,3 0,0*** -100,0 9,5 9,3 2,2 External supervision Individual performance evaluation 22,1 21,9 31,3 42,9 21,8 21,9 -0,5 Based upon (multiple responses possible) Direct observation 62,1 60,7 100,0*** 64,7 54,6 62,5 -12,6 Absence rate 14,7 13,8 37,5 171,7 18,2 12,5 45,6 Student absences 13,2 12,8 25,0 95,3 27,3 8,3 228,9 Class size 9,1 7,4 50,0** 575,7 18,2 4,2 333,3 Parent satisfaction 11,1 9,6 50,0** 420,8 0,0 12,5* -100,0 Student learning 40,7 39,4 75,0** 90,4 45,5 37,5 21,3 Exam performance 14,6 12,8 62,5*** 388,3 27,3 8,3 228,9 Teacher behavior 66,9 67,0 62,5 -6,7 54,6 70,8 -22,9 Other 21,5 22,3 0,0*** -100,0 27,3 20,8 31,3 Note: data are weighted. Differences are relative to public and rural public schools, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. 67 Table 39. Supervision content Diff. Urban Rural Diff. Niger Public Private (%) Public Public (%) Supervisions in 2014/15 2.9 2,9 3,8* 31,0 4,8 2,4*** 100,0 Supervisions in 2015/16 0.9 0,9 1 11,1 1,4 0,8 75,0 Days since last supervision 261,5 263,2 200,0 -24,0 215,8 273,4 -21,1 Visit duration (hours) 3,0 3,0 2,4 -20,0 2,4 3,2 -25,0 Supervision: Used a template 49,6 49,1 71,9*** 46,4 65,5 45,6*** 43,6 Met with teachers 74,0 74,1 71,0 -4,2 70,2 75,0 -6,4 Met with community 28,9 29,6 3,2*** -89,2 8,5 34,3*** -75,2 Observed teaching 49,3 48,2 87,1*** 80,7 59,6 45,7* 30,4 Checked attendance roster 66,8 66,5 77,4 16,4 70,2 65,7 6,8 Checked civil service roster 43,9 43,8 48,4 10,5 38,3 45,0 -14,9 Checked lesson plans 57,8 57,4 71,0 23,7 55,3 57,9 -4,5 Checked inventory 39,0 39,5 19,4*** -50,9 27,7 42,1* -34,2 Checked the SMC action plan 42,6 42,8 35,5 -17,1 29,8 45,7** -34,8 Reviewed the SMC annual report 36,2 36,6 22,6* -38,3 21,3 40,0*** -46,8 Debriefed the director 82,7 82,7 83,9 1,5 85,1 82,1 3,7 Debriefed the staff 79,5 79,8 69,2 -13,3 67,5 82,6* -18,3 Left written feedback 47,6 47,3 57,7 22,0 52,5 46,1 13,9 Written feedback was available 0,6 0,6 0,9 50,0 0,7 0,6 16,7 Note: data are weighted. Differences are relative to public and rural public schools, respectively. Superscript (*) denotes that the difference is significant at the 1% (***), 5% (**), or 10% (*) significance level. School management committees are abbreviated “SMC”. 68 REFERENCES Kremer, M., E. Duflo, and P. Dupas. (2011), “Peer Effects, Teacher Incentives, and the Impact of Tracking.” American Economic Review 101 (5 (August 2011): 1739 -1774. Hanushek, E. A. and L. W ößmann, (2006), Does Educational Tracking Affect Performance and Inequality? Differences- in- Differences Evidence Across Countries*. The Economic Journal, 116: C63–C76. doi:10.1111/j.1468-0297.2006.01076.x INS (2014), « Présentation des résultats globaux définitifs du Quatrième (4ème) Recensement Général de la Population et de l’Habitat (RGP/H) de 2012», Institut National de la Statistique du Niger. Johnson, Cunningham, and Dowling (2012) “Draft Final Report, Teaching Standards and Curriculum Review”. PASEC/CONFEMEN (2014), “PASEC 2014 : Performances des systèmes éducatifs en Afrique Subsaharienne francophone : compétences et facteurs de réussite au primaire», Conférence des ministres de l’Education des pays ayant le français en partage. Vargas, E. and J. De Laat (2003), “Do differences in teacher contracts affect pupil performance? Evidence from Togo”, World Bank Working Paper Number 26955. World Bank (2004), World Development Report: Making Services Work for Poor People. Document of the World Bank