Document of The World Bank Report No.: 86028 PROJECT PERFORMANCE ASSESSMENT REPORT BANGLADESH PRIMARY EDUCATION DEVELOPMENT PROJECT II (P074966) April 14, 2014 IEG Public Sector Evaluation Independent Evaluation Group ÿþii Currency Equivalents (annual averages) Currency Unit = Bangladesh Taka (Tk) 2003 US$1.00 Tk58.15 2004 US$1.00 Tk59.51 2005 US$1.00 Tk64.32 2006 US$1.00 Tk68.93 2007 US$1.00 Tk68.87 2008 US$1.00 Tk68.59 2009 US$1.00 Tk69.03 2010 US$1.00 Tk69.64 2011 US$1.00 Tk74.15 2012 US$1.00 Tk80.30 Abbreviations and Acronyms ADB Asian Development Bank DFID Department for International Development GDP Gross Domestic Product ICR Implementation Completion Report IEG Independent Evaluation Group IEGPS IEG Public Sector Evaluation ICRR Implementation Completion and Results report M&E Monitoring and Evaluation MDGs Millennium Development Goals PAD Project Appraisal Document PEDP I First Primary Education Development Program PEDP II Second Primary Education Development Program PEDP III Third Primary Education Development Program PPAR Project Performance Assessment Report SWAP Sector-Wide Approach UNICEF United Nations Children Fund Fiscal Year Government: July 1-June 30 Director-General, Independent Evaluation Ms. Caroline Heider Director, IEG Public Sector Evaluation Mr. Emmanuel Jimenez Manager, IEG Public Sector Evaluation Mr. Mark Sundberg Task Manager Mr. Erik Bloom 111 Contents Principal Ratings.................. . ....................... ...... vi Key Staff Responsible........... ............................... vi Preface........................................................ ix Summary.......................................................xi Background...................................................xi Objectives and Design............................................xi Implementation, Impacts, and Risks..................................xii Bank and Borrower Performance.................................... xiii Lessons..................................................... xiv 1. Primary Education in Bangladesh: Background and Context..................1 Socio-Economic Context............................... ................... 1 Economic growth.................................... .................. 1 Poverty ................................................ ..... 2 Demographics ......................2..........................2 Education in Bangladesh......................3...... ...............3 Challenges Facing the Education Sector in Bangladesh...................... 5 The World Bank and Development Partners in the Education Sector... .............. 6 Education projects prior to the Primary Education Development Project.............. 6 The Primary Education Development Program..........................7 2. Objectives, Design, and their Relevance .................................9 The World Bank's Program Objectives................................10 PEDP II and the World Bank Program Design................. ..........11 Program Organization and Fiduciary Arrangements......................14 Relevance of Objectives...........................................17 Relevance to Bangladesh...................... ..................17 Relevance to the World Bank..................................... 17 Relevance of Design............................................. 17 Technical design and causal chain.................................. 18 Program coverage, design, and planning..............................18 3. Implementation................................................. 19 This report was prepared by Erik Bloom and Md. Rafi Hossain who assessed the project in February, 2013. The report was peer reviewed by David Evans and Javier Luque and panel reviewed by Denise Vallancourt. Viktoriya Yevsyeyeva and Richard Kraus provided administrative support. iv Initial Phase.............. .................................... 20 The Midterm Review ............................................ 21 After the Midterm Review .................................... ..... 22 4. Achievement of the Objectives ...................................... 22 Objective 1: Improve quality of primary education................ ......... 23 Was there more Focus on Quality?................................. 23 Did Teacher Performance Improve? ................................ 24 Did Textbook Distribution Improve?.......... ................ ..... 26 What Changes were seen in Learning Outcomes9 ......................... 27 Objective 2: Improve Equitable Access to Primary Education ................ 28 Did Physical Access Increase for all Students? .................... ..... 28 Were Stipends Provided for Poorer Students? ..................... ..... 29 Did Access Increase for Students with Disabilities? .................. 30 How and why did Internal Efficiency in Primary Education Change? .... ..... 31 What was the Program's impact on coverage? ................ ......... 33 Did Poor Students Benefit from the Program? .................... ...... 36 Did Children from Hill Tribes Benefit from the Program? ................. 36 Objective 3: Improve the efficiency of primary education through a SWAP........... 37 Were Better Tools for Sector Coordination Developed? ........................ 37 Was There Greater Harmonization9 ................................. 37 Was there Enhanced Sector Stewardship? ............................ 38 5. Efficiency .................................................... 39 Administrative and Transaction Costs.................................. 39 External Efficiency ......................................... ..... 40 6. Ratings ...................................................... 42 Outcome..................................................... 42 Risk to Development Outcome ...................................... 43 Bank Performance.......................................... ..... 44 Quality at Entry.............................................. 44 Quality of Supervision ................... ................. 45 Borrower Performance ........................................... 46 Government Performance ........................................... 46 Implementing Agency Performance ................................. 47 Monitoring and Evaluation................................ ......... 48 M&E Design ........................ ................. 48 V M&E Implementation ..................................... ..... 49 M&E Utilization ................................................... 51 Safeguards .................................................... 51 Fiduciary Compliance............................................. 52 7. Conclusion and Lessons........................................... 53 References ...................................................... 57 Annex A. Basic Data Sheet............... ..................... ..... 63 Annex B. Key Performance and Other Indicators ...................... 67 Annex C. List of Persons Met........................................ 70 Boxes Box 1. The Government's Primary Education Strategy ........ ................ 6 Box 2. Sector-Wide Approaches.......................9.... ............9 Tables Table 1. Key Economic Data ............................. ..... ............ 1 Table 2. Population and Demographics .......................... ......... 2 Table 3. Organization of the Bangladesh Primary Education System... .................. 4 Table 4. Sub-Components and Activities under Component 1........... ....... 12 Table 5. Sub-Components and Activities under Component 2........... ................... 13 Table 6. Activities under Component 3 ............................... 13 Table 7. Activities under Component 4 ............................... 14 Table 8. Financing of PEDP II by Source................................. 16 Table 9. Timeliness of Textbook Distribution ................................ 26 Table 10. Evolution of Pass Rate for the Primary Completion Examination .... ..... 27 Table 11. Repetition Rate by Grade............ .................. ..... 32 Table 12. Dropout Rate by Grade....................................... 33 Table 13. Primary School Enrollment Rates with Different Population Estimates..... 34 Table 14. Education Coverage from the Demographic and Health Surveys ... ...... 35 Table 15. Enrollment by Income Level .................................. 36 Table 16. Estimated Ex Ante Internal Rates of Returns ........ ............... 41 Table 17. Project Outcomes ................... ................. 43 Table 18. Key Performance Indicators ........................... 68 Figures Figure 1. Adult Literacy Rate, 1981 to 2011 .......................... ..... 3 Figure 2. Net and Gross Enrollment Rates in Primary Education ................ 33 vi Principal Ratings ICR* ICR Review* PPAR Outcome Moderately Satisfactory Moderately Satisfactory Satisfactory Risk to Development Significant Significant Significant Outcome Bank Performance Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Borrower Peforme Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Performance * The Implementation Completion and Results (ICR) report is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEG product that seeks to independently verify the findings of the ICR. Key Staff Responsible Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Hena G. Mukherjee Michelle Riboud Christine I. Wallich Completion Susan Opper Amit Dar Ellen A. Goldstein Vii IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank's self-evaluation process and to verify that the Bank's work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank's lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG's use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation's major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project's objectives are consistent with the country's current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project's design is consistent with the stated objectives. Efficacy is the extent to which the project's objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.  ix Preface This Project Performance Assessment Report (PPAR) assesses a primary education project in Bangladesh, the Second Primary Education Development Program, commonly known as PEDP II. This program was financed by several development partners, under the leadership of the Asian Development Bank. The World Bank approved its contribution on February 24, 2004 and the program closed on June 30, 2011. The World Bank's contribution was US$145 million. The program was the principal instrument for development partners to support primary education and was the first Sector Wide Approach (SWAP) for education in Bangladesh. This evaluation was done in conjunction with an evaluation of the Health, Nutrition, and Population Sector Development Program, which is a SWAP operating in the health sector in Bangladesh. This report was prepared by Erik Bloom, Senior Economist, IEG and Md. Rafi Hossain, Consultant. The findings are based on any extensive review of the literature, project reports, and a field visit from February 20 to March 17, 2013, which was done concurrently with an evaluation mission of the Health Sector Support Program. The mission visited Dhaka and the immediate environs and was carried out during a period of hartals (political protests) that limited the possibility of field visits during the mission. From April 16 to 19, 2013, Md. Rafi Hossain visited several primary schools in the Chittagong Division. As much as possible, the PPAR cites publically available documents, and when appropriate, it refers to interviews and internal documents. Following standard IEG procedures, the report was sent to the government officials and agencies in Bangladesh for review and comments before being finalized. No comments were received.  xi Summary Background This Project Performance Assessment Report evaluates the impact of the World Bank's contribution to the Second Primary Education Sector Program (PEDP II) in Bangladesh from 2004 to 2011. PDEPII was one of the largest education Sector Wide Approach Projects (SWAPs) ever undertaken, consolidating a large number of externally-financed primary education projects into a joint operation. Under the SWAP, ten development partners provided financing. In some respects the Program has been transformational, reshaping the relationship between the Government of Bangladesh and development partners, bringing greater strategic coherence to primary education policies, and establishing a new M&E system that has been embraced by the government. Shortcomings in PEDP II's implementation also provide lessons for future SWAPs in education and other sectors. The Ministry of Primary and Mass Education focuses on pre-primary, primary education, non-formal, and adult education. Primary education was made compulsory in 1990 and Bangladesh has seen major advances in the education levels of its population. In 1993, only 14 percent of women had completed primary level or higher. By 2011, 42 percent of women had at least completed primary education. As coverage increased, the sector's focus has shifted to improving learning results and targeting vulnerable children that were left out of the system. Primary education in Bangladesh consists of five grades, with students ideally entering at the age of six. Bangladesh has a large number of primary education systems, both with public and private financing and ownership. The government provides partial or total financing for most primary education, although only around 59 percent of children attend government schools. At the time of project preparation, the sector faced several challenges: * Incorporating excluded groups, such as poor children in urban slum and remote areas (including ethnic minorities) as well as children with disabilities. * Improving education quality as assessments showed that most primary students Bangladesh did not meet minimum learning standards. * Strengthening governance of the sector, which was highly centralized with most decisions made at the Ministry, which had a limited M&E and analytical capacity. Objectives and Design The objectives of PEDP II were closely aligned with the government's strategy for education and remain highly relevant in the government's education strategy. The Program and its successor were identified in the World Bank's country strategy and the goals of improving coverage and quality were, and remain, quite relevant. Xii Financed by the government and a number of development partners, PEDP II was organized as a Sector Wide Approach (SWAP). Under this approach, the government and development partners agreed to finance a common program and to harmonize procedures. The objective of a SWAP is to achieve better coordination and lower transaction costs, along with greater country ownership. The Asian Development Bank was designated as the lead development partner. This arrangement led to a significant reduction in the number of projects with an estimated twenty projects gradually consolidated into PEDP II. The World Bank's share was US$150 million. The central objectives of the Program were (i) to raise coverage and (ii) to improve education quality. As a SWAP, the Program covered most aspects of the formal primary education in the Ministry. The design was built around a logical framework and had a reasonable causal chain. As designed, the Program had a very detailed list of activities that limited its flexibility. Likewise, its design also did not fully take into account the Ministry's initial weakness. Implementation, Impacts, and Risks Initially, implementation was slower than expected. Both the Ministry and development partners had to adapt to the SWAP, which required a change in staffing and improved coordination by both parties. The Ministry had to create a Program Coordination Unit, a Finance and Procurement Division, and Monitoring and Evaluation Cell. Both development partners and the government had to learn how to work on a common program instead of individual plans and developing new administrative procedures. In response to a slow start, the mid-term review introduced some major changes to address these shortcomings. One major focus of the review was on efforts to speed up implementation of "stuck" elements. The mid-term review revised the key performance indicators and contracted a third party evaluator to improve the evaluation of the growing database. After the mid-term review, the Program saw a substantial improvement. The government transferred additional high-level civil servants to support the Program and the Ministry increased its focus on collecting and analyzing data. There were some delays in procurement in the first two years. Although audits were carried out in a timely fashion, there was often a long delay to respond to the observations. Under the Program, coverage improved significantly. The gross enrollment rate increased from 94 percent (2005) to 109 percent (2011) and net the enrollment rate increased from 87 percent to 95 percent. There is evidence that coverage increased for the poor students, students with disabilities, and ethnic minorities. A large part of this increase can be attributed as the Program led to improved education efficiency, and the average time required for a primary student to graduate decreased from 8.1 to 7.2 years. The Program also introduced a focus on education quality and learning in primary education and contributed to the increase in learning assessment results seen in primary education. The Program led to the creation of the Primary Completion Examination in 2009, which helped raise the prestige of primary education as well as emphasizing that quality was a major focus. During the Program period, the pass rate has been increasing at a time when the coverage of the examination has also increased. However, there are still significant differences by region and socio-economic groups. The Program xiii introduced the National Student Assessment, whose design was finalized in 2011. The assessment shows children continued in school to learn between grades 3 and 5, which is by no means guaranteed and fails to occur in many education systems in low income countries. The Program supported a number of efforts that are aimed at leading to better teaching, such as improving the recruitment of teachers. It also supported a significant increase in subject-training for existing teachers, although there was little change in pedagogy training. More teachers received a teacher's certificate. The Program made a major contribution to strengthening sector-wide monitoring and evaluation (M&E) capacity. At the start of the Program, the M&E capacity of the Ministry was quite low. Although progress was slow at first, the Ministry developed a strong capacity to collect and analyze information. The logical framework identified a long-term objective to reduce poverty as well as program-specific objectives. It outlined the outputs to achieve these objectives. Each output had a set of indicators to monitor progress as well as assumptions behind each output; for the most part, these indicators were appropriate for the outcome. M&E implementation was strong and showed constant advances from inception to the closing of the Program. The Program supported conducted a "baseline survey" in late 2005 to all primary schools, which was followed by an annual school census. With support of the Program, the Ministry introduced instruments to measure student learning. Starting in 2009, the Ministry starting publishing its Annual Sector Performance Report that brought together data from different sources. The reports, which outlined the expected causality chain from the inputs and activities to the medium-term and long-term outcomes, presented a comprehensive overview of data and the sector's performance against key performance indicators. Despite these positive results, there are many risks to the development outcome. The Program has broad political support but its continued success requires a high level of human resources within the Ministry. Current civil service rules make career advancement difficult in the Ministry, which tends to weaken the long-term commitment of staff. Ongoing political disputes between political parties may delay the implementation of agreed reform. Bank and Borrower Performance Although the World Bank played a limited role in developing many aspects of the SWAP, it made contributions that improved the preparation process, including the results framework, the fiduciary process, and the economic analysis. Major issues during the preparation of the Program included the lack of data, the low capacity of Ministry, and agreements on joint procedures. Initially, the government was reluctant to enter into a SWAP arrangement and did not, at first, provide the Ministry with the necessary staff to carry out the Program. As the government developed greater ownership, it responded by assigning senior staff to the Xiv Ministry. The government was the largest provider of resources and these resources were generally provided in a timely and predictable fashion. At the start of the Program, the Ministry did not have sufficient capacity to implement elements of the Program. In the first two years, the Ministry was slow in hiring necessary consultants and there were delays in procurement. After the midterm review, turnover was kept to a minimum and the government did its best to ensure that the Program had adequate staff, particularly at the highest levels. The Ministry was able to mobilize its local network to work with schools to collect data and it took the lead in developing the Primary Completion Examination, implementing the new teacher recruitment model, and reforming teacher training. All of these initiatives required coordination between the Ministry at the national level and the local education officials and schools. Lessons In a SWAP, the leadership of development partners requires a mix of "hard skills", "soft skills", and flexibility. PEDP II represented the first successful SWAP for education in Bangladesh, bringing greater coherence to external support and reducing transactions costs for the Government. To succeed, it required flexibility and adaptability from all sides. As the Program got underway, there appeared to be a concerted effort to move beyond institutional debates and to focus on implementation and technical issues. This sort of process requires trust and can only be gained from experience. Government ownership needs to be fostered. The Program's experience showed that ownership is a subtle concept. There was no doubt that the government had a high level of ownership of the Program's objectives and had long given primary education a high priority. However, this did not mean that the government fully understood the sector- wide approach and with the changes it would bring to project management. Development partners could be more subtle in understanding ownership and in identifying steps necessary to strengthen the understanding of new ideas and proposals. No sector is an island and it is important for sector programs to coordinate with efforts to strengthen public administration and governance. In the long run, Bangladesh needs to reform how civil servants are hired and remunerated. This is beyond the remit of a sector program but points out the importance of the sector to engage in the larger discussions about public management and administration. It may make sense for development partners to include public administration specialists or economists on their team to ensure that they can maintain a meaningful dialogue with the appropriate central ministries as well as with other development project and programs. A SWAP is an evolutionary process that requires flexibility and adaptation to succeed. Development partners need to be flexible ion the design of a SWAP and not expect a fully developed institution at the start of the process. In the end, the government and partners agreed to be selective. While the Program included "stretch" targets, many aspects of its design were realistic given Bangladesh's circumstances. In the end, the partners and the government agreed to continue with the "project approach" to manage procurement and financial management. PEDP II set the stage for innovations that were incorporated into PEDP III, such as the use of pooled funds within the government xv budget instead of through a system of special accounts and more flexible Program leadership. With support, governments can transform M&E systems from a pro forma system of collecting data to a powerful tool for monitoring and decision making. The Program showed that it is possible to develop a working M&E system that not only generates data but uses the data to guide policy and strengthen the understanding of the results chain. Although consultants played a major role in preparing the final report, the Ministry has made an important effort to generate data at the student and the school level and to institutionalize this process. Moving forward, there is a need to look at impacts and to make adjustments in individual initiatives, including the primary education stipend program as well as continued research is needed on the best way to design and implement training programs for teachers. Caroline Heider Director-General Evaluation  1 1. Primary Education in Bangladesh: Background and Context 1.1 Education plays a central role in the Millennium Development Goals (MDGs) and in the overall development process. Education is both an important investment in a child's future and a key input in economic growth-countries that have invested in education have seen gains in economic growth (Hanushek and Woessmann 2012) and global competitiveness (Schwab 2012). Education's capacity to lift children out of poverty and its contribution to promote development and economic growth justify public subsidy. Globally, the World Bank has been engaged in the education sector since the 1960s and Bangladesh has been one of the biggest beneficiaries. This Project Performance Assessment Report (PPAR) reviews the impact of the World Bank's Second Primary Education Sector Program in Bangladesh from 2004 to 2011. Socio-Economic Context ECONOMIC GROWTH 1.2 Despite significant economic growth, Bangladesh remains a low income country. When Bangladesh gained independence in 1971, it had gross domestic product (GDP) per capita of US$92 (US$383 in constant 2010 dollars), making it the eighth poorest country in the world. By 2010, the per capita income was US$1,652, placing Bangladesh above the average for low income countries (World Bank 2012). 1.3 Economic growth has been driven by both the industrial sector and services. Central to the industrial growth is the rapid expansion of garment production, accounting for 78 percent of total exports (International Monetary Fund 2011). The second major source of foreign exchange is from remittances, which have grown substantially and now account for 10 to 11 percent of GDP. Table 1. Key Economic Data 1992 2000 2003 2005 2006 2007 2008 2009 2010 GDP per capita (2010 US$) 860 1,076 1142 1,293 1,361 1,432 1,505 1,575 1,652 GDP growth rate (percent) 5.0 5.9 5.3 6.0 6.6 6.4 6.2 5.7 6.1 Currentaccountbalance 0.6 -0.6 0.3 -0.3 1.9 1.3 1.2 4.0 2.1 (percentage of GDP) Net development assistance 5.6 2.4 2.5 2.1 1.9 2.1 2.4 1.3 1.3 (percentage of GDP) Net worker remittances (ptorkertag e s ofG2.9 4.2 6.1 7.2 8.8 9.6 11.2 11.8 10.8 (percentage of GDP) Source: All data from (World Bank 2012). 2 POVERTY 1.4 While Bangladesh remains a low income country with an estimated 47 million people living below the poverty line, it has seen a constant and dramatic reduction in poverty. In 1991, an estimated 59 percent of the population lived below the poverty line; by 2010, this had decreased to 32 percent. Extreme poverty has dropped even more sharply, from 41 percent in 1991 to 18 percent in 2010 (Government of the People's Republic of Bangladesh 2011). The decline in poverty has been relatively well-distributed and has benefited both poor and less poor regions and the Gini index has been relatively stable between 0.30 to 0.35 since 1995. The gap between urban and rural areas remains significant although it has been gradually closing (Government of the People's Republic of Bangladesh 2011). 1.5 Increasing real wages in rural areas in the past decade as well as increasing wages in urban areas from 2001 to 2005 have made an important contribution in reducing poverty (Zhang, et al. 2013). The growth in remittances has also played a major role in transferring resources to poorer households. Changes within the household have also played a major role, as families become smaller and more women enter the workforce (Government of the People's Republic of Bangladesh 2011). Women account for 80 to 90 percent of the employees in this garment sector and the increase in female employment in Bangladesh has played an important role in improving the status of women and reducing poverty (Government of the People's Republic of Bangladesh 2012). DEMOGRAPHICS 1.6 With the exception of a few small island countries and city states, Bangladesh has the highest population density in the world, at 964 inhabitants per square kilometer (Government of the People's Republic of Bangladesh 2011). Since independence, Bangladesh has undergone a major demographic transition. While the population growth rate is still positive, it has slowed down substantially. This, combined with the increase in life expectancy, has led to a gradually aging population. In 1971, 45 percent of the population was under the age of 15, by 1991 this decreased to 42 percent, and to 31 percent by 2011 (World Bank 2012). Table 2. Population and Demographics 1974 1981 1991 2001 2011 Population (millions) 71.5 87.1 106.3 124.4 142.3 Population growth rate 2.6 2.3 2.0 1.6 1.3 Percentage urban 9.4 15.4 20.2 24.0 28.4 Source: Data on the urban population are from (World Bank 2012). Other population estimates are taken from (Government of the People's Republic of Bangladesh 2011) and author's calculations. 1.7 Although the majority of the population of Bangladesh is Bengali, Bangladesh has a significant number of ethnic minorities. The most recent census estimates that around 1.1 percent of the population belongs to a tribal (ethnic minority) group (Government of the People's Republic of Bangladesh 2011). These groups have distinct languages, cultures, and customs, and in some cases, religion. They are primarily located in the hilly areas of the southeastern region and the north (Rahman, et al. 2012). In 2010, an estimated 89.5 percent 3 of the population was Muslim, with most of the remaining population Hindu (Bangladesh Bureau of Statistics 2011). Education in Bangladesh 1.8 The education sector in Bangladesh is divided between two ministries: the Ministry of Primary and Mass Education, which focuses on pre-primary, primary, non-formal, and adult education and the Ministry of Education, which focuses on secondary and higher education as well as a few small primary school sub-systems. 1.9 Primary education has played an important role in nation building since the country became independent in 1971. The government nationalized most primary schools in 1974. In 1990, Parliament approved the Compulsory Primary Education Act, which gradually introduced mandatory primary education. In 1992 the government created the Division of Primary and Mass Education within the Prime Minister's Office. Around 1997, the government shifted its focus to hard-to-reach children and education quality (Unterhalter, Ross and Alam 2003). The Division was converted into a Ministry' in 2003. For several years, the Prime Minister served as the Minister, indicating the priority that the government gave to primary education (The Study Team 2009). Figure 1. Adult Literacy Rate, 1981 to 2011 100 - 90 so 70 60 50 4-0 30 20 10 0 1981 1991 2001 2011 NFemale NTotal Source: (World Bank 2012) 1.10 Bangladesh has seen major advances in the education levels of its population. Figure 1 shows the adult literacy from 1981 to 2011 for the total adult (over the age of 15) population and for adult females. . The adult literacy rate is a cumulative measure that reflects long terms changes in education. In 1981, barely a quarter of the population was literate; adult women had a literacy rate of 18 percent, leading to a gender parity index of 1 For the sake of simplicity, both the Primary and Mass Education Division and the Ministry of Primary and Mass Education will be referred to as "the Ministry." 4 2 2.21. By 2011, the total adult literacy rate increased to 57 percent while the female adult literacy was 52 percent, which implies a gender parity index of 1.17. 1.11 In 1993, only 14 percent of women had a primary level education or higher (S. M. Mitra 1994). By 2011, 42 percent of women had at least primary education. Among women in the top wealth quintile, 64 percent had at least primary education compared to 18 percent in the poorest quintile (NIPORT, Mitra and Associates, and MEASURE DHS 2013). Thus by 2011, the poorest quintile of women had a higher level of education than all women in 1993. 1.12 Primary education in Bangladesh consists of five grades, with students ideally entering at the age of six. Primary education is offered through a large number of systems, both with public and private financing and ownership. The pre-primary education system typically covers children from the ages of three to five. Table 3 outlines the types of primary schools that operate in the country, indicating the source of financing, the ownership, and share of enrollment of each type of school system. The government provides partial or total financing for most primary schools and around 59 percent of children attend government schools. Public financing varies by system but generally covers most recurrent costs. Table 3. Organization of the Bangladesh Primary Education System Percentage of Type of School Totlenrole Ownership Source of Financing Total Enrollment Goverment Schools 58.4% Public Fully public Experimental Schools 0.1% Public Fully public Attached Primary Schools 2.8% Private Partial public Registered Non-Govemment Schools 22.0% Private Partial public Government Alia Madrassas 0.0% Public Fully public Independent Primary Madrassas 5.9% Private Partial public Recognized Non-Govemment Alia 6.8. Private Partial public Madrassas Private Schools 1.0% Private Private NGO Schools 0.2% Private Private Community Schools 2.7% Private Partially public Notes: Attached primary schools are "attached" to secondary schools, which are typically registered non-government schools, receiving public financing. Partial public financing generally includes full or partial payments for teacher's salaries. The government provides text books for all children. Source: (BANBEIS 2012, World Bank 2010) 1.13 Government and registered non-government schools are organized in six regional education boards that coordinate examinations and define the curriculum. The Madrassas Education Board, which operates under the Ministry of Education, carries out a similar function for madrassas schools. Registered non-government schools function like charter schools, largely financed by the government and operating with a high degree of autonomy. 2 This calculated by dividing the male adult literacy rate with the equivalent female adult literacy rate. 5 Each one has a school management committee that is responsible for hiring staff and the budget. Registered non-government schools generally have fewer certified teachers and there are other concerns about their quality (Japan Bank for International Cooperation 2002). 1.14 Secondary school follows primary education and typically has seven grades divided into three sections-junior secondary (three grades), secondary (two grades), and senior secondary (two grades). At the secondary level, an estimated 96 percent of students attend schools in a registered non-governmental school (either secular or madrassa), which are private with significant public financing (World Bank 2010). Challenges Facing the Education Sector in Bangladesh 1.15 Despite the progress that Bangladesh has made in reducing poverty and improving human capital, the primary education sector faced many challenges at the start of the Program. These were described in the government strategies (Government of the People's Republic of Bangladesh 2002, Government of the People's Republic of Bangladesh 2002). 1.16 Coverage. At the time of preparation, the coverage of primary education was high with broad gender and geographic equity (by division and rural-urban). One challenge that the sector faced was how to incorporate excluded groups, including urban slum children, children with disabilities, and children in remote areas including tribal (ethnic minority) children (Asian Development Bank 2003). While most children enter primary school, the repetition and dropout rates were high and many students never finish primary education. Some of these children are often enrolled in a NGO or community school (around 2.9 percent of enrollment); the quality of these schools varies widely (Ardt, et al. 2005). 1.17 Education quality. In addition to getting children to enter and graduate from primary school, the education system has to prepare children with the skills and competencies necessary for work and life. At the time of preparation, national assessments showed that most primary students Bangladesh did not reach minimum learning standards (World Bank 2008). Many factors contributed to the low level of learning. There were many untrained and unqualified teachers in the system and teachers were poorly paid and motivated. As a result, classes were usually delivered as lectures, with a heavy focus on the memorization of facts. In addition, there were delays in the receipt of textbooks and learning materials, which often arrived late in the school year. Many schools had serious infrastructure problems. While there had been several assessments of student learning, there was no system of monitoring learning on a consistent basis (Asian Development Bank 2003). 1.18 Education governance. The primary education system was quite centralized with most decisions made by the Ministry. Local education authorities largely focused on supervision. As a result, staffing decisions (hiring, training, and transferring) were made with little reference to local needs. The Ministry had a limited monitoring and evaluation (M&E) and analytical capacity which further reduced its ability to respond to local needs (Asian Development Bank 2003). Finally, there were a large number of development partners that were largely uncoordinated that imposed significant transaction costs on the government (Jennings 2007). 6 Box 1. The Government's Primary Education Strategy The Education for All: National Plan of Action II (2003 to 2015) is the government's overarching strategy for primary education (Government of the People's Republic of Bangladesh 2002, World Bank 2008). The strategy outlines twelve sector targets as well as describing the actions to be carried out by each of the three sub-sectors (pre-primary, primary, and non-formal education). The plan does not include a detailed implementation strategy. The total cost of the program from 2005 to 2015 was estimated to be US$3.6 billion in the development budget, which excludes most salary costs. In 2002, the government also prepared its primary educations strategy, the Second Primary Education Development Program 2003-2008 (commonly known as the Macro Plan) as a "coordinated and integrated sub-sector approach" (Government of the People's Republic of Bangladesh 2002). The Macro Plan was designed to support the government's National Plan of Action for Education for All and to serve as a coordination device for development partners. The Macro Plan has dual objectives to "...increase primary school, participation, and completion" and "...improve the quality of student learning..." In order to reach these objectives, the Plan proposed a combination of increased investment and reform to strengthen the sector's capacity.The Macro Plan establishes a number of minimum standards (known as Primary School Quality Levels). These standards cover teachers (the pupil-to-teacher ratio and the level of training and education), infrastructure (size of classrooms, presence of sanitary facilities), learning materials (availability of textbooks and teacher inputs), among others. These outputs were also outlined in the National Plan of Action for Education for All. The Macro Plan estimates the inputs required to reach these minimum levels, dividing the program into five components, each with an estimated five year budget: (1) Quality improvement through organizational development and capacity building (US$126.1 million); (2) Quality in schools and classrooms (US$631.5 million); (3) Quality improvement through infrastructure development (US$700.6 million); (4) Improving access to quality schooling (US$665.9 million); and (5) Implementation, management and monitoring (US$31.2 million). Development partners were expected to contribute US$600 million. This is consistent with the investment proposed in the National Plan of Action for Education for All for 2005 to 2010. The Macro Plan had a logical framework, which brings together the logical frameworks for each of the components. Each of the components had a detailed list of interventions and activities. The World Bank and Development Partners in the Education Sector 1.19 The World Bank has a long history in Bangladesh's education sector, starting soon after independence. Since independence in 1971, the World Bank has approved 36 projects in the education sector. The first project, the Agricultural and Rural Training Project, was approved in 1976 and focused on skills for the rural population. Currently, it has projects and programs in most areas of the sector, including primary, secondary, and tertiary education, as well as non-formal education. EDUCATION PROJECTS PRIOR TO THE PRIMARY EDUCATION DEVELOPMENT PROJECT 1.20 The Universal Education Project, approved in 1980, and the Primary Education Project, approved in 1985, appear to be the first projects focusing on primary education (Unterhalter, Ross and Alam 2003). In 1990, the World Bank approved the General 7 Education Project (P009514), with a US$159.3 million credit. This project received co- financing from four bilateral development agencies (the Netherlands, Norway, Sweden, and the United Kingdom) as well as three United Nations agencies. In coordination with the World Bank, the Asian Development Bank financed a parallel project, the Primary Education Support Project, for $68.3 million. In total, this coordinated sector intervention included $310 million in financing and provided support both the "hardware" and "software" side of primary education, with components focusing on construction and renovation, satellite school pilots, teacher training, curriculum and textbook development, and strengthening primary education system management (The Study Team 2009, Operations Evaluation Department 2008). Several development partners provided support outside of the coordinated framework, including the governments of Japan and Italy as well as the Islamic Development Bank, the Organization of Petroleum Exporting Countries, and the European Community. THE PRIMARY EDUCATION DEVELOPMENT PROGRAM 1.21 While the General Education Project was implemented as a traditional investment project, it increased interest in developing a Sector-Wide Approach (SWAP) for primary education (see Box 2). This led to the Primary Education Development Program (known as PEDP I), led by the World Bank. However, after a period of tense discussions, the government decided against establishing a SWAP and opted to continue with a series of coordinating programs as was done under the General Education Project. The Project Appraisal Development for PEDP I indicated that "...the Government was reluctant to adopt time-slice financing [a SWAP] and requested donors to finance discrete components within the sector program" (World Bank 1998). 1.22 PEDP I consisted of a cluster of 27 projects, both funded by the development partners and the government.3 Despite efforts to run PEDP I as a coordinated program, the consensus is that this was not successful. Although there were joint annual reviews, there was no overall operational plan and each individual project was free to determine its implementation strategy. In practice, each project coordinated directly with the government (Jennings 2007). There was no attempt to harmonize procedures (Ahmed 2011). 1.23 The Primary Education Development Program II (known as PEDP II) was designed to follow PEDP I. The government remained skeptical about undertaking a SWAP for primary education due to concerns about the Ministry's implementation and coordination capacity. At the same time, problems with on-going health sector SWAP4 contributed to the government's doubts (Mufad 2008). In this context, development partners decided to advance slowly with the introduction of a SWAP in PEDP II. At the same time, many other development partners indicated their unwillingness to continue their support for primary education without a SWAP (Jennings 2007). 3 Many of the government projects were, in turn, financed by the World Bank through reimbursement of the development budget. 4 The Health and Population Sector Program (P037857), approved on June 30, 1998. 8 1.24 The Education Local Consultative Sub-Group served as a coordinating body for development partners working in the education sector. In March, 2001 the Consultative Sub- Group discussed the appropriateness of a SWAP. "We need to rebuild trust in our collective working relations with [the government]. We can do this best by continuing to avoid SWAP jargon... This would include being truly receptive to opportunities for the achievement of a more comprehensive approach..." and "...we do not wish to press for a sector-wide approach for its own sake.. .not one prescribed by donors" (March 15, 2001 minutes cited in Jennings, 2007). 1.25 Within this context, the Asian Development Bank (ADB) begun to prepare its follow- on project to PEDP I. Through a project preparation technical assistance, the ADB supported the government in developing a strategy for primary education, the Second Primary Education Program 2003-2008 Final Plan which is commonly known as the Macro Plan (Operations Evaluation Department 2008). Given the ADB's role in supporting preparation of the Macro Plan and its status as an "early mover" in the design of a new primary education project, both the government and development partners agreed that ADB should be lead partner. This decision was driven more by circumstances than design (Jennings 2007). 9 Box 2. Sector-Wide Approaches As the name implies, a Sector Wide Approach (SWAP) is an approach to organizing development support for a specific sector (Kostermans and Geli n.d). While definitions of SWAPs vary, there seems to be a general consensus that a SWAP should encompass a number of characteristics, including (World Health Organization 2006, Ahmed 2011): * A clear sector policy and strategy; * A government-led process for donor coordination at the sector level; * A medium-term expenditure program and annual budget; * A performance-monitoring system to measure progress; * A process for moving towards harmonized fiduciary systems; and * A broad consultation mechanism that involves all stakeholders. A well-designed SWAP is expected to bring a number of benefits. First, it can contribute to reducing the transaction costs for the government as it eliminates partner-specific fiduciary rules and multiple implementation arrangements. Second, a SWAP can ensure consistency across projects to ensure that partners are not "cherry-picking" issues and are working together. Finally, a SWAP can built ownership since it mainstreams the government program and aligns external support to the government's program (Buse and Gwin 1998). In practice setting up a SWAP can require several cycles (World Health Organization 2006). For development partners, a SWAP may represent additional work as they spend more time coordinating both with the government and with other development partners. It can also be difficult for a development partner to explain its individual impact and attribution within the context of SWAP to their domestic stakeholders. How can SWAPs be evaluated? A recent approach (Vaillancourt 2012) proposes evaluating SWAPs against three broad criteria: * Establishment of better tools for sector coordination. The SWAP should help improve the relevance of strategies and expenditure programs. * Greater harmonization and alignment of development assistance. The SWAP should improve the coordination of the development partners and encourage the greater use of country systems. * Enhanced sector stewardship. Greater ability to measure results and accountability in the sector and give the government a central leadership role. These improvements should, in turn, facilitate a stronger and more efficient management of the sector leading to greater impact. 2. Objectives, Design, and their Relevance 2.1 Evaluating a SWAP can be complicated due to the large number of actors and the difficulty assigning individual attribution of the program's impact. For the purpose of this evaluation, the government's overall program (including the government financing and the program financed by the World Bank and other partners) will be referred to as the Program. The consortium of development partners financing through the SWAP is referred to as PEDP II or the SWAP. Finally, the share financed by the World Bank (through its International 10 Development Association) will be called the World Bank Program or the World Bank's contribution to PEDP II. Likewise, the share financed by the Asian Development Bank will be referred to as the ADB Program. In practice, the evaluation has to focus on outcomes that attributable to the Program, PEDP II, and the World Bank's Program. In this evaluation, primary education will be referred to as the primary education sector or the sector. The World Bank's Program Objectives 2.2 The legal agreement between the Government and the World Bank (the Development Credit Agreement) provides the Program's objective as (page 20): The objective of the Project which is an integral part of the [Second Primary Education Development Program], is to reduce poverty through universal primary education and to contribute to sustainable socio-economic development and equity as envisaged in the Millennium Development Goals, through the provision of quality primary education to all children in [Bangladesh]. 2.3 This objective was largely taken from the ADB's statement of the Program's long- term objective (Asian Development Bank 2003). For the ADB, this was not intended to be the Program's immediate objective but rather a statement of its longer term contribution. 2.4 The World Bank's Project Appraisal Document has a complementary objective drawn from the government's Primary Education Strategy, the Macro Plan (pages 4-5): The overall program objective is to improve quality, equitable access, and efficiency in primary education through a sub-sector program approach. Specifically, the program will assist the Government ofBangladesh to: (a) improve the quality of teaching and learning, and raise student achievement; (b) increase access to schooling for the disadvantaged; and (c) strengthen planning and management of primary education, including establishing a national monitoring and evaluation system for primary education. 2.5 In presenting the World Bank Program's results framework, Annex 1 of the Project Appraisal Document (Page 34) introduces an alternative formulation of the objective: Consolidate gains in education and equity and support [the government's] efforts to improve educational quality. 2.6 The objective in the legal agreement is not clear, mixing education outcomes, such as coverage and quality, with longer term objectives such as poverty and sustainable socio- economic development. While all three objectives are complementary, IEG's standard evaluation practice is to give precedence to the legal agreement. The World Bank's Guidance Note on Project Preparation states that: "If there are differences of views between the lawyer, other reviewers, and/or the Task Team Leader as to the consistency of the project documents with the negotiated legal agreement(s), the negotiated legal agreements(s) prevail absent a reopening of negotiations to address the discrepancy (paragraph 4 Resolution of Differences)." IEG policy uses the legal 11 2.7 Based on this analysis, the evaluation will focus on three specific objectives that are derived from the legal agreement and the PAD: (a) Improve the quality ofprimary education. The primary education system will be more effective in imparting knowledge and skills. Specifically, the Program will aim to improve the quality of teaching and learning and raise student achievement. (b) Improve equitable access to primary education. The Program will promote access to schooling for the disadvantaged in order to support Bangladesh's effort to reach MDG # 2 (universal primary education) and MDG #3 (no gender discrimination in education). (c) Improve the efficiency ofprimary education through a SWAP. Under this objective, the program aims to strengthen the planning and management of primary education, including establishing a national M&E system for primary education. PEDP II and the World Bank Program Design 2.8 The SWAP had four components, all of which were supported under the World Bank Program. Under each sub-component, a large number of activities were proposed. For example, the activity "Improve the capacity for full-time initial teacher training at the upazila (sub-district)6 level" under sub-component 2.2 includes: (i) reviewing the functions of the upazila resource centers; (ii) completing construction of resource centers; (iii) improve the system of recruiting staff for the resource center; (iv) piloting the introduction of computer and internet at the resource center; and (v) transfer funding for the resource centers from the capital budget to the revenue (current) budget. The components are described below. 2.9 Component 1: Quality Improvement through Organizational Development and Capacity Building (Total estimated cost: US$78 million; World Bank contribution: US$13.5 million). This component focuses on improving management and governance in the primary education sector at the central and local levels. It aimed at supporting investments which would result in improved educational management including good governance of the primary education system. It specifically had a focus on increasing authority and accountability at the district, upazila, and school levels while addressing fiscal and supervisory irregularities. agreement as is it benchmark and reviews complementary documents for clarification: "[When] the financing agreement's statement of objectives is unclear ... describe the outcomes and benefits that the operation was expected to achieve, drawing on the PAD or program documents.... Those intended outcomes will then be the basis for the evaluation." 6 Bangladesh currently has 7 divisions, divided into 64 districts. These districts, in turn, are divided into 500 upazilas and 509 thanas. An upazila is a unit of local government, usually translated as "sub district." In urban areas, sub-districts are called thana. Bangladesh currently has 7 divisions, divided into 64 districts. These districts, in turn, are divided into 500 upazilas and 509 thanas. 12 2.10 The component had four sub-components to enhance: (i) capacity of the Ministry of Primary and Mass Education and the Directorate of Primary Education; (ii) information management system capacity; (iii) field capacity at divisional, district and upazila levels; and (iv) organizational and management capacity at school level. Table 4. Sub-Components and Activities under Component 1 Sub-Component Activities 1.1 Enhancing Capacity of the * Functional analysis and programs to strengthen Financial Ministry of Primary and Mass Management, M&E, and Training Divisions Education and the Directorate of * Support human resource actions and training at the Ministry Primary Education * Better incentives for primary teachers 1.2 Enhancing Capacity of * Expand data collection to local and school levels EMIS * Incorporate the geographic information system database 1.3 Enhancing Local Capacity * Strengthen district level capacity to manage primary education * Strengthen upazila level capacity to manage primary education 1.4 Enhancing Organizational * Support the role of school management committees and Management Capacity at * Support the preparation of school level improvement plans School Level * Strengthen local data collection, planning and management Source: PAD, World Bank, 2004. 2.11 Component 2: Quality Improvement in Schools and Classrooms (Total estimated cost: US$510.8 million; World Bank contribution: US$28.1 million). This component is the core intervention in learning and education quality. It focused on specific quality interventions and strengthening institutions that influenced the quality of teacher training, including the National Academy of Primary Education, Primary Teaching Institutes, and Upazila Resource Centers. This was to be complemented by efforts to revise the curriculum and textbooks as well support to improve the delivery of textbooks. The curriculum reform included a move towards competence-based learning, which emphasizes the learning of skills instead of facts. The Program included a revision of textbooks to support new curriculum as well as efforts to improve the distribution of textbooks. 2.12 The three subcomponents focused on: (a) improving the learning environment through the provision of better quality textbooks and hiring an additional 35,000 teachers to reduce the pupil-teacher ratio and increase contact time to 750 hours per school year; (b) improving the quality of pre-service and in-service teacher training as well as training for head teachers; and (c) increasing community awareness to garner participation and support for raising quality standards, as well as establishing a school support fund to finance small investments in infrastructure, training, and other activities to improve the school. 13 Table 5. Sub-Components and Activities under Component 2 Sub-Component Activities 2.1 Improving the Learning 0 Strengthen the National Curriculum and Textbook Board Environment 0 Revise competency-based exams * Production, storage, and distribution of textbooks 2.2 Improving the Quality of 0 Strengthen the National Academy of Primary Education Initial and In-Service Teacher 0 Strengthen primary education training institutes Training and Support 0 Improve the capacity for full-time initial training * Improve teacher recruitment, incentive, and certification * Head teacher training 2.3 Increasing Community 0 Public awareness campaign Awareness (with parallel 0 Revise the role of school management committees funding from UNICEF) * Introduction of community-based construction * Establishment of a school support fund Source: PAD, World Bank, 2004. Component 3: Quality Improvement through Infrastructure Development (Total estimated cost: US$335.4 million; World Bank contribution: US$56.3 million). The main objective of this component is to provide adequate facilities conducive to good quality education through increased equitable access to infrastructure, facilities, and equipment. 2.13 The component was organized by activity groups rather than by sub-components. This component included all support for improving existing and providing new infrastructure, facilities, furnishings and equipment as necessary in government education offices, training institutes, resource centers, and schools. It also included support for construction supervision and hiring architecture and engineering firms. The main activities included: (a) classroom construction and renovations, based on a national survey of the status of school infrastructure; (b) water and sanitation facilities, including arsenic testing at schools and ensuring separate latrine facilities for girls and teachers; (c) primary teacher training institutes; (d) upazila offices and upazila resource centers; (e) physical infrastructure improvement at the Ministry and other education institutions. Table 6. Activities under Component 3 Activity Group Activities Classroom Constriction and r Minimum of two model schools constructed per upazila Renovation 0 30,000 new classrooms * Renovation of existing classrooms Water and Sanitation Facilities f 15,000 tubewells for drinking wells * 15,000 latrines Infrastructure for the Education l Renovate and upgrade all 53 primary teacher training institute System Renovation of upazila offices and upazila resource centers i Physical infrastructure for the Ministry Source: PAD, World Bank, 2004. 14 2.14 Component 4: Improving and Supporting Equitable Access to Quality Schooling (Total estimated cost: US$ 890.9 million; World Bank contribution: US$ 3.5 million). This component supported efforts to increase enrollment in primary education with a focus on disadvantaged children and children with special needs who had never attended formal primary schooling, or dropped out before completing Grade 5. The two sub-components aimed to: (i) enhance institutional capacity of the Primary Education Directorate at the central and local levels to promote inclusive education, and (ii) improve primary school access and retention for disadvantaged children through stipends for the poorest families and matching grants for food and dietary supplement programs, the establishment of pre-primary classes in rural primary schools, and innovations for tribal children in remote areas. Table 7. Activities under Component 4 Sub-components Activities 4.1 Enhancing Institutional * Establish posts for special needs education Capacity at Central and * Appoint special education specialist at the district level Decentralized Level * Appoint gender specialists in the Training Division 4.2 Improving Access for * Support for stipends to poor students Disadvantaged Children * Support for community-financed food supplementation * Establish kindergartens ("baby classes") Source: PAD, World Bank, 2004. PROGRAM ORGANIZATION AND FIDUCIARY ARRANGEMENTS 2.15 The development partners designated Asian Development Bank (ADB) as the lead partner in PEDP II. As the lead agency, ADB was responsible for coordinating the operational aspects of the program and serve as the primary interface with the government. ADB was responsible for releasing funds under its supervision and also provided advice to the World Bank regarding its disbursement. ADB was also responsible for reviewing audit reports and provided the final approval of these reports, following the advice of other partners. ADB would serve as the permanent deputy chair of the Education Local Consultative Sub-Group to ensure technical consistency. While the ADB was responsible for all operational decisions, the World Bank still maintained fiduciary independence as required by its own rules (Asian Development Bank 2003, World Bank 2004). 2.16 In the area of financial management, the Program introduced three important elements in its design. First, there was agreement that there would be a single external auditor, which would lower the cost of the audit. Second, the common audit would provide, for the first time, a single snapshot of the financial management of the sector. Third, the Program would introduce an internal audit to the Ministry. The Ministry introduced standardized financial management procedures and distributed the Financial Management Manual to all concerned stakeholders. It also established the Finance and Procurement Division to supervise the Program's adherence to the guidelines. The government's Foreign Aided Project Audit Directorate, supported by an independent audit firm, was responsible for conducting annual audits. The audit report would be submitted to ADB six months after the end of the government's fiscal year. ADB would issue the no objection. In addition, the 15 Program was expected to have an internal audit carried bi-annually using an independent firm. Initially, the Ministry had no capacity to carry out internal audits (World Bank 2004). 2.17 The implementing agency was the Primary Education Directorate in the Ministry of Primary and Mass Education. Working under a Program Steering Committee, the Director General of Primary Education would be the Program Director and would lead a Program Coordination Unit. In addition, development partners were to be supported by the newly created the Program Liaison Unit, which was organized by ADB but was designed to be functionally independent and would be located outside of the premises of ADB's resident mission (Mufad 2008). ADB also provided a separate technical assistance grant to support the Program's implementation (Operations Evaluation Department 2008). Unlike traditional investment projects, the implementation would be carried out by line directors and newly created the Management Information System Cell. The Program Coordination Unit, with staff financed by the Ministry, would focus on planning and financial management (Asian Development Bank 2003). 2.18 The government and development partners agreed there was still a need for traditional project supervision of procurement and financial management. Under PEDP II, four separate accounts were opened for the purpose of project financing. The ADB opened two foreign exchange accounts, one for the ADB credit and another for ADB-managed grants from other development partners. A third foreign exchange account was opened for the World Bank Program. Finally, the government opened a local currency account (World Bank 2004). As local currency resources were needed, they would flow into the local currency account. The World Bank and ADB would handle almost all of the foreign exchange purchases through their foreign exchange accounts.7 While fund flows were complicated, the government felt that this arrangement simplified disbursements compared to the previous situation with independent projects. The design also tried to avoid some of the pitfalls with the health SWAP, which included both pooled and non-pooled funds and multiple reporting systems (Mufad 2008). Table 8 shows financing by source. 2.19 The World Bank provided an International Development Association credit of US$150 million, making it the largest external contributor along with the government of the United Kingdom (DFID). As is typical with SWAPs, the bulk of the expected finance was from the government; in this case, 64 percent of the estimated US$ 1.62 billion. Figure 3 outlines financing by source, including government contribution. As is typical with education project, most of the financing was for local expenditures, which accounted for 89 percent of total financing. The non-pooling partners accounted for only around 1 percent of total financing. Foreign exchange expenditures accounted for $200 million and were entirely financed by development partners (World Bank 2004, Asian Development Bank 2003). The government was responsible for all taxes owed by the project (Mufad 2008). 2.20 A large portion of local procurement was carried using the public procurement framework, the Public Procurement Regulation of 2003, including most infrastructure and other local expenditures (Asian Development Bank 2003). International procurement and larger local procurement packages would use the guidelines of either the ADB or the World 7 A small amount would be managed directly by non-pooling partners. 16 Bank, depending on the nature of the good. For example, most international consultants were to be paid using ADB financing and guidelines while paper and textbooks were to be purchased using World Bank financing and guidelines (Jennings 2007). In developing the Program, the government and ADB agreed on a detailed procurement plan including expected dates, size of procurement packages, and disbursement categories that is reflected in the ADB's program document (Asian Development Bank 2003). Table 8. Financing of PEDP II by Source Initial Commitment Final Commitment Partner (US$ Million) (US$ Million) The Government of Bangladesh 297 380 Development Banks Asian Development Bank 100 110 World Bank 150 145 Pooling Partners (funds managed by the ADB) Canada 20 53 The United Kingdom 150 112 The European Commission 100 116 The Netherlands 50 46 Norway 40 36 Sweden 29 34 Other Financing Partners Australia (through UNICEF) 12 27 Japan 3 5 Total 951 1,065 Source: (World Bank 2011). 2.21 The World Bank's legal agreement had only two disbursement categories: (i) "Eligible local expenditures under AOPs" (approximately two-thirds of World Bank financing) and (ii) "ICB goods" (approximately a third of World Bank financing). The first category financed good included in the Annual Operation Plan. The government and development partners agreed on the Annual Operation Plan as a guide for expenditures. Through this category, the World Bank financed provision of equipment, apart from larger purchases that required international competitive bidding, technical advisory services, and operational expenses and recurrent costs. It allowed the World Bank to pool financing with ADB-managed funds and was supervised by the ADB. The second category financed goods purchased through international competitive bidding and was supervised by the World Bank, without pooling of resources. 2.22 At the time of preparation, the World Bank expected a relatively smooth disbursement of funds, with US$24 million disbursed in the July 1, 2004-June 30, 2005 fiscal year (FY2005), representing around 16 percent of the total. Subsequent years were expected to be in the range of US$18 million to US$30 million (World Bank 2004). Although the 17 ADB program document does not have an estimate of disbursement, its implementation schedule is consistent with the World Bank's estimates and is based on a relatively fast start of the Program (Asian Development Bank 2003). Relevance of Objectives 2.23 The relevance of the objectives of the World Bank Program is rated high, reflecting its close alignment to the government's strategy and the central role that the Program plays in the World Bank's country strategy. RELEVANCE TO BANGLADESH 2.24 The objectives outlined in the project document and the legal agreement reflected the educational needs of Bangladesh well. This included filling the gap in enrollment with a focus on disadvantaged groups. The objective also focused on improving quality, which remains a major challenge facing the primary education sector. Finally, the objective focused on improving the administration of the primary education, which is still a major challenge facing the sector. 2.25 PEDP II was guided by the Macro Plan. The SWAP was consistent with a number of relevant documents, including laws, development strategies, and education strategies (Government of the People's Republic of Bangladesh 2002). The SWAP's objective matches those of the Macro Plan. The World Bank's legal agreement made specific reference to the government's strategy to meet the Millennium Development Goals as well as the Macro Plan. This remains the case as the current government strategy makes reference to the Program and outlines the need for more work along the same line (World Bank 2011). RELEVANCE TO THE WORLD BANK 2.26 At the time of preparation in 2003 and 2004, the Country Assistance Strategy Progress Report identifies the need to increase the number of students that graduate from primary school and to improve the quality of primary as major challenges that the country faced. The Progress Report also explicitly referenced the Program, although not in great detail, possibly because the World Bank was not leading the Program (World Bank 2003). At the time that the Program closed, in 2011, the World Bank's Country Assistance Strategy continued to identify the importance of improving education quality while ensuring that students graduate. The Strategy explicitly identified the Program as the center of the World Bank's strategy in primary education (World Bank 2010). Relevance of Design 2.27 The relevance of the World Bank Program design is rated modest, reflecting a credible causal chain combined with an overly detailed list of activities that limited flexibility. The World Bank adopted the same design as the Macro Plan and the ADB's program. The Program had a fully developed logical framework that was developed for the Macro Plan (see Box 1) and was used by the ADB in its program document. The World Bank took this logical framework to develop its results framework. For the first two objectives, the 18 design included all of the necessary elements while there are some elements missing from the third objective. The Program was carefully designed, with detailed activities and costing. While this strengthened the credibility of the causal chain, it limited flexibility and weakened the possibility of strategic planning. TECHNICAL DESIGN AND CAUSAL CHAIN 2.28 The first objective was to increase the quality ofprimary education. The design focused on quality from several different angles. Following the proposal in the Macro Plan (see Box 1), the Program was built around establishing minimum standards for facilities, learning materials, and staff and developing techniques to measure the results, through components 1, 2, and part of component 3. Efforts to improve the quality are likely to lead the groundwork for improved learning and primary education quality. Teachers are generally considered the center of the "education production function" and the Program included significant investment in teacher training, teacher recruitment, and the teacher training system. This was complemented by efforts to improve other inputs such as the curriculum, textbooks, resources for schools, and supervision. The indicators in the results framework were reasonable given the state of the sector during preparation. The logical framework did not include targets for learning or education quality; this is not surprising since the Program planned to strengthen and reform the learning assessment system, which barely existed when the Program began. 2.29 The second objective was to improve equitable access to primary education. The design (component 4 and part of component 3) focused on reaching marginalized population groups.8 This included targeted scholarships, improving infrastructure to attract and retain students, as well as programs focused specifically on vulnerable children (indigenous children and children with disabilities). The scholarship program aimed to raise the demand for primary education and took into account the research and impact evaluation in scholarships and conditional cash transfers in Bangladesh and other countries. Others actions focused on lowering the effective economic cost of primary education since it costs less for children to attend closer schools. The indicators in the results framework were well designed. 2.30 The third objective was to improve the efficiency ofprimary education through a SWAP. For this objective, the program design was somewhat weaker. The program included elements to strengthen the capacity of the Ministry (for example, establishing M&E and internal audit units) as well as setting up a structure to coordinate development partners. The ADB was given the responsibility for managing the administrative process on the development partners' side. PROGRAM COVERAGE, DESIGN, AND PLANNING 2.31 After much discussion, the government requested that non-formal education not be included in the SWAP. The formal division between formal and non-formal primary 8 The PAD defines disadvantaged groups as children poor families, children with disabilities, and children living in tribal areas (World Bank 2011). 19 education was well-established in the government system. The government actively promoted non-formal and "second chance" education to focus on hard to reach children through a different department in the Ministry (Unterhalter, Ross and Alam 2003). As a result, a number of development partners, led by the World Bank, continued to fund non- formal primary education through other projects in what has been called a "two-pronged approach" (IOB 2011, DFID Bangladesh 2011). While this could be considered a shortcoming, it reflects the political reality. 2.32 The World Bank's program document identifies several sector issues, related to access, quality, and finance. While the Program addresses most of these issues, it does not provide much of a focus on the lack of transparency in the allocation of resources and an unbalanced distribution of these resources, with 97 percent of the revenue (current) budget for salaries and benefits and 3 percent for materials and other critical inputs. The program included two indicators that measured the increase in spending on primary education although it does not appear that the Program had the means to track or encourage these increases. 2.33 Given the concerns of the government about its capacity and the concerns of development partners about the transparency of public procurement and financial management, the decision to use a "project approach" as opposed to directly reimbursing the government for its expenditures or providing budget support was reasonable. While the use of multiple accounts and different procurement rules was complicated, it represented an important advance from the multiple systems in place prior to PEDP II (Mufad 2008). It also used government systems for a significant portion of procurement. 2.34 The Program had very detailed and carefully costed plans and activities. While this was consistent with government budget procedures, it limited the Program's flexibility. The high level of planning limited the sector's capacity to learn from experience. Given the importance of new activities and innovation in the Program, this undermined the potential impact and design. 3. Implementation 3.1 The Program's implementation can be divided into three periods: (i) the initial phase, which covers the period after the World Bank Program's approval (2004) and before the mid- term review (2007); the mid-term review itself and its immediate aftermath; and (iii) the Program's final phase, running from the mid-term review to the formal closing in 2011. The initial phase saw significant problems as both the Ministry and development partners introduced a SWAP, which required changes in both staffing and procedures. In response, the mid-term review introduced some major changed aimed at addressing these shortcomings. The final phase saw a substantial improvement on a number of fronts due to the increased attention from the government and the restructuring of the Program. 20 Initial Phase 3.2 The ADB's Program was approved by its Board of Director November 3, 2003 and became effective on April 19, 2004. Since ADB was the lead partner, it was responsible for administering most of the co-financing. The World Bank Program was approved by its Board of Executive Directors on February 24, 2004 and became effective on May 24, 2004. This was slightly later than the expected date of March 15, 2004. The ADB's Program was designed to be completed by December 31, 2009, with the final utilization of funds before June 30, 2010, which is essentially the same period as the World Bank. The Program Liaison Unit was created with financing from ADB's technical assistance budget, with a coordinator appointed in March, 2004. Bangladesh was hit by a major flood in August 2004, which affected around 60 percent of the country, causing an estimated US$ 7 billion of damage (BBC News 2004). As the Ministry turned its immediate attention on rebuilding school, the Program's implementation was effectively delayed. 3.3 There were a number of advances during the first years of implementation. The Program led to a significant reduction in the number of projects operating under the Directorate of Primary Education, with an estimated twenty projects gradually consolidated into PEDP II. The government argued that this led to a reduction in its transaction costs working with development partners (Mufad 2008). By the time of the first Annual Review (May, 2005), the government had started to establish the institutional framework for coordinating the SWAP. Most notably, the Ministry established its Program Coordination Unit and the Finance and Procurement Division. 3.4 The initial implementation was slower than expected. In the view of the government, the financial aspects of the Program were implemented one to two years behind schedule (Mufad 2008). The Program received its first advance (from ADB administered grant funds) in August, 2004, with further advances from the ADB and World Bank Programs in October and December, 2004. With the low level of spending, there was no need to reimburse spending during the first fiscal year (FY2004). By June 2005, only 6.7 percent of World Bank funds were disbursed compared to an initial projection of 16 percent. 3.5 The lack of staff capacity was a major concern during the first years of implementation. While the Program Coordination Unit was created relatively quickly, there were delays in appointing technical personnel combined with frequent changes in personnel. Likewise, development partners expressed similar concerns about the staff in the newly- created Finance and Procurement Division.9 3.6 By many accounts, the government's understanding of the Program remained a problem during the first years of the Program (Mufad 2008). Although there was agreement on the objectives and the activities, there was a lack of understanding of the sector-wide approach (Directorate of Primary Education 2009). This led to delays in contracting key specialists for the Program Coordination Unit. Likewise, there were differences in the understanding the meaning of quality in primary education. Development partners linked the 9 This was discussed during the Inception Mission in 2004 and the first Joint Annual Review Mission in 2005. 21 quality objective to learning outcomes. In contrast, the government was more focused on physical quality. Since this is an important element of PEDP II, this was a major concern during the first years of implementation. 3.7 The Ministry developed a procurement plan in the first year. However, progress on procurement was slow at first although not severely so. Cumulative disbursement in the first two fiscal years (FY2003-2004&FY2004-2005) was 8 percent of total Bank financing (around US$11.9 million) compared to a planned 10 percent (US$15 million). While most procurement was done on a pooled basis (with the World Bank contributing a fixed percentage of the total amount), the World Bank did provide 100 percent of the financing for some specific procurement packages requiring foreign exchange. For the procurement of textbooks, the government requested a onetime waiver of procurement guidelines. The World Bank did not grant this request. 3.8 The broad outline for the disbursement of funds was developed during the preparation of the Program and the World Bank started disbursing in third quarter of 2004 (calendar year, equivalent to the first quarter of fiscal year 2005). However the harmonized rules for disbursement were not finalized until 2005. The first audit was returned with a qualified opinion and while it took a while to resolve the qualification, the general impression was that the audit did not raise substantial concerns. The Midterm Review 3.9 Although some progress had been made by 2007, the consensus was that PEDP II needed some major mid-course adjustments. In addition to a greater focus on M&E (see below), there were concerns about the capacity of the ADB's Liaison Unit. Although participants felt that the Ministry had developed a greater understanding of the sector-wide approach, internal records show that development partners had some concerns about the level of Ministry ownership of the Program. The mid-term review was carried out from October 29 to November 12, 2007 with a wrap-up meeting held on December 2, 2007. Given the broad nature of PEDP II, the review included participation from a large number of government and development partner representatives, divided into six working groups. 3.10 In some areas, such as civil works, there had been advances and the Program was advancing fairly well. A large number of staff (19,000) in the primary education sector had been transferred from the development (capital) budget to the revenue (recurrent) budget. Around 24,800 new teachers were recruited in the primary education sector, against the target of 35,000 new teachers in the Program. Approximately 45,000 teachers had received one year teaching certificates and another 19,000 were undergoing training. However in other areas, such as the development of M&E and of an education assessment system, the Program saw little progress. One of the major focuses of the review was on efforts to speed up the implementation of the "stuck" elements, building on a review made of the Program in 2006. 3.11 The government and the development partners agreed to several changes during the mid-term review. There was an agreement that the key performance indicators needed to be adjusted but there no actual agreement on the necessary changes (these were changed after the mid-term review). Related to this, there was an agreement to contract an independent 22 reviewer to improve the quality of analysis. At the same time, the partners agreed that the Program Liaison Unit would be integrated into the ADB's country office. This responded to concerned about the Unit's design and its unclear mandate (Operations Evaluation Department 2008). After the Midterm Review 3.12 After the midterm review, the pace of implementation improved substantially. The consensus, from interviews and internal documents, is that the government increased its focus on the Program. At the same time, it appeared that the World Bank and ADB had largely resolved institutional differences and worked collaboratively. 3.13 The quality of M&E improved substantially. The Ministry introduced the Primary Completion Examination and continued to improve the National Student Assessment. The Annual School Census was' mainstreamed, which enhanced the focus on quality and equity. Development partners reported that the Ministry had sufficient capacity to carry out the Program, particularly with senior managers although there were still problems with middle- level managers. At the local level, the government introduced a change in teacher recruitment and expanded the coverage of pre-service training. 3.14 By most accounts, implementation was relatively smooth and relatively well coordinated among the partners. Central to the supervision was a comprehensive Annual Sector Performance Review, which was designed to be an analysis of the Program's advance against its logical framework. Although the Review was coordinated by ADB, the World Bank played an active role, focusing particularly on education quality. 3.15 The World Bank closed its Program on June 30, 2011. During the same year, development partners developed the Third Primary Education Development Program (PEDP III)o for US$5,860 million, with US$609.4 million of external financing (of which the World Bank provided US$300 million). Unlike PEDP II, PEDP III did not have a formal lead partner and used a rotating chair. Most resources were placed in a common pool, simplifying the fund flow. PEDP III uses performance-based financing extensively. 4. Achievement of the Objectives 4.1 In a SWAP, a number of development partners work together towards a joint objective under a government strategy. Given this context, the evaluation of the efficacy will focus on the attribution of the overall Program's impact on the objectives. When appropriate, the evaluation will identify the contribution of the World Bank-financed Program and the SWAP on reaching the objectives. In practice, it is difficult to identify the specific contribution of any one development partner as most procurement was jointly financed and 10 The World Bank's portion of the Third Primary Education Development Program (P 113435) was approved on August 25, 2011, became effective in December 2011, and is expected to close in December 2014. 23 most technical assistance was jointly commissioned. The Table 18 in Annex B summarizes some of the Program's key performance indicators. Objective 1: Improve quality of primary education 4.2 This objective is rated substantial, with increased learning assessments, better teacher policy, improved curriculum and distribution of resources leading to increasing learning outcomes. The Program strengthened the learning assessment system, which increased the awareness of education quality. The Program also supported increasing the minimum qualification to become a teacher while it strengthened the teacher training system, focusing on improving subject knowledge. The Program supported efforts to expand single- shifts schools, increasing the learning time. Finally, the Program improved the quality and delivery of textbooks. These actions are typically an important of the causal chain to improve education quality. Initial evidence shows that primary students are learning more. There has been a general increase in the pass rate of the grade 5 Primary Completion Examination. The National Student Assessment also shows that there has been progress in learning within the primary education cycle, with students increasing their learning from grade to grade. WAS THERE MORE Focus ON QUALITY? 4.3 The Program introduced a number of instruments to measure student learning in the classroom, leading to a greater focus on learning and quality. Prior to the Program, it was not possible to measure trends in quality. NGOs had carried out several learning assessments to help identify learning gaps in primary education. The only government assessment available at the primary level was the Scholarship Examination. This was a test given to about 20 percent of student, identified as the top students, to determine eligiblity for a merit scholarship. Under the Program, the Ministry introduced the Primary Completion Examination for students finishing grade 5 in 2009. This census-based examination can be used to evaluate the performance of teachers and schools in addition to serving as a high- stake examination for students to advance to secondary school. 4.4 Since its introduction, the Examination has been given to almost all graduating primary student regardless of the system. The results are distributed to local education authorities, schools, teachers, and parents. Interviews suggest that the Primary Completion Examination helped to raise the importance of primary education in the mind of parents and education providers. Teachers now feel pressure to ensure that all grade five students are ready for the examination. Previously, teachers focused on those students that were selected to take the Scholarship Examination. Interviews suggested that is common for grade five to receive additional hours of classroom contact to prepare them for the examination. As argued below, the examination may have also contributed to the changing pattern of primary school dropout (more dropouts in grade 5 and fewer dropouts in previous grades). While this may have the unanticipated negative effect of strengthening the focus on "factual learning" or recall in the short run, the Examination has also highlighted the importance of education quality in a tangible way. The gradual introduction of competence-based questions, which started after the Program closed in 2012, should help to reduce this tendency. 24 4.5 With support from the Program, the Ministry also introduced a sample-based evaluation of learning for students in grades 3 and 5, the National Student Assessment. This was the first attempt to create a long-term institutional instrument. In many education systems, sample-based evaluations are used to provide education authorities with detailed information that is used to develop new curriculum and learning materials. The National Student Assessment was carried out in 2006, 2008, and 2011 (Government of the People's Republic of Bangladesh 2012). Since this was a new instrument, changes were made to the design as the Ministry piloted different approaches. The assessment evaluates the student's knowledge of Bangla (Bengali) and mathematics. Although the three different assessment pilots were all competency-based and measured problem-solving skills, they are not compatible (Government of the People's Republic of Bangladesh 2012). According to current plans, the 2011 National Student Assessment will be used as the model for future assessments. As part of the pilot process, the 2011 assessment was designed to be nationally- representative. Future assessments will provide greater geographic representation (Government of the People's Republic of Bangladesh 2012). DID TEACHER PERFORMANCE IMPROVE? 4.6 The Program supported increased teacher training and improved the way teachers are recruited, which is likely to lead to improved teaching. High quality teachers are critical for learning. Evidence suggests that much of the variation in learning is due to the variation in teacher quality. Studies have shown that there are long-term positive impacts of having a high quality teacher, such as a lifetime increases in income (Hanushek and Rivkin, The Distribution of Teacher Quality and Implictions for Policy 2012, Chetty, Friedman and Rockoff 2011). At the same time, there is no straightforward way to measure the quality of teachers or of teaching. A review of how to measure teacher quality showed the need for multiple indicators, monitoring of learning outcomes, and possibly observation of teachers and students. Likewise, creating high quality education involves working with teaching training programs, a strong recruitment system, and continuous human resource development (Nelson 2012). These elements can be captured by a robust accreditation or certification system, which incorporates formal education and training, on-the-job experience, and direct observation. 4.7 Even on a sample basis, the country does not yet have the tools to measure teacher quality. The country is in the process of developing a credible teacher accreditation system, although this is likely to take some time. This evaluation uses the available data on teacher quality and look at the Program's impact on several inputs into the production of high quality teachers: recruitment, pre-service training, and in-service training. 4.8 The Program made a substantial contribution to improve how teachers are recruited and deployed at the primary level. It directly supported the introduction of merit-based teacher selection. Prior to the reform, much of teacher selection was done based interviews which, in practice, were neither fair nor competitive. The weight of examinations in selecting teachers was increased under the Program. The Program improved the Ministry's capacity to register and track primary teachers. It also contributed to improving the transparency of contracting teachers (Wescott and Breeding 2011). This contributed to an increase in the number of new teachers with university degrees. By 2011, around 50 percent 25 of new teachers had university degrees (World Bank 2011). While there is no formal baseline, this appears to be a significant increase in the level of education of primary teachers. In 1995, although most government primary teachers have two year qualification, only 13 percent of primary teachers had university degrees (World Bank 2000). 4.9 The Program supported a number of activities to increase the capacity of incoming teachers. In particular, the sector aimed to increase the number of teachers with at least the pre-service Certificate in Education. From 2005 to 2011, the share of teachers (including head teachers) with the Certificate increased from 76 percent to 85 percent. It is likely that at least some of these gains were due to the Program as it appears that most of these gains happened between 2009 and 2011. However, without historic data before the Program, it is difficult to draw strong conclusions about the Program's contribution. Evidence also suggests that some new teachers are still able to enter the system without this training (Government of the People's Republic of Bangladesh 2012). The follow-on program, PEDP III, is continuing to improve the quality of pre-service teacher training building on research and reforms carried out during PEDP II (World Bank 2011). 4.10 The Program also supported efforts to improve the quality of teacher training institutes. The National Academy of Primary Education, the lead agency responsible for research and development in primary education, became an autonomous organization and received investment to improve its infrastructure and equipment. This would allow the Academy to control its own budget and make human resource decisions on its own, which is a major reform given the centralized nature of the education sector. In addition, the Program supported the rehabilitation and expansion of all of the Primary Teacher Training Institutes, in addition to supporting additional training for instructors (World Bank 2011). 4.11 The Program invested substantially in in-service training. Under the Program, there was a substantial increase in the proportion of teachers who received the full five-day subject-based training, increasing from around 30 percent in 2005 to around 80 percent in 2011. The proportion of teachers receiving training on pedagogy has remained relatively stable at around 40 percent (Government of the People's Republic of Bangladesh 2012). Overall, the Program financed the training of 429,000 teachers (World Bank 2011). 4.12 There is evidence that the teacher contact time in primary education has been increasing due to the Program. Ensuring that students have adequate instructional time is a key element in improving quality, particularly for low performing schools (Mourshed, Chijioke and Barber 2010). The Program has supported efforts to create single shift schools, where students can attend school the whole day instead of in a half-day shift. In 2005, only 12 percent of government schools were operating on a single shift. By 2011, this had increased to 21 percent. There is indirect evidence that the number of contact hours has increased, largely due to the increase in full-time schools (Government of the People's Republic of Bangladesh 2012). 4.13 Teachers are only effective if they are at school with students. There has been little systematic tracking of teacher absenteeism in Bangladesh but there are a few stylized facts that come from research and studies. First, at any time about one-sixth of teachers are likely to be missing from school, with half due to short absences (for example, training and 26 sickness) and half for long term absences (for example, maternity leave and courses). Most schools are big enough to manage by shifting around the workload. Second, official statistics show that unexcused teacher absence is rare (Government of the People's Republic of Bangladesh 2012). Third, in addition it appears to be relatively common for teachers to arrive late, particularly in rural areas (Government of the People's Republic of Bangladesh 2012). International literature shows that the teacher absenteeism has a negative impact on learning (P. Glewwe, et al. 2011). There is no evidence that teacher absenteeism changed during the Program. DID TEXTBOOK DISTRIBUTION IMPROVE? 4.14 The Program contributed to a substantial improvement in the delivery of textbooks to students. Traditionally, textbooks arrived late to students, often in the last three months of the school year. The Program focused on ensuring that textbooks arrived in a timely fashion and at no cost to the student. The primary education sector in Bangladesh saw substantial improvement in the distribution of textbooks. Table 9 outlines the timeliness of textbook distribution. Administrative records also show that in 2011, all students in primary education had free access to a copy of all of the needed textbooks during the school year (Government of the People's Republic of Bangladesh 2012). Although these indicators were included in the Program's M&E from the start, the development partners did not establish a formal target value. The government's target was for free textbooks to be available for all students. The World Bank indicated in its PAD that the goal was to have textbooks available in the first week of school. This was obviously aspirational and was not tracked during the life of the Program nor reported in the ICR (World Bank 2004, World Bank 2011). Table 9. Timeliness of Textbook Distribution School Year Some textbooks by January All textbooks by March 2005-06 18o 7400 2008-09 6000 91%o 2011-12 98% 98% Source: (Government of the People's Republic of Bangladesh 2009, Government of the People's Republic of Bangladesh 2012). 4.15 The evidence on the impact of improving access to textbook is generally positive but depends on the context. Overall, most studies show that the provision of textbooks and other pedagogical material has a positive impact on learning in developing countries. However, this requires that textbooks be distributed to students and to be of an appropriate level for the recipients. Unlike many countries, textbooks in Bangladesh are prepared in the native language (Bangala). The use of foreign "official" language, such as English or French, has been associated with low impact for textbooks (P. Glewwe, E. Hanushek, et al. 2011, Sabarwal, Evans and Marshak 2012). Likewise, the Program invested in efforts to improve the quality and relevance of textbook and is linking them to the ongoing curriculum reform. Thus, it is reasonable to assume that improving textbook distribution will have a positive impact. 27 WHAT CHANGES WERE SEEN IN LEARNING OUTCOMES? 4.16 The data show some medium term improvements in learning outcomes. The Primary Completion Examination provides a census measure of learning starting in 2009. Although the 2011 National Student's Assessment is not consistent with the previous assessments, its design allows some inference about trends in learning. 4.17 Results on the Primary Completion Examination show a positive trend. The Examination was introduced in 2009 for students in grade 5. The Primary Completion Examination included Bangla (Bengali), Mathematics, English, Social Science, General Science, and Religious Studies. The Examination is scored from 0 to 100 per subject, for a total score between 0 to 600. The design of the Examination has not changed since its introduction and virtually all students in the country present the exam, allowing consistent comparisons across schools, school systems, and geographic regions. On an aggregate level, the Ministry only releases the percentage of students that pass the examination." The pass rate has been increasing at a time when the coverage of the examination has also increased. Although the pass rate is relatively high, there are significant differences across the country and by socio-economic groups. The original design had a number of indicators focusing on achievement in the Scholarship Examination (World Bank 2004); since this examination was eliminated and replaced with the Primary Completion Examination, these indicators were not followed. Table 10. Evolution of Pass Rate for the Primary Completion Examination Total Number of Exams Percentage Pass for Percentage Pass of Eligible Year Presented Participating Students Students 2009 1,979,895 88.8% 81.8% 2010 2,207,197 91.2% 80.9% 2011 2,457,918 96.7% 90.0% 2012 2,757,492 96.9% 89.9% Source: (Government of the People's Republic of Bangladesh 2009, Government of the People's Republic of Bangladesh 2012). 4.18 The results from the National Student Assessment suggest students are increasing their learning from grade to grade. The Program's design included the development of a sample-based student assessment by 2009; the first version of the National Student Assessment was carried out in 2006. The 2011 National Student Assessment was designed to measure the increase in student learning between grade 3 and grade 5. Thus the assessment graded the performance of students using a common measurement scale (Government of the People's Republic of Bangladesh 2012). While it would be expected that children would increase their learning between grades 3 and 5, there is no guarantee that this will be the case and there are many examples when virtually no learning happens in school (Boone, et al. 2013). The results show that there was significant learning between grades 3 and 5 in both Mathematics and Bangla. The mathematics score of the median (50th percentile) student in grade 5 would be at about the 94th percentile in grade 3 (Government of " The reports to schools and parents include the overall score. 28 the People's Republic of Bangladesh 2012). Some students drop out between grade 3 to grade 5 and it is reasonable to assume that the better students tend to stay school. However, school dropout is not large enough to explain the increase in learning. Objective 2: Improve Equitable Access to Primary Education 4.19 This objective is rated substantial as the enrollment rate has increased for primary education with an increase in internal efficiency as well as successful efforts to maintain gender parity and include poor students and children with disabilities. The Program did not track the progress of Hill Tribe children, however its actions almost certainly led to an increase in their enrollment as well and secondary statistics also suggest increasing enrollment for these children. 4.20 The Program's design aimed to expand infrastructure in areas where coverage was low. In practice, new schools were constructed around the country, with all regions benefiting equally. The Program improved the physical infrastructure of schools, which makes them more attractive to students. The Program financed stipends that were reasonably well targeted to poor students and were likely to have been effective incentives. All of these initiatives were likely to increase the demand for primary education and keep children in school longer. 4.21 The dropout rate decreased, particularly in the lower grades and the repetition rates have generally stayed the same. Both of these suggest that students are more willing to stay in school in order to finish at least a few grades in the primary education cycle. Thus the Program was largely successful, particularly in keeping students in school for a long time. Most of the gains were in public and mixed public-private schools that were supported by the Program. DID PHYSICAL ACCESS INCREASE FOR ALL STUDENTS? 4.22 Overall, the Program supported investment in schools and classrooms in poor areas, although the focus could have been improved. In this context, increasing access means efforts to reduce the effective cost of primary education and efforts to make the education system more "attractive" to students. Access for students with disabilities is discussed below as it was given special attention. 4.23 One of the Program's main focuses was to strengthen infrastructure, including building classrooms in remote areas and ensuring that schools met minimum standards. The original target for classroom construction was 30,000 and the Program supported the construction of 40,440. In the design, priority was given to areas with limited access defined as: (i) remote areas (school is difficult to reach or far from the upazila center); (ii) underserved populations (groups that are considered to be underprivileged, particularly living in difficult areas); and (iii) areas with ethnic minorities (largely in the Chittagong Hill Tracts). With exception of the focus on ethnic minorities, these priorities not well-defined. It appears that in practice, investments were not targeted in these three areas and the evidence suggested the different regions in the country benefited more or less equally (Government of the People's Republic of Bangladesh 2012). While the construction of new schools was not 29 focused on high priority areas, the evidence suggests that it did help to increase the number of adequate classrooms in these areas. Thus while the Program did not have the desired pro- poor design, poorer areas certainly benefited and given their greater need for new infrastructure, this investment was a positive development. 4.24 The Program contributed to improving the standards of classrooms. The Program's primary school standards defined the characteristics of a properly constructed classroom as: (i) built with durable materials; (ii) sufficiently large; (iii) well-lit and ventilated; and (iv) accessible for students with disabilities. Most public schools (around 90 percent) are built with durable materials and this did not change during the Program. Around 51 percent of classrooms in public primary schools are sufficiently large, of which 46 percent were constructed under the Program. 4.25 Another consideration that affects the accessibility is the presence of toilets, especially separate toilets for boys and girls. While the data show that most schools have at least one toilet (around 98 percent for public primary schools), the phrasing of the question is inconsistent. At the time the Program closed, 51 percent of schools had separate toilets for boys and girls. Reports suggest that this represented an increase due to the Program (Government of the People's Republic of Bangladesh 2011). WERE STIPENDS PROVIDED FOR POORER STUDENTS? 4.26 One of the main activities of the Program was the provision of stipends to poor primary students. While this activity began before 2005, the Program has expanded its coverage. In 2011, there were approximately 7.6 million recipients compared to 4.3 million in 2005; coverage increased from 28 percent to 40 percent of total primary enrollment (Government of the People's Republic of Bangladesh 2012). According to operational rules, the recipient of the stipend must meet at least one of five conditions: (i) being landless of nearly landless; (ii) having a parent who is a day laborer; (iii) belonging to female-headed household; (iv) living in a household that derives its income from fishing or several artisanal occupations; or (v) living in a household that derives its income from sharecropping (Baulch 2010). 4.27 Initial evidence suggests that stipends were reasonably well targeted although there were some leakages to non-poor students. Estimates of these leakages vary and many are based on data from before the Program (Government of the People's Republic of Bangladesh 2011). While there have not been any rigorous studies on the impact of the stipend program, it is likely to have a positive effect. The stipend program replaced the previous Food for Education Program that was established in 1993. The Food for Education Program provided grain to the family of students in primary education. Careful studies have shown that this program had an impact on enrollment and on reducing dropouts (Sukontamarn 2010, Meng and Ryan 2010). This effect appears to be stronger for more "marginal" students. In other countries, there is ample evidence that conditional cash transfers have an impact on keeping students in school. In countries where enrollment is high, like Bangladesh, these programs can help keep children in school (Akresh, de Walque and Kazianga 2013). 30 DID ACCESS INCREASE FOR STUDENTS WITH DISABILITIES? 4.28 For the first time in public education in Bangladesh, the Program started to mainstream inclusive education and increase access for students with disabilities. Bangladesh, like most low-income countries, provides very limited services for students with disabilities. The Program highlighted the needs of students with disabilities in primary education and included this in the Program objective. Even students with moderate disabilities are at a disadvantage in school and are more likely to abandon school (World Health Organization & World Bank 2011). Many students with disabilities are simply excluded from the education system while others are segregated from the regular education system. At the same time, many teachers have negative attitudes about students with disabilities (Malak 2013). The Program made two important contributions: (1) increasing attention on the needs of students with disabilities and (2) increasing investment to equip facilities for students with disabilities. 4.29 A module on inclusive education was included in the standard curriculum offered in the Primary Training Institutes. Likewise, inclusive education is being incorporated into in- service training that happens at the local level (Malak 2013). The Program also supported efforts to include facilities schools for students with disabilities as part of the definition of minimum standards. Schools had to report the number of students with disabilities they had in the Annual School Census12 as well as report on the facilities that the schools had for students with disabilities. That requirement focused the attention of schools and local education authorities on the presence of students with disabilities (Government of the People's Republic of Bangladesh 2012). The Program's primary school standards include accessible facilities for students with disabilities. While many of the newly constructed classrooms meet these standards, it appears that many head teachers did not understand the definition (Government of the People's Republic of Bangladesh 2012). 4.30 A UNICEF review on inclusive education in Bangladesh argues that the Program made an important contribution in improving the status of students with disabilities (Hossain n.d). [The Program] has brought to [Ministry] the language of social inclusion and a heightened awareness and understanding of its importance where it barely existed before...... It has begun to take the first steps on the long road of bringing special needs children into schools. It has built thousands of new primary schools that incorporate ramps for use by children with disabilities. It has begun the task of supporting capacity development across [the Ministry] in terms of social inclusion issues and activities, including in monitoring and evaluation. 4.31 The number of students with disabilities in both Government Primary School and Registered Non-Government Primary Schools increased from 46,000 in 2005 to 91,000 in 2012 (Government of the People's Republic of Bangladesh 2012, Government of the People's Republic of Bangladesh 2006). While this is likely to be a small proportion of all children with disabilities, which is estimated to be about 6 percent of the population of children or 12 The school census divided disabilities into physical, visual, hearing, speaking, and cognitive. 31 around 990,000, it represents an important start that is rarely seen in low income countries (Danish Biharziasis Laboratory 2004). It exceeded the coverage target of 59,000 students with disabilities. HOW AND WHY DID INTERNAL EFFICIENCY IN PRIMARY EDUCATION CHANGE? 4.32 Overall, the internal efficiency of primary education in Bangladesh has improved. There is a higher completion rate and more children are finishing the earlier grades.13 4.33 The coefficient of efficiency measures the ratio between the actual numbers of pupil- years to the ideal number of pupil-years needed to reach a given grade (UNESCO Institute for Statistics 2009, Government of the People's Republic of Bangladesh 2009). A completely efficient system would have a measure of 100 percent, with a lower rating indicating less efficiency.. In 2005, the coefficient of efficiency was 60.6 percent and it took an average of 8.1 years of pupil-years to complete the five year cycle. By 2011, the coefficient had increased to 69.1 percent with an average of 7.2 pupil-years to complete the education cycle. This met the Program's target. 4.34 It is likely that the stipends contributed to the increase in internal efficiency. Since repetition has dropped, it does not appear that students have stayed in school longer just to receive the stipend. Changes in quality are also likely to have played an important role in increasing internal efficiency. The Program's actions contributed to an increase in learning (see above) and as well as providing more information through the Primary Completion Examination and the school management committees. Improvements in the quality of teachers will lead to a decrease in repetition as students are better able to take advantage of the time in school. This is consistent with evidence that education quality has led to an improvement in graduation in different countries (Hanushek, Lavy and Hitomi 2006). 4.35 Economic factors also contribute to the internal efficiency in the education sector. Changes in income and wages can have an unexpected impact on enrollment. While education is a normal or luxury (superior) good (demands increases as income increases), an increase in wages may drive children out of school and into work. In Bangladesh, it appears that the increase in wages in the garment industry has led to an increase in enrollment for girls (Health and Morbarak 2012). 4.36 Repetition dropped in most grades but the Program did not meet all of its targets. A major source of internal inefficiency is grade repetition. This could be due to poor schools results or extended absence from school. The Program set targets of reducing the repetition to 10 percent or less in grades I to 4 and to 5 percent in grade 5. For grades 1 to 4, 13 In the education sector, internal efficiency refers to the optimal number of school years per graduating student. In other words, a student is expected to graduate from fifth grade in five years. Common measurements of internal efficiency include the repetition rate, the dropout rate, as well as aggregate indicators of enrollment (UNESCO Institute for Statistics 2009, Government of the People's Republic of Bangladesh 2009). 32 the changes in repetition were small and none reached their targets. Grade 5 saw a significant decline in repetition and met its target. Table 11 shows the repetition rates. Table 11. Repetition Rate by Grade Repetition Percentage Grade 2005 Target for 2011 Actual Results in 2011 1 12.3 10.0 10.7 2 11.0 10.0 10.3 3 13.7 10.0 14.2 4 11.4 10.0 13.5 5 5.7 5.0 3.5 Source: (Government of the People's Republic of Bangladesh 2012). 4.37 During the Program period, the graduation rate at the primary level increased and largely met its target. Early departure from school has been major problem in Bangladesh, even at the primary level. While most children attend primary education, many never graduate. In 2005 the cycle completion rate was estimated at 52.8 percent, meaning that only about half of students that entered grade 1 graduate. By 2011 completion rate had increased substantially to 70.3 percent. While there are different measures of completion, the general trends remain the same (Government of the People's Republic of Bangladesh 2012). 4.38 Table 12 shows the specific dropout rate per grade in 2005 and 2011. The Program set a target of a reduction of 2 percentage points per year for grades I to 4. The overall trends show a clear reduction in grades 1 to 4, although the targets were not met. The baseline for grade 5 is not clear, with some data sources reporting no dropout for this grade. While this is not possible, the general consensus is that the dropout rate for grade 5 has increased over time, possibly substantially. Thus the new pattern seems to be that dropout in the early grades has been effectively reduced and most children that drop out do so in grade 4 and 5. The overall dropout rate decreased from 47.2 percent to 39.7 percent, which is close to the target of 37 percent. This is an improvement since it means that more students graduates and that students that do drop tend to have more education than primary level dropouts in the past. 4.39 This change in dropout patterns may partly result from the introduction of end-of- cycle examination, with teachers encouraging students to stay until school until the examination. Despite the increase in dropouts in grade 5, the overall impact is a reduction in the total dropout rate and an increase in the average education level of children. 33 Table 12. Dropout Rate by Grade Dropout Percentage Grade 2005 2011 1 12.9 4.1 2 8.8 3.0 3 13.4 4.4 4 16.0 7.4 5 n.d. 11.1 Overall 47.2 39.7 Source: (Government of the People's Republic of Bangladesh 2012). WHAT WAS THE PROGRAM'S IMPACT ON COVERAGE? 4.40 By 2011, Bangladesh made impressive progress towards the goal of universal primary education. Measuring the evolution of enrollment rates in Bangladesh is not straight forward. The most common measures of enrollment, the net enrollment and the gross enrollment rates, both require a reliable numerator (the number of students enrolled) and denominator (the number of youth in a given age group). Estimates of rates vary depending on their source and what indicator is used. 4.41 The Program had increasing enrollment as a goal, in an effort to reach universal primary education. Both gross and net enrollment became key performance indicators that were monitored regularly as part of the annual review. For the gross enrollment rate, the Program put a target of 98 percent; the actual figure was 108 percent. Likewise, for the net enrollment rate the Program put a target of 90 percent; the final rate was 95 percent. Figure 2 shows the gross and net enrollment for primary education as monitored by the Program. Figure 2. Net and Gross Enrollment Rates in Primary Education 115 110 105 100 95 90 875 2005 2006 2007 200B 2009 2010 2011 - Net Erollment Rate -Gross Erollment Rate Source: (Government of the People's Republic of Bangladesh 2012). 34 4.42 The above data depends on the quality of the underlying data, both the administrative data that reports the number of students and population data. In 2005, there were 16.2 million students in grades 1 to 5 and by 2011, this number increased to 18.4 million. At the same time, the number of children between the ages of 6 and 10 decreased from 17.3 million to 16.6 million (Government of the People's Republic of Bangladesh 2012). An alternative estimate, from the United Nations Population Division, has the number of children from 2005 to 2012 has basically remained stable (Government of the People's Republic of Bangladesh 2012). A second alternative estimate can be made from the Demographic and Health Surveys, which show that the share of children between ages 5 and 9 decreased from 12.8 percent in 2004 to 12.2 percent in 2011 (NIPORT, Mitra and Associates, and ORC Macro 2005, NIPORT, Mitra and Associates, and MEASURE DHS 2013). With some assumptions, it is possible to estimate the size of the population cohort. Table 13 presents the estimate of gross and net enrollment rates under these two sets of assumptions about the population size. With the adjusted data, the trend remains the same, with an increase in both net enrollment and gross enrollment rates. The evidence suggests that Bangladesh is definitely moving in the direction of the universal primary education. Table 13. Primary School Enrollment Rates with Different Population Estimates Based on United Nations Population Based on Demographic and Health Estimates Surveys GER NER GER NER 2005 93.7 87.3 97.3 90.6 2011 106.4 93.3 106.2 93.1 Note: GER=Gross Enrollment Rate; NER=Net Enrollment Rates Source: Author s Calculation, based on Government of Bangladesh 2012 and NIPORT, etc., 2013. 4.43 Program data show that girls have slightly higher enrollment rates than boys although the difference is small (Government of the People's Republic of Bangladesh 2012). Survey data corroborates the general trend-Table 14 shows education coverage statistics from the Demographic and Health Surveys.14 Overall, there are no significant differences between the enrollment of boys and girls or between the rural and urban populations. Statistics from the World Bank's Education Statistics Database suggest that gender equality was reached in the 1990s. Evidence from the Demographic and Health Survey also show that this gender equality exists at the secondary level as well-in 1996, girls accounted for 49.8 (out of a total enrollment rate of 65.6 percent) of enrollment and in 2007, this share was 56.9 percent (out of a total enrollment rate of 71.8 percent). 14 These are estimated using the coverage rate, which differs from more commonly used measures and tends 35 Table 14. Education Coverage from the Demographic and Health Surveys 1993 1996 1999 2004 2007 2011 National Coverage at the Primary Level 73.7 75.8 79.3 84.4 86.3 87.7 Coverage of Girls 72.5 76.3 80.3 86.2 88.0 88.6 Rural Coverage at the Primary Level 73.9 75.5 79.2 84.9 86.4 87.9 Coverage of Girls 72.7 76.1 80.6 86.9 88.3 88.8 Note: data refers to the percentage of children between the ages of six and ten who are in school. The result is likely to be lower than the gross enrollment rate. Sources: (Mitra, et al. 1997, NIPORT, Mitra and Associates, and ORC Macro 2001, NIPORT, Mitra and Associates, and ORC Macro 2005, NIPORT, Mitra and Associates, and Macro International 2009) and authors calculations. 4.44 To what extent did the actions of the Program supported the increase in primary education? In the absence of the Program, would education enrollment have increased by the same amount? Since the Program was national in scope and included interventions that affected private education as well as public education, it can be difficult to assign attribution. To answer these questions, it is necessary to understand if the enrollment was driven by an increase in coverage by the private sector and thus not attributable to the Program. 4.45 Most of the increase in enrollment took place in schools that were covered by the Program. While the coverage of private education has increased, it remains quite small and focused on the non-poor population where coverage of primary education probably already universal. In 2005, there were 158,000 students in private schools", which represented 0.9 percent of all primary students. In 2011, this increased to 223,000 or 1.2 percent. 4.46 Another potential source of the enrollment growth comes from NGO or non-formal primary schools. These schools focus on hard to reach children and children that have dropped out early. Due to their nature, it is hard to track the growth of these schools. Students may move between the formal and non-formal sector several times. Many NGO schools focus on older children as "second chance education" which further complicates the comparison. In 2011, the Program estimated that around 2 percent of total enrollment was in NGO schools (Government of the People's Republic of Bangladesh 2012). Although it appears that the coverage of these schools has increased, it is unlikely that the overall growth in enrollment could have been due to the increase in coverage by non-formal schools. 4.47 Most madrassa schools are publically financed and are not under the control of the Ministry of Primary and Mass Education. In 2005, around 2.0 million students (12.3 percent of total enrollment) were enrolled in various types of madrassa schools. By 2011, the number of students had dropped to 1.1 million students (5.7 percent of enrolled students). This change in enrollment may be due to the increased availability of government and NGOs schools (Asadullah, Chaudhury and Josh 2009). 1 Private schools here are defined as "non-registered non-government private schools." Registered non-government schools received substantial support from the government and are explicitly covered by the program. 36 DID POOR STUDENTS BENEFIT FROM THE PROGRAM? 4.48 Poorer students particularly benefited from the Program. Given the relatively high level of enrollment when the Program began, most of the increase in enrollment occurred among both the urban and rural poor. The ICR reported the enrollment rates by income quintile, based on the Household Income and Expenditures Surveys in 2005 and 2010 (World Bank 2011) and shows a significant increase in enrollment for the poor and narrowing of the gap between the richest and the poorest quintiles. This is shown in Table 15. 4.49 The Program supported the construction of new classrooms (although not necessarily new schools) in poorer areas of the country. This is likely to have improved the access for the poorest students. In addition, efforts to improve the quality are likely to have the biggest impact on the poorest students and schools and also lead to an improvement in school participation. Probably the most important activity was the expansion of the primary student stipend. Although the targeting was not perfect, it did largely benefit the poorest students and was an important element to encourage students to stay in school. Table 15. Enrollment by Income Level Year 20 Percent Poorest 20 Percent Richest 2005 58% 80% 2010 72% 81% Source: World Bank, 2011. DID CHILDREN FROM HILL TRIBES BENEFIT FROM THE PROGRAM? 4.50 There is evidence suggesting an increase in primary education coverage in tribal areas. Collecting data from the Chittagong Hill Tracts is notoriously difficult and many development projects have had difficulties documenting their impact in this part of the country. While the availability of data appears to be improving, it remains difficult to measure the impact of the Program on these areas. 4.51 The Program's Indigenous People's Plan identified the net enrollment rate of 57 percent in the Chittagong Hill Tracts, with a gender parity of 0.90. There is wide disparity among the different tribes, ranging from 12 percent to 57 percent, while the Bangla population has a rate of 61 percent. This is based on data collected in 2001 (World Bank 2004). A survey carried out by the Chittagong Hill Tracts Development Facilities estimated that in 2008 around 65 percent of children are in primary school (Human Development Resource Centre 2009).16 The survey did not report any noteworthy difference by gender. Using data from around 2010 or 2011, UNICEF reported a net enrollment rate of 82 percent in the Hill Tracts, with a gender parity of 1.0; during the same year, the national net enrollment rate was around 95 percent (UNICEF n.d.). Data from the annual school census in 2010 shows a similar trend, with the net enrollment rate estimated to be 93 percent and a 16 These results exclude the population that did not respond to the question. Although this measure differs from the net enrollment rate, it should be broadly consistent with that measure. 37 gender parity of 1.04.17 Although there are questions of data quality and compatibility, the overall picture is one of increasing coverage of the Hill Tribes and a reduction or elimination of gender inequality. Objective 3: Improve the efficiency of primary education through a SWAP 4.52 This objective is rated substantial; by the end of the Program, the Ministry had better tools to supervise primary education, coordination was strengthened, and the government had greater stewardship of the sector. 4.53 The conceptual model outlined in Box 2 provides a framework to evaluate the objective. This objective will be evaluated against three criteria: (i) the establishment of better tools for sector coordination; (ii) greater harmonization and alignment of development assistance; and (iii) enhanced system stewardship. There were no key performance indicators on this objective and the analysis here is largely qualitative. WERE BETTER TOOLS FOR SECTOR COORDINATION DEVELOPED? 4.54 The Program improved coordination, under the Ministry's leadership. The Program Coordination Unit was designed to support the Director General and to focus on tasks that were related to the management of large SWAP. It effectively replaced the project management units of approximate twenty projects. While there is a long history of jointly financed projects as well as previous attempts to develop a SWAP, this was the first successful SWAP. The Development Partner Consortium played an important role in coordinating technical work among the partners as well as permitting a forum to discuss operational and fiduciary issues. The Macro Plan combined aspirational goal with a well- developed and costed roadmap that made the investment Program quite transparent. WAS THERE GREATER HARMONIZATION? 4.55 The Program reduced the transaction costs for the Ministry and aligned different development partner's systems. PEDP II represented an important evolution from PEDP I, moving away from projects and multiple project implementation units. The creation of the Program Liaison Unit responded to the development partners' desire to work through a single entity and to have a single representative. The Macro Plan and the ADB's program document carefully identified and costed the different activities to be financed under the Program. Although ADB rules allow expenditures to be adjusted, this was a shortcoming. By its nature, a sector program needs flexibility. 4.56 Although costs have not been quantified, the government reported that it generally saw a reduction in its overall transaction costs (Mufad 2008). In its concept note for PEDP III, the Ministry stated that "[h]armonisation among donors has significantly reduced transaction costs (in terms of both time and money). Where [donors] have harmonised their procedures... .this has been appreciated" (Directorate of Primary Education 2009). The ADB's evaluation of the education sector argued that the Program lowered the transaction 17 Calculated by the World Bank from unpublished Annual School Census data. 38 costs for the government while it increased costs for development partners collectively, particularly the transaction costs within the development partner consortium. WAS THERE ENHANCED SECTOR STEWARDSHIP? 4.57 By the end of the Program, country ownership was high. At the start of the Program, the Ministry had a low level of ownership (Mufad 2008). With time, the Ministry learned how to manage the new approach and it gradually developed a greater sense of ownership, appreciating its advantages in terms of reduced cost and increased coordination (Mufad 2008). The Ministry's concept note for PEDP III made ample reference to PEDP II as increasing ownerships and helping the Ministry "mainstream" its relationship with development partners instead of seeing the relationship as temporary (Directorate of Primary Education 2009). Interviews with development partners on the ground suggest that the government has a greater sense of "being in the drivers' seat" and is more confidant in coordinating the sector. An independent evaluation by the ADB (Operations Evaluation Department 2008) makes a similar argument. 4.58 The Program strengthened the Ministry's capacity to lead the sector. Before the Program began, the sector was organized as a division within the Prime Minister's office. The Program supported the newly created Ministry in a number of areas that are needed to improve coordination. This included developing a new M&E which eventually had significant analytical ability and was able to produce a high quality report (the Annual Sector Performance Review) as well analyze other data sources, such as the school census and a number of student assessments. Component 1 included a number of initiatives to strengthen the Ministry's capacity. The Program was incorporated within existing structures with the implementation carried out by the Director General of Primary Education and the relevant line directors. The Program rightly focused on strengthening human resources in the Ministry, including professional development for staff and filling vacancies. The organization of the public sector in Bangladesh was an important external constraint, as the Ministry of Primary and Mass Education did not have its own cadre of staff. Staff members generally have to seek promotions outside of the Ministry and thus the turnover is high. The Program planned to address this issue by creating a cadre within the Ministry that would allow staff members to advance within the sector. 4.59 Component 1 also included efforts to strengthen monitoring and evaluation (M&E) from a rather low level. At the Program's start, the Ministry of Primary and Mass Education had very little monitoring capacity and the Ministry of Education only collected minimal data on primary education. The Program planned to support both the central capacity to analyze data as well as the capacity of local authorities and schools to both collect and analyze data. This is consistent with the comments received from the internal review. While the World Bank's PAD and the Macro Plan were consistent in identifying the importance of M&E, the program document does not provide much detail of what the necessary steps are to strengthen M&E on the ground. The Ministry carried out an annual school census but had no capacity to analyze it. By the Program's end, the Ministry was carrying out an improved annual school census as well as an end-of-cycle examination. In addition, the Ministry published a series of indicators that were available for the system as well as at the local and school levels. This is discussed in greater detail in Section 6 of this PPAR. 39 4.60 The Program also strengthening the local level's capacity in primary education. Bangladesh is often described as having the world's largest unitary school system and traditionally has been highly centralized. The Program introduced a number of activities to change the relationship between the local level and schools and to empower schools. 4.61 The Program enhanced the role of the upazila education officer to collect data for the new M&E system. Under the Program, the upazila became a center of in-service training as well as support to teachers and head teachers. The Program supported the initiative at the upazila level that brought together teachers to work with more experienced peers on specific subject matters. In system that has traditionally focused on veridical relationships, this is major change. At the same time, schools received support to develop school-level improvement plans (called SLIPs). Given the highly centralized nature of primary education, these initiatives represent significant changes in how education is delivered. 5. Efficiency 5.1 Efficiency is rated as substantial as the Program was able to deliver good value to Bangladesh by reducing transaction costs while increasing the future prospects of primary school graduates. During the Program, the Ministry only had to deal with a single consortium instead of a large number of development partners. While the Program still had significant administrative costs, it appears to be less than existed in the past. There were few reported cases of corruption. At the same time, estimates of the Program's external efficiency (calculated by a cost-benefit analysis) show that the Program made a positive economic contribution. Administrative and Transaction Costs 5.2 Overall, the SWAP provided good value for Bangladesh. The SWAP directly led to the elimination of a number of smaller projects that the government considered to be an administrative burden. According to discussions with present and former government officials, it contributed to a reduction in transaction costs for the government as it created a common approach for procurement and disbursements as well as a single reporting and monitoring framework. Interviews and comments show that there was a reduction in overall "transaction-related" work. Government officials also reported that they did not experience any "downsides" to the SWAP and have enthusiastically supported a follow-on program. This is reflected in the Ministry's concept note for PEDP III that notes the improvement in harmonization leading to a reduction in the transaction costs (Directorate of Primary Education 2009). 5.3 Under the SWAP, most of the administrative burden was transferred to the Asian Development Bank and, to a lesser extent, the World Bank and the rotating chair of the Primary Education Consortium (Operations Evaluation Department 2008). At times, it appeared as if the ADB was not able to keep up with the work requirement, particularly at the beginning of the implementation process (Ahmed 2011). Interviews also suggest that other development partners increased the amount of time that they spent on coordination. It is not 40 possible to quantify whether the overall burden on development partners increased or decreased under the program. 5.4 The Program did suffer from delays, particularly during the first phase. The delay in the program's implementation prior to the mid-term review represents an inefficient use of resources. However these delays did not appear to be significant in terms of resources disbursed or the overall implementation of the Program. According to the World Bank internal reports, disbursements were largely as expected, with the World Bank program only falling behind by one to three months, with the biggest delays coming later in the Program. The project was extended by one year, which is common for World Bank-financed activities in Bangladesh and elsewhere. At the time that the Program was closed, the World Bank cancelled around US$5 million of its credit. This appears to be related to exchange rate gains as the credit was denominated in Special Drawing Rights. 5.5 Spending more time on preparation could have strengthened the Ministry and its capacity to implement the Program, which, in turn, could have accelerated implementation. However, this would not have replaced the important "learning-by-doing" process that was necessary for the Ministry to adapt to the new approach. The long preparation time and delayed implementation period were probably necessary costs to start the SWAP. 5.6 There were some issues with transparency and governance, but these did not pull the Program off track. Transparency and governance has been a concern for Bangladesh and has affected its international competitiveness (U4 2012), which has raised concerns for the development community in the country (Wescott and Breeding 2011). There were a number of allegations made, particularly in the first years of implementation. A review of civil works led to a declaration of misprocurement in 15 civil works contracts, leading to the World Bank cancelling approximately US$130,000 of the credit. Interviews showed that the World Bank and other development partners were quite active in reviewing audit reports and contracts and often brought their concerns to the ADB or to the government. 5.7 IEG's report on governance in Bangladesh reports favorably on the Program's impact on transparency in the hiring of teachers: During the 2006-09.. .period the Bank's investments in reforming teacher selection were successful. Teachers are now selected through a merit-based process. Improvements in teacher registration and recruitment have resulted in more transparent allocation of teachers to schools. This has reduced opportunities for corruption in teacher deployment... (Wescott and Breeding 2011). Teachers account for the bulk of resources spent on primary education and improvements in the efficiency of teacher contracting are likely to represent a significant savings. External Efficiency 5.8 An ex-post evaluation of economic benefits estimates the Program's rate of return from 16 to 23 percent. The Asian Development Bank and the World Bank worked 41 together developing an economic analysis of the Program's impacts. The PAD (World Bank 2004) provides an excellent overview of this analysis and the underlying assumptions that were used. The Program's ICR contains a new analysis, updating the original work using actual data. The original analysis focused on both the primary education sector's internal efficiency (the sector's capacity to educate more children with the same resources) as well as its external efficiency (the value of education for individuals once they leave school). 5.9 The initial analysis in the PAD is based on the assumption that the Program could improve both the internal efficiency (leading to more graduates) and learning (leading to better prepared graduates). Specific assumptions were included population growth, expected changes in the enrollment rates, reductions in repetition and dropout rates, and increases in education quality due to Program interventions that in turn leads to an increase the in private return to education. Using these assumptions, the economic analysis estimated a net present value of the Program (US$1.22 billion) and an internal rate of return (13.9 percent). The economic analysis also tested the assumptions ("sensitivity analysis") to provide a range of alternative net present values and internal rates of return. Under the low set of assumptions, the impact was reduced by 50 percent and under the high set of assumptions, the impact was increased by 50 percent. Table 16 provides the range of estimates for the ex ante rate of returns. Table 16. Estimated Ex Ante Internal Rates of Returns Assumptions about External Efficiency Assumptions about Internal EfficiencyLoMeimHg Low Medium High Low 6.1% 13.7% 20.4% Medium 6.1% 13.9% 20.7% High 6.4% 14.2% 21.1% Source: (World Bank 2004). 5.10 Given the high level of primary education coverage in 2005, the analysis shows that the estimated rate of return is largely driven by the external efficiency of primary education. At the time of appraisal, the estimates showed that even with a low set of assumptions, the Program would have a positive economic impact. 5.11 The ICR recalculated the rate of returns based on end-of-the-project data, including data on the number of students that completed primary education. The actual internal efficiency result was between the medium and high assumptions. The ICR made assumptions about the external efficiency of the Program, which took into account the probability of employment and the impact of education quality on wages. With these assumptions, the Program's impact ranges from a low estimate with an internal rate of return of 16 percent and a net present value of US$2,216 million (lower probability of employment, low wage premium for quality) to a high estimate with an internal rate of return of 23 percent and a net present value of US$5,472 million (higher probability of employment, high wage premium for quality). The ICR's estimates are close to the high end of the estimates in the PAD. This is broadly in the range of estimates of the rate of return for primary education interventions (Shafiq 2007). 42 5.12 The assumptions in both the PAD and ICR were reasonable and are based on common approaches used in economic analyses of education projects. However, since the rate of return estimates only focused on the project's benefit to the wages of individuals, they underestimate the economic impact and the efficiency in several areas: * Improving the coverage and quality of primary is likely to lead to better outcomes in secondary education, with more students entering and successfully graduating. It should also increase the quality of secondary education. This will have a larger impact on employment prospects and wages. * Macroeconomic literature shows a clear relationship between education and economic growth that goes beyond the impact of education on wages. This includes impacts both on labor productivity that are not captured by individual workers and through its impact on innovation. 6. Ratings Outcome 6.1 With most of the relevance, efficacy, and efficiency ratings are substantial or better, the Program is rated as satisfactory. This reflects a World Bank Program that was well aligned with the government's objectives and with the primary education SWAP. The design was practical and was the product of the government's and partners' learning from experience. After many of the initial shortcomings were resolved and the government's ownership of the Program increased, the Program's impacts were significant at both the intermediate and the final levels. Finally, the pooling of resources and cooperation by the partners led to important efficiency in implementation. 43 Table 17. Project Outcomes Outcome Rating Remarks Relevance Relevance of High The World Bank Program's objectives are closely aligned with Objective the government's objectives and the World Bank's country strategy. Relevance of Design Modest While the Program's targets were ambitious, it laid out a logical causal chain. The monitoring and evaluation framework was adequate for the first two objectives but not the third. An overly detailed list of activities and inputs reduced the Program's flexibility. Efficacy Objective 1: Improve Substantial The Program supported tracking of quality as well as incentives primary education for schools to focus on quality. It also supported quality quality initiatives, including teacher recruiting and training, the dis- tribution of texts. Teacher contact hours have increased. The initial signs show an improvement of learning. Objective 2: Improve Substantial The program supported a variety of efforts that contributed to equitable access to increasing education efficiency and supported the country's primary education efforts to reach universal primary education by 2015. Objective 3: Improve Substantial The program provided significant support to strengthening the efficiency of M&E. In addition, the program supported the creation of a primary education number of new units that improved capacity at the central and through a SWAP local levels. Efficiency Substantial For the World Bank, the investment in the Program provided good value for the money. Although the Program's implementation was initially slow, this picked up and its resources were effectively utilized. Project Outcome Satisfactory Risk to Development Outcome 6.2 The risk to development outcome is rated as significant as the program faces both political risks and the possibly that senior civil servants will leave the Ministry. 6.3 Despite changes in the control of government during the preparation and implementation, there was broad political support for the Program and its initiatives. Regardless of the political party, the government has traditionally given primary education a high level of priority. For many years, the sector was placed directly under the Prime Minister's Office. After the Ministry of Primary and Mass Education was created, the sector continued to receive high-level of attention with the Prime Minister acting as the Minister. 6.4 In Bangladesh, the transition from one government to another is often fraught with violence and can involve a reversal of previous policies. The Civil Service serves to maintain 44 continuity as government control changes. In the case of the Program, this has helped maintain a technical focus at a high level. 6.5 Following the closing of PEDP II, another Program (PEDP III) was approved. The new SWAP received a similar level of commitment from development partners and built on the activities of PEDP II. Many of aspects of PEDP II have been institutionalized, such as the Primary Terminal Examination, the primary student stipend program, the competitive selection of teachers, and the increased role of M&E at the school, local, and national level. While these initiatives may be modified in the future, it seems quite likely that they will remain in place regardless of future changes in the government. Likewise, the government has always provided the necessary financing for primary education and this is unlikely to become an issue. 6.6 Beyond the institutional support, continued success of the Program requires that the Ministry maintains continuity with its human resources. Unlike in several other ministries, the Ministry of Primary and Mass Education does not have its own dedicated cadre of civil servants, which makes career advancement difficult and tends to weaken staff commitment to the Ministry. There is some evidence that at the end of PEDP II, many key staff members left the Ministry, leading to significant skill-gaps (World Bank 2011). This may put some accomplishments at risk and it may blunt or slow down future reform. Bank Performance 6.7 This section will address the specific performance of the World Bank in this process (the World Bank's value added). The overall Bank performance is rated moderately satisfactory reflecting strong support to supervision that balanced some initial problems during the preparation phase. QUALITY AT ENTRY 6.8 Quality of entry is rated moderately unsatisfactory. 6.9 The World Bank was largely absent from the original discussion of the primary education SWAP. The Program was the second attempt to develop a SWAP for the primary education sector. The first attempt, PEDP I, was not successful as the government decided not to support a SWAP. As development partners were preparing for the next phase, the consensus was that the multiple project approach had failed and that further assistance should be done through a SWAP. The Primary Education Consortium developed a strategy to convince the government to accept a SWAP. The World Bank was not an active participant in this process. 6.10 The preparation process was complicated, given the presence of different development partners and the government's concerns about the benefits of a SWAP. The process advanced slowly and in the end it was decided that several elements of the process could be finalized after the Program was approved and underway. While there is certainly room for setting up a process that encourages "learning by doing," too many elements were not fully designed (Operations Evaluation Department 2008). 45 6.11 The World Bank made important contributions to improve the preparation process. Since the development partner's Program was developed jointly, it is difficult to identify the value added of any particular partner. The ADB coordinated the work. According to interviews and internal documents, the World Bank played an important role strengthening the results framework, particularly related to the collection of baseline data. The World Bank also worked closely with the ADB to develop a good quality economic analysis that was used by both Banks. The Indigenous Peoples' Plan was prepared with support from the ADB and was reviewed by both Banks. 6.12 The World Bank contributed to the overall Program through strengthened fiduciary safeguards. Although most expenditures were to be financed jointly by both Banks (the ADB using its own credit and a large trust fund), the World Bank carried out its own fiduciary review to ensure that all procurement was done in a manner consistent with World Bank guidelines. In the case of internationally competitive bidding, the World Bank would lead on the procurement of goods since the ADB and the World Bank guidelines are not consistent. 6.13 At the time of preparation, the World Bank's ownership of the Program seemed weak. Interviews and documents at the time showed that there was tension between the World Bank and other development partners. This was largely based on concerns that the World Bank was not working at the same pace as the government and other development partners. Since the World Bank was not active in the initial preparation of the Program, it appeared to be slowing the process (Jennings 2007). The World Bank was also criticized for working to establish a parallel project to be financed by the Fast-Track Initiative/Education for All that would exist outside of the pooled arrangement in the SWAP and cover some of the same elements. This project was eventually dropped. The World Bank also publically circulated the internal minutes of its quality enhancement review that were critical of the process. This, in turn, caused concern among development partners that the World Bank was essentially "second guessing" decisions that had already been made by consensus (Jennings 2007). This led to a high-level meeting of World Bank and ADB sector management in Manila in April 2003.19 Much of the World Bank's criticism could have been handled better in a private context. QUALITY OF SUPERVISION 6.14 Reflecting the dedication that the World Bank gave to support and improve the quality of the Program, the quality of supervision is rated satisfactory. 6.15 Both the Asian Development Bank and the World Bank posted internationally- recruited sector specialists in Dhaka, a first for both organizations. From project approval to closing, the World Bank had three task team leaders as well as several technical and fiduciary 18 The Quality Enhancement Review was held on March 24 and March 28, 2003. The World Bank circulated the minutes to the ADB and other development partners on April 7, 2003. 19 Internal documents and interviews suggest that while this meeting helped reduce the tension between the World Bank and ADB, other partners were concerned because of their exclusion from the meeting. 46 staff based in Dhaka. Government officials and development partners indicated in interviews that the World Bank's staff were technically skilled. The World Bank's internal Quality Assessment Group acknowledged the team's hard work. It commended the World Bank's solid supervision with its focus on resolving implementation problems and implementing a complex agenda in financial management and safeguards. 6.16 Despite the friction between the World Bank and others, the World Bank played an important and positive role prior to the midterm review. As noted above, significant additional work was needed after the official program launch. The government and the development partners needed to finalize agreements on the procurement and disbursement rules. The World Bank played an important role in improving the financial management of the Program. This included careful review of some of the larger procurement packages (particularly paper for textbooks) as well working with DFID to push for greater compliance with the annual audit.20 At the same time, the World Bank was concerned about the Program's limited M&E capacity and expressed this concern both externally (for example, with communications with the government and ADB) as well as internally (for example, supervision reports). This was a major theme of the World Bank's participation in the mid- term review and the ADB took steps to improve the Annual Performance Review. There were clear indications that the Ministry's capacity was improving from a very low starting point. By all accounts, the World Bank was quite active in technical discussions but did not appear to provide many formal technical inputs. 6.17 Supervision after the midterm review. The midterm review marked an important turning point in the project. With improvement in many areas as well as greater government ownership, the development partners were able to focus more on technical issues. This, in turn, led to greater development impact. For example, the development partners worked closely with the government to develop a robust sample-based assessment system. 6.18 By all accounts, relations among the development partners improved. The World Bank also was able to focus more on technical issues, preparing several policy reports as well as starting research that continued after PEDP II closed. Towards the end of the Program, much effort was spent to develop the follow-on program, PEDP III. As a reflection of the improving relations among development partners, PEDP III did not need to designate a lead partner, assigning tasks to individual partners. Borrower Performance 6.19 Both the government and implementing agency (The Ministry of Primary and Mass Education) provided effective support, the overall quality of borrower performance is rated moderately satisfactory. GOVERNMENT PERFORMANCE 6.20 The government performance is rated moderately satisfactory due to its support for budget and the Program. The government was the largest provider of resources and 20 This observation is based on interviews as well as internal program documents. 47 these resources were generally provided in a timely and predictable fashion. There were no problems reported with the financing of the complementary Revenue Budget (which went primarily for salaries) necessary for the sector. Initially, the government was reluctant to enter into a SWAP arrangement and did not, at first, provide the Ministry with the necessary staff to carry out the Program. This was reflected in the slow development of the M&E system and in carrying out several of the institutional changes that had been agreed to under the Program. The government initially developed the Program using a Project Proforma (the government's project's document) that was designed for the traditional project approach and was not sufficiently flexible for a SWAP. After the mid-tern review, the government supported changes to its design to ensure that the Ministry had more flexibility. 6.21 As the government developed greater ownership, it responded by assigning senior staff to the Ministry. By all accounts, after the midterm review, turnover was kept to a minimum and the government did its best to ensure that the Program had adequate staff, particularly at the highest levels. 6.22 The development partners felt that the government was addressing the various agreements and covenants. The government provided important support to efforts to improve the transparency of teacher recruitment, which required support from the government that went beyond what the capacity of the Ministry was able to implement on its own (Wescott and Breeding 2011). The same commitment was needed to grant autonomy to the National Academy of Primary Education. The aide memoires indicate that the Program Steering Committee met on a regular basis and the Ministry and the government remained actively engaged. IMPLEMENTING AGENCY PERFORMANCE 6.23 Given its support throughout the Program and its willingness to adopt reforms, the performance of the implementing agency (The Department of Primary Education of the Ministry of Primary and Mass Education) is rated moderately satisfactory. 6.24 During the preparation period, the Ministry did not have a good understanding of what a SWAP was and how it would work in practice, as evidenced in the first aide memoires. Likewise, the Ministry did not have the necessary human resource capacity to implement the Program as it was designed. In the first two years, the Ministry was slow in hiring necessary consultants. In addition, there were delays in procurement. While audits were carried out in a timely fashion, it often took an extended period (a year or longer) to respond to all of the audit observations. 6.25 As the Ministry developed a greater understanding of the Program and of the SWAP, it played an active role in supporting the Program. For M&E, this included leading the 2005 Baseline Survey and the subsequent annual school census. The Ministry also was able to mobilize its local network at the district and upazila level to work with schools to collect data (World Bank 2011). The Ministry also took the lead in developing the Primary Completion Examination, implementing the new teacher recruitment model, and reforming teacher training. All of these initiatives required coordination between the Ministry at the national level and the local education officials and schools. 48 Monitoring and Evaluation 6.26 Starting from virtually no M&E capacity, the Ministry developed an impressive capacity, supported by the Program and the Monitoring and Evaluation of the Program is rated high. 6.27 At the start of the Program, the M&E capacity of the Ministry of Primary and Mass Education was almost non-existent and only limited data were collected by development partners or the Ministry of Education. Although progress was slow at first, the Ministry developed a strong capacity to collect and analyze information. The World Bank played a major role in pushing for better data. The Program presents a good example of using data and the M&E in a living results framework and its achievement exceed what was expected at the time of preparation. M&E DESIGN 6.28 Although the Program had a well-developed logical framework, with baseline and target, the Ministry's capacity to monitor and evaluate the sector was extremely limited. The Program had a formal logical framework that identified a long-term objective as well as program-specific objectives. It also outlined the outputs needed to achieve these objectives. Each output had a set of indicators to monitor progress as well as assumptions behind each output (Asian Development Bank 2003).21 For the most part, these indicators were appropriate. The targets set for most indicators were realistic and represented "stretch targets" that were consistent with the country's trends and were likely to be achievable if the Program performed as designed. 6.29 The World Bank played a major role in supporting the development and in supporting the implementation of the M&E system. There were some minor differences in the World Bank's results framework from the Program's logical framework and a few indicators were not realistic. The World Bank's key performance indicator for quality calls for the number of students who take the Primary Scholarship Examination to increase from 20 percent in 2002 to 50 percent in 2009 and that the pass rate increases from 25 percent to 80 percent. While it is certainly realistic to increase the number of students who take the examination, the increase in passing rate in only seven years was very ambitious. Since the primary cycle lasts five years, this essentially requires the primary education system have implemented a major quality reform by 2004, when the 2009 graduating cohort starts school. Considering that the Program was not expected to start until 2004 (the ADB approved its program in October 2003 and World Bank in January 2004), this was not realistic. Curiously, the ADB and the rest of the Program had a somewhat less ambitious target with a passing rate target of 40 percent in 2009. While this would also be a substantial challenge, it is a more realistic target (Asian Development Bank 2003, World Bank 2004). Another example was the indicator in the output "improved teaching and learning," was starting in 2005, all textbooks and [didactic] material are available... by the first week. While there was no baseline for this 21 The program's logical framework is based on the ADB's program document. The World Bank adopted a similar approach in its results framework although the World Bank's higher level objective focused on improving the skill level of the economy, following its Country Assistance Strategy. 49 indicator, in 2005 it was reported that only 18 percent of students started receiving textbooks by January 1 (Government of the People's Republic of Bangladesh 2011). 6.30 An important weakness in the design was the Ministry's low capacity, including the lack of a functioning unit to collect data and carry out analysis. At the time that the SWAP was designed, most of educations statistics were collected by the Ministry of Education and not by the Ministry of Primary and Mass Education. The SWAP called for the new evaluation division to carry out a relatively sophisticated M&E. Although capacity building was provided, the expectation of how long it would take the Ministry to develop a high quality M&E was not realistic. M&E IMPLEMENTATION 6.31 The implementation of the monitoring and evaluation system showed constant advances and by the end of the Program, the Ministry had a strong and well-sustained system. By the time of the first Joint Annual Review Mission, in April 2005, there were serious shortcomings with M&E. Although the M&E Cell existed, it did not have a Director and was understaffed. The Ministry had not established a National Assessment Cell, which was required in the legal covenants. The National Assessment Cell plays an important both in monitoring changes in learning as well as providing feedback for the design of curriculum and learning material. 6.32 The Ministry developed a set of key performance indicators that would serve to measure the impact of the project at the national, local and school levels. These were based on the result frameworks from the project documents. In addition to developing the key performance indicators, the Program supported a "baseline survey" in late 2005 that was applied to all primary schools. Prior to the survey, training was provided to local education officers at the district and upazila levels. These officials were responsible for distributing to the school and reviewing the results. The data were processed at the national level by the newly established M&E Cell (Government of the People's Republic of Bangladesh 2006). This survey served as the model for future annual school censuses. It also represented a significant improvement in the quality and relevance of the school census as well as an important to step to increase the ownership of the M&E process.22 This was followed by reports on the 2006 and 2008 school censuses. 6.33 With support of the Program, the Ministry also introduced instruments to measure student learning. First, the Ministry worked to develop a sample-based evaluation of learning for students in grades 3 and 5, known as the National Student Assessment. While there had been several earlier attempts to measure the quality of learning, the National Student Assessment was the first attempt to standardize this process. It took several rounds to develop a successful model. Assessments were carried out in 2006, 2008, and 2011. As the Ministry worked on the model, adjustments were made in the assessment and the three assessments were not compatible (Government of the People's Republic of Bangladesh 22 The annual school census was introduced by the Effective School for Enhanced Education Management Project (known as ESTEEM), which was financed by DFID and was eventually incorporated into the SWAP. 50 2012). The current plan is to use the 2011 National Student Assessment as the model for future assessments. The assessment evaluates the student's knowledge of Bangla (Bengali) and mathematics. It measures the student's competence in these two subject materials as opposed to just measuring knowledge of curriculum. 6.34 In addition, the Ministry introduced a Primary Completion Examination (a terminal or end-of-cycle examination) in 2009. This examination replaced the Scholarship Examination, which was given to a select number of primary school graduates. The Primary Completion Examination covered most of the primary curriculum, with subjects in Bangla, Mathematics, English, Social Science, General Science, and Religious Studies. In order to provide quality control, the examination was given in an exam center (nearly 6,200 in 2011 for 87,800 schools). By 2011, almost all students were taking the examination upon finishing the primary school cycle. Starting in 2012, the examination will start to include some evaluation of competencies. Thus, it may be an issue of consistency in the future but will give a better sense of the impact of primary education (Government of the People's Republic of Bangladesh 2012). 6.35 The Ministry had detailed information on the number and location of stipends supported by the Program. However, the M&E system was not able to track the quality of the stipend's targeting. As Bangladesh does not have a universal poverty census, it is only possible to carry out an ex-post evaluation of targeting. The Program financed a survey of the targeting and the analysis is ongoing (Government of the People's Republic of Bangladesh 2011). 6.36 In 2009, the Ministry started publishing its Annual Sector Performance Report, which brought together data from different sources. The Review builds on previous reports such as the School Census Report. The Review presented a comprehensive overview of data and the sector's performance against key performance indicators. As part of the discussion of data, the reports considered alternative sources of data (such as household surveys) to validate or improve the data that came from the School Census and other administrative sources. For example, the reports regularly compared the number of participants in the Primary Completion Examination to estimate enrollment in primary systems outside of the purview of the Ministry. 6.37 The quality of the Annual Sector Performance Review is quite high. The Annual Reviews are closely based on the Program's logical framework (with activities and outputs as well as short-term, medium-term, and long-term outcomes), working through the causal chain and analyzing progress. The Reviews also discuss the quality of data used in the result frameworks and present various alternative measures, when appropriate. Although consultants play a major role in preparing the Annual Review, it has become mainstreamed and continues after the end of PEDP II. The Annual Review depends heavily on data produced by the Ministry using capacity that was developed by the Program. Although the Annual Review did not report on the enrollment of students from Hill Tribe Areas, data were available from the Annual School Census. 51 M&E UTILIZATION 6.38 The reports organized and data using a logical framework that follows the expected causality chain from the inputs and activities to the medium-term and long-term outcomes. This approach made helped policy makers and development partners to identify not only where they were, in terms of indicators, but also where they need to advance. It also helped the Program to focus on indicators that were related to the outcomes and under its control. 6.39 The government's Concept Paper for PEDP III (Directorate of Primary Education 2009) as well as the World Bank's PAD for PEDP III (World Bank 2011) make clear use the data developed during PEDP II and propose numerous course changes to reflect this. For example, there is an attempt to improve the placement of new schools. PEDP III has placed the Annual Reviews in the center of its operations. Safeguards 6.40 For PEDP II, both the World Bank and Asian Development Bank triggered their respective indigenous peoples safeguards. Both development banks have similar policies and cooperated in preparing the Indigenous Peoples Plan, known as the Plan for Expanding the Primary Education of Tribal Children. The Plan was developed following a field visit to Chittagong Hill Tracts that included consultations with a large number of stakeholders. Stakeholders identify two main constraints facing tribal children-the need for children to work at a young age and problems speaking Bangala. On the supply side, it was hard to find teachers who are willing to work in the Hill Tracts, particularly female teachers, and facilities are of particularly low quality (World Bank 2004). 6.41 Around I to 1.5 percent of Bangladesh's population are members of indigenous ethnic minorities. These are primarily located in the Chittagong Hill Tracts, which consists of three districts, as well as smaller groups located in other districts in the north and west of the country. There are 45 recognized ethnic or tribal groups, of which 13 live in the Hill Tracts. The Hill Tribes have distinct cultures and languages; around 50 percent are Buddhist. These groups are typically poorer and have lower levels of human capital than the national average. They are often effectively excluded from the national economy. The Constitution of Bangladesh and different laws guarantee equal rights for ethnic minorities (World Bank 2004, Directorate of Priamry Education 2010). 6.42 The Plan proposed additional attention on tribal children, with around 2,300 classrooms to be built in the Hill Tracts. In addition, the Program was to develop the capacity of local tribal institutions with unspecified support. The Plan also proposed to train additional teachers and to prepare supplementary learning materials. The Plan did not include an M&E framework (World Bank 2004). 6.43 The Indigenous Peoples Plan prepared for the Third Primary Education Program expressed its general satisfaction with the Plan for PEDP II and adopted most of the elements of the original plan (Directorate of Priamry Education 2010). While new school classrooms were built in the Hill Tracts, there is no evidence that this area was given priority (Government of the People's Republic of Bangladesh 2012). Internal World Bank 52 supervision reports continually gave compliance with the safeguard a moderately satisfactory rating. Starting with PEDP III, the Annual School Census started tracking whether a school was located in an area with a high number of tribal children. Fiduciary Compliance 6.44 The Ministry's Finance and Procurement Division received substantial support from the Project Liaison Unit. It was generally successful in performing its assigned tasks, including the management of three US dollar accounts and common pool accounts (involving pooled, non-pooled, and parallel funding) as well as the submission of Quarterly Financial Management Reports. However there were a number of shortcomings in financial management, including delays in consolidating documents from field offices and reconciliation and the use of multiple payments depending on the source of financing (the World Bank, the ADB, the ADB trust fund, or the government). 6.45 As of January 2012, all audit observations from FY2003-04 to FY2006-07 were resolved, but 19 observations from FY2007-08 to FY2009-10 remained unresolved. The audit for FY2010-11 was still ongoing at the time the project closed. Experience from PEDP II improved the Ministry's awareness of the importance of the speedy settlement of audit observations to mitigate risks to financial loss. 6.46 Of the original credit of US$150 million, the undisbursed balance after the final disbursement in October 2011 was US$13.2 million, most of which was from exchange gain from the weakened US dollar versus the SDR and from the weakened Bangladesh Taka. Even taking into account the exchange gain, the project was 91.7 percent disbursed (see Annex A for more detail). 6.47 Using ADB and World Bank procurement guidelines, efforts were made from the start to improve the Ministry's procedures for the procurement of civil works, goods and services, particularly for ICB procedures. The Ministry reinforced procurement management by: (i) distribution and utilization of approved procurement manuals; (ii) ongoing provision of short training courses to concerned officials at central ministry and decentralized levels; and (iii) creation of a permanent, separate evaluation committee for the procurement of goods and services, of which two members were from other ministries. 6.48 There was only one serious incident of note, and it was satisfactorily resolved. A Post Procurement audit for FY2003-04 and FY2004-05, confirmed by subsequent reviews by the Development Partner Procurement and Finance Working Group and the full development partners consortium, found inappropriate tendering and collusive practices in 15 of 17 contract packages for civil works. This led to a declaration of misprocurement and cancellation of the 15 contracts. The Ministry responded rapidly and reiterated the government's zero tolerance policy on procurement irregularity and carried out the appropriate sanctions against the contractors involved. In February 2009, in accordance with provisions in the legal agreement, the World Bank canceled the share of the credit that had already been disbursed against the 15 contracts, approximately US$130,000. 53 7. Conclusion and Lessons 7.1 The Second Primary Education Development Program is a good example of development partners and the government working together in a difficult environment and accomplishing measurable and attributable impacts. 7.2 The Program was one of the largest education SWAPs ever proposed. It appears to be the first time that the World Bank participated in an education SWAP without being the lead partner. The Program was largely successful in promoting increased access and quality, while strengthening many aspects of the education system. There are several lessons about harmonization and working effectively in strengthening education systems. 7.3 In a SWAP, the leadership of development partners requires a mix of "hard skills", "soft skills", and flexibility. PEDP II represented the first successful SWAP for education in Bangladesh, bringing greater coherence to external support and reducing transactions costs for the Government. To succeed, it required flexibility and adaptability from all sides. While the Program's logical framework was well-designed, the Program began with several elements that were not fully developed and the Ministry was not entirely convinced by the benefits of the SWAP. The World Bank expressed concern about the lack of preparation and indicated that there was a high risk of failure. The Program was able to adjust to Bangladesh reality by building trust with the Ministry and making adjustments. This included changing the role of the Project Liaison Unit, changing how M&E is implemented and increasing the Ministry's involvement, and allowing adequate time to develop the capacity of government agencies. At the same time, development partners did not ignore technical issues and provided support to the Ministry in a number of areas, such as developing a new assessment system and improving the recruitment of teachers. 7.4 While the ADB was the lead partner, many partners provided value added. The World Bank proved to play an important role in supporting fiduciary processes and participating in the technical dialogue. In the design phase, there appeared to be a significant amount of tension that got in the way of productive dialogue. But as the Program got underway, there appeared to be a concerted effort to move beyond institutional debates and to focus on implementation and technical issues. This sort of process requires trust and can only be gained from experience. For PEDP III, development partners agreed to eliminate the role of the "lead" development partner. 7.5 Government ownership needs to be fostered. While there is agreement that ownership is an important determinant of success, the concept of ownership is often seen as a dichotomous process-either there is ownership or there is not. The Program showed that reality is more subtle. There was no doubt that the government had a high level of ownership of the Program's objectives and had long given primary education a high priority. However, this did not mean that the government fully understood the sector-wide approach and with the changes it would bring to project management. 7.6 The World Bank expressed repeated concerns about the government's ownership of the Program, for example the use of consultants to prepare the Macro Plan and concerns that the government initially raised about a SWAP. The World Bank also raised concerns about 54 the role of overseas consultants in the design of the Macro Plan and the M&E system. In retrospect, these concerns seem exaggerated. Much like any other institution that uses external consultants, the Ministry used the expertise of these consultants to implement what was essentially its own program. Rather than seeing ownership as dichotomous outcome, development partners could be more subtle in understanding ownership and in identifying steps necessary to strengthen understanding of new ideas and proposals. 7.7 No sector is an island and it is important for sector programs to coordinate with efforts to strengthen public administration and governance. While the Program was able to achieve strong results, it required concerted intervention by the government. One of the most important elements in PEDP II was the government's decision to appoint senior officials to lead the Program. At the end of the Program, many of these officials were assigned to other Ministries. This reflected the demand of other parts of the government for good staff ("the pull") as well as the limited scope for promotion and developing a career in the Ministry of Primary and Mass Education ("the push"). While appointing senior officials led to stronger implementation and showed the importance that the government gave to the sector and the Program, it is not a long term strategy. 7.8 In the long run, Bangladesh needs to reform how civil servants are hired and remunerated. This is beyond the remit of a sector program but points out the importance of the sector to engage in the larger discussions about public management and administration. It may make sense for development partners to include public administration specialists or economists on their team to ensure that they can maintain a meaningful dialogue with the appropriate central ministries as well as with other development project and programs. This will allow a program to influence areas that are outside of its direct control. 7.9 A SWAP is an evolutionary process that requires flexibility and adaptation to succeed. During preparation, many development partners focused on the SWAP's faults, such as the exclusion of secondary education and non-formal education or the Ministry's lack of understanding of the Program. While these are valid concerns and should be part of the debate, they should not take away from the need to be flexible and not to expect a fully developed institution at the start of the process. In the end, the government and partners agreed to be selective. While the Program included "stretch" targets, it design was realistic given Bangladesh's circumstances. In the end, the partners and the government agreed to continue with the "project approach" to manage procurement and financial management. 7.10 PEDP II set the stage for innovations that were incorporated into PEDP III, such as the use of pooled funds within the government budget instead of through a system of special accounts and more flexible Program leadership. While this new approach has its own risks, it shows a willingness to take risks and to learn from previous lessons. 7.11 With support, governments can transform M&E systems from a pro forma system of collecting data to a powerful tool for monitoring and decision making. The Program showed that it is possible to develop a working M&E system that not only generates data but uses the data to guide policy and strengthen the understanding of the results chain. Although consultants (contracted by the government through the Program) played a major role in preparing the final report, the Ministry has made an important effort to generate data 55 at the student and the school level and to institutionalize this process. The government also took the lead in developing, for the first time, instruments to measure education quality. An M&E system is not easy to develop and requires a significant investment. Most World Bank- financed projects included investment and capacity building to support M&E. The experience from the Program shows that it is important to engender an understanding of why M&E is important and provide clear evidence of its utility. Otherwise, the M&E system is simply an exercise in collecting data to fill out tables for administrative reasons. 7.12 Moving forward, there is a need to look at impacts and to make adjustments in individual initiatives. For example, the primary education stipend program appears to have made a contribution to keeping vulnerable children in school. However, there is much room for improvement and to retarget its focus and benefits. Likewise, continued research is needed on the best way to design and implement training programs for teachers. PEDP II M&E reporting largely focused on annual gains. While this was a necessary step to develop an M&E from scratch, there is now room to develop longer-term final indicators.  57 References Ahmed, M. 2011. The Sector-wide Approach in Bangladesh Primary Education. Sussex, United Kingdom: Create Pathways to Access. Research Monograph No. 57. Institute of Education and Development, BRAC University, Dhaka, Bangladesh. Akresh, R., de Walque, D., and Kazianga, H. 2013. Cash Transfers and Child Schooling: Evidence from a Randomized Evaluation on the Role of Conditionality. Washington, DC: The World Bank. Ardt, K., Hastings, C., Hopkins, K., Knebel, R., Loh, J., and Woods, R. 2005. Report on Primary Education in Bangladesh: Challenges and Successes. Bangladesh Bureau of Education Information and Statistics & Bangladesh Ministry of Women and Children Affairs. Dhaka, Bangladesh. Asian Development Bank. 2003. Report and Recommendation of the President to the Board ofDirectors on a Proposed Loan to the People's Republic ofBangladesh for the Second Primary Education Development Program. Manila, Philippines: Asian Development Bank. BANBEIS. 2011. Report on Sample Vital Registration System-2010. Ministry of Planning. Dhaka, Bangladesh. . 2012a. Gross and Net Enrollment Rate in Primary Education. Bangladesh Bureau of Educational Information and Statistics. Dhaka, Bangladesh. . 2012b. Primary Enrollment, 2007. Dhaka, Bangladesh: Bangladesh Bureau of Education Information and Statistics. Dhaka, Bangladesh. Baulch, B. 2010. Evaluating the Long-Term Impact of Anti-Poverty Interventions in Bangladesh. Retrieved January 22, 2014, from IFPRI: http://www.ifpri.org/sites/default/files/publications/ifpricprcnote_baulch.pdf BBC News. 2004 (August 3). Bangladesh Appeals for Flood Aid. BBC New. Beatty, A., and Pritchett, L. 2013. From School Goals to Learning Goals: How Fast can Student Learning Improve? Washington, DC: Center for Global Development. Boone, P. F., Jandhyala, K., Jayanty, C., Johnson, S., Ramachandrin, V. S., and Zhan, Z. 2013. The Surprisingly Dire Situation of Children's Education in Rural West Africa. Cambridge, MA: National Bureau of Economic Research. Buse, K., and Gwin, C. 1998. "The World Bank and global cooperation in health: the case of Bangladesh." The Lancet:665-669. Card, D., and Kruger, A. 1992. "Does School Quality Matter? Returns to Education and the Characteristics of Public Schools in the United States." Journal ofPolitical Economy: 1-40. Chaudhury, N., Hammer, J., Kremer, M., Mularidharan, K., and Rogers, H. 2004. Roll Call: Teacher Absences in Bangladesh. Washington, DC: World Bank. Chetty, R., Friedman, J., and Rockoff, J. 2011. The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood. Cambridge, MA: National Bureau of Economic Research. Danish Biharziasis Laboratory. 2004. Disability in Bangladesh: A Situation Analysis. Dhaka, Bangladesh. DFID. 2011. BRAC Strategic Partnership Arrangement. Dhaka, Bangladesh: DFID. Directorate of Primary Education. 2009. PROG3 Concept Paper. Dhaka, Bangladesh: Ministry of Primary and Mass Education. . 2010. Indigenous Peoples Framework. Dhaka, Bangladesh: Ministry of Education. . 2012. 2011 National Student Assessment for Grades 3 and 5 National Report. Dhaka, Bangladesh: Government of the People's Republic of Bangladesh. ANNEX A 58 Glewwe, P., Hanushek, E., Humpage, S., and Ravina, R. 2011. School Resources and Educational Outcomes in Developing Countries: A Review of the Literature from 1990 to 2010. Cambridge, MA: National Bureau of Economic Research. Glewwe, P., Kremer, M., and Moulin, S. 2007. Many Children Left Behind?: Textbooks and Test Scores in Kenya. Washington, DC: World Bank. Government of the People's Republic of Bangladesh. 2002a. "National Plan of Action II- Education for All in Bangladesh." Retrieved October 15, 2013, from National Plan of Action II (NPA II): http://www.mopme.gov.bd/index.php?option=com_content&task=view&id=452&Itemid=472 . 2002b. "Second Primary Education Development Program." Dhaka, Bangladesh: Primary and Mass Education Division. . 2006. "Report on the Baseline Survey of Second Primary Education Development Programme." Dhaka, Bangladesh: Ministry of Primary and Mass Education. . 2009. "Bangladesh Primary School Census Report 2008." Dhaka, Bangladesh: Ministry of Primary and Mass Education. . 2011 a. "Bangladesh Primary Education Annual Sector Performance Report 2011." Dhaka, Bangladesh: Directorate of Primary Education. . 201 1b. "Population & Housing Census 2011." Dhaka, Bangladesh: Bangladesh Bureau of Statistics, Ministry of Planning. .201 1c. "Sixth Five Year Plan." Dhaka, Bangladesh: Planning Commission. . 2012a. "2011 National Student Assessment for Grades 3 and 5 National Report." Dhaka, Bangladesh: Directorate of Primary Education. . 2012b. "Bangladesh Primary Education Annual Sector Performance Report 2012." Dhaka, Bangladesh: Directorate of Primary Education. . 2012c. "The Millenium Development Goals - Bangladesh Progress Report 2011." Dhaka, Bangladesh: General Economics Division, Bangladesh Planning Commission. Hanushek, E., and Rivkin, S. 2012. "The Distribution of Teacher Quality and Implications for Policy." American Review ofEconomics:131-157. Hanushek, E., and Woessmann, L. 2012. "Do better schools lead to more growth? Cognitive skills, economic outcomes, and causation." Journal ofEconomic Growth:267-321. Hanushek, E., and Zhang, L. 2006. "Quality-Consistent Estimate of International Returns to Skill." Cambridge, MA: National Bureau of Economic Research. Hanushek, E., Lavy, V., and Hitomi, K. 2006. Cambridge, MA: National Bureau of Economic Research. Health, A., and Morbarak, A. 2012. "Does Demand or Supply Constrain Investment in Education? Evidence from Garment Sector Jobs in Bangladesh." Dhaka, Bangladesh. Hossain, M. J. Undated. Special Education in Bangladesh. Retrieved May 15, 2013, from National Institute of Special Education: http://www.nise.gojp/kenshuka/josa/kankobutsu/pub_d/d-279/d-279_10.pdf Human Development Resource Centre. 2009. 'Socio-Economic Baseline Survey of the Chittagong Hill Tracts.' Dhaka, Bangladesh: UNDP. Independent Evaluation Group. 2006. Bangladesh Fourth Population and Health Project (Credit 2259-BD) and Health and Population Program Project (Credit 3101-BD). Project Performance Assessment Report. Washington, DC: The World Bank. International Development Association. 2004. Development Credit Agreement (Second Primary Education Development Program) between People's Republic ofBangladesh and International Development Association. Dhaka, Bangladesh: World Bank. International Food Policy Research Institute. 2009. Why Some Households in Rural Bangladesh Stay Poor, While Others Move Out of Poverty. Retrieved May 8, 2013, from 59 http://www.ifpri.org/publication/why-some-households-rural-bangladesh-stay-poor-while-others- move-out-poverty-key-finding International Monetary Fund. 2011. Bangladesh Article IV Consultation. Washington, DC: International Monetary Fund. IOB. 2011. The two-pronged approach: Evaluation ofNetherlands support to primary education in Bangladesh. Hague, the Netherlands: Ministry of Foreign Affairs of the Netherlands. Japan Bank for International Cooperation. 2002. Bangladesh Education Sector Overview. Dhaka, Bangladesh: JBIC. Jennings, J. 2007. Planning of the Second Primary Education Development Programme (PEDP II) in Bangladesh. Dhaka, Bangladesh: UNICEF Regional for South Asia. Kostermans, K., and Geli, P. Undated. The Sector- Wide Approach in Action: The Example of the Bangladesh Health, Nutrition, and Population Sector. Washington, DC: The World Bank. MacDonald, L. 2013. How to Avoid Another Bangladesh Factory Disaster. Retrieved May 7, 2013, from Global Prosperity Wonkcast: http://www.cgdev.org/blog/how-avoid-another-bangladesh-factory- disaster-kimberly-elliott Malak, M. 2013. "Inclusive Education Reform in Bangladesh: Pre-Service Teachers' Response to Include Students with Special Educational Needs in Regular Classrooms." International Journal of Instruction:195-214. Mas-Colell, A., Whinston, M., and Green, J. 1995. Microeconomic Theory. New York, NY: Oxford University Press. Meng, X., and Ryan, J. 2010. "Does a food for education program affect school outcomes? The Bangladesh case." Journal ofPopulation Economics:415-447. Mitra, S. M. 1994. Bangladesh Demographic and Health Survey, 1993-1994. Calverton, Maryland: National Institute of Population Research and Training (NIPORT), Mitra and Associates, and Macro International Inc. Mitra, S., Al-Sabir, A., Cross, A., and Jamil, K. 1997. Bangladesh Demography and Health Survey, 1996- 1997. Dhaka, Bangladesh and Calverton, MD: NIPORT, Mitra and Associates, and Macro International. Mourshed, M., Chijioke, C., and Barber, M. 2010. How the world's most improved school systems keep getting better? New York, NY: McKinsey & Company. Mufad, C. 2008. Presentation on Aid Harmonization in the Second Primary Education Development Program (PEDP II). Dhaka, Bangladesh: Available on www.1cbangladesh.org. Nelson, J. 2012. Aligning Human Capital Systems to Improve Teaching Quality and Student Success. Austin, Texas: Education First. NIPORT, Mitra and Associates, and ORC Macro International. 2005. Bangladesh Demographic and Health Survey 2004. Dhaka, Bangladesh and Calverton, MD: NIPORT, Mitra and Associates, and ORC Macro. . 2009. Bangladesh Demographic and Health Survey 2007. Dhaka, Bangladesh and Calverton, MD.: NIPORT, Mitra and Associates, and ORC Macro. NIPORT, Mitra and Associates, and MEASURE DHS. 2013. Bangladesh Demographic and Health Survey 2011. Dhaka, Bangladesh and Calverton, MD: NIPORT, Mitra and Associates, and Measure DHS. NIPORT, Mitra and Associates, and ORC Macro. 2001. Bangladesh Demographic and Health Survey, 1999-2000. Dhaka, Bangladesh and Calverton, MD: NIPORT, Mitra and Associates, and ORC Macro. OECD. 2002. Education at a Glance. Paris, France: OECD. OPCS. 2010. Implementation Completion and Results Report: Guidelines. Washington, DC: World Bank. ANNEX A 60 Operations Evaluation Department. 2008. Education Sector in Bangladesh: What Worked Well and Why under the Sector-Wide Approach. Manila, Philippines: Asian Development Bank. Quisumbing, A., and Baluch, B. 2009. Assets and poverty traps in rural Bangladesh. Washington, DC: International Food Policy Research Institute. Quisumbing, A., Baluch, B., and Kumar, N. 2011. Evaluating the Long-term Impact of Antipoverty Interventions in Bangladesh: An Overview. Washington, DC: International Food Policy Research Institute. Rahman, S. A., Kielmann, T., McPake, B., and Normand, C. 2012. 'Healthcare-seeking Behavior among the Tribal People of Bangladesh: Can the Current Health System Really Meet Their Needs?' Journal ofHealth, Population, and Nutrition: 353-365. Sabarwal, S., Evans, D., and Marshak, A. 2012. Textbook Provision and Student Outcomes: The Devil in the Details. Retrieved April 23, 2013, from http://www.globalpartnership.org/media/textbooks/day3/GPE_Textbook_Workshop_Shwetlena_P resentationOctober 2012.pdf Saxena, S., and Salze-Lozac'h, V. 2010. Competitiveness in the Garment and Textiles Industry: Creating a supportive environment. New York, NY: The Asia Foundation. Schwab, K. 2012. The Global Competitiveness Report. Geneva, Switzerland: World Economic Forum. Shafiq, M. 2007. "Household Rates of Returns to Education to Rural Bangladesh: Accounting for Costs, Child Labor, and Option Value." Education Economics:343-358. Sukontamarn, P. 2010. "Bangladesh's food for education program: the effects on two groups of targeted households." Education Economics:79-91. The Study Team. 2009. Rapid Assessment and Stock Taking: Second Primary Education Development Program. Dhaka, Bangladesh: Education Consortium. U4. 2012. U4 Expert Answer. Oslo, Norway: Anti-corruption Resource Centre. UNESCO. 2006. EFA Monitoring Report 2005. Paris, France: UNESCO. UNESCO Institute for Statistics. 2009. Education Indicators: Technical Guidelines. Paris, France: UNESCO. UNICEF. Undated. Chittagong Hill Tracts. Retrieved January 15, 2014, from http://www.unicef.org/bangladesh/CHT.pdf Unterhalter, E., Ross, J., and Alam, M. 2003. "A Fragile Dialogue? Research and Primary Education Policy Formulation in Bangladesh, 1971-2001." Compare: A Journal of Comparative and International Education:85-99. Vaillancourt, D. 2012. In Sweet Harmony? A Review ofHealth and Education Sectorwide Approaches (SWAps) in the South Pacific. Washington, DC: World Bank. Wassner, B. 2012. In an Unlikely Corner of Asia, Strong Promise of Growth. New York Times. Wescott, C., and Breeding, M. 2011. "Bangladesh: World Bank Engagement on Governance and Anticorruption." Washington, DC: Independent Evaluation Group. World Bank. 1998a. "Project Appraisal Document for a Proposed Interim Trust Fund Credit in the Amount of SDR 111 Million Equivalent to the People's Republic of Bangladesh for a Primary Education Development Project." Washington, DC: The World Bank. . 1998b. "Project Appraisal Document on a Proposed Interim Trust Fund Credit in the Amount of SDR 111 Million Equivalent to the People's Republic of Bangladesh for a Primary Education Development Project." Washington, DC: Education Sector, South Asia Region. . 2000. "Bangladesh Education Sector Review, Volume III. Dhaka, Bangladesh: The University Press Limited." Washington, DC: The World Bank. 61 . 2003. Memorandum of the President of the International Development Association and the International Finance Corporation to the Executive Directors on a Country Assistance Strategy for the People's Republic of Bangladesh. Washington, DC: World Bank. . 2004. "Project Appraisal Document on a Proposed Credit in the amount of SDR 103.2 million (US$ 150 million equivalent) to the People's Republic of Bangladesh for a Primary Education Development Project II." Washington, DC: World Bank. . 2008a. "Poverty Assessment for Bangladesh: Creating Opportunities and Bridging the East-West Divide." Washington, DC: The World Bank. . 2008b. "Education for All in Bangladesh: Where Does Bangladesh Stand in Achieving the EFA Goals by 2015." Dhaka, Bangladesh: The World Bank. . 2010a. "Bangladesh Public Expenditure and Institutional Review." Washington, DC: World Bank. . 20 1Ob. "Country Assistance Strategy for the People's Republic of Bangladesh for the Period FYI 1-14." Washington, DC: World Bank. . 201 1a. Implementation Completion Report on a Credit in the Amount of of SDR 104.2 million (US$ 150 million equivalent) to the People's Republic of Bangladesh for a Primary Education Project II. Washington, DC: World Bank. . 2011b. Project Appraisal Document on a Proposed Credit int eh Amount of SDR 187.5 million (US$300 million equivalent) to the People's Republic of Bangladesh for a Third Primary Education Development Program. Washington, DC: World Bank. . 2012. World Development Indicators. data.worldbank.org. . 2013. Gini Index. Retrieved April 30, 2013, from Data: http://data.worldbank.org/indicator/SI.POV.GINI World Health Organization & World Bank. 2011. "World Report on Disability." Geneva, Switzerland: World Health Organization. World Health Organization. 2006. "A guide to WHO's Role in Sector Wide Approaches to Health Development." Geneva, Switzerland: World Health Organization. Yardley, J. 2012. "Export Powerhouse Feels Pangs of Labor Strife." New York Times. Zhang, X., Shahidur R., Kaikaus A., Mueller V., Lee H. L., Lemma S., Belal S., and Akhter A. U. 2013. "Rising wages in Bangladesh." IFPRI discussion papers 1249, International Food Policy Research Institute (IFPRI).  63 ANNEX A Annex A. Basic Data Sheet Primary Education Development Project II (P074966) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 951.00 1,065.02 112.00 Loan amount 150.00 145.28 97.00 Cancellation 0.00 13.53 0.00 Cumulative Estimated and Actual Disbursements FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 Appraisal estimate (US$M) 14.68 37.71 67.22 98.17 128.40 150.00 150.00 150.00 Actual (US$M) 11.93 38.94 58.73 78.21 105.49 125.51 138.38 145.28 Actual as % of appraisal 81.26 103.26 87.36 79.66 82.15 83.67 92.25 96.85 Date of final disbursement: 12/05/2011 Project Dates Original Actual Initiating memorandum 08/06/2002 01/30/2003 Negotiations 09/09/2003 12/15/2003 Board approval 11/06/2003 02/24/2004 Signing 01/29/2004 03/22/2004 Effectiveness 05/24/2004 05/24/2004 Closing date 06/30/2010 06/30/2011 ANNEX A 64 Task Team Members Names Title Unit Responsibility Specialty Lending Hena G. Mukherjee Sr. Education Specialist SASED Task Team Leader Amit Dar Sr. Economist SASED Education Yoko Nagashima Education Specialist SASED Education Irajen Appasamy Operations Officer SASED Operations Supervision/ICR Toufiq Ahmed Procurement Specialist SARPS Procurement Burhanuddin Ahmed Sr. Financial Management Specialist SARFM Financial Syed Rashed Al-Zayed Economist SASED Operations, M&E Irajen Appasamy Senior Operations Officer AFTED Operations Hon-Chan Chai Consultant SASED Education Helen J. Craig Lead Human Development Specialist AFTED Task Team Leader Subrata S. Dhar Senior Operations Officer SASED Operations Zafrul Islam Lead Procurement Specialist SARPS Procurement Pema Lhazom Senior Operations Officer EASHH Operations Bertha M. Mburugu Program Assistant LCSPS Operations Hena G. Mukherjee Consultant SASED Education Yoko Nagashima Sr Education Spec. SASED Education Rajat Narula Sr Financial Management Specialist EAPFM Financial Susan Opper Sr Education Spec. SASED Task Team Leader Md. Mokhlesur Rahman Senior Operations Officer SASED Operations Chau-Ching Shen Senior Finance Officer CTRFC Disbursement Nazma Sultana Program Assistant SASHD Operations Kishor Uprety Sr Counsel LEGES Legal Ayesha Y. Vawda Sr Education Spec. SASED Education Suraiya Zannath Sr Financial Management Specialist SARFM Financial 65 ANNEX A Staff Inputs (staff weeks) Stage of Project Cycle Staff Time and Cost (Bank Budget Only) US$ Thousands (including Lending travel and consultants costs) FY02 3.9 44.30 FY03 76.74 436.64 FY04 57.64 243.10 FY05 -0.04 Total: 724.00 Supervision/ICR FY04 4.14 19.50 FY05 44.49 153.04 FY06 39.62 157.28 FY07 40.66 124.09 FY08 48.39 181.78 FY09 49.93 206.13 FY10 41.68 185.44 FY11 8.68 45.92 FY12 6.15 47.20 Total: 1,120.38 Other Project Data Borrower/Executing Agency: Follow-on Operations Operation Credit no. Amount Board date (US$ million) Primary Education Development Program III P113435 300.0 August 25, 2011  67 Annex B. Key Performance and Other Indicators The program had a number of sources for key performance indicators. The World Bank had a results framework that was monitored on a regular basis. At the same time, the Program had its own set of key performance indicators that was monitored on an annual basis through the Annual Sector Performance Report. Since both were derived from the logical framework in the Macro Plan, there are only small differences between the two. Table 18 provides a summary of these indicators. The data come from the Program's M&E system, as reported in the Annual Sector Performance Report and in the World Bank's results monitoring system. The table indicates which indicators are officially classified as key performance indicators. It also gives an assessment of the "level" of the indicators on the results chain, ranging from outputs, intermediate outcomes, and final outcomes, which directly relate to the Program's objectives. These are the judgment of the PPAR and may differ from their position on the results framework. ANNEX B 68 Table 18. Key Performance Indicators Baseline Target Final Indicator Source Type alue Value Val Remarks Value Value Value Indicators Related to Objective #1 (Improving the Quality of Primary Education) Literacy (Bangla test PAD/ KPI, 53 58 69 Final results from the 2008 National Student Assessment. New assessments are not consistent with original series. Results are Numeracy (math test PAD KPI, 38 58 63 similar for boys and girls. Included as a PDO level indicator. scores) /ICR FO Schools with single shift PAD/ OUT 9% 28% 20% Target substantially revised upward during implementation system ICR (from 11% to 28%). Included as an intermediate indicator. System for learning PAD/ Before 2006, the National Student Assessment was not an assessment ICR official instrument. Included as an intermediate indicator. Pupil-teacherratio PAD/ KPI 54:1 48:1 47:1 Targets exceeded by 2% ICR Some textbooks There are a number of different indicators of textbook delivered by January availability. All have shown similar advances. Pass rate for completion Assessment introduced in 2009. Pass rate is for participating examination students. Indicators Related to Objective #2 (Improve equitable access to primary education) Number of stipends PAD/ KPI, mn. 7.9 mln. Initial target was to keep the stipend program at the same level ICR OUT mln. Gross enrollment rate PAD/ KPI, 940 98% 1080 Gender parity ratio was 1.09 at the end of the program ICR FO Net enrollment rate PAD/ KPI, 87% 90% 950% Gender parity ratio was 1.06 at the end of the program. ICR FO Completion rate PAD/ KPI, 52% 55% 60% Gender parity ratio was 1.10 at the end of the program. ICR ro Transition rate to grade PAD/ KPI, 92%o 96%o 9800 Transition is based on estimates from both primary and lower_ 69 ANNEX B Baseline Target Final Indicator Source Type alue Value Val Remarks Value Value Value 6 ICR IO secondary data system. There is likely to be measurement error in the data. Special need students PAD/ FO 8,800 11,800 15,000 Data probably has significant measurement error ICR Disadvantaged students KPI ASPR FO' 46,000 59,000 83,000 Generally refers to children with disabilities. Student absenteeism PAD/ KPI, 22% 18% 17% Boys are more likely to be absent their girls. ICR IO Repetition rate, grade 1 PAD/ KPI, Boys/Girls: Boys/Girls: ICR IO 13%/ 12% <10% 12%/ 11% The PAD had targets for all grades for both boys and girls. Although there were reductions in many of the rates, only one of Repetition rate, grade 5 PAD/ KPI, Boys/Girls: <5% Boys/Girls: the targets was met. Boys and girls had similar performance. ICR IO 6%o/5 5% 4%/ 4% Dropout rate, grade 1 ASPR KPI, 13% 4% IO Dropout rate declined from all grades, except for grade 5 where Dropout rate, grade 5 ASPR KPI, 00 110 it increased substantially. I0 Primary cycle efficiency PAD/ 7.5. . 10 8.1 years 8.0 years Only small increase in efficiency. ICR years Other indicators Spending on education PAD/ 2.8% 2.3% of I PCR80 of GDPGD ICR KI28 ofGPof GDP GDP These indicators do not measure the impact of the program on primary education and do not form part of the causal chain to Share of education for PAD/ KPI 37% 45% 45% achieve the objectives. primary education ICR Note: Data are from the program documents (PAD/ICR or the Annual Sector Performance Report (ASPR). Key performance indicators in the Annual Sector Performance Report are referred to as KPI. In the view of the PPAR, final outcome indicators that measure the objectives are referred to as FO. Intermediate outcomes are referred to as 10. Indicators that measure outputs are referred to as OUT. 70 Annex C. List of Persons Met M. Musharraf Hossain Cabinet Secretary and former Cabinet Division, Bhuiyan Primary and Mass Education Government of Bangladesh Secretary Joint Secretary and former Joint Ministry of Education, Project Director, PEDP II Government of Bangladesh Additional Secretary and Ministry of Primary and Shyamall Kanti Ghosh Director General, Primary Mass Education, Education Government of Bangladesh Ahasan H. Mansur Executive Director Policy Research Institute, Government of Bangladesh National Academy for A.K.M. Abdul Awal Director General, Former Paning adeDeveop Mazumder Primary Education Secretary Gonent of Ba'ladesh Govemnment of Bangladesh Christine Wallich Senior Advisor and Former IEG (World Bank) Country Director Amit Dar Manager, Education Sector, World Bank South Asia Hena G. Mukherjee Sr. Education Specialist and World Bank Former Task Team Leader Ayesha Vawda Senior Economist, Task Team World Bank Leader Shobhana Shohale Senior Operations Officer World Bank Susan Opper Consultant, former Task Team World Bank Leader Syed Rashed Al Zayed Economist World Bank Subrata Dhar Sr. Operations Officer World Bank Dr. Rudaba Khondker Program Head-Education BRAC Monoser Hossain Khandker Program Coordinator-Education BRAC Profulla Chandra Barman Program Coordinator- Education BRAC Naima Sultana Monika Asst. Manager- Visitor BRAC Communication Gregory Adams Acting Office Director USAID Mohammed Shahidul Islam Education Team Leader USAID Jouko Sarvi Education Practice Team Leader ADB Brajesh Pant Lead Education Specialist ADB Ayako Inagaki Principal Education Specialist ADB Fazle Rabbani Education Adviser DFID Naveed Ahmed Chowdhury Social Development Adviser DFID James Jennings Regional Education Adviser AusAid (South Asia) Tara Painter Second Secretary (Development) CIDA Ali Md. Shahiduzzaman Education Advisor CIDA Tahera Jabeen Senior Education Advisor CIDA