THE WORLD BANK OPERATIONS EVALUATION DEPARTMENT Rationales and Results in Secondary Education Investments: A Review of the World Bank's Portfolio, 1990-2001 Gillian Perkins Director-General, Operations Evaluation: Gregory K. Ingram Director: Ajay Chhibber Manager, Sector and Thematic Evaluations: Alain Barbu Task Manager: Martha Ainsworth 2004 This paper is available upon request from OED. The World Bank Washington, D.C. ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. OED assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. OED Working Papers are an informal series to disseminate the findings of work in progress to encourage the exchange of ideas about development effectiveness through evaluation. The findings, interpretations, and conclusions expressed here are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank cannot guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply on the part of the World Bank any judgment of the legal status of any territory or the endorsement or acceptance of such boundaries. Contact: Operations Evaluation Department Partnerships & Knowledge Programs (OEDPK) e-mail: eline@worldbank.org Telephone: 202-458-4497 Facsimile: 202-522-3125 http:/www.worldbank.org/oed iii Contents Abbreviations and Acronyms .................................................................................................v Executive Summary.............................................................................................................. vii 1. Focus and Method of Review............................................................................................1 Definition of the Secondary Education Portfolio...........................................................1 Sources and Methods.....................................................................................................2 2. Context................................................................................................................................3 Bank Strategy for Secondary Education........................................................................3 Trends in Lending for Secondary Education .................................................................5 3. Findings...............................................................................................................................7 Where has the Bank Financed Secondary Education and Why?...................................7 How Are Sector Strategy and Country Conditions Reflected in Secondary Education Project Design?..................................................................11 What Factors Have Influenced Performance of Secondary Education Projects?...............................................................................................17 4. Conclusions.......................................................................................................................25 Definition of Objectives and Indicators.......................................................................25 The Strategy for Low-income Countries......................................................................25 "Hardware" Versus "Software".................................................................................26 Program Implications..................................................................................................27 Annex. Supplementary Tables..............................................................................................29 Boxes Box 1: Examples of Expected Project Benefits at the time of Appraisal ............................... 12 Box 2: Projects in Korea Helped to Promote Efficiency and Private Participation in Vocational Schooling...................................................................................................... 22 Box 3: Good Monitoring and Evaluation for Secondary Education in High- and Low- Capacity Environments................................................................................................... 24 iv Tables Table 1: Aggregate Project Costs Allocated to Secondary Education...................................... 6 Table 2: Lending for Secondary Education in Relation to Country Income Level and Primary Gross Enrollment Rate ................................................................................. 7 Table 3: Distribution of Projects by Country Conditions....................................................... 10 Table 4: Regional Distribution of Projects Supporting Secondary Education ....................... 10 Table 5: Objectives of Secondary Education Projects............................................................ 14 Table 6: Frequency of Different Project Objectives by Region ............................................. 15 Table 7: Subsectoral Scope of Secondary Projects................................................................. 15 Table 8: Project Costs by Expenditure Category.................................................................... 15 Table 9: OED Ratings for Education Projects Closing FY1990-2001 ................................... 17 Table 10: Performance Ratings of Secondary Education Projects ......................................... 18 Table 11: Performance by Project Design Characteristics ..................................................... 19 Table 12: Performance by Type of Objective......................................................................... 20 Figures Figure 1: Distribution of Education Commitments by Subsector............................................. 5 Figure 2. Lending Commitment Amounts by Education Level................................................ 6 Figure 3: Shifts in Regional Shares of Secondary Education Commitments ........................... 9 v Abbreviations and Acronyms AFR Sub-Saharan Africa Region CPIA Country Policy and Institutional Assessment EAP East Asia and Pacific Region ECA Europe and Central Asia Region FY fiscal year GER gross enrollment rate LAC Latin America and the Caribbean Region M&E monitoring and evaluation MDB multilateral development bank MNA Middle East and North Africa Region OED Operations Evaluation Department SAR South Asia Region vii Executive Summary 1. Over the past two decades, the Bank's strategy for the education sector has given less specific attention to secondary education than to primary and tertiary levels. Well before the Education For All initiative, this strategy gave priority, on economic and social grounds, to expanding access at the primary level; and it advocated lending for secondary education mainly in countries that already had universal access to primary schooling. 2. Several factors have prompted a rapid rise in the share of lending to general secondary education since the mid-1990s and are now putting a renewed focus on strategy at the secondary level. First, demand for secondary places is growing as primary completion rates are raised. Second, equitable and sustainable financing and management of secondary education, especially in low-income countries, remains a challenge. Third, the role of secondary education in economic and social development is being reassessed in relation to globalization and competitiveness in the information age. Fourth, rapid changes in technology and labor markets are driving changes in secondary education. The Bank is carrying out studies to reexamine the issues for developing countries and its own assistance strategy in the subsector. This OED paper contributes a review of project experience and lessons learned from the Bank's lending for secondary education from 1990 through 2001. 3. Bank lending for secondary education has taken a variety of channels, usually in combination with assistance to other subsectors, and there is no clear definition of the "secondary education portfolio." This study has defined it by identifying the 45 projects closing between fiscal 1990 and fiscal 2001 that allocated an equal or higher share of costs to secondary (lower and/or upper, general and/or vocational) than to pre- or post-secondary levels. The review draws on Edstats and OED project databases, project documents, and OED evaluation reports to address three questions. Where has the Bank financed secondary education and why? 4. For projects completed from 1990 to 2001 ­ the subject of this review ­ the distribution of lending broadly conformed with the Bank's strategy of supporting secondary education in countries that had already achieved wide coverage at the primary level: middle- income countries with high primary enrollment rates were more prominent in borrowing for secondary education than for other levels and there was a relatively low concentration of secondary projects in the South Asia and Africa regions. Country population size was also a factor, with large countries borrowing more often for projects dedicated exclusively to the secondary subsector. In Asia and Latin America, country selection for secondary education lending was also influenced by the activities of other multilateral development banks. In the more recent portfolio ­ projects that closed recently or are still active ­ the picture has changed substantially with a greatly increased share of lending for secondary education to low income countries in South Asia and Africa. 5. The rationale for investing at the secondary level also varied by country conditions, although a substantial minority of the projects presented no explicit rationale relating to development constraints or benefits beyond the education system itself. In countries with viii high secondary enrollment rates (usually middle-income countries), two approaches were evident: improve the employability and productivity of school leavers through support to vocational secondary schools, when specific shortages of vocational and technical skills had been identified and high unemployment was evident among the graduates of academic secondary schools; or increase the country's international competitiveness by improving the quality of general secondary education so as to raise the overall productivity and further trainability of the labor force. In countries with low secondary enrollment rates, two different approaches prevailed: meet specific shortages of educated manpower, including in the public sector, by raising rates of secondary school completion; or improve the social conditions of the poor and reduce inequity by expanding access to secondary education. How are sector strategy and country conditions reflected in secondary education project design? 6. Shifts in the balance of project objectives and components over the review period followed the direction of the Bank's sector strategy, incorporating a stronger focus on "software" for education quality and sector management, and on poverty issues. Still, even in the latter period, only a minority of secondary education projects had explicit poverty or equity objectives. The balance of secondary project activities shifted away from civil works, but the average share of costs allocated to civil works remained higher than in the education portfolio as a whole. Other elements of sector strategy, particularly the promotion of private schools, and cost recovery with scholarship schemes, did not feature prominently in project design. Support to secondary education in Africa, in low-income countries, and in countries with small populations, was more often combined in projects that included assistance also to the primary and/or tertiary sectors. What factors have influenced performance of secondary education projects? 7. Since objectives were often not clearly defined or related to relevant indicators of outcomes, outcome ratings tended to be based largely on the extent to which project components were implemented as planned. On outcome ratings, the secondary portfolio compared well with the education portfolio as a whole and with Bank-wide averages, with 89 percent of projects rated Satisfactory, while 67 percent were rated Likely or Highly Likely on sustainability. Performance on institutional development impact was considerably weaker, with fewer than half the projects rating Substantial or High, but still comparing well with sector and Bank averages for the period. 8. Country conditions were a major factor in project performance, with outcomes of the secondary education projects being even more sensitive to country income level than for the education portfolio as a whole. Projects focusing heavily on civil works for expansion of schooling capacity had better outcomes on average than other secondary projects but much weaker institutional development impact. Some 60 percent of the projects that aimed to improve education quality or relevance were largely successful in this ­ at least in terms of implementing the relevant components as planned ­ but only one in three projects succeeded in their efforts to improve sector efficiency, financing, planning, and/or management. In project completion and evaluation reports, good performance was attributed frequently to ix national ownership, government commitment and implementing capacity. Common factors in weak performance were overly complex project design and underestimation of institutional or political constraints. KEY ISSUES AND PROGRAM IMPLICATIONS 9. There are major limitations to a review of the project portfolio as a tool to focus on the issues for a specific level of education. Since the great majority of projects combine assistance to more than one level or subsector category, the relevant portfolio cannot be clearly identified. As a result, many of the findings are common to the education sector as a whole rather than specific to the secondary subsector. Also, most of the secondary-specific issues relate to policy options, and the analytical work underlying policy recommendations, which are only partially revealed through project documents. A full assessment would require a country-based review of relevant sector work as well as projects, and interviews with those involved in policy dialogue. 10. Within these constraints, this review does point to several areas warranting further analytic work and more attention in future strategy and operations: · the need for better definition of project objectives, projected benefits, and monitoring indicators; · strategies to address the particular challenges for development of secondary education in low-income countries, where project performance has been relatively weak; · analysis of why sectorwide investment projects, incorporating assistance to primary and tertiary as well as secondary levels, have tended to be unsuccessful; · more attention to the necessary conditions for successful investments in "software," especially in improving sector management and efficiency, which have performed much less well than "hardware" investments in secondary education projects. 11. Addressing these concerns could depend on deeper and broader country-level analysis and on more selectivity in the scope of interventions. Better analysis within and beyond the education sector is needed (i) to draw explicit linkages between expected project outcomes and broader development goals for economic growth, social development, and poverty reduction, (ii) to monitor distributional and poverty impacts, even when poverty reduction is not a specific project objective, (iii) to assess real constraints and opportunities and ensure that project priorities and design match locally perceived priorities and institutional capacity. The Bank's support to secondary education would be greatly strengthened through systematic prior analysis and ongoing tracking (including graduate tracer studies) of the linkages with labor force conditions and poverty/equity impacts. 1 1. Focus and Method of Review 1.1 This review seeks to identify recent trends and lessons learned from the Bank's experience in lending for secondary education through the 1990s. Specifically, it addresses the following questions: · In which countries and regions has the Bank financed secondary education and why has the Bank lent for secondary education projects in the countries it has? · How have secondary education project objectives and components reflected (i) the evolution of Bank strategy for the education sector and (ii) country conditions? · What can be said about the determinants of performance of secondary education projects according to country conditions, project objectives and design? DEFINITION OF THE SECONDARY EDUCATION PORTFOLIO 1.2 The term "secondary education" is broadly used to refer to the grades between primary and tertiary education, typically grades 6 or 7 to 12, with variations between countries in the structure of schooling and the pupil age range. Secondary school programs may be academic, diversified, comprehensive, or vocational, and may be designed to prepare students for tertiary education, white-collar jobs or manual occupations. In addition to the aims of increasing labor productivity and meeting a country's manpower needs, secondary education may have social objectives, such as supporting social cohesion and democratic participation (promoting knowledge, attitudes, and behaviors for good citizenship), enhancing life skills (health, nutrition, parenting), and reducing youth crime. 1.3 Distinctions are often made between lower and upper secondary levels and between general and vocational secondary education. Lower secondary may be known as middle, intermediate, or junior high school, while in some cases the term secondary is applied only to upper secondary or high school. At the same time, the primary/secondary distinction is overlaid by the increasingly prevalent distinction between "basic" and "post-basic" levels, with the definition of "basic" varying among countries from primary grades only to the inclusion of some or all secondary grades. For this review, we have used the broad definition of secondary education, to include lower and upper, general and vocational. 1.4 Bank lending for secondary education has used a variety of channels: investment projects focused exclusively on the secondary subsector; projects combining support to secondary with primary and/or tertiary levels; and, increasingly, through sectorwide and even multisectoral investment and adjustment operations. There is no one clear definition, therefore, of the Bank's "secondary education" portfolio.1 This review has used the appraisal allocation of project costs between pre-secondary, secondary, and post-secondary education (from the Bank's Edstats database2), as the best available indicator of the significance of the 1. The official "secondary education" subsector classification of projects is not a reliable indicator for this purpose ­ it has been applied inconsistently and does not identify all projects with substantial secondary components. 2. EdStats is a comprehensive database of education statistics that provides single-point access to educational data from a variety of national and international sources (such as UIS and OECD). It also compiles Bank data 2 secondary subsector in a project. The review focuses primarily on the 45 investment projects (see Annex Table A2) that closed from between fiscal 1990 and fiscal 2001 and allocated an equal or greater share of costs to (general and/or vocational, lower and/or upper) secondary than to either pre-secondary or post-secondary education.3 Of all the Bank's education projects approved since 1980, one in five has met these criteria, while about one in three has included some assistance to the secondary subsector. 1.5 The review does not cover support to secondary education through adjustment operations. Five of the ten education sector adjustment loans closing from FY1990-2001, all in Sub-Saharan Africa, included secondary education among listed subsector categories. One was combined with the health sector. Objectives of these operations relating to secondary education included: restructuring of the system and introduction of curriculum changes; restructuring of public finance in favor of the primary level; and increasing student retention rates. In no case was secondary education the main focus of the operation. SOURCES AND METHODS 1.6 The review questions were addressed first through quantitative analysis of project information from Edstats and OED project databases, together with country-level information from World Development Indicators and Country Policy and Institutional Assessment (CPIA) ratings, and summary information on project objectives and components from OED's project Evaluation Summaries. Patterns were identified, although statistical analysis was limited by the small size of the sample (45 projects in 34 countries) combined with the large number of relevant variables. Issues emerging from these findings were then followed up through review of information in the project appraisal and completion documents, and in OED project assessment reports, to shed light on the rationale behind these investments, and the factors influencing outcomes. These sources, alone, provide only a partial picture of how policy issues specific to the secondary sector have been addressed. They point to the need for further work, including a review of relevant country-level sector work and interviews with those involved in policy dialogue, and an assessment of what changes or improvements are evident in the active portfolio. on lending operations and pertinent education topics and provides links to a large number of education websites worldwide (see http://www1.worldbank.org/education/edstats/). 3. The breakdown of project costs by subsector in the Edstats database includes: preprimary, primary, secondary vocational, secondary general, university, science and technology, polytechnic, technical institute, teacher training, vocational, labor market/social security and employment, institutional development, and "other" (such as textbooks). To estimate the significance of the secondary level for a project, it can be misleading to take simply the percentage of costs allocated to secondary vocational and/or secondary general education because a substantial share of costs may be, for example, for textbooks or teacher training for the secondary subsector. Also, there appears to be some inconsistency in whether costs of teacher training, textbooks, or institutional development, are allocated to the relevant education level(s) or to separate categories. Therefore a "secondary" project, as defined for this study, includes projects that report an equal or greater share of costs to vocational and/or general secondary than to either pre- or post-secondary categories, but this share may be less than 50 percent of the total if costs are also allocated to categories that are not differentiated by level (such as teacher training and other). The costs are the total project costs estimated at appraisal, not the loan amount and not actual expenditures. 3 2. Context BANK STRATEGY FOR SECONDARY EDUCATION 2.1 The Bank's strategy for the education sector over the past two decades has given less specific attention to secondary education than to the primary or tertiary levels. From the 1980s, two factors contributed to increasing the priority given to investment in primary education: the growing influence of rate of return analysis, replacing manpower planning as the basis for public investment decisions in education; and growing emphasis on the poverty and equity impacts of Bank operations. The same factors prompted the Bank to promote cost recovery and reduction in public subsidies at the tertiary level, because of the high private returns at this level and the inequitable incidence of public expenditure. Secondary education, falling in the middle, was considered a priority mainly in middle-income countries that already had universal access to primary or basic education. 2.2 Before the 1980s, Bank support to secondary education had focused on the expansion of capacity and enrollments to meet manpower needs, primarily through financing school construction and equipment. The need for more attention to education quality and internal efficiency was signaled in the 1980 Sector Policy Paper, which proposed that the Bank finance secondary expansion only in those countries that had achieved adequate education quality at acceptably low unit costs. 2.3 A policy note in 19864 focused on the misallocation of public resources across levels of schooling, highlighting the inequity and inefficiency of subsidies for higher education. It prescribed cost recovery (with credit markets) at this level and the reallocation of public resources to the primary level, where the social returns were highest. In secondary education, it proposed increased cost recovery accompanied by selective scholarship schemes, and the encouragement of private and community schools to improve education quality and efficiency through competition. 2.4 Endorsing this position, the 1995 strategy paper5 advocated cost-benefit analysis to determine country-specific public expenditure priorities and reinforced the priority to primary education, where social rates of return were found to be highest in most developing countries. The focus was shifted, however, from "primary" to "basic" education, to include lower secondary grades in many cases. Basic education would continue to receive the highest priority in the Bank's education lending to countries that had not yet achieved universal literacy and adequate access, equity, and quality at that level; as the basic education system developed in coverage and effectiveness, more attention could be devoted to upper secondary and higher levels. 2.5 On the question of vocational versus general secondary education, the Bank's position has been increasingly firm since the 1980s in discouraging support to vocational secondary schooling, due to its high unit costs and mixed outcomes. Bank policy studies in 4. Financing Education in Developing Countries, World Bank, 1986 5. Priorities and Strategies for Education, World Bank, 1995 4 the early 1990s6 highlighted the poor cost effectiveness of vocational school programs and their weak linkage with employment opportunities, while the 1995 strategy paper cited the much higher returns to general than to specialized vocational secondary education. 2.6 Since the mid-1990s, the value of the Bank's use of rate of return analysis in the sector came under question and its role in informing sector strategy has diminished. The most recent sector strategy paper, in 1999, takes a more holistic, sectorwide perspective.7 Many of its prescriptions are applicable to the secondary as to other levels: identifying context-specific reform alternatives; better monitoring and empirical research to guide reform efforts; focus on participation and client ownership; comprehensive analysis but with selectivity in the actions supported; emphasis on quality through continuing the shift from "hardware" to "software;" concentration on development and poverty impact, knowledge management, and partnerships. Little specific attention is given to secondary education itself. The global priorities for the Bank are basic education for girls and the poorest, early childhood intervention, innovative delivery including the application of new technology, and systemic reform to boost achievement, reform governance, and promote non-government investment. For post-basic education, the strategy emphasizes private participation in school financing and provision, to promote competition and innovation and allow public resources to be better targeted to the poor. 2.7 Within this framework, the Bank's regional education strategies reflect some variation in the issues and priorities related to secondary education. The strategy for Latin America and the Caribbean (LAC) has the most specific focus on the secondary level, aiming to support alternative approaches to increasing secondary access, together with curriculum reforms to improve relevance to the global economy. The lack of relevance of much secondary education, including vocational education, to requirements of the market economy is also identified as a major issue for the Middle East and North Africa region (MNA) along with gender and other social gaps in participation at post-compulsory levels. In the Sub-Saharan Africa region (AFR), distance learning and secondary math and science are highlighted as areas where the knowledge base needs to be strengthened. For South Asia (SAR), improving access for girls at the secondary level is a high priority. The strategy for East Asia (EAP) focuses on equity and quality concerns at all education levels and emphasizes the need for diversified strategies related to specific country conditions. For Europe and Central Asia (ECA), the focus is on systemic change across all education levels to allow re-alignment to market demands, reform of governance, and improved efficiency, equity, and sustainability. 2.8 Bank sector strategy has yet to address in depth many of the issues specific to the secondary education level, and to relate structural, curriculum, financing, and governance options to specific development objectives ­ human, social, or economic ­ in different country contexts. Not all borrowers have been persuaded by the Bank's arguments against vocational secondary education ­ an issue that may warrant a fresh (and less dichotomous) approach in the light of country-specific conditions and objectives. Inequities in access to secondary schooling, and the strength of vested interests, call for particularly careful assessment of the distributional and poverty impacts of Bank assistance to the subsector. Globalization and the 6. Vocational and Technical Education and Training: A World Bank Policy Study, 1991, and Skills for Productivity, The World Bank, 1993 7. Education Sector Strategy, The World Bank, Human Development Network, July 1999 5 requirements for competitiveness in the information age have raised new challenges for the secondary sector. The need for more work has been recognized, in particular through the Education Network anchor's planned FY04 strategy paper on secondary education, and a major ongoing study of this subsector in Sub-Saharan Africa, also due to be completed in FY04. TRENDS IN LENDING FOR SECONDARY EDUCATION 2.9 The broad trends in education lending since the 1960s (Figure 1) reflect the evolution of the Bank's sector strategy. Most of the Bank's early lending for education, in the 1960s and 1970s, was directed to general and vocational secondary schooling. This share diminished in the 1980s, in favor of the tertiary and primary subsectors, and shrank further in the early 1990s, as yet higher priority was given to financing primary education. Several factors, however, prompted a renewed focus on the secondary subsector and a rapid increase in the share of lending to general secondary education in the late-1990s: growing demand for secondary places as primary completion rates are raised; the challenge of financing and managing secondary education on an equitable basis, especially in low-income countries; and the need to adapt secondary schooling to match rapid changes in technology and labor markets, with the focus on "lifelong learning." Figure 1: Distribution of Education Commitments by Subsector 100% Other 80% High education Teacher training 60% Vocational post-secondary education 40% Vocational secondary education General secondary education 20% Primary education 0% Pre-primary 1963- 1970- 1980- 1990- 2000- 1969 1979 1989 1999 2003 Source: World Bank Edstats database 2.10 The share of education lending allocated to the secondary level reached an annual peak of 32 percent in 1997, since when it has leveled off around an average of 16 percent. In absolute terms, the amount of lending for secondary (Figure 2) peaked in 1995-1997, at US$1,169 million for the 3-year period, falling to US$ 715 million for 2000-2003 ­ a fall of 38 6 percent, compared with the 14 percent decline in overall lending for education between the same periods.8 Figure 2. Lending Commitment Amounts by Education Level 6000 Other 5000 Post-secondary Secondary 4000 Pre-secondary (millions) $SU 3000 rrentuC2000 1000 0 1983-1985 1986-1988 1989-1991 1992-1994 1995-1997 1998-2000 2001-2003 Source: World Bank Edstats database 2.11 Measured by project costs at appraisal, the aggregate share of commitments to secondary education within education operations grew from about 13 percent in projects closing in the last decade (the subject of this review) to 22 percent in the active portfolio. In line with sector strategy, this growth has been entirely in general secondary schooling ­ financing for secondary vocational education has all but disappeared in the active portfolio (Table 1). This subsectoral distinction may have become redundant, as the dichotomy between vocational and general secondary schooling is less sharp and the curriculum relevance debate incorporates vocationalization of general secondary education. Table 1: Aggregate Project Costs Allocated to Secondary Education (current US$) Education Projects Closing FY1990-2001 Projects Closing Project cost allocation Approved pre-FY90 Approved FY90+ from FY02 or still (n=122) (n=110) Active (n=181) US$m. % US$m. % US$m. % Vocational secondary 216 2% 420 3% 34 0% General secondary 846 8% 1,710 11% 8,137 22% All secondary 1,062 11% 2,130 14% 8,171 22% All education 9,959 100% 15,091 100% 36,450 100% Source: Edstats project database of cost allocation at appraisal by subsector, aggregation across all education sector projects. 8. These data include only those projects that have been defined as education projects, under the Education Sector Board. The recent fall in overall lending for education has been offset by an increase in lending for education through multi-sectoral projects mapped to other sectors, such as Social Protection (social fund projects) and Poverty Reduction and Economic Management (adjustment loans including Poverty Reduction Support Credits). 7 3. Findings WHERE HAS THE BANK FINANCED SECONDARY EDUCATION AND WHY? Characteristics of Borrowing Countries 3.1 The Bank's sector strategy over the review period advocated lending for secondary education in countries that already had effective primary education systems with broad coverage. For the period covered by this review ­ operations closing from FY1990-2001, and hence approved between the mid-1980s and mid-1990s ­ the characteristics of borrowing countries did show a tendency in this direction. Of 136 low-income and middle-income countries, 86 borrowed for education, and 54 of these had at least some borrowing for the secondary level (Table 2 and Annex Table A4). On average ­ and many exceptions are concealed by the averages ­ countries borrowing for secondary education had higher gross enrollment rates at the primary level, stronger policy and institutional conditions,9 and higher per capita incomes than those borrowing to finance other levels of education. Effectiveness of the primary system, as measured by youth literacy rates, appears to have been less of a constraint to including some support to the secondary subsector within education projects, but countries borrowing for dedicated secondary-only projects did have higher rates of youth literacy. Countries not borrowing at all for any level of education (50 of the 136 countries) had, on average, higher incomes and enrollment rates but weaker policy and institutional conditions. Table 2: Lending for Secondary Education in Relation to Country Income Level and Primary Gross Enrollment Rate Projects closing 1990-2001 No. of Av. GNI Av. Av. countries per capita Youth Prim. US$ illiteracy GER Having education project(s) with no costs to 32 841 27% 79% secondary Having education project(s) with some but not all 36 1,267 27% 89% costs to secondary Having education project(s) with all costs to 18 1,788 17% 93% secondary Source: World Development Indicators and Edstats project database 3.2 Population size ­ economies of scale ­ may also have been a factor (Annex Table A9). Highly populous countries have been most likely both to borrow for education and to finance development at the secondary level through dedicated, secondary-only projects: countries with populations above 50 million represented one in eight of the 136 countries for which data are available, but one in five of the countries borrowing for education, and one in three of the countries with secondary-only projects. When countries of medium or smaller population size borrowed for secondary education, this was more often through a project combining assistance to other education levels. 9. As measured by Country Policy and Institutional Assessments (CPIA) made annually by the Bank, based on indicators of current policy conditions and institutional capacity. 8 Specialization Among Development Banks 3.3 Some country specialization, or division of labor, between the multilateral development banks (MDBs) in the education sector also appears to have been a factor in determining which countries the Bank focused on for support to secondary education, at least in Asia and Latin America. Of 23 Asian countries with recent MDB-funded secondary education projects, only about four had borrowed from both the World Bank and the Asian Development Bank. In LAC, even fewer countries borrowed for secondary education from both the World Bank and the Inter-American Development Bank. In Africa, it was much more common for countries to borrow both from the World Bank and the African Development Bank. Recent Shifts in the Regional Distribution of Commitments 3.4 The regional distribution of lending commitments for operations closing from FY 1990-2001 reflected the dominance in borrowing for secondary education of middle-income countries with high primary enrolment rates (Figure 3). East Asia (EAP) accounted for a full 40 percent, in dollar terms, of the lending for secondary education while the Middle East and North Africa (MNA) and Latin America and Caribbean (LAC) Regions also featured prominently. Africa (AFR) and South Asia (SAR), on the other hand, with more countries still falling far short on primary enrollments, accounted together for only 15 percent. Lending for primary-secondary education in Eastern Europe and Central Asia (ECA) transition countries took off at a relatively late stage in the transition, explaining the small representation of ECA in the this period ­ that is, among projects that were approved from the mid-1980s to mid-1990s. 3.5 The picture shifts radically for more recent lending on operations that closed very recently or are still active. South Asia now accounts for the largest regional share of commitments for secondary education, due to a massive operation in Pakistan and large projects in Bangladesh ­ both low-income countries. Sub-Saharan Africa's share has also grown substantially, through commitments to 18 low-income countries including Ethiopia, Mali, Mauritania. In rough per capita terms, borrowing for secondary education increased substantially in all regions except EAP, while MNA continued to be the largest borrower per capita (excluding the high income countries in the region), and AFR and SAR saw the biggest increases ­ 420 percent and 1130 percent respectively. 3.6 The focus of this review is on the earlier portfolio of projects that have been closed and evaluated but it is notable that low-income countries without universal primary enrolment have subsequently become much more significant in borrowing for secondary education. The shift may reflect improvements in primary enrolment rates since the mid- 1990s in a number of low income countries as well as the more holistic approach to education assistance advocated in the 1999 Sector Strategy Paper. In line with sector policy, commitments for vocational secondary education, which had been highly concentrated in upper-middle income countries (Korea, Turkey, Brazil, Mexico), shrank to a mere US$34 million in the recently closed and active portfolio. 9 Figure 3: Shifts in Regional Shares of Secondary Education Commitments Projects Closing FY90-01 More Recent and Active Projects SAR AFR 6% AFR 9% SAR 17% MNA 27% 19% EAP 12% EAP 40% LAC MNA ECA 20% 17% 14% ECA LAC 6% 13% The Bank's Secondary Education Projects 3.7 Among the 236 education projects completed in FY1990-2001, roughly one in three allocated any costs to the secondary level and one in five allocated more resources to secondary than to pre- or post-secondary levels. Of the 45 secondary education projects, as defined for this review,10 closing in FY1990-2001, 31 were in countries with estimated 1995 primary gross enrollment rates (GERs) above 90 percent; only two were in countries (Ethiopia and Tanzania) with primary GERs below 70 percent at the time. Secondary projects were rare relative to other education projects in low-income countries and in countries with high rates of adult illiteracy (Table 3). This distinction is smaller in the more recent portfolio of active projects: the proportion of "secondary projects" located in low income countries increased from 33 percent to 48 percent, compared with the new average of 52 percent for all education projects (Table A6). Vocational secondary projects, which had been especially concentrated in upper-middle income countries, are virtually absent in the current education portfolio. 10. Projects allocating an equal or greater share of costs at appraisal to (general and/or vocational, lower and/or upper) secondary than to either pre-secondary or post-secondary levels. 10 Table 3: Distribution of Projects by Country Conditions Share of projects (closed FY1990-2001) that were in Country characteristic countries with that characteristic All education projects Secondary projects (n=236) (n=45) Low income 53% 33% High adult illiteracy (>30%) 49% 31% High CPIA rating11 37% 47% 3.8 The regional distribution of these secondary education projects, with a relatively small concentration in the South Asia and Africa portfolios, reflected the dominance of middle-income countries with high primary enrollment rates. Of the 45 "secondary education projects," which closed from FY1990-2001, the largest number (13) were in EAP, followed by (10), LAC (8), MNA (7), SAR (4), ECA12 (3)(Annex Table A4). The share of secondary projects within each regional portfolio (Table 4) was highest in EAP, and lowest in SAR and AFR ­ the predominantly low-income regions where universal access to primary education is still a significant challenge and labor markets are less developed. Table 4: Regional Distribution of Projects Supporting Secondary Education (percent of each Region's projects) AFR EAP ECA LAC MNA SAR Total All projects approved FY80-02 All education projects 137 95 33 107 54 51 477 Projects with any secondary 39% 36% 61% 32% 39% 25% 37% All or mainly secondary projects 15% 22% 24% 20% 24% 14% 19% Closed FY1990-2001 All education projects 66 53 15 42 29 31 236 Projects with any secondary 41% 36% 40% 29% 48% 16% 35% All or mainly secondary projects 15% 25% 20% 19% 24% 13% 19% US$ m. allocated to secondary educ 276 1,301 195 638 606 178 3,194 Closed FY 02 or Active All education projects 46 31 18 51 18 19 183 Projects with any secondary 39% 45% 78% 43% 39% 42% 45% All or mainly secondary projects 13% 23% 28% 24% 28% 16% 21% Note. Project categories overlap: "Projects with any secondary" includes the "all or mainly secondary projects," i.e., those allocating a larger share of costs to secondary than to either primary or tertiary." 11. Country Policy and Institutional Assessments made annually by the Bank, based on indicators of current policy conditions and institutional capacity. A high CPIA ranking indicates good policy and institutional conditions, a low one poor conditions. 12. Lending for primary-secondary education in ECA transition countries took off at a relatively late stage of the transition, explaining the small respresentation of ECA in this period -- in projects that were approved from the mid-1980s to mid-1990s. 11 3.9 Support to vocational secondary education was more highly concentrated in middle- income countries than support to general secondary. The nine projects with the greatest share of resources allocated to vocational secondary were all in middle-income countries (predominantly in LAC and EAP), with high primary enrollment rates, middle or high literacy rates, and middle or high CPIA ranking. 3.10 The typical form of Bank assistance to secondary schooling in middle-income countries was a series of consecutive projects. Only thirty-four different countries were represented in the portfolio of 45 projects because as many as 8 countries ­ including 4 in EAP and 2 in MNA ­ borrowed for two or more secondary education projects within the review period alone. At least 35 of the 45 projects were following and/or immediately succeeded by other secondary education projects in the country concerned ­ including all the MNA projects, and all in EAP and LAC that were not in very small countries. The stand-alone projects were predominantly in low-income countries or in countries with very small populations. 3.11 Shifts have occurred in the regional distribution of secondary projects, including a substantial increase in the relative significance given to secondary education in ECA, LAC, and SAR. LAC increased both its share of global education projects ­ from 18 percent among projects closing in FY1990-2001 to 28 percent among the recently completed or active portfolio, and the share of secondary projects within its portfolio, replacing East Asia as the region with the largest number of secondary education projects. ECA emerges as the region with the strongest relative focus on the secondary level, with 14 of its 18 projects including at least some cost allocation to secondary education.13 HOW ARE SECTOR STRATEGY AND COUNTRY CONDITIONS REFLECTED IN SECONDARY EDUCATION PROJECT DESIGN? Country Conditions and Development Rationale 3.12 The broad rationale for investing in secondary education, as expressed in the 45 project appraisal reports, was most often based on labor market requirements in the country concerned. In a few cases it was based on the need to reduce inequity and/or improve conditions of the poor. Some examples are noted in Box 1. But more than one-fifth of the appraisal reports, in making the case for investment at the secondary level and identifying project benefits, made little or no reference to country-specific factors and development constraints beyond the education system itself. Among the projects that did link expected project benefits to specific broader development needs and objectives in the country, the following groups can be identified (in addition to two projects geared to post-conflict reconstruction): 3.13 In countries with mid to high secondary enrollment rates: · Meet shortages of specific technical or vocational skills among secondary school leavers, to raise their employability and productivity. The projects in this category 13. Many of the ECA countries combine primary and secondary schooling in single schools, explaining the large share of projects with at least some cost allocation to secondary. 12 supported vocational secondary schooling or vocationalization of general secondary schooling. In some cases, youth unemployment was a driving concern, with insufficient white-collar jobs and tertiary places for graduates of academic secondary schools. In the upper-middle income countries, private sector demand and international competitiveness were strong factors. · Raise the overall quality and productivity of the labor force to allow competitiveness in the global economy, when the quality and/or relevance of general secondary education was found to be inadequate to produce graduates with the cognitive and problem-solving skills and trainability needed by modern sector employers. All of these projects were in middle-income countries (most upper-middle) with high secondary GERs and mid or high CPIA rankings. Box 1: Examples of Expected Project Benefits at the time of Appraisal · Korea, Vocational Schools Development Project: By ensuring that training was attuned to the high technology equipment used in the industrial, commercial, and agricultural sectors, the project would increase the value of graduates to prospective employers, and assure the continued employability of graduates. · Malaysia, Third Primary and Secondary Education Sector Project: The projected benefits were mainly qualitative: the quality of inputs and the emphasis on research, M&E, and management would enhance the system's ability to produce graduates equipped with the skills, attitudes, and values required to maximize the productivity of the large public and private investments planned in knowledge-based industries, vocational/technical training, and higher education. Greater flexibility would prepare the education system to meet the skills demands of an increasingly complex, competitive, and volatile economic environment. · Maldives, Education and Training Project: The project would alleviate shortages of skilled and educated manpower; increase availability of people with adequate education levels to take up training opportunities; increase access to secondary education opportunities, especially for girls; and increase responsiveness of education and training to labor market needs. · Ghana, Community Secondary Schools Construction Project: The project would improve access to senior secondary schools by youths from underserved areas through increasing enrollments; support the community initiative approach; and increase community ownership of schools, helping to ensure that schools would be used for a wide range of activities of importance to the community. 3.14 In countries with low secondary enrollment rates: · Meet specific shortages of educated manpower primarily in the public sector, when higher rates of secondary school completion were needed to alleviate a shortage of educated manpower including for public sector and teaching positions. These projects supporting general secondary education, all in countries with particularly low secondary GERs, negligible unemployment of secondary graduates, and employment of expatriates in essential public sector positions. Five of the projects were in small island nations. The countries varied in income level. 13 · Reduce social and economic inequity by helping to meet the social demand for secondary education among less privileged groups, improving social conditions for the poor. The borderline is difficult to draw between projects in this category and those that made no specific reference to development benefits beyond the education sector, or to existing labor market conditions, but supported expansion of access with implied benefits for equity. Projects' Stated Objectives 3.15 Systematic treatment of project objectives is difficult because of lack of consistency, and sometimes clarity, in the way project objectives were framed. In some cases, objectives were stated in general terms, such as "improve the quality of secondary education" or "increase efficiency in education management," although often only project outputs ­ inputs to the education system ­ were planned to be monitored and their link with project objectives was assumed. In others, objectives were highly specific and related to outputs rather than impacts, as in "to undertake priority studies in the education sector." Often they were framed as "assist the government in its plan to.... by providing..." A clear connection in the appraisal document between the stated objectives and the expected benefits was not always evident. Objectives and Design Relative to Bank Sector Strategy 3.16 In terms of project objectives and design, Bank strategy for the sector has indicated the following for secondary education: balancing objectives for expansion of enrollments with increasing focus on education quality and internal efficiency, financing the "software" needs ­ including training and technical assistance to support systemic reform ­ as well as construction and equipment; promoting quality, relevance, and efficiency through encouraging private and community schools, and by promoting general over vocational secondary education; improving equity in public expenditure by increasing cost recovery and selective scholarship schemes when secondary coverage is not broad. The portfolio reflected some but not all of these themes: the focus on quality and sector management increased over the review period, and the share of expenditure on civil works declined, as did lending for vocational secondary schooling; but the emphasis in projects on private funding or delivery of secondary schooling, cost recovery, or scholarship schemes was relatively small.14 3.17 All the projects under review were approved before publication of the 1999 strategy paper, which advocated concentration on development and poverty impacts and selectivity in the actions supported, and gave priority to education of girls and the poorest, innovative delivery including use of new technology, and support to systemic reform with better monitoring and empirical research to guide reform efforts. Project objectives have indeed shifted over the review period to incorporate a stronger focus on sector management and poverty issues. Still, only a minority of secondary education projects had explicit poverty or equity objectives. 14. These elements of the strategy may have been pursued through non-lending operations. 14 3.18 The hierarchy of objective types, as indicated by the frequency with which each occurred among the 45 secondary projects (Table 5), was headed by (i) improving education quality and (ii) improving sector efficiency and/or management ­ the great majority of projects included both these objectives. Predictably, the objective of expanding enrollments was less frequent in upper-middle income countries ­ that is, the countries most likely to have achieved broad secondary coverage. Less predictably, and worth further investigation, poverty, equity, and gender objectives were particularly rare in low-CPIA countries. 3.19 Separating the projects into those approved before FY90 (24) and those from FY90 (21) reveals some important shifts in emphasis over the time period, in line with the sector strategy to focus on development software and poverty impact. The objective of expanding enrollment capacity dropped from two-thirds to half of the projects, while the percent with a poverty reduction objective doubled, but was still less than half of projects. (The first of these shifts would be explained in part by the higher share in the later period of projects in countries that already had relatively high enrollment rates.) Table 5: Objectives of Secondary Education Projects (45 projects closed FY1990-2001) Percent of projects that included each type of objective Objective type Approved pre-FY90 Approved FY90+ Total (n=24) (n=21) (n=45) Quality 96 90 93 Efficiency/planning/man. 83 90 87 Expansion of capacity 67 48 58 Labor market relevance 33 38 36 Poverty/equity 21 43 31 Gender 17 10 13 Regional Variations 3.20 Regional variations in sector strategy are evident in the differences between regions in secondary project objectives (Table 6). Some of the differences are clearly related to prevailing country conditions and priorities. For example, gender objectives were most prevalent in SAR and MNA, and expansion of capacity in MNA, SAR, and AFR. Others are more difficult to explain. All the SAR projects included explicit poverty or equity objectives, compared with few or none in EAP and ECA, and only one in four in LAC. MNA projects had a particularly strong emphasis on expansion of capacity and less than other regions on quality or efficiency. 15 Table 6: Frequency of Different Project Objectives by Region Percent of projects that included each type of objective Region Efficiency/ Expansion of Relevance to Poverty/ (no. of projects) Quality management capacity labor market equity Gender AFR (n=10) 90 80 70 40 40 10 EAP (n=13) 92 92 54 31 15 15 ECA (n=3) 100 67 33 67 0 0 LAC (n=8) 100 100 25 38 25 0 MNA (n=7) 86 71 86 14 29 29 SAR (n=4) 100 100 75 50 100 75 Variation in Project Scope 3.21 Table 7 indicates the subsectoral scope of the 45 secondary education projects, more than half of which also included assistance to pre- and/or post-secondary education. Combining assistance to general and vocational subsectors was relatively rare: vocational secondary was more likely to be combined with tertiary education, and general secondary with primary. Four of the seven projects that included primary, secondary and tertiary components were in low-income countries, and five were in the Africa region. Table 7: Subsectoral Scope of Secondary Projects (Number of projects closed FY1990-2001) General Vocational General and Total Secondary Secondary Voc'l Second. Secondary alone 11 6 2 19 With primary 11 0 2 13 With tertiary 3 2 1 6 With prim. And tertiary 4 1 2 7 Total 29 9 7 45 Balance of Project Expenditures and Activities 3.22 The secondary education projects, on average, allocated a higher share of expenditures to civil works (45 percent) and equipment (27 percent) than did the education portfolio as a whole, and smaller shares for textbooks, training, and technical assistance (Table 8). The share of resources for training and technical assistance in secondary projects was below the average for all education projects. Table 8: Project Costs by Expenditure Category Share of Aggregate Project Costs All educ Secondary projects closed FY1990-2001 projects Approved pre-FY90 Approved FY90+ Total closed (n=24) (n=21) (n=45) FY1990-2001 Civil works 52% 40% 45% 32% Furniture & equip. 28% 27% 27% 23% Textbooks 1% 9% 6% 10% Technical assistance/training/ 11% 14% 13% 18% services Other 8% 10% 9% 17% 16 3.23 Based on the breakdown of project costs by expenditure category, the composition of project activities changed between projects approved pre-FY90 and the later cohort: · The share of expenditure on civil works declined, although it remained above the average for education projects. · The significance of textbooks in total costs increased substantially. 3.24 A review of project documents indicates that, after civil works and equipment, teacher training was the most common activity, financed by four in five of the secondary projects, followed by curriculum reform (close to half the projects) and assessment systems (about one in three projects). Decentralization, school-level management, and management information systems, were each supported by about one in five projects. The focus of activities shifted in the latter part of the review period (projects approved from FY90): · More projects aimed to support assessment systems, management information systems, and financing reform. · New elements became significant: support to decentralization, school-level management, and community participation. · More attention was given to labor market information and employment services. 3.25 Despite the shifts in this direction, only about half of the projects that aimed to improve relevance of secondary education to labor markets appeared to include significant activities related to labor market information or employment services. And, although official Bank strategy had promoted private participation in financing and provision of secondary education, and cost recovery with selective scholarships at least in upper secondary, these issues were mostly addressed through proposed studies rather than implementation activities. Regional Variation in Structure of Secondary Projects 3.26 The above shifts are consistent with the evolution of sector strategy over the period but they are also related to regional patterns, since most of the African projects were approved before FY90, while most projects approved since FY90 were in EAP and LAC. Regional variations in the selection of project components and activities included the following: · The typical African project included support to several subsectors (five of the ten projects included primary, secondary and tertiary) and focused on civil works, equipment, textbooks and teacher training, usually with a central policy element, and sometimes with support to curriculum reform or assessment. · LAC projects focused on standards and curriculum specifically in the secondary subsector through a coordinated package of interventions ­ including textbooks, teacher training, curriculum reform and assessment systems ­ sometimes also with governance reform/decentralization. · EAP and MNA projects gave less emphasis than projects in other regions to curriculum reform. 17 · AFR, MNA and ECA had the highest shares of projects that combined a secondary component with larger-scale assistance to pre- or post-secondary levels (26 percent, 24 percent and 20 percent respectively, compared with only 3 percent in SAR). WHAT FACTORS HAVE INFLUENCED PERFORMANCE OF SECONDARY EDUCATION PROJECTS? 3.27 The OED project ratings cited in this section ­ for outcome, sustainability, and institutional development impact ­ are based on the evidence available at or shortly after project completion and depend in large part on the quality of monitoring and evaluation under the project. For most of the projects in this portfolio, the definition of objectives was weak and evidence on outcomes or impacts was sparse. Ratings were thus based largely on whether project components were implemented as planned. In effect, projects were rated "Satisfactory" on "outcome" when they met most of the project output targets, such as the number of schools or classrooms built, textbooks or equipment supplied, or staff trained. Project Ratings Relative to Sectorwide and Bankwide Averages 3.28 The outcome of the vast majority (89 percent) of projects in the secondary portfolio was rated satisfactory (Moderately Satisfactory, Satisfactory, or Highly Satisfactory), which compares well with the education portfolio as a whole over the same period. For sustainability, 67 percent rated Likely or Highly Likely. Performance on institutional development impact was considerably weaker, with fewer than half the projects rating Substantial, but still better on average than for the education sector or Bank as a whole (Table 9).15 Table 9: OED Ratings for Education Projects Closing FY1990-2001 Secondary Educ All Education Bankwide Projects Projects Average Outcome: % Satisfactory 89 77 70 Instit'l Dev Impact: % Substantial 47 34 36 Sustainability: % Likely 67 54 51 Performance Relative to Country Conditions 3.29 Project performance in the education sector as a whole has been especially sensitive to country income level and institutional conditions.16 In the case of secondary education, income level appears to have been a particularly important factor. Compared with the education portfolio as a whole, secondary education projects have, on average, yielded similar or worse outcomes than other education projects in low-income countries and somewhat better in low-CPIA 15. See Table A10. Out of the 45 projects, 4 were rated Highly Satisfactory on their outcome, 27 Satisfactory, 9 Moderately Satisfactory, 1 Moderately Unsatisfactory, and 4 Unsatisfactory. On sustainability, 1 project was rated highly likely, 29 were rated likely, 6 were rated unlikely, and 9 were rated uncertain. Institutional development impact for 21 projects was rated substantial, 18 were rated modest, 5 were rated negligible, 1 was not rated. 16. See OED Annual Review of Development Effectiveness 2001, p.47. 18 countries. Outcome was Satisfactory or better in 92 percent of projects in upper-middle income countries, but in only 47 percent in low-income countries (Table 10). Outcomes were no better overall for projects in low-income countries with mid or high CPIA ratings. Policy and institutional conditions, as measured by the CPIA index, appear to have had a greater influence on project sustainability and institutional development impact: only two of the seven projects in low-CPIA countries achieved Likely sustainability or substantial institutional development impact. This sensitivity of outcomes to country conditions is reflected in the differences between Regions in project ratings: Africa accounted for eight of the 15 projects in low-income countries. Table 10: Performance Ratings of Secondary Education Projects (Projects closed FY1990-2001) By Region N= Outcome17 Sustainability ID Impact % Likely or Highly % High or % Satisf. Or % Mod. % Highly Satisf.18 Likely Substantial Satisf. Unsatisf.19 AFR 10 40 30 30 30 60 EAP 13 85 15 0 69 46 ECA 3 33 67 0 67 67 LAC 8 76 11 13 75 50 MNA 7 86 0 14 86 29 SAR 4 75 25 0 100 50 By CPIA Group High 21 81 14 5 76 67 Mid 17 59 29 12 71 29 Low 7 57 14 29 29 29 By Income Level Upper-middle 14 92 8 0 86 71 Lower-middle 16 69 25 6 75 38 Low 15 47 33 20 40 33 Low income with 8 38 49 13 50 38 mid/high CPIA Total 45 69 20 11 67 47 Note: "Percent Satisfactory" ­ the summary indicator used most often in analysis of OED ratings and in Table 9, above ­ is not explicitly stated in the table but can be deduced by subtracting the figure in the Unsatisfactory column from 100% (or by adding the two preceding columns). Some improvements in secondary project performance are evident over the period under review (Annex Table A10). Comparing ratings of projects approved pre- and post-FY90, average performance on both outcome and sustainability (not on ID impact) improved noticeably. This may be explained by country conditions rather than project factors since six of the seven projects in low-CPIA countries fell in the earlier period. 17. Possible outcome ratings are: Highly Satisfactory, Satisfactory, Marginally Satisfactory, Marginally Unsatisfactory, Unsatisfactory and Highly Unsatisfactory. 18. Includes Highly Satisfactory and Satisfactory 19. Includes Marginally Unsatisfactory, Unsatisfactory and Highly Unsatisfactory 19 Performance by Project Characteristics 3.30 Looking at differences in performance related to project scope and activities (Table 11), projects combining primary, secondary, and tertiary elements performed very poorly. Country conditions may have been an important factor in these outcomes: notably, four of the seven projects concerned were in low-income countries, three also with low CPIA ranking. 3.31 Vocational secondary projects, and projects with a quality improvement "package" (supporting curriculum reform, teacher training, textbooks, and assessment system), performed well on average; more of these projects were in upper-middle income countries. Projects with a major focus on civil works were rated somewhat better than others on outcomes but were less likely to achieve Substantial institutional development impact. Table 11: Performance by Project Design Characteristics By subsectoral scope N= Outcome Sustainability ID Impact % Satisf. Or % % % Likely % Highly Satisf. Unsatisf. Mod. Substantial Satisf. Secondary only 19 74 5 21 74 47 Gen. Sec. Only 12 67 8 25 75 50 Voc. Sec. Only 6 83 0 17 67 50 Sec + primary 13 77 0 23 77 62 Sec + tertiary 6 83 0 17 83 67 Sec + prim + tert 7 29 57 14 14 0 By project activities >75% expend. On civil works 21 72 10 18 71 43 + furn and equip. <50% expend. On civil works 8 63 13 24 63 63 + furn. and equip. Quality "package"* 8 88 13 9 75 50 Total 45 69 11 21 67 47 *Project included support for curriculum reform, teacher training, textbooks and assessment system Performance Relative to Project Objective 3.32 Although there are no official ratings for performance on individual objectives or components, comments on each are typically included in implementation completion reports and evaluation documents, permitting at least a rough assessment. The figures in Table 11 are based on a review of these documents by the author. Due to the lack of information on indicators linked to objectives, these figures tend to indicate whether relevant components were implemented as planned rather than the extent to which the project achieved the expected (or assumed) outcomes and impacts. Three in five of the secondary projects aimed to expand enrollment capacity and the great majority (92 percent) were found to have succeeded on this objective. Projects were much less successful in meeting objectives related to sector efficiency, management, and planning: only one-third of projects were fully successful in this respect. 20 Performance on quality and relevance objectives fell in the middle with some 60 percent of projects succeeding ­ at least in implementing the components as planned. Table 12: Performance by Type of Objective Type of Objective N= Successful* Partly Succ* Unsuccessful* Expansion of enrollments 26 92% - 8% Poverty/equity/gender 16 63% 25% 13% Labor market relevance 16 63% 19% 19% Education quality 42 60% 31% 10% Efficiency/management/plan 39 33% 36% 31% * These are not official "ratings." They were drawn from comments in project completion and evaluation reports. Factors in Outcome Success 3.33 The country and project factors determining project performance are difficult to separate, but the information in project completion and evaluation reports reveals some relevant issues. Factors in Satisfactory Outcomes 3.34 Strong government commitment, political stability, and capacity and continuity of Bank and project implementation staff, were all mentioned often. The combination of top- down initiatives (curriculum and assessment, supply of inputs) with bottom-up, demand- driven interventions (such as school-managed improvement funds and book selection) was found to be an important factor in projects to improve education quality and/or equity, as in the Chile Secondary Education project, which was rated Highly Satisfactory. In two overlapping and very similar projects in Korea, focusing on provision and utilization of equipment in vocational schools, Highly Satisfactory outcomes were attributed to good preparation (prior to appraisal, approval by the National Assembly was secured and technical assistance was provided to prepare for shifting implementation responsibilities to local bodies), and to the capacity, experience, and commitment of the borrower. 3.35 Although projects in low-income countries performed relatively poorly, five were rated Satisfactory and one Highly Satisfactory, four of these in countries that also ranked low on policy and institutional conditions. Common factors were realism in project ambitions, commitment of the Ministry of Education, high-quality project staff, and flexibility in responding to changing conditions. Success of the (Highly Satisfactory) Solomon Islands Education II project, after a difficult start, was attributed to reducing the project scope and ambitions, placing emphasis on planning and policy formulation, good use of technical assistance, and intensive Bank supervision with frequent missions and effective coordination with other agencies. The use of piloting and demonstration proved a good approach in post- conflict conditions in Angola. 21 Factors in Unsatisfactory Outcomes 3.36 Lack of government/national ownership and underestimation (or lack of adequate assessment) of institutional constraints were the most prevalent factors cited. Over- complexity of project design, failure to anticipate or address the need for legal and regulatory changes, and lack of an adequate stakeholder assessment and participation strategy, were weaknesses that affected outcomes even in relatively favorable country conditions. The Moderately Unsatisfactory outcome of the Mauritius Education Sector project, for example, was attributed in part to shifts in government and changes in personnel, but mainly to neglect of institutional weaknesses in the course of project design and implementation, overestimation of institutional capacity and of government commitment to some components, and lack of attention to dialogue and consensus building. Some projects ran into difficulties when decision-making and implementation responsibilities had not been aligned from the outset with the decentralization framework. The Unsatisfactory outcome of the Morocco Basic Education project (half the large loan was cancelled) was attributed to its overly ambitious scale, weak government ownership, lack of adequate preparation in costing alternative strategies or identifying implementation issues, and overestimation both of Ministry financial and operational capacity and of the demand for upper basic schooling. 3.37 Although borrower performance was rated worse on average in low-income than middle-income countries, the Bank's performance, especially in preparation, appears to have been a particular factor in the poor outcomes in low-income countries. Almost two in three projects in low-income countries were rated unsatisfactory for Bank quality at entry, compared with one in three for projects in middle-income countries. Of the 11 projects in low-income countries that were rated by OED20 for Bank performance on quality at entry, seven (five of them in Africa) were rated Unsatisfactory and only one of the seven had better than Moderately Satisfactory outcomes. But only three of these seven projects were also rated Unsatisfactory for borrower performance in preparation. Supervision performance in low-income countries was rated somewhat better, with 30 percent Unsatisfactory compared with 23 percent in middle-income countries. 3.38 Unrealistic project scope, complexity, and/or schedule were found to be common problems throughout the portfolio, and particularly in low-income and low-CPIA countries. All the projects in low-income countries were either reduced in scope during the course of implementation and/or failed to fully implement some components. Unsatisfactory outcomes in four of these projects were attributed in varying measure to political or economic instability or weaknesses in government and project unit performance, but also to inadequate project preparation: over-ambitious scale; failure to mobilize political commitment or community participation, or to establish capacity and procedures for procurement and financial control; insufficient study of demand-side needs, cost effectiveness, and recurrent financing capacity. Lack of continuity of Bank staff and/or weak supervision were also noted in three of the projects. 3.39 Projects that included primary and tertiary as well as secondary components ­ accounting for only 15 percent of the portfolio but for four of the five Unsatisfactory projects ­ were found to be excessively complex and fragmented in design. This sectorwide approach 20. The Bank's Quality Assurance Group (QAG) did not report on Quality at Entry for any of the projects covered in this review. 22 was clearly ineffective in the difficult country conditions where it was mostly applied. Bank performance in preparation was rated Satisfactory for only one of these projects ­ Angola Education I ­ the design of which was unusual. Project activities, beyond rehabilitation of infrastructure and training in education management, comprised implementation of a demonstration model for basic schools and teacher training. The relative success of the project (not all its proposed activities were completed and sustainability was uncertain) despite civil unrest, macro instability, and lack of finance for the sector, was attributed to strong Bank and borrower performance in preparation, and commitment of the Ministry of Education and project implementation staff. Performance on Policy Reform 3.40 To assess the Bank's role in helping client countries address the policy issues for the secondary sector would require a country rather than project focus and would need to include a review of non-lending activities and interviews with those involved. This project-based desk review shows only part of the picture, in so far as individual projects contained and reported on policy-related objectives and components. Analysis of sector policies was often presented in some detail in appraisal reports, but projects usually addressed these issues only indirectly, by financing relevant studies and capacity building, rather than supporting actual implementation of new policies. Technical assistance, training activities and studies related to policy development were among the least successful project activities; they were not always carried out as planned and completion reports yield few examples of studies whose recommendations were adopted. Difficulties included the lack of demand and country ownership, and poor quality control. 3.41 The three secondary projects in Jordan, rated Satisfactory overall, exemplify some of the constraints that were commonly reported in ICRs: the Bank was unrealistic in expecting a study (of questionable quality) to lead to financing reforms; dialogue and consensus building needs for policy change were underestimated; training alone, without institutional incentives, proved insufficient to promote effort and change by teachers on new curriculum and teaching approaches. Box 2: Projects in Korea Helped to Promote Efficiency and Private Participation in Vocational Schooling Two overlapping projects to support vocational secondary education in Korea during the 1990s (Vocational Education Project, 1991-96, and Vocational Schools Project, 1992-98) were successful in improving efficiency through better management and maintenance of equipment and close monitoring of unit costs. They also strengthened links between employers and schools. Less participation by private schools was achieved through the projects than had been anticipated, but the projects did contribute to a growing government commitment to encourage private provision. Studies were completed and recommendations implemented, leading to better policies and management for quality and efficiency. The Highly Satisfactory performance of these projects was attributed to the strength of Bank-borrower experience in the sector, consistency of the Bank team, and the borrower's capacity and commitment. 3.42 Considering the emphasis in Bank sector strategy documents on equitable and sustainable financing ­ through promotion of private secondary schools, combined with cost recovery and selective scholarship schemes in public schools ­ the projects in this portfolio indicate that the 23 impact through lending operations has been limited, at least for general secondary education. Few general secondary projects focused strongly on these measures (beyond funding relevant studies) and none was conspicuously successful. In Mauritius, for example, the project included loans and other incentives for expansion of private schools but government commitment was too weak to support implementation. More progress was made through vocational projects, as in Mexico, China and in Korea (Box 2), in cost containment, cost recovery, and through increasing involvement of the private sector. Most vocational projects that undertook curriculum reform aimed to involve employers in the process, usually with good results. Project Rationale, Objectives, and Monitoring Indicators 3.43 From the appraisal documents, several of the projects had no explicit linkage to broader development objectives beyond the education system (see para. 3.16-3.17, above) and, for many of those that did, the linkage was vague, and not related to country-specific information or analysis. Overall, projects that focused on supporting economic growth by overcoming identified shortages or shortcomings in the skills, future trainability, employability, or productivity of graduates had better outcome ratings than those that sought in general terms to reduce poverty by expanding access to secondary education, or to promote development of the human capital needed for economic development, with little reference to the relevant country-specific labor market constraints and opportunities. 3.44 Most of the projects in this portfolio were approved before monitoring plans were a requirement in appraisal reports and it is likely that more recent projects have overcome at least some of the weaknesses found here. Project completion and evaluation reports indicate the variable quality of evidence of performance on different types of objectives: 3.45 Increasing access. Achievements on this objective are relatively straightforward to specify and measure ­ the number of new places created, the quality and efficiency of construction and equipment, gross and net enrollment rates, facility utilization rates ­ although only the first of these was consistently reported for the projects in this portfolio. Poverty, equity and gender objectives were most often pursued through construction and equipping of facilities in under-served areas, with the same indicators. None of the projects sought to track wider impacts on productivity or social indicators. Several implementation completion reports commented on the lack of attention to recurrent cost implications and financial sustainability. 3.46 Labor market relevance. Improving the relevance of schooling to labor market requirements was a common objective of vocational secondary projects, but the quality of evidence of achievement varied greatly ­ sometimes the benefits were assumed and not documented at all. Graduate employment rates were the most frequently used indicator. The use of tracer studies and employer surveys was confined to the vocational secondary projects. Of the general secondary projects that aimed to meet changing labor market needs by improving the quantity and/or quality of general secondary education (as in Mauritius Education Sector, Oman III), little or no evidence was provided of results beyond the education sector. 3.47 Education quality. Performance on quality objectives was typically judged on the basis of project outputs (numbers of teachers trained or textbooks delivered) rather than education outcomes or impacts. This can be attributed partly to inherent difficulties in impact assessment, including the time lag before impacts are evident, but also reflects vagueness in the definition 24 of quality objectives and inadequate attention to monitoring and evaluation in project design and implementation (the latter commented on frequently in both implementation completion reports and OED evaluations). Planned curriculum reforms were not linked to specific or monitorable expected benefits. Some form of teacher training was included in the great majority of projects but systematic evaluation of its effectiveness or impact was rare; and when relevant studies were included in the project they were often not implemented as planned. Two exceptions, with relatively strong monitoring of learning outcomes, are noted in Box 3. Box 3: Good Monitoring and Evaluation for Secondary Education in High- and Low- Capacity Environments The Chile Secondary Education Project, one of the later projects in the portfolio, provides a model for M&E. It was unusual not only in the high quality of monitoring, but also in emphasizing the development of learning systems as an integral aspect of improving education quality. Every activity was independently evaluated in the course of implementation, facilitating mid-course refinements, dissemination, and synergies at the school level; test results in math and language were used to monitor equity and target at- risk schools. The country's strong institutional capacity was a major enabling factor. The Angola Education 1 Project faced extremely difficult institutional conditions, civil conflict, and macroeconomic instability. Project ambitions were necessarily limited, but monitoring and evaluation was relatively strong. The Ministry of Education carried out a comparison of achievements in selected project and non-project schools, comparing changes in transition and completion rates, the number of pupils acquiring basic literacy and numeracy within 4 years, and the number reaching satisfactory achievement levels. A beneficiary assessment was carried out to reveal perceptions of the quality of education in project schools. 3.48 Financing and efficiency. Projects aimed to promote a variety of measures to increase efficiency of the secondary education system: cost recovery, double shifting, measures to reduce student dropout and repetition, consolidation of schools, reducing the share of boarding places, changes in teacher utilization, promotion of private schools, and others. From project completion reports for general secondary projects, the relevant project- financed activities and policy measures were often not implemented as planned, while the reports themselves rarely included detailed analysis of sub-sector expenditures, unit costs or cost effectiveness. Vocational secondary projects were stronger in this respect. Efficiency and cost effectiveness of the activities financed by the project itself were given little attention in most implementation completion reports. 3.49 Planning and management capacity. This category covers a wide range of sub- objectives and project activities to support management information systems, curriculum development and assessment systems, decentralization, capacity for budgeting, economic and financial analysis, planning, and policy development. Monitoring was largely confined to tracking actual versus planned implementation of proposed training, technical assistance, and studies. Good independent evaluation of their effectiveness or impact was very rare. In many cases, the proposed activities could not be implemented as planned, making monitoring of outcomes redundant. 25 4. Conclusions 4.1 There are major limitations to a review of the project portfolio as a tool to focus on the issues for a specific level of education. Since the great majority of projects combine assistance to more than one level or subsector category, the relevant portfolio cannot be clearly identified. As a result, many of the findings are common to the education sector as a whole rather than specific to the secondary subsector. Also, most of the secondary-specific issues relate to policy options, and the analytical work underlying policy recommendations, which are only partially revealed through project documents. A full assessment would require a review of relevant country sector work and interviews with those involved in policy dialogue. Within these constraints, this review does point to several areas warranting further analytic work and more attention in future strategy and operations: · the need for better definition of project objectives, projected benefits and monitoring indicators; · strategies to address the particular challenges for development of secondary education in low-income countries, where project performance has been relatively weak; · analysis of why sectorwide investment projects, incorporating assistance to primary and tertiary as well as secondary levels, have tended to be unsuccessful; · more attention to the necessary conditions for successful investments in "software," especially in improving sector management and efficiency, which have performed much less well than "hardware" investments in secondary education projects. Addressing these concerns could depend on broadening country-level analysis and being more selective in the scope of interventions. DEFINITION OF OBJECTIVES AND INDICATORS 4.2 Absent clear objectives and outcome targets, project outcome ratings tend to be based on whether components have been implemented as planned rather than whether objectives have been achieved. This indicates a need for more clarity and consistency in the way project objectives are stated and are related to wider development benefits, to project monitoring indicators, and to development of the country's monitoring and assessment capacity. Projects with quality objectives were found to lack country-relevant baseline and impact indicators for measuring change in learning over time and between regions and social groups, and assessing the cost effectiveness of strategies. Despite the importance of cost containment for access and sustainability, secondary projects rarely tracked unit costs and budget development or monitored efficiency indicators. THE STRATEGY FOR LOW-INCOME COUNTRIES 4.3 The recent focus on Millennium Development Goals and Education For All, although re-affirming the priority to primary education, has helped to increase awareness of the needs at the secondary level. Aside from the pressure of demand from primary school completers as access is expanded, the shortage of secondary-educated manpower is seen as a constraint to economic growth and to the availability of adequately educated teachers for a growing 26 primary system. This presents critical questions as to when and how much lending should be directed to secondary schooling in low-income countries, and how this lending can be made more effective. 4.4 Although secondary education projects as a whole performed well relative to other education projects, outcomes were particularly sensitive to country income level ­ more than half the projects in low-income countries had significant shortcomings. Usually, the approach in these countries was to include secondary components in projects with a broader scope, even including both primary and tertiary subsectors (especially in Africa). More work is needed to investigate whether the sectorwide nature of the projects (undue scope, complexity, fragmentation) contributed to the frequent failure of this approach, and what other project features needed to be better adapted to country conditions. Confronting the particular issues in poorer countries (for cost containment, equitable financing, sustainability, and relevance to labor markets), taking account of institutional capacity, may require more selectivity, piloting, and targeted interventions, to develop alternative approaches for financing and delivering post-primary education beyond the elite. The relatively weak ratings for Bank performance in low-income countries indicate some room for incentives to ensure that the most capable staff are applied to the most difficult challenges. "HARDWARE" VERSUS "SOFTWARE" 4.5 More than 90 percent of projects were fully successful on objectives of expanding enrollment capacity, but only 60 percent on education quality objectives, and only one in three on objectives to improve sector efficiency and management. On all types of objectives, the factors in success or failure related mainly to the quality of preparation and design in relation to country conditions, and the need for country ownership. 4.6 For construction and equipping of schools to expand secondary access, the key factors in success were: · Prior analysis of both student demand and tertiary/labor demand to avoid the danger of underutilization of new facilities · Prior assessment and ongoing monitoring of recurrent costs · Identification of sites, or at least clear principles for site selection and acquisition, prior to appraisal · Defining and engaging community participation, where relevant, at an early stage · Accounting for cultural as well as physical factors in building design · Demand-driven/decentralized assessment of individual school needs · Alignment of decision-making and implementation arrangements with the decentralization framework · Ensuring project implementation capacity, especially in procurement and financial management, prior to appraisal. 4.7 On the more difficult challenges of providing the "software" for systemic improvements in quality and efficiency, projects widely underestimated the requisite depth and length of engagement in the sector. In this respect, key factors in success were: 27 · Prior analysis of social demand for secondary education, and demand of tertiary institutions and formal and informal labor markets, to help define priorities · Prior analysis of recurrent costs and the distributional impact of public expenditure, to allow ongoing monitoring of equity, efficiency, and financial sustainability · In-depth prior assessment of institutional and political constraints · Prior investment in stakeholder analysis and dialogue; ensuring prior approval of parliament and/or legal and regulatory changes where relevant · Matching project ambitions and proposed schedule to institutional conditions and strength of local commitment, limiting when appropriate to piloting and demonstration · Combining supply-driven and demand-driven interventions for school improvement; avoiding a uniform approach to improving quality, especially in large countries, and empowering local and school authorities to respond to diverse needs · Ensuring both quality control and local participation/ownership of studies. PROGRAM IMPLICATIONS 4.8 Addressing these issues will require an in-depth and systematic focus on country- specific factors in defining priorities and designing interventions. Since the projects in this portfolio were approved, the Bank has already made relevant changes in practice: decentralization of staff to country offices would be expected to increase the depth of country knowledge applied to designing operations and maintaining dialogue; new programmatic lending instruments allow more flexible support over a longer period of time than traditional projects; and monitoring and evaluation requirements have been strengthened. The findings in this review highlight the importance of distinguishing, as the Bank's 1999 Sector Strategy does, between comprehensiveness in analysis but selectivity in operations/interventions. They also point to the need for better integration of project objectives and design in the broader institutional and poverty reduction framework. 4.9 From appraisal documents, the analytical basis for projects in this portfolio, and especially for those supporting general secondary education, was largely confined to the education sector itself, lacking anchorage in broader analysis of country economic, financial, institutional, social, and labor market conditions. Better analysis within and beyond the education sector is needed (i) to draw explicit linkages between expected project outcomes and broader development goals for economic growth, social development, and poverty reduction, (ii) to monitor distributional and poverty impacts, even when poverty reduction is not a specific project objective, (iii) to assess real constraints and opportunities and ensure that project priorities and design match locally perceived priorities and institutional capacity. 4.10 The Bank's support to secondary education would be greatly strengthened through systematic prior analysis and ongoing tracking of the linkages with (i) labor force conditions and (ii) poverty/equity impacts: · Labor force linkages: Although the majority of secondary education projects have been justified on the basis of anticipated labor force benefits, the linkages have usually been assumed rather than based on solid country-level analysis, while labor force outcomes 28 were rarely monitored and reported for general secondary education projects. Judging from project documents, tracer studies of secondary graduates were extremely rare, but these could produce a wealth of relevant information. · Poverty/equity impacts: Only a minority of projects included specific objectives related to poverty alleviation or equity; and systematic monitoring of poverty and distributional impacts was not evident from the project completion reports. Considering the particular equity issues for secondary education in access, pupil selection, and program/quality differentiation, the Bank should be insisting (regardless of project objectives) on close monitoring of the distributional impact of investments in secondary education ­ in terms of learning and employment/ productivity outcomes and the incidence of public expenditure in the subsector. 29 Annex Annex. Supplementary Tables Table A1: Country Conditions (1990) and Bank Support to Secondary Education (projects closing FY1990-2001) COUNTRY CONDITIONS BANK-FUNDED PROJECTS GNI Youth Prim- CPIA Any Amt. Country Reg. per cap Illit GER Gp educ Sec* Angola AFR 800 92 1 X 2 Benin AFR 370 60 58 1 0 Botswana AFR 2730 17 113 3 X 2 Burkina Faso AFR 290 75 33 2 X 1 Burundi AFR 220 48 73 3 X 1 Cameroon AFR 960 13 101 2 X 1 Cape Verde AFR 980 18 121 3 X 1 Central African Republic AFR 470 48 65 2 X 0 Chad AFR 280 52 54 1 X 0 Comoros AFR 540 43 75 1 X 1 Congo, Dem. Rep. AFR 230 31 70 1 X 0 Congo, Rep. AFR 980 7 133 1 0 Cote d'Ivoire AFR 780 50 67 1 X 0 Equatorial Guinea AFR 350 7 1 X 0 Ethiopia AFR 160 57 33 1 X 2 Gabon AFR 4900 1 0 Gambia, The AFR 310 58 64 2 X 1 Ghana AFR 390 18 75 3 X 3 Guinea AFR 460 37 2 X 0 Guinea-Bissau AFR 230 56 56 2 X 0 Kenya AFR 380 10 95 3 X 3 Lesotho AFR 610 13 112 1 X 1 Liberia AFR 42 29 1 0 Madagascar AFR 240 28 103 3 X 1 Malawi AFR 200 37 68 3 X 1 Mali AFR 270 55 26 2 X 0 Mauritania AFR 550 54 49 2 X 1 Mauritius AFR 2300 9 109 3 X 1 Mozambique AFR 170 51 67 2 X 0 Namibia AFR 1890 13 129 0 Níger AFR 310 83 29 1 X 0 Nigeria AFR 270 27 91 2 X 0 Rwanda AFR 370 27 70 2 X 1 Sao Tome and Principe AFR 430 2 0 Annex 30 COUNTRY CONDITIONS BANK-FUNDED PROJECTS GNI Youth Prim- CPIA Any Amt. Country Reg. per cap Illit GER Gp educ Sec* Senegal AFR 720 60 59 1 X 1 Seychelles AFR 5070 3 0 Sierra Leone AFR 190 50 X 0 Somalia AFR 130 11 1 X 0 South Africa AFR 2890 12 122 0 Sudan AFR 600 35 53 1 0 Swaziland AFR 1120 15 111 2 0 Tanzania AFR 190 17 70 2 X 2 Togo AFR 440 37 109 3 X 2 Uganda AFR 340 30 71 2 X 0 Zambia AFR 450 19 99 1 X 0 Zimbabwe AFR 880 6 116 2 0 Cambodia EAP 140 27 121 1 0 China EAP 320 5 125 3 X 3 Fiji EAP 1850 2 125 3 0 Indonesia EAP 620 5 115 3 X 3 Korea, Rep. EAP 5740 0 105 3 X 3 Lao PDR EAP 200 45 105 1 X 1 Malaysia EAP 2380 5 94 3 X 2 Mongolia EAP 1 97 0 Papua New Guinea EAP 850 31 72 2 X 3 Philippines EAP 740 3 111 1 X 0 Samoa EAP 1040 17 122 2 0 Solomon Islands EAP 730 84 1 X 3 Thailand EAP 1520 2 99 3 0 Tonga EAP 1170 2 0 Vanuatu EAP 1160 96 1 X 3 Vietnam EAP 5 103 2 0 Albania ECA 680 5 100 X 2 Armenia ECA 0 104 0 Belarus ECA 3460 0 95 0 Bosnia-Herz ECA 70 X 0 Bulgaria ECA 2260 1 98 2 0 Croatia ECA 0 85 0 Estonia ECA 0 111 0 Georgia ECA 97 0 Hungary ECA 2880 0 95 2 X 2 Latvia ECA 0 94 0 Lithuania ECA 0 91 0 Annex 31 COUNTRY CONDITIONS BANK-FUNDED PROJECTS GNI Youth Prim- CPIA Any Amt. Country Reg. per cap Illit GER Gp educ Sec* Macedonia, FYR ECA 99 X 0 Moldova ECA 0 93 0 Poland ECA 0 98 2 0 Portugal ECA 6420 0 123 2 X 1 Romania ECA 1720 1 91 1 0 Russian Federation ECA 0 109 X 0 Slovak Republic ECA 3340 98 0 Slovenia ECA 0 108 0 Tajikistan ECA 840 0 91 0 Turkey ECA 2270 7 99 2 X 3 Ukraine ECA 1600 0 89 0 Uzbekistan ECA 0 81 0 Yugoslavia, Fed. Rep. ECA 72 2 0 Argentina LAC 3200 2 106 1 X 3 Barbados LAC 6600 93 2 X 2 Belize LAC 2190 4 111 3 X 0 Bolivia LAC 750 7 95 2 0 Brazil LAC 2780 10 106 2 X 3 Chile LAC 2190 2 100 3 X 3 Colombia LAC 1180 5 102 2 X 0 Costa Rica LAC 1780 3 101 3 X 1 Dominican Republic LAC 880 13 97 1 X 0 Ecuador LAC 970 4 116 2 X 0 El Salvador LAC 940 16 81 1 0 Grenada LAC 2310 1 0 Guatemala LAC 970 27 78 1 X 0 Guyana LAC 380 0 98 1 0 Haiti LAC 410 45 74 1 X 2 Honduras LAC 710 22 109 2 0 Jamaica LAC 1620 9 101 2 X 3 México LAC 2830 5 114 3 X 3 Nicaragua LAC 340 32 94 1 X 1 Panama LAC 2220 5 106 2 0 Paraguay LAC 1190 4 105 2 X 0 Peru LAC 780 5 118 1 X 0 St. Kitts and Nevis LAC 3610 2 0 St. Lucia LAC 2810 2 X 0 St. Vincent and the Grenadines LAC 1740 3 0 Suriname LAC 1330 1 0 Annex 32 COUNTRY CONDITIONS BANK-FUNDED PROJECTS GNI Youth Prim- CPIA Any Amt. Country Reg. per cap Illit GER Gp educ Sec* Trinidad and Tobago LAC 3700 4 97 2 X 0 Uruguay LAC 2850 1 109 3 X 0 Venezuela, RB LAC 2650 4 96 2 X 1 Algeria MNA 2440 22 100 2 X 1 Bahrain MNA 7120 4 110 0 Djibouti MNA 27 38 1 X 1 Egypt, Arab Rep. MNA 810 39 94 1 X 1 Iran, Islamic Rep. MNA 2590 13 112 0 Iraq MNA 2170 39 111 0 Jordan MNA 1390 3 71 2 X 2 Lebanon MNA 1010 8 120 0 Morocco MNA 1030 45 67 3 X 3 Oman MNA 5410 14 86 2 X 3 Saudi Arabia MNA 6620 15 73 0 Syrian Arab Republic MNA 940 20 108 1 0 Tunisia MNA 1430 16 113 3 X 1 Yemen, Rep. MNA 270 50 58 2 X 3 Bangladesh SAR 280 56 72 1 X 0 Bhutan SAR 520 2 X 0 India SAR 390 36 97 3 X 0 Maldives SAR 620 2 134 3 X 2 Nepal SAR 220 53 108 2 X 1 Pakistan SAR 390 53 61 2 X 3 Sri Lanka SAR 470 5 106 1 X 1 Notes: Country conditions: GNI/cap, youth illiteracy rate and primary gross enrollment rate from World Development Indicators for 1990. CPIA (Country Policy and Institutional Assessment) for 1990, 3 = good and 1= poor. Bank-funded projects: Any educ, X = Country had Bank-funded education project closing FY1990-2001. Amount sec., 3 = Bank funded 100% secondary education project, 2 = Bank funded project with >50% costs allocated to secondary education, 1 = Bank funded project with < 50% costs allocated to secondary, 0 = no project costs allocated to secondary education. 33 Annex Dev. do M Neg geN SGNITAR sbuS do do do do M M M M sbuS sbuS Mod sbuS sbuS Subs sbuS do M sbuS sbuS sbuS sbuS do M Subs Mod Mod nstitI Sust. CU L LU CU LU LU CU LU L L UC L L L L CU L CU L L CU L L L S S S S S M M M U U M U S S MS S S S S S M SH SH S S S HS S MS Outcm. sts Sec.* Co %14 64% %94 %15 % % % % % % % % % 100 %55 %85 %25 %76 %36 37% %98 100 50% 100 %001 100 100 %05 %75 100 100 100 50% STSOC ectj sts Pro Co 2.13 44.0 5.73 6.901 6.91 1.65 7.87 0.91 0.55 0.41 114.6 0.19 4.652 177.0 9.921 5.322 3.34 4.84 8.951 0.243 4.87 10.0 17.8 11.3 Net 5.62 23.7 2.13 8.86 0.51 6.84 3.21 6.21 6.32 0.71 56.5 7.35 Commit 6.101 97.3 6.67 5 138.7 6.92 0.92 8.85 136. 3.94 .16 .68 .59 0 1 6 4 5 5 9 4 1 2 5 7 0 1 1 6 8 3 1 3 9 0 Exit FY 200 199 199 199 199 199 199 199 199 199 199 199 200 200 199 1998 199 199 199 1997 199 199 199 200 2 6 8 5 1 6 3 6 1 5 9 0 3 5 5 1 2 9 4 6 9 4 Appr FY 199 198 198 198 199 198 199 198 198 198 198 199 199 199 198 1990 199 199 198 1993 198 198 198 199 n Loa DAI BRDI DAI DAI DAI DAI BRDI DAI DAI DAI DAI DAI DAI DAI BRDI BRDI BRDI BRDI BRDI BRDI IBRD DAI DAI DAI IL L L L M M L SI SIL SIL SIL SIL SIL SI SI SI SIL SIL SIM SIM SIL SI SI SIL LIS LIS MIS IMS SIL SIL SIL c ine sd la FY1990-2001 anaw a bli cibl di ai a a a a ai Gu ya Country ngoA Re Re Ne on otsB urunB thiopE hanaG enK auritius M danawR esi pu pu ys ias w Islan atu ian anzaniaT ogoT Chin Chin Chin Chin Indon Indonesia Mala Malay a Vanu lbaA Korea, Korea, Closed Papu Solom Projects TCEJORP g. R R Re AFR AF AFR AFR AF AFR AFR AFR AFR AFR EAP EAP EAP EAP EAP EAP EAP EAP EAP EAP EAP EAP EAP ECA II T EA II AR ION RO MIN Education & EDUC. EDUC.III I Name Secondary Project IONTACUDE IONAL SEC ION REH VI.CUDE VPTD.CES.CUDE SCHOOLS EDUC.I EDUC.II SCHOOLS EDUCAT AB IV.CUDE YRADNOCES SECT VII DEV. DEVELOPMEN IV.CUDE IONTACUDE ION-IIITACUDE IONTACUDE IONTACUDE BOOKTXET ECHNICALT/LANOITACOV TEACHING POOR IVETCEFFE IN BAS.ED YRADNOCES YRADNOCES PROJ OND. SECOND. EDUC & VOC VOCAT PRIM. PRIM.& YRADNOCES II EDUCAT IONTACUDE A2: LOOHCS 2 9 8 2 4 1 0 7 2 6 7 2 7 6 2 ID 04 14 19 70 95 30 92 21 73 84 47 45 56 63 84 Table Proj P000 P000 P000 P000 P000 P001 P001 P002 P002 P002 P003 P003 P003 P003 P003 378300P 2 7 1 7 0 4 9 16 16 31 36 62 81 26 P004 P004 P004 P004324 P004 P004 P004 P008 34 Annex Dev. SGNITAR sbuS geN sbuS do do do do do do do Subs M sbuS geN Mod sbuS sbuS M M M geN M sbuS RN sbuS sbuS M M nstitI L Sust. CU L L L LH LU UL L L L L CU L L L L L L L L S S M S S S SH U MS S S S S S U S S S S S S M S Outcm. sts % % % % % % % % % % % Sec.* Co % 100 100 100 83% 100 100 %44 56% 100 100 %65 %06 %15 100 100 %55 100 %05 %08 100 %05 STSOC ectj sts 3.9 Pro Co 1.06 7.27 7.862 15.6 4.54 0.702 3.22 11.1 7.04 5.402 9.05 0.221 5.252 0.832 9.32 7.72 6.02 9.71 1.461 0.57 n. Net 2.11 7.55 Commit 4.461 .05 5.31 0.32 3.01 .07 6.82 0.431 1.81 4.73 9.27 5.45 0.51 0.31 5.5 1.8 9.21 7.56 8.55 educatio ofle lev 1 5 1 1 0 1 2 5 1 9 0 3 7 8 2 5 1 6 0 0 7 Exit FY the 200 199 200 200 199 200 199 199 200 199 199 199 199 199 199 199 199 199 200 200 199 to 8 5 5 4 4 5 5 8 3 1 2 4 9 1 4 7 2 9 5 2 0 Appr FY ated 199 198 199 199 198 199 198 198 199 199 198 198 198 199 198 198 198 198 199 199 199 n unrel Loa BRDI BRDI BRDI BRDI BRDI BRDI DAI BRDI BRDI BRDI BRDI BRDI BRDI BRDI BRDI BRDI DAI DAI DAI DAI DAI are IL L L L M SI SIL SI SIL SIM SI SI SIL LIS L L L M SI SI SI SIL SI SIL SIL LIS LIS L SIL SI SIL that cilb y na a a e pu a gar dos n n n es es subsectors Country unH urkeyT exico Lank to rgentiA arbaB razilB hilC aitiH amaicJ amaicJ M ordaJ ordaJ ordaJ orocco M manO manO Re Maldiv Maldiv Pakistan Sri emen,Y cated TCEJORP allo g. Re ECA ECA LAC LAC LAC LAC LAC LAC LAC LAC MNA MNA MNA MNA MNA MNA MNA SAR SAR SAR SAR costs the I ignoresy ION RO 1 AIN.RT ION R secondar SCHOOLS ED URCES V EDUCAT SECONDARY SECT TION III OOLING UCAT to Name AININGRT % Project HTUOY IALRTSUDNI YRADNOCES ED RESO NA of MUH IONTACUDE YRADNOCES IV-A SCT IV VI II III AND IONTACUDE OF ION IV DE RMOFER RNGT COV EDUCAT IONTACUDE RES. EDUCA HUMAN CISAB IONTACUDE IONTACUDE ION EDUCAT IONTACUDE IONTACUDE SCH costs ELDDI M GENERAL ationl 5 8 2 9 7 3 0 8 9 4 0 4 5 0 2 2 6 5 7 4 3 ID 48 95 99 08 33 67 29 44 47 70 26 26 29 46 53 54 86 33 33 39 34 calcu Proj P008 P008 P005 P006 P006 P006 P007 P007 P007 P007 P005 P005 P005 P005 P005 P005 P005 P004 P004 P010 P010 the* 35 Annex Table A3: Objectives of Secondary Education Projects PROJECT OBJECTIVES of to te nt ion r ns cena rk cy/ me ma ge pa acity verty/ nde bor na Proj ID Project Name Region Country Ex cap Po quitye Ge Quality Relev la Efficien ma P000042 EDUCATION I AFR Angola X X X P000149 EDUC.IV AFR Botswana X X X P000198 EDUC.SEC.DVPT AFR Burundi X X X P000702 EDUC.VI AFR Ethiopia X X X P000954 SECONDARY SCHOOLS AFR Ghana X X X X P001301 EDUC.VI AFR Kenya X X X P001920 EDUCATION SECTOR AFR Mauritius X X X X P002217 EDUCATION-III AFR Rwanda X X X P002732 EDUCATION VII AFR Tanzania X X X X X P002846 EDUCATION DEV. AFR Togo X X P003477 TEXTBOOK DEVT EAP China X X P003452 VOCATIONAL/TECH EAP China X X X X P003567 EFFECTIVE TEACHING EAP China X X P003636 BASIC EDUC IN P& M II EAP China X X X P003842 SECONDARY EDUC.I EAP Indonesia X X P003873 SECONDARY EDUC.II EAP Indonesia X X P004162 VOC EDUC PROJ EAP Korea X X X P004167 VOCATIONAL SCHOOLS EAP Korea X X X P004311 PRIM & SEC. EDUC. II EAP Malaysia X X X P004324 PRIM & SEC EDUC. III EAP Malaysia X X X P004367 SECONDARY EDUC EAP PNG X X X P004620 EDUCATION II EAP Solomon Is X X X P004814 EDUCATION EAP Vanuatu X X X X P008269 SCHOOL REHAB ECA Albania X X P008485 YOUTH TRAINING ECA Hungary X X P008958 INDUSTRIAL SCHOOLS ECA Turkey X X X X P005992 SECONDARY ED 1 LAC Argentina X X P006089 HUMAN RESOURCES LAC Barbados X X X P006337 EDUCATION V LAC Brazil X X X P006673 SECONDARY EDUC LAC Chile X X X X P007290 EDUCATION IV LAC Haiti X X X P007448 ED IV LAC Jamaica X X P007479 REFORM OF SECOND. LAC Jamaica X X X P007704 VOC TRNG SCTR LAC Mexico X X X P005260 EDUCATION IV MNA Jordan X P005264 EDUCATION VI MNA Jordan X X X P005295 HUMAN RES. SECTOR MNA Jordan X X P005460 BASIC EDUCATION MNA Morocco X X X P005532 EDUCATION II MNA Oman X X X X Annex 36 PROJECT OBJECTIVES of to te nt ion r ns cena rk cy/ me ma ge pa acity verty/ nde bor na Proj ID Project Name Region Country Ex cap Po quitye Ge Quality Relev la Efficien ma P005542 EDUCATION III MNA Oman X X X X X P005866 EDUCATION III MNA Yemen X X X X P004335 EDUCATION AND TR. SAR Maldives X X X X X X P004337 EDUCATION SAR Maldives X X X X X P010394 MIDDLE SCHOOLING SAR Pakistan X X X X X P010343 GENERAL EDUCATION SAR Sri Lanka X X X X Table A4: Lending for Secondary Education in Relation to Country Conditions EDUCATION LENDING COUNTRY CONDITIONS Projects No. of Av. GNI per Av. youth Av. Prim. Av. Sec. Av. CPIA21 closing 1990- countries capita US$ illiteracy GER GER 2001 Having no 50 2,014 10% 99% 64% 2.66 education project Having educ 32 841 27% 79% 35% 2.77 project(s) with no costs to secondary Having educ 36 1,267 27% 89% 35% 3.16 project(s) with some but not all costs to secondary Having educ 18 1,788 17% 93% 44% 3.50 project(s) with all costs to secondary All countries 136 1,469 20% 90% 46% 2.98 21. Country Policy and Institutional Assessments made annually by the Bank, based on indicators of current policy conditions and institutional capacity. A high CPIA ranking indicates good policy and institutional conditions, a low one poor conditions. Annex 37 Table A4: Regional Distribution of Projects Supporting Secondary Education (number of projects, excluding adjustment operations) Approved post-FY80 AFR EAP ECA LAC MNA SAR Total All education projects 137 95 33 107 54 51 477 Projects with any secondary 54 34 20 34 21 13 176 At least half secondary 21 21 8 21 13 7 91 Closed FY1990-2001 All education projects 66 53 15 42 29 31 236 Projects with any secondary 27 19 6 12 14 5 83 At least half secondary 10 13 3 8 7 4 45 Closed FY 02 or Active All education projects 46 31 18 51 18 19 183 Projects with any secondary 18 14 14 22 7 8 83 At least half secondary 6 7 5 12 5 3 38 Table A5: Regional Distribution of Projects Supporting Secondary Education (percent of projects, excluding adjustment operations) AFR EAP ECA LAC MNA SAR Total Approved post-FY80 All education projects 29% 20% 7% 22% 11% 11% 100% Projects with any secondary 31% 19% 11% 19% 12% 7% 100% At least half secondary 23% 23% 9% 23% 14% 8% 100% Closed FY1990-2001 All education projects 28% 22% 6% 18% 12% 13% 100% Projects with any secondary 33% 23% 7% 14% 17% 6% 100% At least half secondary 22% 29% 7% 18% 16% 9% 100% Closed FY 02 or Active All education projects 25% 17% 10% 28% 10% 10% 100% Projects with any secondary 22% 17% 17% 27% 8% 10% 100% At least half secondary 16% 18% 13% 32% 13% 8% 100% Annex 38 Table A7: Distribution of Projects by Country Rate of Adult Illiteracy (projects closing FY1990-2001) Adult Illiteracy (1995 est.)* High Moderate Low Number Share Number Share Number Share All education projects 114 49% 74 31% 47 20% Projects with any secondary 37 45% 29 35% 17 20% At least half secondary 14 31% 21 47% 10 22% * High = > 30%. Moderate = 11 ­ 30%. Low = <11%. Table A8: Distribution of Projects by Country Policy and Institutional Conditions (projects closing FY1990-2001) CPIA Index (Av. 90-98) High Mid Low Number Share Number Share Number Share All education projects 86 37% 109 46% 40 17% Projects with any secondary 32 39% 38 46% 13 16% At least half secondary 21 47% 17 38% 7 16% Table A9: Distribution of Projects by Country Population Size (all projects approved post FY80) Population Size* Large Medium Small Number Share Number Share Number Share Approved since FY80 All education projects 156 33% 244 51% 75 16% Projects with any secondary 43 24% 93 53% 40 23% Mainly secondary projects 23 25% 44 48% 24 26% Closed FY1990-2001 All education projects 83 35% 115 49% 37 16% Projects with any secondary 16 19% 44 53% 23 28% Mainly secondary projects 11 24% 21 47% 13 29% *Estimates for 1995. Large = > 50m. Medium = 3.5 m. ­ 50 m. Small = < 3.5m. Annex 39 Table A10: Secondary Education Project Performance Ratings (Projects closed FY1990-2001, percent) Outcome Sustainability Institutional Devt. N= HS SA MS US LI/HL UC UN SU MO NE UR By region AFR 10 0 40 30 30 30 30 40 60 30 0 10 EAP 13 23 62 15 0 69 31 0 46 46 8 ECA 3 0 33 67 0 67 33 0 67 0 33 LAC 8 13 63 13 13 75 0 25 50 38 13 MNA 7 0 86 0 14 86 14 0 29 57 14 SAR 4 0 75 25 0 100 0 0 50 50 0 By approval date Pre-FY 90 25 4 60 20 12 56 20 20 52 28 12 8 FY 90+ 21 14 57 19 10 76 19 5 38 52 10 By CPIA group High 21 14 67 14 5 76 19 5 67 29 5 Mid 17 0 59 29 12 71 18 12 29 53 12 6 Low 7 14 43 14 29 29 29 43 29 43 29 By income level Upper-middle 14 21 71 7 0 86 14 0 71 21 7 Lower-middle 16 0 69 25 6 75 19 6 38 50 13 Low 15 7 40 33 20 40 27 33 33 47 13 7 Outcome: HS = Highly satisfactory, SA = Satisfactory, MS = Marginally Satisfactory, US = Unsatisfactory or Marginally Unsatisfactory or Highly Unsatisfactory. Sustainability: LI/HL = Likely or Highly Likely, UC = Uncertain, UL = Unlikely or Highly Unlikely. Institutional Development Impact: SU = Substantial, MO = Modest, NI = Negligible, UR = Unrated.