Report No. 89836 About the Panel The Inspection Panel was created in September 1993 by the Board of Executive Directors of the World Bank to serve as an independent mechanism to ensure accountability in Bank operations with respect to its policies and procedures. The Inspection Panel is an instrument for groups of two or more private citizens who believe that they or their interests have been or could be harmed by Bank-financed activities to present their concerns through a Request for Inspection. In short, the Panel provides a link between the Bank and the people who are likely to be affected by the projects it finances. Members of the Panel are selected “on the basis of their ability to deal thoroughly and fairly with the request brought to them, their integrity and their independence from the Bank’s Management, and their exposure to developmental issues and to living conditions in developing countries.”1 The three-member Panel is empowered, subject to Board approval, to investigate problems that are alleged to have arisen as a result of the Bank having Report and Recommendation ignored its own operating policies and procedures. Processing Requests After the Panel receives a Request for Inspection it is processed as follows:  The Panel decides whether the Request is prima facie not barred from Panel consideration.  The Panel registers the Request—a purely administrative procedure.  The Panel sends the Request to Bank Management, which has 21 working days to respond to the ARMENIA: Second Education Quality and allegations of the Requesters.  Relevance Project (P107772), and The Panel then conducts a short 21 working-day assessment to determine the eligibility of the Requesters and the Request.  Education Improvement Project (P130182) If the Panel recommends an investigation, and the Board approves it, the Panel undertakes a full investigation, which is not time-bound.  If the Panel does not recommend an investigation, the Board of Executive Directors may still instruct the Panel to conduct an investigation if warranted.  Three days after the Board decides on whether or not an investigation should be carried out, the Panel’s Report (including the Request for Inspection and Management’s Response) is publicly available through the Panel’s website and Secretariat, the Bank’s Info Shop and the respective Bank Country Office.  When the Panel completes an investigation, it sends its findings and conclusions on the matters alleged in the Request for Inspection to the Board as well as to Bank Management.  The Bank Management then has six weeks to submit its recommendations to the Board on what actions the Bank would take in response to the Panel’s findings and conclusions.  August 1, 2014 The Board then takes the final decision on what should be done based on the Panel's findings and the Bank Management's recommendations. 1 IBRD Resolution No. 93-10; IDA Resolution No. 93-6. i The Inspection Panel Report and Recommendation On Request for Inspection Armenia: Second Education Quality and Relevance Project (P107772) and Education Improvement Project (P130182) A. Introduction 1. In accordance with the Resolution (“the Resolution”) 1 establishing the Inspection Panel (“the Panel”), the purpose of this Report and Recommendation on Request for Inspection (“the Report”) is to make a recommendation to the Board of Executive Directors (“the Board”) as to whether the Panel should investigate the matters alleged in this Request. The Panel’s recommendation is based on its consideration of the technical eligibility of the Request and its assessment of other factors as provided in the Resolution and the Panel’s Operating Procedures. 2. On May 16, 2014, the Inspection Panel received a Request for Inspection (“the Request”) of the Armenia: Second Education Quality and Relevance Project, APL 2 (“EQRP 2”) and the Education Improvement Project (“EIP”) (“the Projects”). The Request was submitted by 9 NGOs, 2 students and 5 parents from Armenia who requested to keep their identities confidential. The Panel registered the Request and notified the Board and Management on June 5, 2014. Management submitted the Management Response on July 7, 2014. 3. This Report includes the following sections: (a) this introduction, (b) a description of the Project, (c) a summary of the Request, (d) the Management Response, (e) the Panel’s review including the Panel’s determination of the technical eligibility of the Request, in accordance with the 1999 Clarification to the Resolution, 2 and the Panel’s observations on other factors considered, (f) the Panel’s analysis and (g) the Panel’s recommendation to the Board on whether an investigation of the matters alleged in the Request is warranted. 1 International Bank for Reconstruction and Development (Resolution IBRD 93-10) and International Development Association (Resolution 93-6), “The World Bank Inspection Panel”, September 22, 1993 (hereinafter “the Resolution,”) ¶ 19. Available at: http://siteresources.worldbank.org/EXTINSPECTIONPANEL/Resources/ResolutionMarch2005.pdf 2 “1999 Clarification of the Board’s Second Review of the Inspection Panel”, April 1999 (hereinafter “the 1999 Clarification”). Available at: http://siteresources.worldbank.org/EXTINSPECTIONPANEL/Resources/1999ClarificationoftheBoard.pdf B. The Projects Second Education Quality and Relevance Project, APL 2 (EQRP 2) 4. The project aims to improve the quality and relevance of the Armenian school system to meet the challenges of the knowledge society. 3 The Project Development Objectives (PDO) of EQRP 2 are: “(i) enhance school learning in general education and improve school readiness of children entering primary education, and (ii) support the integration of Armenian Tertiary Education system into the European Higher Education Area.” 4 5. The project includes three components: (i) Enhancing the quality of general education, (ii) Supporting tertiary education reforms in the context of the Bologna Agenda, and (iii) Project management, monitoring and evaluation. 5 The Ministry of Education and Science (MOES) is responsible for the implementation of the project supported by a Center for Education Project (CEP) acting as the Project Implementing Unit (PIU). The Project was classified as “Environmental Category C,” and none of the safeguards policies were triggered. 6. The EQRP 2 was approved by the Board on May 12, 2009 for an IDA credit of SDR 17.0 million (equivalent to US$25.0 million) to the Republic of Armenia, with a closing date of November 30, 2014. 6 It was restructured in August 2012 in order to “(i) revise the activities under one of the Project’s components; (ii) reallocate financing across project activities; and (iii) adjust the Project Results Framework (RF) to accommodate the proposed changes.” Neither the PDO nor the applicable policies changed as a result of the restructuring. 7 7. At the time the Panel received the Request for Inspection, about 72% of the credit had been disbursed. Education Improvement Project (EIP) 8. The project aims to support the government of Armenia in two main endeavors: (i) enhancing the quality of general education, and (ii) fostering relevance of tertiary education through mainstreaming of the Competitive Innovation Fund (CIF). 8 The Project Development Objectives (PDO) are “to improve school readiness of children entering primary education, improve physical conditions and the availability of educational resources in upper-secondary schools, and support improved quality and relevance in higher education institutions in Armenia.” 9 3 Project Appraisal Document (PAD) on a Proposed Credit in the amount of SDR 17.0 million (US$25.0 million equivalent) to The Republic of Armenia for a Second Education Quality and Relevance (APL) program, April 15, 2009, p. 4, para. 13 4 Financing Agreement, Second Education Quality and Relevance Project, between the Republic of Armenia and the International Development Association, dated June 5, 2009,Schedule 1, p. 4. 5 Financing Agreement, Schedule 1 for EQRP 2, p. 4. 6 Financing Agreement for EQRP 2, p. 1. 7 Restructuring Paper on a Proposed Project Restructuring of Second Education Quality and Relevance (APL2) Project to the Republic of Armenia, August 2, 2012, p. 4. 8 Project Appraisal Document (PAD) on a Proposed Credit in the amount of SDR 9.8 million (US$15.0 million equivalent) and on a Proposed Loan in the amount of US$15.0 million to The Republic of Armenia for an Education Improvement Project, February 18, 2014, p. 24, para. 3. 9 PAD EIP p. 4, para. 10. 2 9. The project includes three components: (i) Enhancing the quality of general education, (ii) Mainstreaming of the Competitive Innovation Fund (CIF) for Higher Education Institutions (HEI) into full implementation, and (iii) Project management, monitoring and evaluation. The project was classified as “Environmental Category B” and two safeguard policies were triggered: (i) Environmental Assessment (OP/BP 4.01), and (ii) Physical Cultural Resources (OP/BP 4.11). According to the PAD, OP/BP 4.01 was triggered because the Project would support upgrading of selected school buildings, and the rehabilitation work may have some temporary negative impacts typical for reconstruction and rehabilitation of small to medium-size buildings. For this, an Environmental Management Framework (EMF) has been developed. OP/BP 4.11 was triggered to ensure that the process of school selection for rehabilitation would involve verifying if any of the buildings are formally entered into the list of the nations’ historical and/or cultural heritage. 10 10. The EIP was approved by the Board on March 13, 2014 for an IDA credit of SDR 9.8 million (US$15 million equivalent), and an IBRD loan of US$15 million to the Republic of Armenia, with a closing date of September 30, 2019. 11 At the time of writing of this Report, the EIP was not yet effective. The Ministry of Education and Science (MOES) will be responsible for the implementation of the project supported by a Center for Education Project (CEP) acting as the Project Implementing Unit. C. The Request 11. This section summarizes the allegations included in the written Request (the full Request is attached as Annex I). 12. The Requesters allege that the EIP is “…designed in such a way as not to address failures of the previous program… (EQRP 2)” 12 and “…would exacerbate the harm caused by the previous loan (EQRP 2)….” 13 The Requesters believe that EQRP 2 resulted in “substantial harm” 14 to the Armenian education system, especially “…in areas of accountability, governance, quality and accessibility”, and as a result, harmed academics, students and parents, including the Requesters, who will also be affected by the EIP. 13. According to the Requesters, the main focus of EIP is on refurbishing school buildings rather than addressing the shortcomings of the previous program, and it poses a threat of great harm to the education system of the whole country. In their view, “…any new loan…” 15 of the World Bank must consider the problems of a previous loan program and thus address those issues. The Requesters attached several case studies to illustrate the Projects’ adverse impact on students and parents. The alleged harms are described below. 10 PAD EIP, p. 16-17, paras. 50-52. 11 PAD EIP. 12 The Request, p. 1. 13 Ibid. 14 Ibid. 15 Ibid. 3 Quality of Education 14. Disadvantaging Rural Students. According to the Requesters, the reform under EQRP 2 unfairly disadvantaged students from rural areas. They allege that there are only a few high schools in rural areas, and that the quality of schools is lower than that of the urban areas, functioning based on a 12-year curriculum that does not provide specialization. This renders rural students unable to “…comply with the government’s plan to transfer to a credit system in high school courses.” 16 As a result, rural students are at a disadvantage with respect to urban students who have greater access to high schools and therefore, higher education. 15. Ineffective Teachers Training. The Requesters also note that there are many “…professionally unqualified teachers…” 17 but that the teachers’ training aimed at professional development was ineffective as it did not accommodate varying skill types and levels among the teachers. 16. Political Influence on Higher Education. According to the Request, “the governance boards of state universities, state education agencies are represented by government officials, MPs and by the administration of the President at around 50%” 18 , which allows for “…interventions by the state over their finances, management appointments, specialization, and admissions”. 19 The Requesters state that these “undermine academic independence vital for credibility and sustainability of any research and scientific endeavor”. 20 One student claims that a research topic, i.e., “The solutions of current problems in Armenia through political left” 21 was rejected by the Head of the Philosophy Department of National Academy of Science giving the reason that “…he was not going to help the current political opposition in Armenia” 22 and “…cynically…” 23 suggested that the student instead writes on “The behavior of strip dancers in Armenia”. 24 According to the Requesters, a Rector of a university was also wrongfully fired by the Minister of the MOES. 17. State Control over University Finances and Poor Financing Scheme under the Project. The Requesters assert that the state directly controls the budget of state universities, despite the fact that the proportion of state allocation is very small (9-30%). In addition, the Requesters consider that the Competitive Innovation Fund (CIF) and sustainable funding strategy being designed under this project are “…fragmented…” 25 and cannot assist in “…ensuring sustainable funding system…” 26 as they do not address the declining public spending or legal framework regulating the finances of universities. Parents feel that the schools are “not equipped with the necessary technical capacities and do not have proper equipped laboratories and libraries that 16 The Request, p. 10. 17 The Request, p. 14. 18 The Request, p. 7. 19 Ibid. 20 The Request, p. 9. 21 The Request, p. 12. 22 Ibid. 23 Ibid. 24 Ibid. 25 The Request, p. 8. 26 Ibid. 4 would provide for the quality education.” 27 The Requesters state that the quality of education suffers due to insufficient financing. 18. Religious Influence over Higher Education. The Requesters state that the subject standard for the Armenian Church History is “singlehandedly controlled and supervised by Armenian Apostolic Church through Center for Christian Education and Propaganda.” 28 The Requesters allege that religious institutions exercise influence over research topics as illustrated by one case study on a student, where a Reverend’s “…non-scientific…” 29 negative feedback on a thesis was accepted by an academic committee which eventually rejected the thesis and the student did not receive a degree. Discrimination 19. The Requesters say that biased gender roles and “…discriminatory norms (e.g., religious) and perceptions are widely promoted…in textbooks, teaching process and teachers’ attitude.” 30 According to the Requesters, a 10th grade text book shows five basic needs of men and women which are discriminatory in nature (i.e., sexual satisfaction, a charming woman, household management for men and financial support, conversation and honesty for women). The Requesters also allege that schools do not offer opportunities to discuss these discriminatory ideas which could influence the thinking of students. The Requesters state that “gender insensitive content and the constant messaging of traditional gender stereotypes, the teachers deeply biased attitude is particularly damaging for developing the notion of gender equality in youth during their formative years.” 31 The Requesters also say that “ill treatment is widely practiced by both teachers and students towards religious minorities.” 32 Governance 20. Governance and Corruption in Quality Assurance Systems. The Armenian National Quality Assurance Agency (ANQA) 33 is an independent external quality assurance agency. The Requesters allege “there are serious irregularities in its mission, composition and function”, and thus its independence is questioned. In addition, the Requesters allege that quality assurance reforms under the project have been “fully controlled by the government and ruling political party with no dialogue or consultation” 34 with relevant stakeholders and therefore, “met with distrust, if not open opposition…” 35, and eventually resulting in “poor quality reform of higher 27 The Request, p. 14. 28 The Request, p. 11. 29 The Request, p. 12. 30 The Request, p. 10. 31 The Request, p. 4. 32 The Request, p. 6. 33 The government established the Armenian National Quality Assurance Agency (ANQA) in November 2008. MOES also created a Working Group on Quality Assurance, which will assist the consolidation of ANQA during the initial stage. This sub- component will support the ANQA and the WG of QA to establish the external QA system and universities to develop internal QA system. 34 The Request, p. 2. 35 The Request, p. 3. 5 education in Armenia” 36. The Requesters state that the Bank ignored its “…own assessment of the tertiary sector governance issues and irregularities…” 37 and failed to make changes. Lack of Consultations 21. The Requesters state that the “beneficiary and public feedback was ignored both during the implementation, monitoring, and assessment of theEQRP2…” 38 which they feel would have improved both Projects. D. The Management Response 22. This section summarizes the Management Response submitted on July 7, 2014 (the full Response is attached as Annex II). 23. Management does not believe that the Request meets the eligibility criteria and disagrees that the alleged adverse impacts cited in the Request result from the two Projects. It believes that the Requesters are not able to demonstrate that their rights and interests have been or are likely to be adversely affected by the Projects. Management argues that the Request is primarily about broad issues of the education system in Armenia and design preferences for the Project. According to Management, the Requesters appear to define harm as any pre-existing conditions which they view as undesirable and which are not addressed by the Bank supported Projects, which they believe should be. The Response argues that this does not relate to an act of omission by the Bank, nor does it constitute harm referred to in the Panel’s Resolution, rather it represents unfulfilled expectations and do not create harm. 39 24. Management concurs with some of the points made by the Requesters and notes that several interventions under the Projects support reforms that are needed to address the issues raised in the Request. However, in Response to the Requester’s contention that the Bank support should be offered only once issues of the education sector have been addressed, Management notes that according to the Bank’s global experience, waiting precludes the opportunity to support progress and early intervention can improve the pace and quality of their progress. 40 Higher Education 25. Governance Issues and Corruption in the ANQA. Management disagrees with the Requesters’ claims that the Bank failed to address these problems with the Borrower. A report produced by the Bank (Addressing Governance at the Center of Higher Education Reform in Armenia, January 2013) indicates that ANQA, a national quality assurance system, has made important achievements in a short period of time 41 and the government is committed to taking the additional steps to meet the international standards of quality assurance. 36 Ibid. 37 The Request, p. 7. 38 The Request, p. 10. 39 MR, p. 10-12, paras. 16-21. 40 MR, p. 11, para. 21. 41 MR, p. 13, para. 26, section (i). 6 26. Inadequate Public Financing. Management agrees with the Requesters on the importance of ensuring sufficient public funding for the education sector but notes that the Projects are not responsible for the sector’s financing. Management states that despite this, the need to increase public funding …has been regularly brought to the attention of the Ministry of Education and Science (MOES) and the Ministry of Finance (MOF) 42 as elaborated in the Bank report “Public Fiscal Consolidation and Recovery in Armenia.” Management also notes that the Competitive Innovation Fund (CIF) was also established to address some of the financing issues in the sector. 43 27. Alleged Overly-tight Political Control. Management notes that these issues are being addressed through Bank financed Projects and in sector dialogue, and that the government has shown commitment. 44 Management indicates that the above mentioned Report did not find evidence of systemic issues pertaining to …widespread corruption and nepotism…in the sector. 45 28. Alleged Lack of Academic Freedom. Management states that this project does not deal with academic freedom, nor does the Bank have evidence to show that this is a pervasive issue in higher education in Armenia. 46 In Management’s view, the universities’ self-evaluations promoted by ANQA are useful mechanisms to monitor progress on the desired changes within universities. 47 Management also notes that the “case studies” provided by the Requesters do not offer any evidence that the alleged incidents stem from or are aggravated by the two Projects. Secondary Education 29. Alleged Political Control of the Sector by the Government. Management views the Requesters’ claims as not being related to the Projects and that it is difficult to verify the claims without concrete evidence. Management also notes that the appointment of school officials by the Ministry of Education is similar to other education systems in the world. 48 30. Weak Student Evaluation. Management agrees that student evaluation could be significantly improved. However with the help of the Bank and other donors (through the Projects, trust funds and policy dialogue) the government is making efforts to improve the evaluation. 49 31. Putting Rural Students at a Disadvantage. Management notes that although there remain significant …infrastructure and educational equipment needs in both rural and urban areas, 50 the Bank has provided support to address those needs, including the rest of the Bank portfolio that covers support for infrastructure and equipment needs of schools in rural and underserved 42 MR, p. 14, para 26, section (ii) 43 MR, p. 14, para. 26, section (ii). 44 MR, p. 14-15, para 26, section (iii) 45 MR, p. 15, para 26, section (iii) 46 MR, p. 15, para 26, section (iv) 47 MR, p. 15, para26, section (iv) 48 MR, p. 15, para 27, section (i) 49 MR, p. 16, para. 27, section (ii). 50 MR, p. 16, para 27, section (iii) 7 areas. Management further states that while the infrastructure component of EIP focuses on high schools in urban areas, students in rural schools have and will benefit from the system-wide activities supported under the EQRP2 and EIP, such as in-service teacher training, connectivity to internet for all schools, curriculum revisions and improvements, and students’ assessment activities. 51 32. Ineffective Teacher Training. Management states that the support for in-service teacher training provided under the EQRP2 was aimed at leveling the field among teachers with regard to…reforms…including, curriculum reforms, the extension of the education system from 10 to 12 grades, and the introduction of ICT into the learning and teaching process. 52 An impact evaluation of Bank supported in-service teacher training for 2011 and 2012 showed some positive impacts, as well as areas that needed improvements. However, the government decided in 2013 that all in-service teacher training is to be funded by the government and provided by the National Institute of Education (NIE). 33. Lack of Transparency in Curricular and Syllabus Design, Leading to Gender Bias and Discriminatory Behavior. Management notes that the Bank has not provided support for the development of textbooks in Armenia under either of the two Projects. While noting that it is outside the scope of the Projects, the Bank is strongly committed to gender equality and gender inclusion, and will review this issue and raise it with the Borrower if gender bias and discriminatory behavior is confirmed. 34. Discrimination against Religious Minorities. Management states that the EQRP2 and the EIP do not support any links between religion and education, and the discrimination and the potential harm stemming from it is neither caused nor aggravated by the Projects. Management notes that the Bank strongly supports the principle of inclusive development and equal treatment of ethnic and religious minorities. 35. Alleged Lack of Consultations. Management maintains that supervision of the EQRP 2 and preparation of the EIP have included close contact and extensive consultations with various stakeholders, and states that the feedback received was taken into account. In addition, a series of analytical studies to inform the preparation of the project were disseminated and discussed in country in May 2013. 53 Management also notes that the Bank team met with the Open Society Foundation (OSF) of Armenia three times during the EIP preparation. 36. Other Policy Considerations. With regard to OP/BP 13.05 (Supervision), Management asserts that the Projects have been supervised according to the requirements of the policy; Management also asserts that the Projects are anchored in country policy/sector analysis; and reflect lessons learned from the Bank’s experience, as required by OP/BP 10.00 (Investment Lending; Management states that OP/BP 8.60 (Development Policy Lending is not applicable to the Projects.. Management states that OP/BP 13.60 (Monitoring and Evaluation) establishes the requirements for results-oriented monitoring and evaluation, but does not include obligations at the project level. (p.32 MR) 51 MR, p. 17, para 27, section (iii) 52 MR, p. 17, para 27, section (iv) 53 MR, p. 18, para. 28, section (i). 8 37. Given Management’s disagreement with the allegations of harm stemming from policy compliance, Management states that no actions are required to bring the Projects into compliance. However Management adds that it take this opportunity to consider how the issues raised in the Request could be addressed by the Bank going forward to help the education sector in Armenia. In view of this, Management proposes to review and verify allegations of discriminatory content in textbooks and would consider raising the issue with the government with a view to encourage adapting or changing such school textbooks to include gender appropriate content. 54 E. Panel Review of the Request and of the Management Response 38. A Panel team composed of Chairperson Eimi Watanabe, Panel Member Gonzalo Castro de la Mata, and Senior Operations Officer Tatiana Tassoni visited Yerevan, Armenia on July 16- 19, 2014. The Panel met with the Requesters and other allegedly Project affected people. The Panel also met with Government authorities in Armenia including the Minister of Economy, the Minister of Education and Science, officials from the National Center of Education Technology (NACET), representatives of the Armenian National Quality Assurance Agency (ANQA), the Project Implementing Unit, World Bank staff in the Yerevan Country Office, representatives of UNICEF, representatives of the Yerevan State Medical University, and representatives of the Step by Step Benevolent Foundation (involved in the implementation of some of the Projects’ aspects). 39. The Panel wishes to express its appreciation to all those mentioned above for sharing their views and exchanging information and insights with the Panel. The Panel extends its thanks to the Requesters and other allegedly affected people who met with the Panel team. The Panel wishes to thank, in particular, the Ministry of Education and Science for preparing a detailed document, which was helpful in gaining an improved understanding of the issues under review. Particular appreciation goes also to the World Bank Country Director, the Country Manager, and staff in the World Bank Country Office in Yerevan for meeting with the Panel team, discussing the issues and providing relevant information, and assisting with logistical arrangements. 40. The Panel’s review is based on information presented in the Request, on the Management Response, on other documentary evidence, and on information gathered during the visit to Yerevan. This review covers the Panel’s determination of the technical eligibility of the Request, according to the criteria set forth in the 1999 Clarification (subsection 1), and observations on other factors supporting the Panel’s recommendation (subsection 2). Determination of Technical Eligibility 41. The Panel is satisfied that the Request meets all six technical eligibility criteria provided in paragraph 9 of the 1999 Clarifications as detailed below. The Panel notes that its confirmation of technical eligibility, which is a set of verifiable facts focusing to a large extent on the content of the Request as articulated by the Requesters, does not involve the Panel’s assessment of the substance of the claims made in the Request. 54 MR, p. 20, para. 35. 9 42. Criterion (a): “The affected party consists of any two or more persons with common interests or concerns and who are in the borrower’s territory.” The Panel confirms that the Requesters live in the borrower’s territory and share interests that may be affected by Project activities. The Panel considers the requirement of paragraph 9(a) as met. 43. Criterion (b): “The request does assert in substance that a serious violation by the Bank of its operational policies and procedures has or is likely to have a material adverse effect on the requester.” The Requesters assert that in the design of the EIP the Bank failed to address problems identified during the implementation of the EQRP 2, as is required by Bank policy on investment lending regarding the appraisal of a project (OP/BP 10.00). In the Requesters’ view, in spite of the Bank’s own assessment of the education reform underway, the EIP did not include changes to project design in line with the lessons learned from the EQRP 2. In this context, for example, they believe that as a result of the reforms supported by the Bank Projects, rural students were disadvantaged and thus discriminated against in the quality of the education they received compared to urban schools. The Requesters also refer to failures in supervising the EQRP 2 properly, with respect to the quality of teachers training, undue political influence on higher education and gender discrimination in the content and standards in the schools. The Requesters also state that there are irregularities that the Bank should have addressed as part of its supervision with respect to the composition and functioning of ANQA, which, they believe, does not act as an independent agency. The Requesters also assert that consultations with stakeholders about the education reform were inadequate. The Panel is thus satisfied that the requirement of paragraph 9(b) is met. 44. Criterion (c): “The request does assert that its subject matter has been brought to Management's attention and that, in the Requester’s view, Management has failed to respond adequately demonstrating that it has followed or is taking steps to follow the Bank’s policies and procedures.” The Requesters state that they presented their facts and analysis to senior World Bank staff in both Yerevan and in Washington DC; that they met and corresponded with the Country Office Senior Management as well as with other Bank staff but are not satisfied with Management’s Response. The Requesters clarified to the Panel that they met with Bank officials in Washington DC in the Fall of 2011 to discuss the standard of secondary education in EQRP2. The Requesters also state that they met with Bank Management in the Yerevan office in the Fall of 2013 where they raised concerns. The Requesters believe that the Bank’s responses to their complaints did not address their issues adequately. The Panel is satisfied that this criterion has been met. 45. Criterion (d): “The matter is not related to procurement.” The Panel is satisfied that the claims with respect to harm and non-compliance included in the Request for Inspection do not raise issues of procurement under the Project and hence this criterion is met. 46. Criterion (e): “The related loan has not been closed or substantially disbursed.” At the time of receipt of the Request for Inspection, about 72% of the EQRP 2 had been disbursed. The EIP was not yet effective. 10 47. Criterion (f): “The Panel has not previously made a recommendation on the subject matter or, if it has, that the request does assert that there is new evidence or circumstances not known at the time of the prior request.” The Panel confirms that it has not previously made a recommendation on the subject matter of the Request. Observations on Other Factors Supporting the Panel’s Recommendation 48. During the visit to Yerevan, the Panel team met with the Requesters and had the opportunity to discuss in depth their concerns regarding the Projects and the harm that they believe has resulted or will result from them, on different aspects of the Projects’ design and implementation. Observations are made on each of the alleged specific harms resulting from the Projects and the failure of the Bank to properly supervise the implementation of the EQRP 2 and address the shortcomings and integrate the lessons learned from the EQRP 2 into the design of the EIP. 49. High School System. The Requesters claim that the high school system created with the support of the EQRP 2 has aggravated poverty by widening the gap between urban and rural areas and putting the latter at a great disadvantage. The Requesters argue that the EQRP 2 design is discriminatory vis-à-vis non-urban children because: (i) only 7 out of 109 high schools were opened in non-urban areas, (ii) the financing system privileges urban schools which, the Requesters state, receive financing 30% higher than general school 1-12th grade, and (iii) the curriculum in high schools is different and qualitatively better than the curriculum in general schools for the same grades (10 through 12th). Therefore, they maintain, children from rural areas who, for economic as well as transportation reasons, cannot attend a high school are disadvantaged with respect to urban children in terms of quality of education and competitiveness to enter university. 50. Management does not agree, stating that while the infrastructure component of EIP focuses on high schools in urban areas, students in rural schools have and will benefit from the system-wide activities supported under the EQRP2 and EIP, such as in-service teacher training, connectivity to internet for all schools, curriculum revisions and improvements, and students’ assessment activities. 55 51. The Panel understands that the educational reform in 2006 extended the general education system from 10 to 12 years. In the following year, the decision was taken to establish three-year high schools, totaling 109 for the entire country. The result is that there are now two parallel systems for secondary education, one in which children attend school from first to ninth grade and then move on to high school, and the other where children attend the same general school from first to twelfth grade. The Panel was informed during the visit that the 109 high schools established with the support of EQRP 2 are situated prevalently in urban areas (the Panel was informed that some one-third are in Yerevan where 35% of the population resides, at least one in each capital of the ten marzes or provinces, a small number in villages, and the remainder in other cities and towns). High schools were also created with the intent of becoming centers of excellence in specific specialized subjects, such as math, sciences and languages, and to cover the surrounding communities as “catchment” areas. In the villages, some 9th to 12th grade 55 MR, p. 17, para 27, section (iii) 11 children commute to the high schools if accessible, while most other children attend the 1-12th grade general schools. 52. Officials of the PIU informed the Panel that initial plans were to try to introduce high schools in urban and rural areas alike, but for economies of scale and cost-effective considerations, the decision was made to establish high schools only in large communities of 15,000 inhabitants or more, while at village level general schools 1-12th would be kept. The PIU also noted that support under the EQRP 2 was focused on establishing high schools, not on 1- 12th grade schools. All schools were however integrated into an internet IT network system financed by the Bank’s project, as a basis for an MIS supporting the administration of schools. The Panel was also informed that more than 90% of schools have computers, but some twenty schools in remote areas have problems with accessibility. An ICT syllabus is being introduced, and children are accessing the internet in 50 pilot schools. 53. According to the document provided by the Minister for Education, the country’s 109 high schools are located accessibly in central locations in each region and designed as regional high schools in and near population centers, at reasonable distances from rural areas, to minimize the aggregate amount of travel for students from nearby communities. This is a rational approach to high schools followed in most rural areas throughout the world… 56 54. During the visit, the Panel was informed of the rapid demographic decline in Armenia. According to UNICEF, the number of children have halved over the past twenty years, reflecting the overall population decline through emigration as well as the drop in fertility rates. 57 The Minister for Education spoke of the two crises he faced; one financial and the other, the demographic one. Under such circumstances, it is not unreasonable to locate the high schools in centers of population concentration, with the intent of covering a larger “catchment” area of surrounding villages. In addition, the Panel understands that there are inaccessible mountainous villages that are cut off for months on end during the harsh winters; children from such areas would be unable to commute to the towns for their education. 55. The PAD acknowledges that the decision to concentrate high schools in bigger centers presents the “flip side” that students who could have had a chance to go to the new, specialized high schools “will now stay in their local school with, in theory, a more constrained opportunity to learn in comparison to their urban counterparts.” 58 The PAD indicates however, that a Mitigation Fund of AMD 50 Million (ca. US$123,000) was created and transferred to the marzes (provinces) education departments to cover transportation costs. The PAD further notes that the quality of education in high schools is expected to be higher than in regular 1-12th grade schools, but adds that the implementation of need-based grants for rural students to cover transportation costs “would contribute to keep a high enrollment rate in upper secondary education in Armenia.” 59 56 Document from the Ministry of Education and Science received on July 10, 2014. 57 “It is estimated that Armenia has lost up to a quarter of its original population since independence due to emigration,” “currently, out-migration from rural to urban areas (mainly to Yerevan) continues. Some rural communities of Armenia already experience a very strong depopulation.” (Manasyan, H. and G. Poghosyan. Social Impact of Emigration and Rural-Urban Migration in Central and Eastern Europe. Final Report: Armenia. April 2012). 58 EQRP 2 PAD, p. 35, para. 30. 59 EQRP 2 PAD, p. 35, para. 32, section (iii). 12 56. The Panel was informed during the visit that the high school curriculum and that used for grades 10 to 12 in the general schools are currently the same. During a meeting with the Yerevan State Medical University, university officials told the Panel that the number of students from non-urban areas applying to their university has been increasing and students from rural areas reach now over half of their new enrollments. 57. Tutoring. The Requesters claim that the quality of education has decreased over the years forcing parents to resort to expensive private tutoring to guarantee a level of education for their children that would allow them to enter universities. Further, they state that some teachers encourage kids to take tutoring classes that they themselves provide and then discriminate among children who did not attend. 58. The Panel understands that under the reform process supported by Bank financing, a unified exam to enter university has been instituted. According to parents with whom the Panel met, this exam is very difficult and neither the new high school curriculum nor the 1-12th school’s curriculum adequately prepare children for such an exam. 59. Management states that tutoring is not uncommon in countries with competitive university entrance systems. 60 The Panel was also informed through several sources that tutoring is not a recent phenomenon in Armenia but has existed since Soviet times, especially for students preparing to enter university in certain subjects. It was also explained by some that the tutoring was primarily focused on preparing for appropriate techniques for passing the unified university qualifying exam rather than acquiring additional knowledge. These views reflect a different perception from that of the Requesters, for whom the prevalence of tutoring is directly indicative of the declining quality of education in schools. 60. Quality of Teacher Training. The Requesters allege that there are many “professionally unqualified teachers” and that teacher training was not adequate to help teachers implement new methodologies and approaches in the classes and thus offering better education to the teachers. Tight political control over the decision making structure in schools harmed independence and as a result progress in reforming the education system. 61. Management Response states that the support for in-service teacher training provided under EQRP2 was aimed at “leveling the field” among teachers with regard to the many reforms that have taken place, including curriculum reforms, the extension of the education system from grades 10 to 12, and the introduction of ICT into the learning and teaching process. According to Management, an impact evaluation of in-service training showed some positive impacts in the use of interactive teaching methodology but also identified areas for further improvement. 61 62. The note prepared by the MOES states that although the majority of teachers participated in informational training workshops under the Projects, they were neither designed nor expected to have a substantial effect on teaching practices, which as any education professional knows is 60 MR, p. 27, Claim no. 8. 61 MR, p. 17, para. 27, section (iv). 13 a longer, more arduous process. Furthermore, it adds that the number of teachers receiving qualification ranks in 2012 was 51, in 2013 was 96, and in 2014 was 146. A total of 5,925 teachers have been trained under EQRP2 in 2011 and 6,241 in 2012. Evaluation of trained teachers shows that 94% are actively using the interactive and cooperative teaching methods, while only 0.6% continues to use their traditional teaching practices. 63. The Panel notes that all stakeholders agree on the need for further strengthening of teacher training for it to have a sustainable impact. 64. Standards and Curriculum Development. In the Requesters’ view, the EQRP 2’s support for standards and curriculum development has translated into support for education content that is discriminatory, especially with regards to gender issues. They add that content analysis of school textbooks shows that asymmetric representation of gender roles prevails and that discriminatory norms and perceptions are widely promoted both in textbooks, teaching process, and teachers’ attitude. In the Requesters’ view, this does not provide a comprehensive development of individuals as specified in the standard and goal of the subject program. During the meetings with the Panel, the Requesters elaborated on this by adding that the current system for textbook development, design, publication, review and approval lacks transparency and checks and balances; in the words of the Requesters, neither authors, publishers, committees responsible for piloting, (nor) expert reviewers are gender sensitive. The Request gives an example from a 10th grade textbook quoting a psychologist that describes The needs of a man are in sexual satisfaction, rest companion, a charming woman, household management and admiration. For a woman the needs are expressed in tenderness, conversation, honesty and frankness, financial support and devotion to family. In this context, according to the Requesters, the teacher training financed under the Project was inadequate and reinforced discriminatory concepts about the role of women in Armenian society. 65. Management acknowledges that this is a valid and important issue, and while stating that the Bank has not provided support or advice for the development of textbooks and is thus not responsible for its content. Management also emphasizes Bank’s strong commitment to gender equality and gender inclusion. Management further states that it intends to review and verify the Requesters’ claim, and if confirmed, the issue will be raised with the Borrower in the context of ongoing policy dialogue. Management also states that the curriculum revisions envisaged under EIP will provide an additional opportunity to revisit the textbook content issues. 66. Financing of Education. The Requesters claim that the Project component in the EIP providing Competitive Innovation Funds (CIFs) are isolated efforts that do not address the more general issue of declining public spending nor the legal framework regulating finances of universities to ensure more sustainable financing. Management shares the Requesters’ view on the importance of sufficient funding for the education sector and state that they brought this message regularly to the attention of MOES and MoF officials. 62 In Management’s view, the 2.6 percent of the GDP allocated for public expenditure on education is very low compared to most other countries at comparable levels of development. 62 MR, p. 14, para. 26, section (ii). 14 67. Management explains that the CIF aims to increase the diversification of sources of funding of higher education through a competitive and transparent process of allocating funds. The Panel heard from the Yerevan State Medical University highly appreciative comments on funding received from CIF to finance a modern Simulation Center in the Medical Faculty, which would allow it to greatly enhance quality and safety of medical training. The University will provide matching funds to cover building and management expenses. At the same time, there were also comments regarding the cumbersome procedures involved, especially for procurement. 68. The Panel notes that the CIF is based on a “bottom up” approach based on competition and demand, linked to the private sector and based on transparent distribution of funds. The Panel notes that CIF was not designed to address the overall financing issue. 69. Governance Issues – ANQA. The Requesters claim that Project has helped established a body, ANQA, whose composition and function present many irregularities, thus legitimizing a corrupt system that was already pervasive. Further, the accreditation system created by ANQA and supported by the Project has encouraged corrupt practices in universities. During the meeting with the Panel, the Requesters expressed the view that ANQA legitimized or provided window dressing to a corrupt system. 70. Management states that they partly concur with the Requesters’ assessment of the governance challenges facing higher education 63 and that these issues were identified by the Bank, brought to the Borrower’s attention and are being addressed through several Bank- supported Projects. Management also adds that with regards to higher education, the governance issues raised by the Requesters are broadly consistent with the findings of the Bank’s 2013 education report. 64 In the Management’s view, both Bank supported Projects have contributed to gradually improving the governance of higher education. The Panel heard from some of the Requesters their acknowledgement that the unified university entrance exam removed the possibilities of corruption that was pervasive previously when each university was conducting its own entrance exams. 71. The MOES also recognized in its note that the internal quality assurance systems are still under development, acknowledging the critical importance of quality assurance issues and the need for a longer time period to achieve noteworthy results 72. A key issue regarding the Requester’s concern about governance was the composition of the ANQA Board. In their view, the very institution that is intended to assure quality is unable to do so in that as currently constituted, the Prime Minister (PM) is the President of the Board of Trustees, which impacts the independence and credibility of ANQA’s accreditation process. This point was fully recognized by the ANQA representative during the meeting with the Panel, who held the view that the PM’s chairing of the ANQA Board during the transitional stage was desirable in order to ensure that state universities pass through the quality assurance review procedures. Furthermore, the ANQA representative recognized that this goes against the established European standard and will prevent further progress with the Bologna process; thus 63 MR, p. 14, para. 26, section (iii). 64 Addressing Governance at the Center of Higher Education Reform in Armenia, January 2013 15 changes will have to be made. At the same time, it was explained that ANQA has a diversified set of governing bodies for accreditation, with checks and balances that provide assurances that the accreditation committee will act independently from the Board. 73. Now that ANQA is more strongly established, and 15 universities have gone through the accreditation process, the ANQA representative noted that changes can be made to further improve its functioning and governance. 74. Consultation. While the Requesters state that the beneficiary and public feedback was ignored both during the implementation and assessment of the project, Management maintains that close contact was maintained and extensive consultations with various stakeholders have taken place, and states that the feedback received was taken into account. In addition, a series of analytical studies were disseminated and discussed in country in May 2013. 65 Annex 3 of the Management Response lists a series of consultations that were carried based on the Bank’s analytical studies, as well as consultations in preparation of EIP. 75. Monitoring and Supervision. According to the Requesters, the main focus of EIP is on refurbishing school buildings rather than addressing the shortcoming of the previous program, and poses a threat of great harm to the education system of the whole country. In their view, any new loan of the World Bank must consider the problems of a previous loan program and thus address those issues. 76. With regard to OP/BP 13.05 (Supervision), Management asserts that the Projects have been supervised according to the requirements of the policy. Management also asserts that the Projects are anchored in country policy/sector analysis; and reflect lessons learned from the Bank’s experience, as required by OP/BP 10.00 (Investment Lending) Management also attaches an Annex listing how they had applied lessons from previous Projects and international experience into consideration. Management states that OP/BP 8.60 (Development Policy Lending) and OP 13.60 (Monitoring and Evaluation) are not applicable to the Projects. Management states that Policy on Monitoring and Evaluation, OP 13.60, establishes the requirements for results-oriented monitoring and evaluation, but does not include obligations at the project level. 77. While the Panel agrees with Management that OP/BP 8.60 is not applicable to the Projects, the Panel does not share Management’s view on the application of OP 13.60 on Monitoring and Evaluation. In accordance with this Policy, the Bank needs to monitor and evaluate its projects to assess the extent to which its efforts and those of borrowers are making progress toward the objective of reducing poverty and achieving sustainable growth. 66 Management lists actions being undertaken under the monitoring system currently in place on p.32 of the Management Response. The Panel trusts that this monitoring system is being implemented in order to meet the continuing challenges of the sector. 65 MR, p. 18, para 28, section (i). 16 F. Panel’s Analysis 78. According to the Resolution, the Panel’s mandate is to review Requests alleging that the rights or interests of the Requesters “have been or are likely to be directly affected by an action or omission of the Bank as a result of a failure of the Bank to follow its operational policies and procedures with respect to the design, appraisal and/or implementation of a project financed by the Bank.” 79. In making its recommendation on whether an investigation of the claims raised in the Request is warranted the Panel considers, in addition to the technical eligibility criteria discussed in Section E above, the following factors: i. Whether there is a plausible causal link between the harm alleged in the Request and the Project, ii. Whether the alleged harm and possible non-compliance by the Bank with its operational policies and procedures may be of a serious character, iii. Whether Management, in the Panel’s view, has dealt appropriately with the issues raised in the Request and demonstrated clearly that it has followed the required policies and procedures, or Management acknowledged that it did not comply with relevant policies and procedures, and iv. Whether Management has provided a statement of specific remedial actions, and whether, in the judgment of the Panel and taking into account the view of the Requesters, these proposed remedial actions may adequately address the matters raised by the Request. Points iii and iv above do not apply in this case. 80. During its meetings with the Requesters, the Panel had the opportunity to hear concerns regarding the education reform process from different stakeholders, ranging from parents, students, civil society representatives looking at the various aspects of education, and university professors. They all spoke very passionately about their hopes for reform that can guarantee better quality of education for their children and the future of their country. Parents in particular, who experience on a daily basis the challenges of providing a good education, alongside their children, expressed their frustration about the education system they view as flawed and not improving. This perception is apparently widely held; the Panel notes that in the UNDP “Social Cohesion Study” of 2011, some 65% respondents felt that education is worse than 25 years ago, 40% worse than 5 years ago. 67 81. Management, in its Response, stated that "the Bank has been informed that the government remains committed to continue with the long-term reform agenda…" 68 The Panel has noted the Minister’s repeated statements recognizing some of the challenges, but adding the need for a longer time frame required to achieve all the necessary changes. 67 Slide 6, 2011 Social Cohesion Survey CRRC-Armenia, UNDP: http://www.crrc.am/hosting/file/_static_content/projects/social_cohesion_survey/CRRC_UNDP_SocialCohesion%20Presentation _English.pdf 68 MR, p. 13, para. 25. 17 82. The Panel notes that the issues raised by the Requesters in their submission to the Panel are serious, legitimate concerns of citizens about the content and directions of reforms in education that is absolutely critical to the future of Armenian children, and therefore of Armenia. Management, in its Response, indicated that they share some of the critical concerns elaborated in the complaint. In the Panel’s view, the allegations of harm are thus of a serious character. 83. Notwithstanding these considerations, and after careful review of the Request, the Management Response, and taking into account the information gathered during the visit to Armenia, the Panel is unable to establish that the concerns raised by the Requesters, although legitimate, can be considered as instances of material harm that may have resulted from a failure of the Bank to follow its Operational Policies and Procedures with respect to the design, appraisal and/or implementation of the Projects financed by the Bank. 84. In the Panel’s view, the Requesters’ claims point to systemic weaknesses of the education sector in Armenia that the Projects indeed were designed and aimed to address, and the Panel does not consider them as harms resulting from the Projects. 69 Moreover, an in depth analysis by the Panel of the results achieved by the reforms with support of the Projects would amount to an evaluation review that goes beyond the purview of the Panel’s role and mandate. G. Recommendation 85. In light of the foregoing the Panel does not recommend an investigation into the matters alleged in the Request. 86. The Panel appreciates Management’s statement that it intends to verify and review the claims related to gender issues and related insensitivities, and to include them in its policy dialogue with the Borrower. 69 During the Panel’s meeting with one of the Requesters, it was mentioned in connection with the issue of undue religious influence in textbooks that while the issue was not directly linked to the World Bank, if the Bank was financing reform, then it should address the issue. This also does not constitute harm resulting from the Project, as per the Panel’s Resolution. 18 ANNEX I Date: May 16, 2014 To: Executive Secretary, the Inspection Panel 1818 H Street NW, MSN 10-1007, Washington, DC 20433, USA Who we are: N.B. We do not authorize you to disclose our identities. Subject of the Claim On March 13, 2014, the World Bank Board of Executive Directors approved a US$30 million financing for the Education Improvement Program in Armenia. 1 This financing continues two previous 2 loan programs and is supposed to build on the latest program, the 2009-2013 Education Quality and Relevance II Loan Program. We started working on this Claim before the approval of the Education Improvement Program in Armenia with the purpose of postponing its approval and initiating a revision of the scope and purpose of the proposed financing. Our analysis indicates that the Education Quality and Relevance II Loan Program resulted in substantial harm to the Armenian education system, particularly in the areas of accountability, governance, quality, and accessibility. We, all the above signed, believe that the main directions of the Education Improvement Program in Armenia loan project are designed in such a way as not to address failures of the previous program and pose a threat of greater harm to the education system of the whole country and hence each and every one of us. Simultaneously, among the signatories of this claim there are people who are immediately affected by the harmful effects we identify below. Namely, there are those whose career is harmed because of the created and supported governace structure where lack of academic freedom and multiple layers of conflicts of interest are not conducive to the right to critical thinking in research and teaching. There are others, who were simply fired for ideological reasons. There are parents who had to move children to private schools, hire tutors as the high school in its form is not suited to provide adequate education for future higher learning and career (see Attachment). It is important to note that the individual cases are reflection of systemic problems that we prioritize given the nature of our organizations' missions. The need for additional tutoring even for best students is so widespread that the children admitted to universities without private expensive tutoring are exceptions rather than a norm To support the statement, a blitz survey conducted among 95 first and second-year students (i.e. graduates of newly established high school) of one department of Yerevan State University showed that only 4 of them had not had additional tutoring. That is only 1% of all graduates were able to continue higher education at the university with knowledge and skills gained in the high school. We believe the new financing in its curren.t scope and form does not properly account for the vast governance and accountability problems in Armenia's education s~ctor and would exacerbate the harm caused by the previous loan rather than improve the quality of and access to education in Armenia. We believe any new loan needs to take into account the problems with the previous loan program. Simultaneously in view of existing governance inadequacies demonstrated in dubious and conflicting legislative and regulatory frameworks and numerous obvious conflicts of interests (institutional, not personal) at the highest levels of governance, certain preconditions need to be established to satisfy minimal accountability standards before the World Bank moves forward with any new financing. 1 The project will support the improvement of school readiness for children entering primary education and the physical conditions in upper-secondary schools. It will also promote greater links between higher education institutions and labor market in Armenia. Retrieved from http://www.worldbank. org/en/news/press-release/20 14/03/ 13/world-bank-supports-improvements-in-education-in-armenia 2 Education Quality and Relevance I Loan Program, World Bank, 2003-2009. Education Quality and Relevance II Loan Program. World Bank 2009-2013. We presented our facts and analyses to senior World Bank staff in both Yerevan and in Washington DC 3 . r The fact that the World Bank unconditionally approved the new program is particularly surprising given that the Bank's own assessment4 of the previous project concluded that in Armenia there are vast and systemic problems with education governance that hamper any potential for improvement. The World Bank's assessment alone, even without other credible supporting research commissioned by OSF Armenia, 5 provided conclusive evidence that the objectives of the previous financing had not been achieved and that these failures posed risks for further improvement. A detailed chart below specifies how concrete deliverables were not attained and the particular failures of the World Bank procedures in relations to each objective thatwe believe was not achieved (Chart 1). In general, both the World Bank's and above noted outside researchers identified the main systemic problems that caused stagnation of the reform process and degradation of the quality and equity of the tertiary education system, including its key aspects - learning and teaching, research, relevance to the economy, relevance to the needs of a democratic society and sustainability. These flaws are: poor governance and quality assurance, inadequate financing and overly-tight political control and a resultant lack of academic freedom. It is our contention that without a serious effort to address these problems in advance the World Bank's plan to support research through selected university grants will not contribute to improving Armenian universities' research capabilities or their sustainability (from either the financial or credibility points of view), development or contribution to industry and the labor market. While quality assurance was a major component of the previous WB education project, the Central European University's Higher Education Observatory reports that higher education reforms in Armenia have not had a significant impact on the quality of teaching and learning in higher education, or on the content and substance of administrative or governance practices in the field. 6 Moreover, the overall impact of the quality assurance reforms has not been positive; instead quality assurance changes are carefully and fully controlled by the government and ruling political party with no dialogue or consultation possible among relevant stakeholders. 7 Disingenuous quality control is exercised by the state with no nationally defined performance indicators or a comprehensive database to monitor and evaluate higher educational institutions' (HEis) performance. 8 Both the CEU's and the World Bank's own reports strongly emphasize the pernicious practice of the government directly influencing higher education governance by appointing a majority of the members of HEis' governing Councils. 9 The reports further highlight systemic problems pertaining to conflicts of interest in the governing of HEis, widespread .corruption, and nepotism in the field. 10 Quality assurance reforms have been carefully controlled, designed and carried out bureaucratically in a purely top-down manner, and are therefore 3 Meeting with Mr. Jean-Michelle Happi, Country Director, World Bank Armenia office in November 2013. Official correspondence to Mr. Happi following the meeting dated December 9, 2013. Meetings with education team, Armenian ED's office and a round-table presentation at the the World Bank Washington office in DC, February 2014. 4 Kataoka,S; Shahverdyan, A; Harutyunyan, H. 2013 . Addressing Governance at the Center of Higher Education Reforms in Armenia. Washington DC. World Bank. Retrieved from http://documents.worldbank.org/curated/en/20 13/0 1117748657/addressing-govemance- center-higher-education-reforms-armenia 5 Matei, L., Iwinska, J. & Geven, K. 2013 . Higher Education in Armenia Today: a focused review. CEU Higher Education Observatory. Budapest. 6 Matei, L., Iwinska, J. & Geven, K. 2013 . Higher Education in Armenia Today: a focused review. CEU Higher Education Observatory. Budapest. Pp 9-11. 7 Ibid page II . 8 Kataoka,S; Shahverdyan, A; Harutyunyan, H. 2013. Addressing Governance at the Center of Higher Education Reforms in Armenia. Washington DC. World Bank .Page 17. 9 . Matei, L., Iwinska, J. & Geven, K. 2013.Pp. 11 , 12-14.Kataoka,S; Shahverdyan, A; Harutyunyan, H. 2013 . Pp. 24-9. 1 ° Kataoka,S; Shahverdyan, A; Harutyunyan, H. 2013 . Pp.4, 12. Matei, L., Jwinska, J. & Geven, K 2013. Pp. 3, 24 met with distrust if not open opposition and simply lack of understanding by most intended 11 beneficiaries , eventually resulting in poor quality reform of higher education in Armenia. An additional obstacle to quality-based higher education is created by poor financing schemes in the field. Successful reform of existing financing mechanisms, which the government has already committed to implement, will require not only effective allocation of available funding, but a substantial increase in total public funding for higher education. In that case reforms can be expected to have a measurable impact on the quality and competitiveness of higher education. 12 This will only be possible when there is a broad public consensus on the direction of the reforms and trust in HE governace. Another significant structural problem is the overly tight political control of the sector. The reports by CEU and the World Bank indicate identical channels and means through which exaggerated and severely counterproductive control over higher education institutions is exercised. Regulatory ambiguity is consistently mentioned as a factor that allows state control over universities. 13 Moreover, the 2002 Charter of the Ministry of Education and Science establishes that the state is nominally in control of universities despite the fact that the proportion of funds allocated to the sector by the state is very limited. 14 Regarding secondary education - and particularly high school development as planned by the new loan- there are equally strong arguments against the unconditional new funding. The government and the ruling party (Armenian Republican Party) assert political control of the sector throufh control over the appointment of school principals 15 , the composition of school governing bodies 1 and financing mechanisms. 17 Secondary educational institutions and their employees serve as an important administrative resource that is systematically and vastly abused for political ends during elections. 18 The system of student aptitude evaluation is-hugely problematic in principle and provides conflicting or even mutually exclusive results for student learning and skills evaluation. It does not enjoy broad support and trust among either the public or professional educators, a fact amply demonstrated by expert interviews and research. Below in Chart 2 we present in greater detail problematic issues that arose during the implementation of the previous loan and failures by the World Bank to address underperformance and the causes that hampered development. 11 Matei, L., Iwinska, 1. & Geven, K. 2013. Pp. 7, II , 15-16. • 12 Kataoka, S.; Shahverdyan, A.; Harutyunyan, H. 2013. P. 23. 13 Kataoka, S.; Shahverdyan, A.; Harutyunyan, H. 2013. Pp. 8-10. Matei, L., lwinska, J. & Geven, K. . 2013. P. 13. 14 • Matei, L., Iwinska, J. & Geven, K. 2013. P. 13, 25. 15 According to largest Shirak marz data 70% of principals are of ruling Republican party. Policy Forum Armenia.2013. Corruption in Armenia. p31. Accessible from : http://www.pf-armenia.org/document/corruption-armenia 16 School boards are composed of 8 members, 4 of which are representatives of executive and/or local government. The responsibility of School Boards is selection of school principal, financial decisions, approval of suggestions on procurement/alienation of school property, etc. Despite the fact that school principals are selected by the School Boards, contracts with them are processed and signed by supervising body (marzpetaran or Yerevan municipality). Current way of Board composition gives the executive full control over decisions. Most of school Boards are chaired by high level officials from marzpetarans or Yerevan municipality. 17 Per-capita funding mechanism is not based on the measurement of necessary costs needed to ensure quality education of each student. As a measuring principle in educational service provision no quality based criteria are taken into account. Instead, each year available amount for secondary education is mechanically distributed amongst existing schools based on the formula developed and approved by the Ministry of Finances and the Ministry of Education. No clear methodology of these allocations is available, nor a content based list specifying costs associated with each students' educational needs. There is no comparative data on how existing financial mechanisms contribute to and correlate with the achievements of the state education standards and objectives. 18 Republic of Armenia Parliamentary Elections. 2012. OSCE/ODIHR Election Observation Mission. Final Report. Warsaw. pl2. Republic of Armenia Presidential Elections 2013. OSCE/ODIHR Election Observation Mission. Final Report. Warsaw. pp 12-13. Observation of the Parliamentary Elections in Armenia. 2012. Council ofEurope.pp30-38 By not addressing systemic problems and clearly identified failures the new loan threatens to exacerbate those and make the situation irreversible given the vast amounts that are supposed to be spent under flowed premise of reform. While further development of the high school system is stated as a priority in the new loan, it is limited to the physical refurbishment of a limited number of high schools ( 17 out of the existing 102 19) and will not address the issue of quality and accessibility of high schools. Education standards, curricula and the entire system of high school are considered as deeply problematic by both the public and the education community. The curricula and education standards and teachers' professional capacity are of particular concern. The failures of previous standard development have already resulted in gender biased content of education materials and incompliance of civic education with human rights standards and Toledo principles? 0 Along with gender insensitive content and the constant messaging of traditional gender stereotypes, the teacher's deeply biased attitude is particularly damaging for developing the notion of gender equality in youth during their formative years. The level of bias is well demonstrated in a survey on gender socialization among teachers according to which a majority of teachers strive to instill docility and modesty in girls and leadership in boys. Equally troubling is their statement that the educational benchmarks they presume for boys and girls are different with the boys being held to higher standards, with a justification that they need education more than girls 21 • This is in a country where the number of women with higher education has been greater than men for decades and women in technical and scientific specialties constituted almost half of workforce as recently as ten years ago. The process by which the standards, the curricula and syllabus are reformed needs to become open and transparent; the benchmarks must exclude the possibility of propaganda of gender bias and discriminatory behaviors. Yet, there is no such guarantee as the same process that had created the problems is supported through the new loan. In the two charts below we present major points of concern regarding implementation of the previous education versus its stated objectives and deliverables. We also underline particular failure of the WB procedures that contributed to unsatisfactory implementation/result, specifically meaning two things: firstly, the failure to properly identify some of the outlined systemic and damaging problems through proper assessment and hence address them in a timely manner; and secondly, most importantly, when a large number of systemic problems ranging from conflicts of interests, breaching of academic independence and integrity to abundant executive control over the education, had been identified and reported within the WB assessment, failure to not address those in any manner that the harm is minimized and the problems are resolved. Moreover, the problems being systemic and far-reaching into all aspects of governance and quality assurance inadvertently endanger any future support, whereas the World Bank approved the new financing without any reference to the problems and attempt to correct the wrongs. 19 Total number of general education institutions in Armenia in 2013-2014. http ://stat.armedu.am/?section=content&id=2&year=20 13 20 Tsaturyan, R.20 13. Gender Based Stereotypes in Pedagogical Domain. Yerevan. Accessible from http ://www.osf.am/wp- content/uploads/2014/0I/Policy paper Ruzanna Tsaturyan.pdf; Hovhannisyan. H, .Davtyan, A. 2013.,Content Analysis of Armenian Church History Textbooks'. Yerevan. Accessible from http://www.osf.am/programs/policy-fellowship-initiative/policy-papers/?lang=am 21 According to the survey, for 96% of teachers, courage, determination and industriousness are among instilled characteristics for boys; whereas modesty and patjence should be instilled in girls according to 94% of the same teachers. Also 66% of teachers consider boys more capable than girls, while 68% of teachers consider girls to be obedient and modest. Gevorgyan. H. ed. 2011. The Mosaics of Gender Relations. pp35-37. Suggestions to the WB • Suspend implementation of the newly approved financing for education. Initiate thorough revision of the project with genuine involvement of local education commun~ty, public policy experts in design and consultation process. • Condition the WB support upon elimination of conflict of interest, government representation and political control in higher education governance through resignations of high ranking officials (i.e. the president, the prime minister, etc.) from university governing boards, establishment of periodic public reporting mechanisms and separation of political and educational spheres via internal university regulations. • Use clearly defined conditionality tools while working with the Armenian Government. The main conditionality tool in the given context is the reform of the legislative framework to the effect of guaranteeing freedom and independence of HEis in accordance with the requirements of Armenian Constitution and the Law on Education. Particular legal norms include: SNCO law that allows exercise of undue executive control over university financial management, university board governance and management, government Decree that allows appointment of political figures and government officials as board chairs and majority representation in the university governance. • In addition to institutional enhancement of internal quality assurance, address internal quality improvement in practice based on clear measurement tools, such as number of internationally competitive research projects and research outcomes by the faculty and students, random sampling of academic papers written by faculty and students and checking of those against criteria of academic honesty, practice of academic freedoms, continuous and holistic plan of the training of faculty and staff, etc. • In consultation with a wide scope .of beneficiaries design quality measurement tools of specialized high school staff, its professional capacity, school facilities, and access to high schools in support of high school reform. Design further WB support to high school reform based on the measurement. • Prior to deciding main directions of WB support, create a platform where wide circles of beneficiaries will come to reflect upon the directions and means of reform implementation consecutively contributing to the popularization of the education reform agenda itself. Chart 1. Tertiary Education components of the WB Second Loan with described harms and bank procedure failures Description of the Bank Harm Failure of the bank procedures Loan component 1. Establishment and In our assessment (coinciding with the WB's conclusion) the newly The Bank failed to address the Strengthening of the established institution, Armenian National Quality Assurance identified problems to the borrower National Quality Agency allows for political control mechanisms and lack of integrity to resolve them. Moreover, Assurance System in the system. It does not meet standards for independence and its establishment of ANQA is The main goal of the sub- scope of authority is insufficient to fulfill its mission both per considered as a major achievement component is to establish European standards and versus its own charter. of the loan project, despite its National Quality criticism by the Bank itself. Assurance System of Details: Higher Education, as well Established in 2008 Armenian National Quality Assurance Agency Details: as support the (ANQA) functions a consu.tative body for external quality ENQA principles, ANQA own establishment and assurance. There are serious irregularities in its mission, mission statement have been strengthening of its composition and functions. The mission of ANQA is stated in its identified by the Bank as capacity. Two major Strategic Charter22 as "independent from the RA Government and undermining the credibility of the components will be accreditation subject institutions". However, it is neither agency. supported: a) independent nor has the accreditation authority, but is rather establishment or external consultative to the Armenian government. Moreover, it is chaired by quality assurance in line the Prime Minister, who in his tum is the board chair of the with QA principles Armenian State Pedagogical University. Moreover, ANQA is a under European consultative body only and the final decision on accreditation lies standards, support to with the Government. Additionally, ANQA is an associate. member the establishment and of ENQA and aspires for its full membership. However, the level of functioning of ANQA political control in its composition and failure to adhere to the according to the external quality assurance principles are major violations in the government charter and functions of the Agency. Namely, the principle of independence as its capacity building defined in the Standards and Guidelines for Quality Assurance in European Higher Education Area, as "Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and 22 ANQA Strategic Plan 2010-2015. PI!. Retrieved from: http://www.picga.org/en!Portals/O/PicqaFiles/1 .2%20ANOA Transitional Period strategic plan.pdf recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders" 23 . This has also been commented in the ENQA report of ANQA's work in November 2013, where the direct conflict of interest, lack of authority to make accreditation decisions and coaching work have been mentioned as diverted from main mission of an independent external accreditation agencl4 • b) Internal quality The current set up of university governance is regulated through Despite own assessment of the assurance system with ambiguous legal documents that allows overrepresentation of tertiary sector governance issues policies and procedures political entities in various ways into both education content and and irregularities, failure to make for internal mechanisms, management, and uses as a tool for political/ideological control, corresponding changes in program and establishment of QA brings to top down reform implementation, non-inclusive design/concept and continuing to units in universities approaches for academic community. fund fragmented components that have little likelihood to sustain the Outcomes: system. Strengthened capacities Details: of ANQA, MoES Higher The governance boards of state universities, state education agencies To address qualitative, in additional Education and Licensing are represented by government officials, MPs, and by the to quantitative, measurement in Departments, HEis; administration of the President at the level of no less than 50%. 25 Program Result Report and make Universities are managed by Law on Education, Law on Higher and adjustments for future activities. Ensuring day-to- day Postgraduate Education, Law on State Non Commercial Details: effective operation of Orgaillzations, which are in direct contradiction with one another The internal QA centers supported newly established and allow for interventions by the state over their finances, by the Bank program only bodies(ANQA, IQA units management appointments, specializations, and admissions. Such structurally follow the policies and in HE Is) in accordance intervention is in itself anti-constitutional procedures on formal level. with the Bologna The establishment of internal QA units was mostly done at formal Whereas the goal is on the 23 Standards and Guidelines for Quality Assurance in European Higher Education Area.p25.Retrived from: http://www.enga.eu/wp-content/uploads/2013/06/ESG 3edition-2.pdf 24 http://www.anga.am/argata/Portals/O /documents/ANOA%20Reports/ANOA %20Proof"/o20Review"/o20Pane1Report%2028.11.13.F1NAL.pdf 25 The full list is available in Kataoka, Sachiko; Shahverdyan, Anush; Harutyunyan, Hovhannes. 2013. Addressing governance at the center of higher education reforms in Armenia Washington DC. World Bank. Retrieved from http://documents.worldbank.org/curated/en/20 13/0 1/ 17748657/addressing-governance-center-higher-education-reforms- armenia objectives; level with little capacity and internalization of the required tasks. development of internal QA A status report on Bologna in 2012 shows that 56% of universities mechanisms, the measurement for Establishment and do not have a QA concept for their university developmene6 . this is only done through the strengthening of Moreover, the report shows only 18% of universities in Armenia number of QA units established. internationally compatible develop important documents in consultation with faculty and mid- However, the quantitative Quality Assurance system management, which shows that these papers are developed in indicators cannot be considered as in Armenia. narrow circles, do not reflect needs of practitioners and are not measuring the quality system and brought down to this level. quality per se. 2. Constraints in financial autonomy, control over university finances Key risks were identified with a) Designing a restrict universities and fragmented funding for research and re.commendations to the country sustainable strategy for modernization cannot assist in ensuring sustainable funding system, government to resolve them, public financing of HE especially considering decreasing public expenditure on education. however no actions were taken with components of other than "closing eyes" on the legislative changes for the Details: The financing strategy does not address the decrease in the risks. implementation of the public spending and does not offer ways for universities to engage in higher education financing due to legal framework regulating the finances of various activities - strategy's universities27 . The state controls directly the budget of state Details: Situation with governance recommendations; universities, despite the fact that the proportion of state allocation is and funding principles was assessed financing new methods very limited. State universities receive only between about 9 and as a major problem with the higher and mechanisms; 30% of their budgets from the state; the rest of their funding comes education sector. Despite own expansion of students' from student tuition fees 28 • The latest Medium Term Expenditure assessment, the Bank bypasses this financial aid; proper Framework 2012-2014 indicates that total education spending will situation and directs funding into pricing policy and continue declining in the next three years down to 2.6 percent of compartmentalized sector, which methodology of tuition fee GDP, placing Armenia back to among one of the lowest education given the current situation cannot formation spending countries in the region. soar. b) Designing a Details: Absence of research and complete separation of research competitive innovation and teaching is one of the major shortcomings of education system. fund Not only do universities disregard the ipl}JOrtance of research, but 26 Current Status of Bologna Reforms in Armenia. 2012. National Center for Strategic Research in Higher Education. (report available in Armenian only) 27 Kataoka, S.; Shahverdyan, A., Harutyunyan, H.. 2013 . Addressing governance at the center of higher education reforms in Armenia. Washington DC. World Bank. 28 Formulating the Financing Principles and Exploring Alternative Financial Mechanisms for Armenian Tertiary Education. 2008. CHEPS. to develop and implement also do not create favorable conditions for faculty to participate in innovative plans for international programs and engage in research. The above modernizing their mentioned control mechanisms undermine academic independence curriculum and vital for credibility and sustainability of any research and scientific pedagogical methods, endeavor. Reports on how these control levers have been used to develop ECTS, build their ban "politically incorrect" research projects even at the level of fund-raising capacity, approval of topics are abundant. This makes competitive faculty create linkages with leave universities and/or adjusts to antiscientific standards. The industry, and engage in presence of research is a necessary condition for attracting international cooperation competitive faculty, who otherwise prefer to leave the sector. activities. Simultaneously, limited public funding for research (0.2 % of GDP) Outcomes: is primarily allocated to the Academy of Sciences, limiting - Higher education university research capacities. However, the major problem with the sustainable financing component is that apart from lack of funding, there are no strategy mechanisms for research development in universities 29 . Jhis is - Appropriate explained by the extreme workload for faculty/researchers, little documents for resource available for full-fledge research, control over financial designing CIF and transactions, and incomplete internal mechanisms and contradictions legislative documents to various policies and practices that bottleneck the autonomy of package for . universities. submitting to approval 29 Matei. L.., Iwinska J., Geven K. Higher Education in Armenia Today: a focused review. Budapest 2013. CEU Higher Education Observatory Chart 2. Enhancing General Education Component with harms and failures for bank procedures What is bank loan component Harm Failure of the Bank procedures Component 1 • High schools are largely inaccessible for students from rural areas. Enhancing the Quality of Hundred and two out of hundred and nine high schools are • Bank failed in setting proper General Education located in cities and only seven - in rural areas30 . In rural areas monitoring and self-evaluation there are still schools that work based on 12- year curriculum indicators for EQRP2 and relied Subcomponent 4. which is not designed and does not provide specialization. 31 These on the borrower's M&E system Supporting the Implementation schools are not able to comply with the government's plan to (OP 13.60, point 4.) and reporting. of High School Reform transfer to credit system in high school courses. Thus graduates of Irrelevant baseline data were high schools will have double advantage over those in 12-year considered for measuring the Goal: curricula schools. Given higher rates of poverty and vulnerability progress of EQRP2. Increasing the education quality in rural areas, as well as professionally and technically lower • The factual challenges of the high and relevance in high schools capacities of these schools compared to urban ones, the school reform were not taken into and establishment and disadvantage is much greater and the risks for further limiting account during the development improvement of the high school access to higher education is growing higher. stage of the new Education network by assisting in: • The existing per-capita financing mechanism does not ensure Improvement Project (Pl30182), allocation of the relevant financial resources for provision of while the Bank Procedure, defines - Organization of quality streaming education in high school. Students and parents that "During appraisal, the task teaching/learning process in still have to take private tutor classes for preparation to the final team assesses the adequacy of streams. un1.fi 1ed/admiSSion . . exams. 32 the proposed program to - Creation of methodological • Content analysis of school textbooks shows that asymmetric achieve its stated objectives." structure, curriculum, teaching representation of gender roles prevails. Male and female (BP 8.60, point 6). and learning materials for the characteristics, roles, attributes, professions are created and • The beneficiary and public HSs according to the diversified reproduced through images and texts. Discriminatory norms and feedback was ignored both during streams. perceptions are widely promoted both in textbooks, teaching the implementation, monitoring - Raising the professional level process, and teachers' attitude. By the decree of the Minister of and assessment of the EQRP2 30 Distribution of general education institutions in Armenia in 2013-2014 per educational programs available in urban and rural areas http://stat.armedu.am/?section=content& id=63&year=20 13 31 Total number of general education institutions in Armenia in 2013-14 per available implemented educational programs http://stat.armedu.am/?section=content&id=2&year=2013 32 Report on the State Policy on General Education Financing. 2014. Community Finance Officers' Association. Available at http://www.osf.am/wp- content/uploads/20 13/ !!/State-Pol icy-on-General-Education-Financing. pdf of high school teachers Education, the class for the subject of Technology should be results and development of new divided into groups based on gender. For each group there is Education Improvement Project, separate curriculum. About 30% of textbook themes are general while according to the and are meant for both girls and boys. Other themes are gender procedure (OP13.60) in addition specific and are meant for differentiated teaching for boys and to working with borrower, the girls (e.g. the section on culinary, food security and nutrition is Bank should "work with other taught only to girls, so · is the section on aesthetics, whereas the development partners to agree section on construction is taught to boys only). This is justified by on the results expected from further professional specialization for males and females, which development activities and to does not provide a comprehensive development of individuals as harmonize the monitoring, specified in the standard and goal of the subject program33 . reporting, and evaluation • Development of subject standard for the Armenian Church History requirements". is singlehandedly controlled and supervised by Armenian Apostolic Church through its Center for Christian Education and Propaganda. During the teaching process the doctrine of the Armenian Apostolic Church is preached. Discrimination and ill treatment are widely practiced by both teachers and students towards religious minorities34 . • Substantial part of EQRP2 was aimed at the professional development and in-service teachers' trainings. Initial stated aim of the teachers' professional development was the increased quality of education, capability of using ICT technologies and e- content during the teaching process . . However, the trainings conducted based on unified content and format, which was not efficient as teachers' fsrofessional and pedagogical skill, experience and needs vary 5 . 33 Ruzanna Tsaturyan. (2013). Gender Based Stereotypes in Pedagogical Domain. Yerevan. For full version of the paper available here http://www.osf.am/wp- content/uploads/20 14/0 !/Policy paper Ruzanna Tsaturyan.pdf 34 Hovhannisyan. H, .Davtyan, A. 2013.Content Analysis of Armenian Church History Textbooks. Yerevan. Accessible from http://www.osf.am/wp- content/uploads/20 14/04/H_Hovhannisyan_ Research-.pdf 35 Khachatryan. S., Petrosyan. S., Terzyan. G. 2013. Assessment of Teacher Professional Development and Educational Content in the Context of General education Reforms in Armenia. Barev Scientific Educational NGO. Available at http://www.osf.~wp-content/uploads/2014/03/FinalENGAssessmentPDEC.pdf Attachment Case 1: The case shows lack of academic independence for credibility and sustainability of research, censorship of research and politicization in academia In 2012 I contacted the Head of Philosophy Department of National Academy of Science and expressed my wish to write a dissertation within political philosophy field on "The solutions of current problems in Armenia through political left". The head of the department was long trying to convince me that this is not a right topic to take and was cynically suggesting that I write about "The behavior of strip dancers in Armenia". Anyway, it took me three months to discuss and negotiate with him the topic I chose and eventually he refused my suggested topic reasoning that he was not going to help the current political opposition in Armenia. Just to note that there is in fact no political left in Armenia and the opposition he was referring is of neoliberal nature. Not only did the department head violate my basic rights, but he also positioned himself politically as well as cynically. I was long trying to get a written notification from him for rejection, but he kept refusing. Case 2: The case is illustrative of lack of academic freedom in research, irregular mechanisms of research within universities, political control and violation of secularity of university. In 2003 I was admitted as a PhD student at Armenian State Pedagogical University and in 2008 I presented for a pre-defense my thesis paper on "Characteristics of Spiritual Education for High School Students" and received a positive feedback on it. The public defense of my thesis was postponed for 3 years, so for that reason it was re-submitted for another pre-defense in 2011 and eleven months later on May 24, 2012 it was finally set. The Department of Theology at Yerevan State University, which was wrongly selected as a leading entity, provided negative feedback on the thesis, naming it as "something populist". This could not have been legitimate, since the above department had no relevance to pedagogical sciences and was not entitled to provide opinion on the thesis work. It took another 10 months to get the Department of Pedagogy of the same university to take over as a leading entity and to provide a second opinion. This department's opinion was largely influenced by the existing negative feedback of the first department. Also, it was influenced by the opinion given by Reverend Father Vardan Navasardyan, the representative of Apostolic.Church and the Director of the Center for Christian Education. His opinion was non-scientific and should not have been taken into account for the defense of a scientific paper either legally or ethically, however it was accepted by the academic committee. The opponents claimed propaganda elements in the thesis without justifying their position. On February 28, 2013 the public defense eventually took place. Seven of the twelve members of the academic committee voted against my thesis and rejected granting me with a degree. I should note that my work has been presented to a number of scientific institutions at the National Academy of Sciences of Russia and the State Linguistic University of Novgorod and I have received offers to defend it in Russia or in Ukraine. In 2012 I was dismissed from my position at Armenian State Pedagogical University, which according to the university management was in the result of reorganization of the Chair of Social Pedagogy. I claim that the real reason behind this related to the issues of spiritual education in my thesis that was misinterpreted by the above communities as religiosity. I keep getting hints that the defense of my thesis work will never be possible, because I am a member of a religious organization and the academic committee will vote against. Case 3: This case demonstrates that due to dubious legislative field executive control over universities is made possible On April 6, 2012 the Rector of Yerevan State Linguistic University after Brussov received a reprimanding letter from the Minister of Education and Science Mr. Armen Ashotyan. The Ministe~ referred to the breach of duties and unexcused absences by the Rector whereby he, despite the Minister's rejection, participated in a conference in Strasbourg, France from March 19 to March 23, 2012. The Rector received a second reprimand by the Minister on April 17, 2012. The same day the Minister suspended the rector's responsibilities and terminated the employment contract. The Rector's dismissal is a violation of RA Constitution and his labor rights. Firstly, the university autonomy principle specified by the RA Constitution and the Law on Higher and Postgraduate Education was violated that allowed the executive to intervene into the hiring and firing of the university head (by the above laws such a function is prescribed to the university governance board). The Rector's dismissal was, however, possible due to the irregular legal framework, namely the RA Law on State Non-Commercial Institutions that regulates Armenian universities and allows the founder, i.e., the Ministry, to appoint and dismiss rectors and board members. This law therefore conflicts the independence of universities. Moreover, the labor right of the Rector was violated since the termination of his duties and the contract was not done by the relevant body, i.e. his employer. The minister is not entitled legally or practically to draw penalties, since according to the RA Labor Code this function is vested into the employer, i.e., the university board. Parents' authorization letter Subject matter: the cases support the claim that high school as designed and supported by the WB loan has not been realized to the extent that it does not provide minimal sufficient learning outcomes and skills to the students to continue education at universities, which is the main mission of the high school as stated by the Education Quality and Relevance Second Loan Program. We the undersigned parents - citizens of the Republic of Armenia, have suffered a significant harm in the result of insufficient implementation of the high school reforms that were initiated and implemented within the scope of the Education Relevance and Quality II Loan Program funded by the World Bank. We believe that this harm is a result of World Bank's non- compliance with its operational policies on monitoring and evaluation and improper control over the implementation of the ERQP II. In particular: 1. High schools, where our children study/ studied do/did not ensure provision of quality education services. All of us have had to either apply for additional tutoring or transfer children to private schools to get better quality education and preparedness to enter universities. We believe that our children would have never gotten quality education and enter the university if they had stayed at the state high schools. 2. High schools do/did not provide for the quality streaming education that would ensure professional orientation for our children. High schools are not equipped with the necessary technical capacities and do not have proper equipped laboratories and libraries that would provide for the quality education. Computer classes are not fully accessible for children. Teachers do not use the computers during the class. 3. The textbooks and teaching are/were of low quality. Some of textbooks, like Social Science textbooks for 9 to 11 grades, include discriminative norms and are gender insensitive. . For example, in the lOth grade textbook the hypothesis of psychologist U.F.Harley is given according to which there are five basic needs of men and women, fulfillment of which guarantees stability of marriage, while dissatisfaction of those needs may lead to the conflict and even to divorce. The needs of a man are in sexual satisfaction, rest companion, a charming woman, household management and admiration. For a woman the needs are expressed in tenderness, conversation, honesty and frankness, financial support and devotion to family. The textbook and teaching process do not provide children with opportunity to discuss and argue these statements. Hence there is real risk that children may accept them as norm and apply later in life. Another example from the same textbook says that "the societies, such as western ones, where women are fully involved in social life, tum to have a significant decrease in birth rate". Children can take the statement as granted and think that the only mission of women is to ensure population growth, thus making her reproductive function a priority in contrast to women's wish to become a fully integrated member of society. · 4. We all bear ·the harms that result from professionally unqualified teachers, their discriminatory attitudes and practices towards school children, unawareness of and incompetence in applying innovative teaching methods in classrooms. Some of the teachers in our schools do/did not take any action to promote interest of our children in particular subjects (e.g. physics) which resulted in zero motivation among children to study that subject. 5. We faced the situations when teachers proposed us out-of-class private tutoring of our children, which in fact was the compensation for the under-taught content in the classroom. Taking into account th~t some of our children are still enrolled in high schools, and in order to avoid their undue discrimination or harassment, we would request the Inspection Panel and the World Bank to keep our data in strict confidentiality. We hereby authorize above organizations to apply to the World Bank Inspection Panel on our behalf. In consultation with us, the above organizations will embark on compiling and submitting the Claims to the WB and Inspection Panel. Respectfully, Claimants: ANNEX II MANAGEMENT RESPONSE TO REQUEST FOR INSPECTION PANEL REVIEW OF THE ARMENIA SECOND EDUCATION QUALITY AND RELEVANCE PROJECT (P107772) AND ARMENIA EDUCATION IMPROVEMENT PROJECT (P130182) Management has reviewed the Request for Inspection of the Armenia Second Education Quality and Relevance Project (Pl 07772) and Armenia Education Improvement Project (P130182), received by the Inspection Panel on May 16, 2014 and registered on June 5, 2014 (RQ14/03). Management has prepared the following response. July 7, 2014 CONTENTS Abbreviations and Acronyms ......................................................................................... iv Executive Summary .......................................................................................................... v I. INTRODUCTION .................................................................................................... 7 II. THE REQUEST ....................................................................................................... 7 III. BACKGROUND OF THE PROJECTS ................................................................ 8 IV. MANAGEMENT'S RESPONSE .......................................................................... 10 Annexes Annex 1. Claims and Responses Annex 2. Lessons from Previous Bank Funded Projects in Armenia Considered in the Design of the EIP Annex 3. Consultations and Due Diligence Ill ABBREV IA TlONS AND ACRONYMS ANQA Armenian National Center for Professional Education Quality Assurance APL Adaptable Program Loan (used also to refer to Adaptable Program Credit) ATC Assessment and Testing Center BP Bank Procedures CEU Central European University CIF Competitive Innovation Fund EC European Commission EIP Education Improvement Project EMIS Education Management Information System EMP Environmental Management Plan ENQA European Association for Quality Assurance EQRP2 Second Education Quality and Relevance Project ESMF Environmental and Social Management Framework EU European Union FY Fiscal Year GDP Gross domestic product GIZ Deutsche Gesellschaft fur Internationale Zusarnmenarbeit HEI Higher Education Institution ICR Implementation Completion and Results Report ICT Information and Communication Technologies IDA International Development Association IQA Internal Quality Assurance Mo ES Ministry of Education and Science MoF Miriistry of Finance NIE National Institute of Education OP Operational Policy OSF Open Society Foundations PIU Project Implementation Unit POM Project Operational Manual QA Quality Assurance READ Russian Education Aid for Development SABER System Assessment and Benchmarking for Education Re- sults TEMIS Tertiary Education Management Information System TIMSS Trends in International Mathematics and Science Study UNICEF United Nations Children's Fund US AID United States Agency for International Development IV EXECUTIVE SUMMARY The Projects 1. The Education Improvement Project (EIP). The EIP consists of a loan of US$15 million from IBRD and a credit of US$15 million from IDA. The objectives of the EIP are to improve school readiness of children entering primary education, improve physical conditions and the availability of educational resources in upper secondary schools, and support improved quality and relevance in higher education institutions in Armenia. 2. The Second Education Quality and Relevance Project, Adaptable Program Loan (APL) Phase 2 (EQRP2). The EQRP2 consists of a credit of US$25 million from IDA. The objectives of the EQRP2 are to: (i) enhance school learning in general educa- tion and improve the school readiness of children entering primary education; and (ii) support the integration of the Armenian Tertiary Education system into the European Higher Education Area. The Request for Inspection 3. On June 5, 2014, the Inspection Panel registered a Request for Inspection, IPN Request RQ 14/03, concerning the EQRP2 and the EIP. The Request for Inspection was submitted by nine NGOs, two students and five parents from Armenia who requested to keep their identities confidential. 4. The Request alleges that the two Bank financed operations fail to address what the Requesters view as the key issues in the higher education system in Armenia, includ- ing: (i) poor governance and corruption in the Armenian National Center for Professional Education Quality Assurance (ANQA); (ii) inadequate financing; (iii) overly-tight politi- cal control; and (iv) lack of academic freedom. The Request further alleges, similarly, that the two Bank operations fail to address systemic issues in secondary education, in- cluding, (i) political control of the sector by the government; (ii) weak student evaluation; (iii) putting rural students at a disadvantage; (iv) ineffective teacher training; (v) lack of transparency in curricular and syllabus design, leading to gender bias and discriminatory behavior; and (vi) discrimination against religious minorities. The Request also alleges that there was a lack of consultations during project preparation as well as inadequate in- corporation of lessons learned from the previous education project~. As a result of this, the Requesters claim that the projects "pose[s} a threat of greater harm to the education system of the whole country and hence each and every one of us." Management's Response 5. Management does not believe that the Request meets the eligibility criteria. In Management's view, the Requesters are not able to demonstrate that their rights or inter- ests have been or are likely to be adversely affected by the projects as required by the Panel Resolution. Furthermore, Management disagrees that the alleged adverse impacts cited in the Request result from the two Bank-supported projects. v 6. Management notes that the Request/or Inspection is primarily about broad is- sues in the education system in Armenia and that the description of the alleged poten- tial harm consists of existing shortcomings to the system in general, neither specific to nor resulting from the two Bank-financed projects. In Management's view the claims of harm presented in the Request for Inspection: (1) do not relate to the Bank-supported op- erations, but rather to (i) existing conditions in the Armenian education system (quality of higher and secondary education, quality of teachers, urban/rural distribution of schools, quality of text books), and (ii) issues around government or church influence in the edu- cation sector; and (2) are generally based on the assumption that Bank support granted under an operation amounts to the Bank' s endorsement of and responsibility for every issue in the respective sector. The Requesters also suggest that sector issues that are not addressed by the Bank-supported projects would constitute harm caused by the projects. 7. In Management's view the Request/or Inspection is about the Requesters' de- sign preferences for the projects as well as a view that Bank support should be offered only when the education sector is performing well In the Requesters' view, the projects cannot achieve adequate progress under the current situation, and the Bank therefore should not support the education sector in Armenia but rather exit and create precondi- tions for reengagement. The Requesters also present a list of suggested actions for the Bank to take in this regard. These are presented in Annex 1 along with Management's responses. 8. The Requesters appear to misinterpret harm as any pre-existing conditions which they view as undesirable and which are not addressed by the Bank supported projects, and which they believe should be. This, however, does not relate to any "acts or omissions " by the Bank in relation to Bank policies or procedures, and does not con- stitute the harm which is referred to in the Panel Resolution. Hence, in Management's view the harm alleged by the Requesters represents what they view as non- accomplishments of the projects and must be viewed as unfulfilled expectations that do not generate a material deterioration. This cannot "be considered as a material adverse effect " for the purpose of the Panel Resolution. 9. Notwithstanding the above considerations Management welcomes this oppor- tunity to further clarify its support to the Armenian education sector and progress of achievements to date. VI EQRP 2 and EIP I. INTRODUCTION 1. On June 5, 2014, the Inspection Panel registered a Request for Inspection, IPN Request RQ 14/03 (hereafter referred to as "the Request"), concerning the Armenia Sec- ond Education Quality and Relevance Project (EQRP2, PI 07772) financed by the Interna- tional Development Association (IDA) and the Armenia Education Improvement Project (EIP, Pl30182), financed by both IDA and the International Bank for Reconstruction and Development (the Bank). 2. Structure of the Text The document contains the following sections: Section II summarizes the Request; Section III provides background on the two Projects; and Sec- tion IV contains Management's response. Annex 1 presents the Requesters' claims, to- gether with Management's detailed responses, in table format. Annex 2 discusses lessons learned from previous Bank-funded projects in Armenia that were considered in the de- sign of the EIP and Annex 3 provides information on the consultations and due diligence undertaken by the Bank for the two projects. II. THE REQUEST 3. The Request for Inspection was submitted by nine NGOs, two students and five parents from Armenia who requested to keep their identities confidential, hereafter re- ferred to as the "Requesters." 4. The Request alleges that the two Bank financed operations fail to address what the Requesters view as the key issues in the higher education system in Armenia, includ- ing: (i) poor governance and corruption in the Armenian National Center for Professional Education Quality Assurance (ANQA); (ii) inadequate financing; (iii) overly-tight politi- cal control; and (iv) lack of academic freedom. The Request further alleges, similarly, that the two Bank operations fail to address systemic issues in secondary education, in- cluding, (i) political control of the sector by the government; (ii) weak student evaluation; (iii) putting rural students at a disadvantage; (iv) ineffective teacher training; (v) lack of transparency in curricular and syllabus design, leading to gender bias and discriminatory behavior; and (vi) discrimination against religious minorities. The Request also alleges that there was a lack of consultations during project preparation as well as inadequate in- corporation of lessons learned from the previous education projects. As a result of this, the Requesters claim that the EIP "pose[s} a threat of greater harm to the education sys- tem of the whole country and hence each and everyone of us. " 5. Attached to the Request are three testimonials by individuals and a parental au- thorization letter. No further materials were received by Management in support of the Request. 6. The Request contains claims that the Panel has noted may relate to the following Bank Operational Policies (OP) and Bank Procedures (BP): 7 Armenia • OP/BP 4.01, Environmental Assessment; • OP/BP 10.00, Investment Lending; • OP/BP 10.00, April 2013 , Investment Project Financing; and • OP/BP 13.05, Project Supervision. III. BACKGROUND OF THE PROJECTS The Education Improvement Project (EIP) - P130182 7. The Project. The EIP consists of a loan of US$15 million from IBRD and a credit of US$15 million from IDA, and government counterpart funding in the amount of US$7.5 million. It was approved by the Board on March 13, 2014 and is not yet effective. Its closing date is September 30, 2019. 8. Project Objectives. The objectives of the EIP are to improve school readiness of children entering primary education, improve physical conditions and the availability of educational resources in upper secondary schools, and support improved quality and rele- vance in higher education institutions in Armenia. 9. Project Components. • Component 1: Enhancing the Quality of General Education (total estimated cost: US$27 .50 million), which consists of the following sub-components: (1.1) Pro- moting school readiness and equal opportunities at the start of general education; (1.2) Enrichment of upper secondary schools; (1.3) Improving data-collection and monitoring of the education system performance; and (1.4) Supporting further improvements in the quality of education through curriculum revisions. • Component 2: Mainstreaming of the Competitive Innovation Fund (CIF) for Higher Education Institutions (HEI) into full implementation (total estimated cost: US$6.25 million). • Component 3: Project Management, Monitoring and Evaluation (US$3 .75 mil- lion). The Second Education Quality and Relevance Project, Adaptable Program Loan (APL) 1 Phase 2 (EQRP2) - P107772 10. The Project. The EQRP2 consists of a credit of US$25 million from IDA and government counterpart funding in the amount of US$6.25 million. The project was ap- proved by the Board on May 12, 2009 and closes on November 30, 2014. A total of US$19 million has been disbursed to date. 1 APL acronym also used to refer to this Adaptable Program Credit. 8 EQRP 2 and EIP 11. Project Objectives. The objectives of the EQRP2 are to: (i) enhance school learn- ing in general education and improve the school readiness of children entering primary education; and (ii) support the integration of the Armenian Tertiary Education system in- to the European Higher Education Area. 12. Project Components. • Component 1: Enhancing the Quality of General Education (total estimated cost: US$19.66 million equivalent), which consists of the following sub-components: ( 1.1) Promoting school readiness and equal opportunities at the start of general education; (1.2) Supporting further improvements in the quality of education through improved in-service teacher training and professional development; (1 .3) Continuing support to the integration of Information and Communication Tech- nologies (ICT) in the teaching and learning process; and (1.4) Supporting the im- plementation of high-school reform. • Component 2: Supporting Tertiary Education Reforms in the Context of the Bo- logna Agenda (US$7.53 million equivalent), which consists of the following sub- components: (2.1) Establishment and strengthening of the national quality assur- ance system; (2.2) Developing a Tertiary Education Management Information System (TEMIS); (2.3) Strengthening the capacity to implement a sustainable fi- nancing system; and (2.4) Reforming pre-service teacher education. • Component 3: Project Management, Monitoring and Evaluation (US$4.06 million of which US$3.25 million in IDA financing). 13. The EQRP2 has achieved important results by supporting the implementation of reforms that are expected to contribute to improved student learning in both general and higher education. The reforms supported include: (i) preschool expansion through im- plementation of micro-projects to enhance existing preschools or through establishment of preschool services in the poorest and most vulnerable communities, benefitting about 5,000 children; (ii) enhancing the quality of education by providing targeted in-service trainings to about 12,000 general education teachers; (iii) providing internet connectivity to all schools in Armenia and implementing the Education Management Information Sys- tem (EMIS); (iv) supporting high school reform by establishing and equipping resource centers in all of them; (v) providing technical assistance for the establishment of a quality assurance system led by the ANQA; and (vi) development and piloting of a CIF for HEis. The project is currently rated as Moderately Satisfactory for its development objectives. Two out of the three outcome indicafors present a positive trend (improved Early Devel- opment Index of students in preschools and kindergartens receiving grants; and standings of Armenia in the Bologna Scorecard in Higher Education), while the third indicator (per- formance of Armenian students in Trends in International Mathematics and Science Study, TIMSS 2011) .will need to be revised as the field tests for TIMSS were carried out 6 months after the project became effective. All project activities are on track towards achieving the end of program targets. 9 Armenia 14. Preparation of both projects met the requirements of OP/BP 10.0 and OP/BP 4.01. Subsequent supervision of EQRP2 met the requirements of OP/BP 13.05. Notably, les- sons learned during the implementation of the first Education Quality and Relevance Pro- ject (P074503) and the ongoing EQRP2 shaped the design of the EIP (see details in An- nex 2). The Implementation Completion and Results (ICR #00001318) report of the first project highlighted the importance of: (i) conducting open discussions and achieving pub- lic consensus; (ii) using good analytical work to inform project preparation; (iii) intensive and continuous guidance and supervision; and (iv) having a sound and independent moni- toring and evaluation mechanism in place. The EIP' s preparation involved consultations with several education sector and civil society stakeholders (as detailed in Annex 3) and included analytical work covering the whole spectrum of the education system, from pre- school through higher education. The EQRP2 included rigorous evaluation studies for the project's supported preschools and for the in-service teacher training activities. The Bank team also maintained a close dialogue with the government and undertook several super- vision missions (as detailed in Annex 3). Lessons learned during the implementation of the EQRP2 as well as international evidence and best practice were also considered dur- ing the preparation of the EIP, in particular, for the support of the preschool subcompo- nent, for the EMIS to increase accountability, and for the mainstreaming of the CIF as described in Annex 2. IV. MANAGEMENT'S RESPONSE 15. The Requesters' claims, accompanied by Management' s detailed responses, are provided in Annex 1. Eligibility Considerations 16. Management does not believe the Request meets the eligibility criteria. In Man- agement's view, the Requesters are not able to demonstrate that their rights or interests have been or are likely to be adversely affected by the projects as required by the Panel Resolution. Management disagrees that the alleged adverse impacts cited in the Re- quest result from the two Bank-supported projects. 17. Management notes that the Request/or Inspection is primarily about broad is- sues of the education system in Armenia and that the description of the alleged poten- tial harm consists of existing shortcomings to the system in general, neither specific to nor resulting from the two Bank-financed projects. In Management's view the claims of harm presented in the Request for Inspection: (i) do not relate to the Bank-supported op- erations, but rather to existing conditions of the Armenian education system (quality of higher and secondary education, quality of teachers, urban/rural distribution of schools, quality of text books), and to issues around government or church influence in the educa- tion sector; and (ii) are generally based on the assumption that Bank support granted un- . der an operation amounts to the Bank' s endorsement of and responsibility for every issue in the respective sector.. The Requesters also suggest that sector issues that are not ad- dressed by the Bank-supported project would constitute harm caused by the project. 10 EQRP 2 and EIP 18. Management also notes that the individual testimonies (referred to as "case studies'? presented in the Request, are not verifiable, nor do they offer indication or evidence that the alleged incidents stem from the support granted under the two pro- jects, or are related to the projects in any way. These testimonies present personal expe- riences of a few individuals, while lacking evidence that such experiences are occurring in a systemic manner. Further, it is not clear how the two Bank-supported operations could be related to: (i) the individual decisions of professors or faculties regarding the acceptance or refusal of a proposed Ph.D. thesis topic; (ii) the acceptance of a Ph.D. the- sis; or (iii) the dismissal of a Rector. 19. The Request/or Inspection is about the Requesters' design preferences/or the projects as well as a view that Bank support should be offered only when the education sector is performing well. Much of the Request is about the Requesters' views on when and how the education system in Armenia should be supported by the Bank. In the Re- questers' view, the projects cannot achieve adequate progress under the current situation, and the Bank therefore should not support Armenia but rather exit and create precondi- tions for reengagement. The Requesters also present a list of suggested actions for the Bank to take in this regard. These are presented in Annex 1 along with Management's responses. 20. The Requesters appear to define harm as any pre-existing conditions which they view as undesirable and which are not addressed by the Bank supported projects, but which they believe should be. This, however, does not relate to any "acts or omis- sions" by the Bank in relation to Bank policies or procedures, and does not constitute the harm which is referred to in the Panel Resolution. Hence, in Management's view, the harm alleged by the Requesters represents what they view as non-accomplishments of the projects and must be viewed as up.fulfilled expectations that do not generate a µiaterial deterioration.2 This cannot "be considered as a material adverse effect" for the purpose of the Panel Resolution.3 21. Notwithstanding concerns regarding the eligibility of this Request, Manage- ment agrees that some of the points made by the Requesters are valid and notes that several interventions under the projects support the government precisely in the direc- tion of the wide reforms that are needed to address these points. However, the Bank' s global experience shows that educational improvement is gradual and takes time. Hence, based on the Bank's mandate and experience, its support is most effective when applied based on a joint understanding with the Borrower of the required interventions that lead to improvement. Evidence shows that engaging before the circumstances are ideal allows for progress toward the goals of improving quality and relevance of education. Waiting until all issues are addressed precludes the opportlinity to support progress and early in- tervention can improve the pace and quality of their progress. 2 In the Requesters ' own words, in their view the previous education projects' "objectives [.. ] had not been achieved" and "concrete deliverables were not attained;" and that without changes the new project will not "contribute to Armenian Universities' research capabilities or their sustainability[ .. ]." 3 1999 Clarification of the Inspection Panel Resolution, paragraph 14. 11 Armenia 22. Likewise, Management welcomes this opportunity to continue to clarify the is- sues and questions raised by the Requesters. The Bank has met with various stakehold- ers over the past few years, including civil society, where issues of concern have been raised, and has shared many documents. In addition to dissemination of Bank reports on the education sector and engagement with stakeholders during preparation and implemen- tation visits, consultations were carried out in the context of the EIP preparation during 2013. As detailed in Annex 3, consultations were undertaken for: (i) the school readiness enhancement subcomponent; (ii) assessment of the current situation of high school re- forms in Armenia; (iii) the environmental and social management framework for the in- frastructure subcomponent; and (iv) the evaluation of the CIF in its pilot phase for the higher education component. Throughout the concept and preparation stages of the pro- jects, the Bank has continued to provide many opportunities for in-depth discussions with civil society to take into account their suggestions and feedback in the design process. Management's Response to Specific Allegations Raised in the Request 23. The Requesters allege lhat the designs of the· two projects do not resolve system- ic and far-reaching failures of the Armenian education system and request the suspen- sion and revision of the projects. Management disagrees with these allegations and points out that the Bank' s global experience shows that the improvement of education systems is gradual and demands time. Reforms usually meet resistance and controversy, face systemic barriers, and therefore require a constant process of evaluation and feed- back for proper decision-making. Finland and Poland are two examples where efforts to improve education took time before initial results became evident. Indeed, Finland has systematically worked for over 40 years to make its once poorly ranked (1970) educa- tional system one of the leaders in student achievement. Starting in 1989, Poland de- signed a reform, ultimately introduced in 1998, that only began to show overall im- provements in educational performance by 2006, as measured by results in the Program for International Student Assessment (PISA) tests. 24. The Bank has supported the gradual reforms taking place in the education sec- tor in Armenia through education projects, policy dialogue and knowledge exchange. The EQRP2 and EIP build on the reform efforts and achievements made by Armenia to date and contribute to furthering those efforts. For example, in preschool education, the EQRP2 has contributed to the increase in preschool enrollment of 5-6 year old children, · from less than 30 percent in 2007 to 65 percent in the school year 2012-2013, thus im- proving school readiness of Armenian first graders. However, this is still far from reach- ing the government' s objective of 90 percent preschool enrollment by 2017. In general education, the projects have supported and will continue supporting the implementation of the 12-year general education system, including the curriculum revisions, teacher training, and student learning assessment aligned to the new 12-year system; the intro- duction of high schools offering specialized streams to better fit students' academic needs; increased account~bility of and information on the education system to citizens by improving data collection through systems for monitoring education system performance; and by providing continued support for the regular participation of Armenia in interna- tional large-scale student assessments (TIMSS). It is clear, however, that quality of edu- cation remains a challenge, as highlighted by the results in international assessments and 12 EQRP 2 and EIP also by the perception of stakeholders in Armenia. In tertiary education, the projects have contributed to the increased integration of Armenia into the European Higher Edu- cation Area, by supporting the establishment of a quality monitory agency (ANQA) and by contributing to the diversification of sources of funding for innovation for both public and private universities. Important challenges remain in the area of governance of higher education, but the direction of the reforms is encouraging and the support of the Bank to move in such a direction is opportune. 25. In Management's view, waiting until all education sector issues are addressed, as suggested by the Requesters, precludes the opportunity to support progress and in- fluence the pace and quality of that progress. For the reasons offered above, the Bank's support through the EQRP2 and EIP and its policy dialogue with the government are of strategic importance in promoting a gradual improvement process that is taking place in the education sector in Armeriia. The Bank has been informed that the government re- mains committed to continue with the long-term reform agenda, which was recently con- firmed by the content of the Program for the Government of the Republic of Armenia submitted to Parliament on May 19, 2014. 26. On the allegations regarding the Higher Education System in Armenia (i) Governance issues and corruption in the ANQA: The Requesters allege that "there are serious irregularities in ANQA mission, composition and func- tion, " and thus its independence is questioned. In addition, the Requesters allege that quality assurance reforms under the project have been ''fully controlled by the government and ruling political party with no dialogue or consultation " with relevant stakeholders therefore, "met with distrust, if not open opposition, " and eventually resulting in ''poor quality reform of higher education in Armenia. " The Requesters state that the Bank ignored its "own assessment of the tertiary sector governance issues and irregularities " and failed to make changes. Management disagrees with the Requesters' claims that the Bank failed to ad- dress these problems with the Borrower to resolve them. The Bank has produced a report titled, "Addressing Governance at the Center of Higher Education Re- form in Armenia" (January 2013) that has been widely consulted and disseminat- ed in-country. This report has served as the basis for the policy dialogue with the government and other stakeholders on higher education. The report indicates that the ANQA, which was established to strengthen the national quality assurance system in Armenia, has made important achievements in a short period of time, but there are further steps to take in order to become an internationally credible quality assurance agency. In order to meet international standards, ANQA needs to address deficiencies in its governance structure through legislative action that would allow for needed changes to the composition of its Board of Trustees. It should be noted that the Program for the Government of the Republic of Armenia dated May 19, 2014 and presented for the approval of the National Assembly, states the government's commitment towards international recognition of qualifi- cations for joining the "European Quality Assurance Register for Higher Educa- tion" through full membership of the ANQA in the European Association for 13 Armenia Quality Assurance in Higher Education (ENQA). In disagreement with the Re- questers, Management wishes to clarify that the Bank' s diagnostic and assessment work of the ANQA did not identify issues of corr\lption arising from the way it is currently organized (see Item 2 in Annex 1 for further details). (ii) Inadequate public financing: The Requesters allege that successful reform of existing financing mechanisms, which the government has already committed to implement, will require not only effective allocation of available funding, but a substantial increase in total public funding for higher education. The Requesters assert that the CIF cannot assist in "ensuring sustainable funding system " as it does not address the declining public spending or legal framework regulating the finances of universities. Management agrees with the Requesters on the importance of sufficient public funding for the education sector but the projects are not responsible for the government's decisions on financing for the sector. Even though the level of public financing for education is not the Bank's decision, the need to increase public funding has been regularly brought to the attention of the Ministry of Edu- cation and Science (MoES) and ·the Ministry of Finance (MoF). The 2013 higher education report indicates that public funding for higher education is limited and that increasing and diversifying higher education financing is needed. The 2011 "Public Fiscal Consolidation and Recovery in Armenia" report by the Bank makes a specific recommendation to increase spending in education, in particular, to increase non-salary recurrent spending to improve quality. The CJF, piloted in the EQRP2 and mainstreamed in the EIP, and available for public and private universities, aims to increase the diversification of sources of funding for higher education, through a competitive and transparent process of allocating funds. The Bank's experience of supporting these mechanisms in higher education in several projects highlights the value of CIFs in establishing diversified funding mechanisms for higher education that are demand-driven and have transparent procedures, as opposed to designating resources for narrowly de- fined purposes and in a top-down fashion. It also highlights the flexibility of CIFs to respond, with transparent procedures, to specific sector development needs and to reorient resources from investments in teaching facilities to improvement of curricula and the learning process. (See Items 6 and 9 in Annex 1 for more details and lessons learned on CIF in Annex 2). (iii) Alleged overly-tight political control: According to the Requesters, "the governance boards of state universities, state education agencies are represented by government officials, MPs and by the administration of the President at around 50%, " which allows for "interventions by the state over their finances, management appointments, specialization, and admissions. " Management partly concurs with the Requesters' assessment of the governance challenges facing higher education, as identified by the Bank in its engagement 14 EQRP 2 and EIP with Armenia, and those are being tackled through Bank financed projects and education sector dialogue (see Item 4 in Annex 1 for more details). However, contrary to the Requesters' statement, the Bank's 2013 report did not find evi- dence that "further highlights systemic problems pertaining to . .. widespread cor- ruption and nepotism in the field. " Management notes that the government is cur- rently working on the legal framework for higher education. The Program of the Government of the Republic of Armenia dated May 19, 2014 and presented for approval of the National Assembly, calls for increased autonomy of HEis and for greater transparency and accountability in the system, while continuing to imple- ment the integration of Armenia into the European Higher Education Area. (iv) Alleged lack of academic freedom: The Requesters state that the alleged overly-tight political control "undermines academic independence vital for credi- bility and sustainability of any research and scientific endeavor. " The projects supported by the Bank do not include components dealing with ac- ademic freedom, nor does the Bank have evidence that lack of academic free- dom is a pervasive issue in higher education in Armenia. Management' s view is that the universities ' self-evaluations, promoted by the internal quality assurance units supported by the EQRP2, are useful mechanisms to monitor progress on the desired changes within universities. Management also notes that the supporting testimonies provided by the Requesters do not offer any indication or evidence that the alleged incidents stem from or are aggravated by the support granted by the Bank under the two projects (see Item 27 in Annex 1 for details). 27. On the allegations regarding Secondary Education (i) Alleged political control of the sector by the government: The· Requesters allege that "[t}he government and the ruling party (Armenian Republican Party) assert political control of the sector through control over the appointment of school principals, the composition of school governing bodies and financing mechanisms. Secondary educational institutions and their employees serve as an important administrative resource that is systematically and vastly abused for po- litical ends during elections. " These claims are not related to the projects in any way. Management notes that the MoES exercises its oversight role in managing the country's public schools. In the absence of concrete evidence it is difficult to verify the Requesters ' claims that " ... the ruling party (Armenian Republican Party) asserts political control over secondary education" and that "secondary educational institutions and their employees serve as an important administrative resource that is systematically and vastly abused for political ends during elections." Similar to many education sys- tems around the world, the MoES appoints school principals, determines the composition of school governing bodies and defines financing mechanisms (see Item 4 in Annex 1 for details). 15 Armenia (ii) Weak student evaluation: The Requesters allege that "[t]he system of stu- dent aptitude evaluation is hugely problematic in principle and provides conflict- ing or even mutually exclusive results for student learning and skills evaluation. It does not enjoy broad support and trust among either the public or professional educators, a fact amply demonstrated by expert interviews and research. " Management concurs that there is significant room for improvement in student evaluations in Armenia, while also recognizing that the government is making strides toward improving those evaluations with the help of the Bank and other donors. In 2011, Armenia conducted a benchmarking exercise on student assess- ment in its different dimensions through the Bank's System Assessment and Benchmarking for Education Results (SABER) initiative, which helps countries examine and strengthen the performance of their education systems to achieve learning for all. The benchmarking exercise identified possible next steps to fur- ther develop the different types of assessment activities (classroom assessment, national large seal~ assessments, international large scale assessments and exami- nations). The Bank, through projects, trust funds and policy dialogue, is providing support for gradual improvements in -these areas, including the introduction of new forms of assessment of the learning process and of learners' achievements, in compliance with international standards, as detailed in Annex 1, Item 17. With the support of an earlier Bank-financed project, Armenia established the Assessment and Testing Center (ATC) and has participated in all rolinds of TIMSS since 2003. Additionally, national student assessments are being carried out by the ATC, albeit not regularly. Finally, under the Russian Education Aid for Develop- ment (READ) Trust Fund for Armenia, in 2013-2014 a classroom assessment course was developed for tertiary level students preparing to join the teaching pro- fession and a cadre of instructors from various universities has been trained to de- liver this course. (iii) Putting rural students at a disadvantage: According to the Requesters, "[h]igh schools are largely inaccessible for students from rural areas. Hundred and two out of hundred and nine high schools are located in cities and only seven in rural areas In rural areas there are still schools that work based on 12- year curriculum which is not designed and does not provide specialization. [. ..] Given higher rates ofpoverty and vulnerability in rural areas, as well as professionally and technically lower capacities of these schools compared to urban ones, the disadvantage is much greater and the risks for further limiting access to higher education is growing higher. " Management is aware that general education schools in Armenia have severe infrastructure and educational equipment needs, both in rural and urban areas, and Bank engagement has supported efforts to address those needs. While the infrastructure component of the EIP focuses on high schools, which are primarily located in urban areas, the Bank' s portfolio in Armenia is broad and covers schools all over the country, including important support for infrastructure and equipment needs lic funding for higher education. In that all, of which 0.3 percent of GDP is invested in higher case reforms can be expected to have a measurable education. Both amounts are very low compared to impact on the quality and competitiveness of higher many other countries. Management agrees that the education. This will only be possible when there is a Government of Armenia should improve the allocation broad public consensus on the direction of the re- of financial resources to the education sector. This forms and trust in HE governance. conclusion has been highlighted in the Bank' s regular Public Expenditure Reviews and has been regularly brought to the attention of high-level officials in the MoES and MoF. The Bank's 2013 report also notes the need for in- creased and diversified financing for higher education. The CIF, piloted in the EQRP2 and mainstreamed in the EIP, aims to increase the diversification of sources of funding for higher education, through a competitive and transparent process of allocating funds. 7. Enhancing General Education Component with The Bank is aware that schools based on 12-year cur- harms and failures for bank procedures riculum in rural areas have severe infrastructure and Component 1 Enhancing the Quality of General Ed- educational equipment needs. The Bank's portfolio in ucation Armenia is broad and covers schools in rural areas through another project. Subcomponent 4. Supporting the Implementation of High School Reform. The Bank has already financed the rehabilitation or construction of 120 schools 9 and provided 57,508 units Harms : of school furniture (desks, chairs, bookcases, black- High schools are largely inaccessible for students boards and teachers' desks, etc.) worth US$6.12 mil- from rural areas. Hundred and two out of hundred lion for 812 educational facilities in poorer communi- and nine high schools are located in cities and only ties in Armenia. This financing has focused primarily seven - in rural areas. [ ... ] In rural areas there are on rural and underserved areas, and has excluded from still schools that work based on 12- year curriculum coverage the city of Yerevan and the Marz (province) which is not designed and does not provide speciali- centers, where most of the high schools are located. zation. [ ... ]Given higher rates of poverty and vu!- The EIP seeks to improve the infrastructure and safety nerability in rural areas, as well as professionally and conditions of 17 urban high schools and improve edu- technically lower capacities of these schools com- cational equipment in all of them. pared to urban ones, the disadvantage is much great- er and the risks for further limiting access to higher Students in rural schools have and will benefit from the education is growing higher. system-wide activities supported under the EQRP2 and EIP, such as in-service teacher training, connectivity to internet for all schools, curriculum revisions and im- provements, and student assessment activities. 8 References on CIF in higher education: Saint, W. 2006. Innovation Funds for Higher Education: A Users' guide for World Bank Funded Projects. The World Bank; Fehnel, R. 2004. Higher Education Reforms and Demand Responsive Innovation Funds: Dimensions of Difference. The World Bank. 9 Armenia has about 1,390 public schools. 26 EQRP 2 and EIP No. Claim Response 8. Parents' letter: I. High schools, where our children Management recognizes that there are still several study/ studied do/did not ensure provision of quality challenges for improving the quality of general edu- education services. All of us have had to either apply cation in Armenia. The projects seek to address many for additional tutoring or transfer children to private of those challenges. schools to get better quality education and prepared- The EQRP2 and EIP aim to raise learning outcomes ness to enter universities. We believe that our chi!- through their different activities. However, private dren would have never gotten quality education and tutoring is not expected to disappear, regardless of im- enter the university if they had stayed at the state provements in quality, as it is caused by a high-stakes high schools. university entrance exam - the Armenia Unified En- trance Exam - which is used for university admission. The widespread use of tutoring in addition to public education is not a phenomenon that is particular to the Armenian context. Tutoring coexists with some of the best public education systems in the world (e.g., Korea and Japan). In fact, tutoring is usually prevalent in up- per secondary education in countries that have compet- itive, high-stakes exams for university admission, which is the case in Armenia. 9. The existing per-capita financing mechanism does Management agrees with the Requesters on the im- not ensure allocation of the relevant financial re- portance of sufficient funding for the sector. This sources for provision of quality streaming education message has been regularly brought to the attention in high school. Students and parents still have to take of the government The 2011 "Fiscal Consolidation private tutor classes for pteparation to the final uni- and Recovery in Armenia " report by the Bank makes a tied/admission exams. specific recommendation to increase spending in edu- cation, in particular, to increase non-salary recurrent spending to improve quality. At 2.6 percent of GDP, public expenditures on educa- tion in Armenia are very low compared to most other countries. As a consequence, primary and secondary education "levels are underfunded and the Bank has brought this to the attention of high-level officials in the MoES and MoF. See response in Item 6 above, which addresses a similar claim for the case of tertiary education. 10. Parents' letter 3 and 5. High schools do/did not pro- The poor infrastructure and lack of availability of vide for the quality streaming education that would educational equipment at high schools is a concern ensure professional orientation for our children. High shared by parents, the MoES and the Bank. The EIP schools are not equipped with the necessary technical focuses on improving infrastructure and the availabil- capacities and do not have proper equipped laborato- ity of educational equipment for student safety and ries and libraries that would provide for the quality improved learning conditions. education. Computer classes are not fully accessible With the support of the EIP, major rehabilitation works for children. Teachers do not use the computers dur- will be conducted in 17 high schools - selected on the ing the class. We faced the situations when teachers basis of objective criteria, see Item 19 - which do not proposed us out-of-class private tutoring of our chil- meet Armenia's construction and safety standards (in- dren, which in fact was the compensation for the eluding seismic stability). The project will also support under-taught content in the classroom. the provision of contextualized digital learning materi- als, modem equipment and school furniture to all high schools in Armenia to enrich the learning environment. Teachers' proposals to provide out-of-class private tutoring to compensate for low quality classes are un- acceptable and should be reported throu!!h the aooro- 27 Armenia No. Claim Response priate channels that the MoES has available for these cases. 11. Content analysis of school textbooks shows that The Bank has not provided support or advice for the asymmetric representation of gender roles prevails. development of textbooks or the financing of text- [ ... ] Discriminatory norms and perceptions are wide- books. Hence, neither of the Bank supported projects is ly promoted both in textbooks, teaching l'rocess, and responsible for the cited contents in textbooks. teachers' attitude. [ . .. ] This is justified by further However, the Bank is strongly committed to gender professional specialization for males and females, equality and gender inclusion, and believes that pro- which does not provide a comprehensive develop- gress toward gender equality is a prerequisite to end- ment of individuals as specified in the standard and ing extreme poverty and boosting shared prosperity goal of the subject program. (http://www.worldbank.org/en/ topic/gender). Man- agement strongly supports gender sensitive education in all of its dimensions, and intends to further review and verify the Requesters' claim to ascertain if and where, and to what extent, gender discriminatory con- tent exists in school textbooks. If confirmed, the issue will be raised with the Borrower in the context of on- going policy dialogue, including implementation sup- port visits for the projects. With regard to other aspects ofreform related to quali- ty, the EQRP2 invested in the establishment ofre- source centers (libraries, computer labs, books and teaching materials) in each high school as well as train- ing of librarians and teachers. The EIP is designed to support improvements in teaching and learning condi- tions through: (i) further curriculum revisions in gen- eral education by investing in technical assistance (working groups composed of subject experts, school teachers, university professors, and experts from the NIE) and consultative workshops; it is envisaged that these consultative workshops will enable collection of direct feedback from all key stakeholders throughout the entire period of project implementation; and (ii) provision of contextualized digital learning materials, modem equipment and school furniture to be used in classrooms in all high schools in Armenia, including training of teachers on using modem teaching method- ologies and digital learning materials in the classrooms. 12. Parents' letter 4. The textbooks and teaching See response in Item 11. above. are/were of low quality. Some of textbooks, like So- Management acknowledges that this is a valid and cial Science textbooks for 9 to I I grades, include important issue. As indicated above, the Bank is discriminative norms and are gender insensitive .. For strongly committed to gender equality and gender in- example, in the lOth grade textbook the hypothesis of clusion, and stands ready to further engage on this is- psychologist U.F.Harley is given according to which sue in its dialogue with the MoES. In addition, the en- there are five basic needs of men and women, ful- visaged curriculum revisions to be supported under the fillment of which guarantees stability of marriage, EIP provide a valuable opportunity to revisit textbook while dissatisfaction of those needs may lead to the content issues, as textbooks will need to be aligned conflict and even to divorce. The needs of a man are with the revised curriculum. Moreover, as detailed in in sexual satisfaction, rest companion, a chamiing its Project Appraisal Document, the EIP will promote woman, household management and admiration. For gender sensitivity and prioritize the inclusion of worn- a woman the needs are expressed in tenderness, con- en, the poor, the disabled and other vulnerable groups versation, honesty and frankness, financial support across all project activities, includin~ project consulta- 28 EQRP 2 and EIP No. Claim Response and devotion to family. The textbook and teaching tions and other mechanisms for stakeholder engage- process do not provide children with opportunity to ment. Direct consultation with beneficiaries, their rep- discuss and argue these statements. Hence there is resentatives and other stakeholders throughout project real risk that children may accept them as norm and implementation will provide an opportunity to engage apply later in life. Another example from the same in discussions on gender issues and gender biases or textbook says that "the societies, such as western barriers. These consultations will also provide an op- ones, where women are fully involved in social life, portunity for any group or individual to voice their tum to have a significant decrease in birth rate". concerns on textbooks and on aligning textbooks to the Children can take the statement as granted and think revised curriculum. Finally, the EIP will ensure that it that the only mission of women is to ensure popula- monitors its gender impact, and the project results tion growth, thus making her reproductive function a framework includes gender-disaggregated indicators on priority in contrast to women's wish to become a school readiness, preschool enrollment and high school fully integrated member of society. attendance. 13. The failures of previous standard development have See response in Items 11 and 12 above. already resulted in gender biased content of educa- tion materials and incompliance of civic education with human rights standards and Toledo principles. Along with gender insensitive content and the con- stant messaging of traditional gender stereotypes, the teacher's deeply biased attitude is particularly dam- aging for developing the notion of gender equality in youth during their formative years. The level of bias is well demonstrated in a survey on gender socializa- tion among teachers according to which a majority of teachers strive to instill docility and modesty in girls and leadership in boys. Equally troubling is their statement that the educational benchmarks they pre- sume for boys and girls are different with the boys being held to higher standards, with a justification that they need education more than girls. This is in a country where the number of women with higher education has been greater than men for decades and women in technical and scientific specialties consti- tuted almost half of workforce as recently as ten years ago. 14. Development of subject standard for the Armenian The EQRP2 and the EIP do not support any links Church History is singlehandedly controlled and between religion and education and the Bank strong- supervised by Armenian Apostolic Church through ly supports the principle of inclusive development. its Center for Christian Education and Propaganda. This particular situation and the potential harm During the teaching process the doctrine of the Ar- stemming from it is neither caused by nor aggravated menian Apostolic Church is preached. Discrimina- by the Bank-supported projects. tion and ill treatment are wi