Document of The World Bank Report No: ICR0000731 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-45390 JPN-27719) ON A LOAN IN THE AMOUNT OF US$34.7 MILLION TO THE HASHEMITE KINGDOM OF JORDAN FOR HIGHER EDUCATION DEVELOPMENT PROJECT (HEDP) December 26, 2007 Human Development Group Middle East and North Africa Region Jordan Higher Education Development Project CURRENCY EQUIVALENTS (Exchange Rate Effective as of Board Date) Currency Unit = Jordanian Dinar (JOD) 1.00 JOD = 1.42 US$ US$ 1.00 = JOD 0.71 Fiscal Year January 1 ­ December 31 AN Area Network JOD Jordanian Dinar BAU Al-Balqa Applied University JUST Jordan University of Science and Technology Ministry of Higher Education and Scientific CAS Country Assistance Strategy MHESR Research CC Community College MIS Management Information System CIDA Canadian International Development Agency MOP Ministry of Planning Directorate of Computers, Information, and DCIS MTR Mid-term Review Statistics National Centre for Human Resources DO Development Objectives NCHRD Development FC Fund Committee NICS National Information Center FDC Faculty Development Center NSC National Steering Committee FDF Faculty Development Fund PAD Project Appraisal Document GDLN Global Development Learning Network PC Personal Computer GDP Gross Domestic Product PDU Program Development Unit GOJ Government of Jordan PIAC Project Implementation Advisory Committee HEAC Higher Education Accreditation Council PIU Project Implementation Unit HEC Higher Education Council PMR Project Management Report HEDF Higher Education Development Fund RSS Royal Scientific Society Sustaining and Extending Technical and HEDP Higher Education Development Project SETVET Vocational Education and Training International Bank for Reconstruction and IBRD SIL Specific Investment Loan Development ILS Integrated Library System TA Technical Assistance IRR Internal Rates of Return TVET Technical Education and Training IT Information Technology UIU University Implementation Unit ITF Information Technology Fund Vice President Daniela Gressani Country Director Joseph Saba Sector Director Michal Rutkowski Sector Manager Mourad Ezzine Project Team Leader Adriana Jaramillo ICR Team Leader Linda K. English Jordan Higher Education Development Project Implementation Completion and Results Report CONTENTS A. Basic Information ..................................................................................................................1 B. Key Dates...............................................................................................................................1 C. Ratings Summary...................................................................................................................1 D. Sector and Theme Codes .......................................................................................................2 E. Bank Staff ..............................................................................................................................2 F. Results Framework Analysis..................................................................................................3 G. Ratings of Project Performance in ISRs ................................................................................7 H. Restructuring (if any).............................................................................................................8 I. Disbursement Profile..............................................................................................................8 1. Project Context, Development Objectives and Design...........................................................9 2. Key Factors Affecting Implementation and Outcomes........................................................13 3. Assessment of Outcomes .....................................................................................................17 4. Assessment of Risk to Development Outcome....................................................................22 5. Assessment of Bank and Borrower Performance.................................................................22 6. Lessons Learned...................................................................................................................24 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners......................25 Annex 1: Project Costs and Financing .....................................................................................26 Annex 2: Outputs by Component.............................................................................................27 Annex 3: Economic and Financial Analysis.............................................................................34 Annex 4: Bank Lending and Implementation Support/Supervision Processes ........................37 Annex 5: Beneficiary Survey Results.......................................................................................39 Annex 6: Stakeholder Workshop Report and Results ..............................................................40 Annex 8: Comments of Co-financiers and Other Partners/Stakeholders .................................46 Annex 9: List of Supporting Documents..................................................................................47 MAP A. BASIC INFORMATION Jordan Higher Education Country: Jordan Project Name: Development Project ID: P069326 L/C/TF Number(s): IBRD-45390,JPN-27719 ICR Date: 11/26/2007 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: SIL Borrower: JORDAN Original Total USD 34.7M Disbursed Amount: USD 33.0M Commitment: Environmental Category: C Implementing Agencies: National Center for Human Resources Development Co-financiers and Other External Partners: B. KEY DATES Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/07/1997 Effectiveness: 10/07/2000 10/07/2000 Appraisal: 09/25/1998 Restructuring(s): Approval: 02/29/2000 Mid-term Review: 06/12/2003 Closing: 12/31/2005 06/30/2007 C. RATINGS SUMMARY C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory 1 C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Satisfactory Overall Bank Overall Borrower Performance: Moderately Satisfactory Performance: Moderately Satisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Performance Indicators any) Rating Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any time No Quality of Supervision None (Yes/No): (QSA): DO rating before Satisfactory Closing/Inactive status: D. SECTOR AND THEME CODES Original Actual Sector Code (as % of total Bank financing) Central government administration 1 0 Tertiary education 99 100 Theme Code (Primary/Secondary) Education for the knowledge economy Primary Primary E. BANK STAFF Positions At ICR At Approval Vice President: Daniella Gressani Kemal Dervis Country Director: Hedi Larbi Inder Sud Sector Manager: Mourad Ezzine Marlaine Lockheed Project Team Leader: Adrian Jaramillo Francis Steier ICR Team Leader: Linda K. English ICR Primary Author: Linda K. English 2 F. RESULTS FRAMEWORK ANALYSIS Project Development Objectives (from Project Appraisal Document) The project will initiate improvements in the quality, relevance, and efficiency of higher education in Jordan, and will support the Kingdom's program to reform sector governance. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Indicator Baseline Value Values (from Achieved at approval Revised Completion or documents) Target Values Target Years Indicator 1 : Quality: Increased use of new teaching methods, educational media and technology, and improved ratings of faculty teaching by students in universities. Teaching was traditional lecture method, technology Value use in classrooms ranged Increase technology Technology use in quantitative or from 20% to 1% and by use in classes to classroom is not Qualitative) professors from 0% to 70% (on average) widespread. 20 %, student evaluations were not used. Date achieved 10/07/2000 10/07/2000 10/24/2004 06/30/2007 Target achieved. New teaching methods were introduced in 13 new and 11 revised Comments undergraduate programs as a result of the Higher Education Development Fund (incl. % (HEDF), technology use in class rooms is embryonic according to the ICT evaluation, achievement) Faculty Development Centers set up to train teaching staff in modern teaching methods. Student evaluations have been carried out in all universities. Indicator 2 : Relevance: increased enrollments in new programs supported by HEDF 2150 students enrolled in the first Value Competitive funding year of 11 new quantitative or mechanism was not used as No specific targets undergraduate Qualitative) means to select which new programs; and 5200 programs will be developed. in 13 restructured programs 13 Masters programs introduced Date achieved 10/07/2000 09/01/2004 06/30/2007 3 Target partially achieved. The number of students enrolled in the first year of new programs was approximately 2150. Data was not collected to monitor on a multi-year Comments basis. Revised programs reached approximately 5200 students. 30 percent of sub- (incl. % projects made links with enterprises and social organizations. The competitive achievement) mechanism was not used in the first two of three rounds of the competitive fund and the modified selection process for the first two rounds worked more like a matching fund facility and thus did not ensure that the selected programs were the most relevant from the country perspective. Indicator 3 : Efficiency of public resource use: share of expenditures for academic programs increases from 1999 baseline to target. Improvement in Ratio of Value Ratio of administrative staff ratio of administrative to quantitative or to teaching staff was 2.7 to administrative to teaching staff Qualitative) 1 in 2000. teaching staff to 1 to worsened slightly to 2.0 2.8 to 1 in 2005. Date achieved 10/07/2000 07/01/2007 06/30/2007 Comments Target Not achieved Universities have not reduced non-teaching to teaching ratios (incl. % which worsened during the project from 2.7:1 to 2.8:1. At the same time, student faculty Achievement) ratios have worsened from 23 in 2000 to 31 in 2004 and 38 in 2005 in public universities. Management and governance: transparent mechanism used by Higher Education Indicator 4 : Council for allocation of recurrent resources (formula) and new program investments (HEDF); time for student registration reduced by 10% by 2005 from baseline hours in 1999. Financial resources to Government using a universities allocated formula funding Dated covenant through annual negotiated mechanism based onfor Value budget process without student headcount implementation Formula funding quantitative or explicit consideration to and per-program of competitive abandoned; Qualitative) access, equity and student unit cost; funding was competitive fund efficiency; competitive fund competitive fund eliminated after introduced. did not exist; time for introduced and time mid-term students to register was on to register reduced review average 55 minutes. by 30% Date achieved 10/07/2000 06/30/2007 06/30/2007 Comments Partially achieved Transparency and efficiency in budget management not established (incl. % as formula funding for allocating resources to public universities was abandoned; achievement) competitive fund mechanism introduced in last of three competitive rounds; no data on time to register. 4 Indicator 5 : Quality Community Colleges: Employer Satisfaction with skills of graduates improved over baseline Employer survey conducted during preparation Language labs Value perceived college programs Share of students installed but degree quantitative or to provide a low level of having competent of student Qualitative) instruction, academic English skills from knowledge and skill 0% to 30 % competency not development not relevant to known. the workplace Date achieved 10/07/2000 10/07/2000 10/24/2004 Comments Target not achieved. A Tracer study and employer attitude study were carried out in (incl. % 2004 under the SETVET project financed by CIDA. The study results were not achievement) conclusive and since only four programs in four colleges were revised in 2006, it is too early to know the impact in the workplace. Indicator 6 : Relevance in Community Colleges: Increased enrollment in new associate degree program. Four new programs Enrollments in new were introduced in Value Increase enrollments each of 4 community quantitative or associate degree programs in new associate colleges in 2006. Qualitative) to increase from 10,000 in 1999 degree programs to Enrollment in 15,000 community colleges did not increase. Date achieved 10/07/2000 09/01/2004 06/30/2007 Target Not achieved under the SETVET project, four new associate degree programs Comments were developed in collaboration with the private sector and introduced in 2006. The (incl. % number of students enrolled in public community colleges has remained stagnant at achievement) about 10,000 per year over the past 5 years. Enrollment in all community colleges has fallen from approximately 29,000 to 25,500. Management and governance: Complete the organizational and management structure Indicator 7: and operational manual for Al-Balqa University and policy and procedures manuals by 2003. Community colleges are run New organizational structure clarifying Value by A Baqua university with the governance and quantitative or little autonomy or management for Not completed Qualitative) involvement by management in decision- community colleges making and policy and procedures manuals Date achieved 10/07/2000 07/01/2007 06/30/2007 5 Comments Target Not achieved Options for organizational and management structure were (incl. % designed but final decision not taken. A Community College Deans' Council set up. Achievement) Policy and procedures manuals have not been approved. (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Indicator Baseline Value Values (from Achieved at approval Revised Target Completion or documents) Values Target Years Indicator 1 : ICT has been adopted to enhance quality and efficiency of teaching and learning throughout the public university system. 100% of universities connected through 0% of universities had Wide Wide Area Network Value Area Network and LAN 100% of universities and LANs installed; (quantitative connectivity; students have had Wide Area 100% of faculty have or Qualitative) access to PC for 10 Network personal computers minutes/week; 1:10 to 1:50 connectivity. (PC) and 100% of computer to professor ratios students have access to PC for 5 hours/week. Date achieved 01/01/2000 06/30/2005 06/30/2007 Comments Target surpassed. Program fully implemented by 2005 and surpassed as two new and (incl. % additional universities were included. An ICT evaluation concluded that student achievement) utilization of ICT is 70 percent for coursework; utilization of ICT by teachers is limited; an integrated library system is operational. Indicator 2 : ICT has been adopted to enhance quality and efficiency of administration and decision- making capabilities at the policy maker's level. None of the public Value universities use a computer The implementation MIS established in (quantitative based Management of the MIS community colleges or Qualitative) Information System (MIS) but not operational in linking them to each other completed universities and to the MHESR. Date achieved 10/07/2000 07/01/2005 Comments (incl. % Partially achieved. Technical specification and bidding documents prepared by closing achievement) date. No university utilizing a computer based MIS. Indicator 3 : Pedagogical practices have been enhanced through establishment of Faculty Development Centers in all public universities. Value FDCs being used to FDCs established in (quantitative No FDCs in any public enhance all universities, or Qualitative) university. pedagogical training courses practices. offered. Date achieved 10/07/2000 06/30/2005 06/30/2007 Comments Target partially achieved. Universities financed 146 faculty development activities 6 (incl. % carried out by FDCS primarily on classroom instruction, methods of teaching, and achievement) instructional media, and evaluation and 12,500 teacher evaluations were carried out between 2004 and 2006 in all universities. Evaluations of FDC and HEPF concluded that impact on pedagogic practices has been limited to date. A strategic plan for renewal of the community college system developed and being Indicator 4 : implemented; and organizational structures and practices aligned to meet requirements of the strategic plan. Community colleges produce graduates with high unemployment rates and strategic plan Organizational plan and procedures Value No agreed upon strategic formulated, manual prepared but (quantitative plan or direction for planning procedures not being or Qualitative) community colleges developed and documented; annual implemented. No work plans based on strategic plan strategic plan in developed place; revised organizational plan in place. Date achieved 10/07/2000 06/30/2005 Comments Partially met. MIS is in place and options for organizational structure developed, but no (incl. % strategic plan is in place, there is no agreement on admissions policy, transfer policy to achievement) universities and how to meet demands of labor market. G. RATINGS OF PROJECT PERFORMANCE IN ISRS No. Date ISR Actual Disbursements Archived DO IP (USD millions) 1 06/23/2000 Satisfactory Satisfactory 0.00 2 11/27/2000 Satisfactory Satisfactory 0.35 3 05/25/2001 Satisfactory Satisfactory 1.38 4 10/23/2001 Satisfactory Satisfactory 1.46 5 12/18/2001 Satisfactory Satisfactory 1.46 6 05/16/2002 Satisfactory Satisfactory 4.71 7 06/27/2002 Satisfactory Satisfactory 4.71 8 11/12/2002 Satisfactory Satisfactory 6.62 9 04/21/2003 Satisfactory Satisfactory 7.87 10 10/16/2003 Satisfactory Satisfactory 11.41 11 04/28/2004 Satisfactory Satisfactory 13.04 12 10/29/2004 Satisfactory Satisfactory 17.13 13 04/29/2005 Satisfactory Moderately Satisfactory 18.21 14 12/30/2005 Satisfactory Moderately Satisfactory 22.34 15 06/29/2006 Satisfactory Satisfactory 25.66 16 12/22/2006 Satisfactory Satisfactory 29.72 17 06/26/2007 Satisfactory Satisfactory 31.79 7 H. RESTRUCTURING (IF ANY) Not Applicable I. DISBURSEMENT PROFILE 8 1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN 1.1 Context at Appraisal The Higher Education Development Project (HEDP) was prepared during a period of economic difficulty in Jordan as regional tensions caused a slowdown in trading partner economies, reduced demand for Jordanian exports (potash, phosphates and fertilizers) and adversely affected private investment in Jordan. The Government's economic reform program aimed at increasing growth and employment rates by diversifying and improving the competitiveness of the Jordanian economy, reducing the role of the state in productive sectors and liberalizing markets. The business community was to be the engine of growth and part of the Government's strategy to support private enterprises was to improve the quality of education and training and the skills of new graduates entering the labor force. Although Jordan was well-endowed with university graduates at the time the project was being prepared, their skill mix and the nature of their pedagogical training resulted in shortages of managerial and mid- level modern technical skills which included innovative problem-solving capacity. The problems of quality and efficiency at the university level were affecting Jordan's international competitiveness. Universities and colleges lacked modern information technology systems, laboratories and libraries required for modernizing programs of study. Pedagogic approaches were theoretical and most programs were not relevant to the needs of high-growth export sectors. Enrollments in community colleges were declining sharply and unemployment among graduates was over 25 percent. The university sector lacked effective policy-making advisory boards to assist decision makers in reforming the sector and making resource allocation decisions. There was no strategy to link the allocation of resources in public university programs to the needs of the labor market. In 1988, the Government initiated a successful basic education reform program for which the Bank provided two loans. By the mid-1990s, the Government began to focus on higher education and developing a comprehensive strategy to improve the learning and relevance to meet market needs (including applied research and quality assurance of private higher education). This strategy focused on improving quality of university training, the financial sustainability of universities, governance and management of the sector, reviving community colleges and promoting research and the role of private universities. In August 1998, Parliament enacted a higher education law expanding the mandate of the Higher Education Councils (HEC) by giving them full responsibility for regulating higher education and abolishing the Ministry of Higher Education. A Higher Education Accreditation Council (HEAC) was created and Boards of Trustees with private sector participation were established for each public university. Finally, to bring colleges into a professionally relevant environment, Parliament approved the creation of Al-Balqa' Applied University (BAU) in March 1997, grouping all public two-year community colleges with a four year college and gave BAU the mandate of strengthening applied technology training. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The project development objectives were to initiate improvements in the quality, relevance, and efficiency of higher education in Jordan, and support the Kingdom's program to reform sector governance. The project was to: (i) establish system-wide modern information technology (IT), management information systems (MIS), and library infrastructure for higher education; (ii) support a Higher Education Development Fund that would provide grants to public universities for innovative and economically relevant sub-projects and for improving teaching and learning; (iii) initiate reforms of higher education governance, including the introduction of formula-based allocation of higher education recurrent funding from the Government; and (iv) support the rationalization of the community college sub-sector through the new Al-Balqa Applied University (BAU). 9 B. Key Performance Indicators The key performance indicators as stated in the Minutes of Negotiations and in the project logical framework in Annex 1 of the PAD are as follows: Quality · Increased use of new teaching methods, educational media and technology and improved ratings of faculty teaching by students. · Employer satisfaction with skills of graduates of community colleges improved. Is there any evidence of improved employer satisfaction? Relevance · Increased enrolments in new university programs support by the competitive fund. · Increased enrollments in new associate degree programs. Efficiency of Public Resource Use · The share of expenditures for academic programs increases from 1999 to target. Management and Governance · Transparent mechanisms used by HEC to allocate recurrent resources (formula-funding) and new program investments. · BAU/CC completes organizational design and operational policy manual by 2003 MS. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The development objectives were not revised. 1.4 Main Beneficiaries The main beneficiaries were: (i) In the short-term, students in the eight public universities and 21 community colleges (70 percent of which are female) as well as teachers and administrators in the public university system. (ii) In the medium and long-term, employers will benefit, as will students enrolled in the private universities from the creation of the HEAC. 1.5 Original Components (as approved) The project had four components and nine sub-components: Component 1 - Initiate improvements in quality, relevance and efficiency (US$46.092 million): The first component covered interventions designed to support the entire higher education sector and was divided into two sub-components: Sub-component 1.1 - System-wide Support: This sub-component aimed to improve the quality, relevance and efficiency of teaching, learning and administration in universities. System-wide support was to be provided to: (i) establish a robust, standards-based information technology infrastructure (IT), including hardware, software, and applications; inter- and intra-university connectivity, and global 10 connectivity through the Internet to provide students and faculty with access to national and global research data and information; and the ability to collaborate electronically with educators on both the national and global levels; (ii) install high-speed fiber optic lines, a mix of shared and switched network technologies and various application servers linking public universities together; (iii) design and implement a modern integrated management information system needed in decision-making by the HEC; (iv) develop an inter-university library system with greater automation, networking facilities and staff training; and (v) set up Faculty Development Centers (FDC) in eight universities and training for faculty developers. Sub-component 1.2 - Higher Education Development Fund (HEDF): The objective of the HEDF was to support improvements in the quality and relevance of higher education and to align higher education given by Jordanian public universities with the needs of a market driven economy. The Fund was initially designed to provide grants through three windows: (i) competitive grants; (ii) a Faculty Development Center (FDC) Fund; and (iii) an Information Technology (IT) Fund. The project planned to finance advisory and technical services, study tours, contract research, comprehensive service centers, training, incubator services, laboratory and computer equipment and multi-disciplinary centers. Component 2 - Initiate Improvements in Governance (US$1.911 million): This component was to strengthen the Government's institutional capacity to implement governance reforms as well as universities' planning and management capacity, and had 3 subcomponents. Subcomponent 2.1 - Higher Education Council Secretariat: The objective of this sub-component was to strengthen the policy formulation capacity of the HEC. This sub-component was to support the establishment of the HEC Secretariat and to develop its technical mandate and skills to inform and advise the HEC on policy matters by: (i) providing TA to help set up the analytical functions of the HEC Secretariat, including the Funding Unit, a Policy Advisory Unit, and a Statistical Unit; (ii) providing equipment to support these functions; (iii) technical assistance to design student loan and scholarship schemes to ensure the financial sustainability of the sector, while improving access and equity; and (iv) local and overseas staff training. Sub-component 2.2 - Higher Education Accreditation Council (HEAC): This sub-component aimed to promote the process of accreditation and evaluation of public and private universities and their programs of study. The sub-component was to support the technical assistance necessary to make the HEAC operational including setting up disciplinary committees and procedures for evaluation and accreditation, studies, participation of stakeholders in the development of methodologies for external and self-evaluation of universities and programs and the preparation of an evaluation calendar for universities and program accreditation. Sub component 2.3 - Improved Planning and Management Capacity at the University Level: The objective of this sub-component was to bring all planning and management staff and process in public universities in line with the upgraded practices at the national level, and consistent with the new accountability requirements put in place by the HEC and HEAC. Training of university managers, administrators and selected faculty in management and financial management, and the strengthening of relations with the private sector was to be financed. 11 Component 3 - Reform of the Community College System/Al-Balqa Applied University (US$5.622 million) The GOJ established Al-Balqa Applied University (BAU) to be responsible for the academic, management and administrative affairs of all community colleges. This component was to support the reform process in governance and management, program development, human resources management and facility upgrading, and had four subcomponents. Sub-component 3.1 - Governance and Management: This sub-component was to support: (i) the development of the BAU organizational structure and manual of procedures; (ii) an assessment of management information systems for BAU and its associated community college system; (iii) acquisition of hardware and software to develop a financial, human resources and student management information system; and (iv) fellowships were to be provided for the BAU System Analyst and training for MIS users. Sub-component 3.2 - Program Development: BAU committed itself to a process of program renewal and was to introduce new employment-driven streams at the Associate Degree level in the fields of technology, business and finance, natural resources and applied social sciences. The sub-component was to finance: (i) fellowships for staff in the Program Development Unit (PDU) to manage the program renewal process; (ii) five PDU staff were to be given fellowships to establish a competency-based curriculum; and (iii) technical assistance to establish procedures for the unit and guide the development process for the first year of the new Associate Degree Programs. Sub-component 3.3 - Human Resources Development (HRD): This sub-component supported three mechanisms to strengthen its teaching staff, including the adoption of lead instructors for sector and sub- sector programs making up the Associate Degree program framework. Activities to be supported included fellowships to lead instructors for professional and pedagogic upgrading, short term secondments to business or industry, and in-country professional development workshops in specialized areas relevant to the new Associate Degree programs. The investment program was to provide technical assistance to prepare the HRD plan, together with funds for local training. Sub-component 3.4 - Faculty and Equipment Upgrade: The planned BAU Associate Degree program was to build technical skills and competencies on a foundation of numeracy, literacy, communications, computer applications, and English comprehension. The sub-component was to finance computer, science and language and specialized labs as well as equipment, books and audio visual equipment. Component 4: Project Implementation Capacity (US$1.509 million): The component supported project implementation capacity and was financed by the Government, including: · Staffing of the Project Implementation Unit (PIU) (Project Manager, Fund Coordinator, IT coordinator, Procurement Office, Financial Officer). · Administrative costs (peer reviewers of HEDF, two evaluation conferences and expenses related to evaluation activities. · Incremental staff at University Implementation Units (UIU). · Support to NSC, IT/MIS committee, Library Committee and ad hoc advisory committees. 1.6 Revised Components The components were not formally revised during the project. 12 1.7 Other Significant Changes At the time of negotiations, the Government decided to reduce the amount of the appraised project by US$9 million, primarily for technical assistance and to find other grant resources to finance these planned activities. These changes were not reflected in the Project Appraisal Document (PAD) and the performance indicators remained unchanged. Mostly affected were the technical assistance for the MIS and library in subcomponent 1.1, Component 2 and Sub-components 3.1, 3.2 and 3.3. The Government argued that it could not afford to borrow resources for technical assistance. The Bank team did not want to revise the scope of the project or reappraise performance indicators as 18 months had already lapsed and the Bank team was concerned that such demands would jeopardize negotiations. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 Project Preparation, Design and Quality at Entry The quality at entry of the project is assessed as moderately unsatisfactory, based on the following assessment. Project Design: The project design was suitable as it identified the key policy reforms that needed to be put in place and targeted project activities at both the sector level and the university level. The project introduced mechanisms to change behaviors in universities to improve quality including the introduction of a funding formula, the use of competitive funds to promote relevance and innovation in programs and accreditation mechanisms to improve accountability for results. The project scope focused on both universities and post-secondary education and thus was well placed to address the constraints that Jordan faced in providing the technically skilled labor needed to improve the country's competitiveness. Although in line with the key policies needed, the ambition to introduce system wide finance reforms together with institutional reforms to promote quality and relevance was too ambitious. Participatory Approach: Representatives of universities were involved in the preparation process in both the design stage, in technical committees and in the National Steering Committee (NSC) which was established during preparation. Stakeholders were consulted in the social assessment which influenced the design of the community college component. Lessons learned from previous projects were integrated into the project design, including: (i) the need to take into consideration the quasi-autonomous nature of universities in the project preparation and implementation; and (ii) achieving improvements in quality and relevance at the national level required universities to improve their planning and management practices. Economic and Sector Analysis: A sector report identified the major constraints and provided concrete recommendations for improving sector performance and was widely disseminated and useful for initiating the policy dialogue. The analysis, however, did not include a detailed sector expenditure analysis which would have helped in providing the basis for implementing the formula funding mechanism and relevant indicators. Government Ownership: Factors that hindered a strong sense of government ownership included difficulty in developing a strong consensus for the strategy and policy framework to provide the foundation for governance and financing reforms and thus to achieve project goals. Given that eighteen months elapsed between appraisal and the completion of negotiations and that during negotiations, the project was downsized by US$9 million without identifying alternative resources for technical assistance, indicated less than optimal government ownership for the project and thus commitment to implementing reforms. 13 Assessment of Risks and Mitigation Strategies: Six risks were identified. The risk associated with technical assistance not being mobilized was rated as high. Mitigation strategies to manage the technical assistance risk were inadequate in light of the fact that the Government eliminated most technical assistance at the time of negotiations and it was highly unlikely that sufficient technical assistance could be mobilized from other sources in a short period to ensure adequate project implementation. The risks also did not include the weak consensus by stakeholders and Government to implement reforms and did not take into account the risk of under-funding of the sector in the design of the project even though this was highlighted in the economic analysis. Readiness for implementation was unsatisfactory. Factors that undermined the readiness for implementation included the following aspects: · While the Government was committed to the reform program, the project did not detail the specific activities to assist in the implementation of reforms, e.g., introducing the formula funding mechanism and accreditation systems. The preparation of the governance component was incomplete as activity plans were not developed during preparation and their feasibility appraised. Instead, dated covenants were used to ensure that these action plans were prepared. Thus there was weak provision for implementing reforms. · The ability to successfully attain project objectives without the planned consultancy services was not assessed and key performance indicators were not revised at the time of negotiations when the project was down-sized by US$9 million. Given this substantial cut, it would have been better to reduce the scope of the project. · There was no major policy entity in the Government charged with managing the reform agenda. The decision to use a project implementation unit (PIU) located in a semi-autonomous agency such as the National Centre for Human Resource Development (NCHRD) underestimated the complexity and leadership requirements to successfully implement reforms in higher education. A PIU is not the best entity to manage complex projects in higher education involving major reforms, beneficiaries and implementers together to forge consensus and agree on actions to implement within the academic context. · The PIU was not adequately staffed to carry out all the procurement and financial management activities, adequately manage and implement the competitive fund, and ensure the adequate monitoring and evaluation functions in addition to being responsible for executing policy reforms. · The manual of procedures for the HEDF was prepared by Bank staff and did not involve universities in its preparation. As a result, the relevance of the competitive processes was not adapted to the reality of Jordan and the buy-in of universities was less than optimal when compared to other competitive funds financed by the Bank at this time. The manual did not link the HEDF programs to Government priorities and did not adequately clarify the program objectives and priorities. The project underestimated the importance of having a highly regarded academic Director of the Fund with adequate staff to manage the capacity building, selection process, peer reviewing and monitoring that competitive funds have in other countries. The centralized implementation by the PIU did not help to build university capacity to manage competitive funds. · The monitoring and evaluation framework was not robust as indicators were complex and depended on the universities' capacity to provide timely information including, for example, expenditure allocations to programs, time for students to register and improvements in teaching quality which were impossible to obtain given that the MIS systems did not exist. 14 Conclusion The project was relevant to the needs of the country. Background sector work reflected a good understanding of issues and government legislation creating the HEAC and strengthening the mandate of the HEC provided an improved institutional environment. Sector policy reforms, including the funding formula, competitive fund, accreditation system and those for community colleges were appropriate. . However, the design of the project did not adequately reflect the need for academic and political leadership to steer major reforms in higher education. The inappropriate design of the competitive fund, the failure to adapt the project scope and performance indicators to the reduced project size, and the unsatisfactory readiness for implementation undermined the quality of entry. 2.2 Implementation A number of significant changes were occurring during the early implementation of the project including the reestablishment of the Ministry of Higher Education and Scientific Research and thus the change in the role and mandate of the Higher Education Council and the governance structure of the Higher Education Accreditation Council. New legislation provided University Boards of Trustees with more autonomy. Due to the changing demographics and the rapid improvements in primary and secondary completion rates, the demand for higher education was rising rapidly. The Government also decided to cut funding to universities and at the same time froze admission fees. In this environment of falling public resources and rising student enrollments, the Government was not able to form a consensus on implementing the formula funding mechanism. To cope with expenditure cuts and fee freeze on regular programs, universities introduced parallel programs where students willing to pay higher fees were admitted. In view of this changed policy environment, it was decided at the time of the MTR in June 2003 to: (i) eliminate the dated covenants regarding the action plans for HEC and HEAC and the application of formula funding; (ii) share with the Bank by the end of 2003, the Government's revised strategy. . This strategy would consolidate the dated covenants and address financing in a comprehensive manner; (iii) revise key performance and intermediate indicators to reflect the sector and project reality; (iv) re-allocate approximately US$1.5 million of unallocated resources to be used as technical assistance for the MIS, library and HEDF sub-projects as adequate technical assistance on a grant basis was not forthcoming; (v) change the composition of the National Steering Committee (NSC) to reduce the participation of university vice-presidents who did not have time to allocate to this activity. The PIU's strong implementation capacity to deliver project outputs constituted an impressive strength of the project implementation team, particularly for Component 1. Activities were well planned and implemented within deadlines. Delays in implementing the financing and governance reforms inherent in component 2 and 3 were due to the lack of a policy champion in the government to implement the reform agenda. The creation of the Ministry of Higher Education and Scientific Research and decisions made at the MTR in 2003 improved the policy environment and helped to advance project implementation. At this time it was decided to reallocate un-disbursed resources to provide technical assistance to advance implementation of activities related to the library, MIS and the competitive fund. Resources were assigned to carry out studies to assess policy and program initiatives financed by the project, and to develop a sector strategy. Nevertheless, the project could have benefited from a more strategic emphasis on implementing reforms and monitoring policy implementation and outcomes. 15 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization An M&E framework was prepared during appraisal detailing indicators, reports and evaluations to be carried out. The PIU was responsible for monitoring indicators and presenting reports to the NSC. Following the downsizing of the project, including a major reduction of the technical assistance component, during negotiations, indicators were not changed to reflect the new reality of the project. Indicators were not specific enough for the reforms as was the case for the accreditation system and formula funding mechanism and reforms for community colleges. In some cases, baseline data did not exist. Collecting data depended a lot on universities having a good MIS in place and the responsibility for collecting and analyzing data at universities was not specified. For the first three years of the project, performance and output indicators were not monitored or measured because of unclear responsibilities about who would collect and analyze data which rendered the monitoring framework irrelevant. Regarding evaluation, the initial project design included a tracer study and a labor market review for community colleges which were carried out by the SETVET project financed by CIDA. However, these reports and recommendations did not result in any change. As mentioned in Section 2, following the mid-term review in June 2003, performance indicators were revised in October 2004 and these indicators were regularly monitored. This delay in revising indicators was related to the time it took for the Ministry of Higher Education and Scientific Research to finalize its revised higher education strategy. At this time, it was decided to finance evaluations and on policy and program initiatives including, assessments of the competitive funding mechanism, faculty development centers, ITC programs and studies on higher education governance, financing of higher education and student loans. These studies were subsequently discussed at a Higher Education Forum organized in February 2007 and will form the basis of the Government's next phase of reforms. 2.4 Safeguard and Fiduciary Compliance Procurement: Overall, procurement activities were handled within guidelines and standards acceptable to the Bank: (i) a procurement plan was developed and used for monitoring procurement processes; and (ii) the implementation team carried out the acquisition of goods and services efficiently. A post- procurement review was satisfactory; and (iii) the procurement of equipment and goods was done according to plan and within the original project time-frame and constitutes achievement. Financial management: The financial management capacity at the level of the PIU was maintained at a satisfactory level, and quarterly and annual audit reports were regularly prepared and submitted within deadlines. The system assured accurate monitoring of disbursements of project, Government and other donor financing. Environment protection: The project was classified as Category B. In compliance with Annex 12 of the PAD, safeguard and management procedures were applied. 2.5 Post-completion Operation/Next Phase In terms of sustaining reforms, the creation of the Ministry of Higher Education and Scientific Research and the autonomous Agency for Quality Assurance and Accreditation (AQAA) with a broadened mandate has provided impetus for continuous policy reform. The Ministry is putting together a new policy framework and strategy to improve the governance of higher education institutions and the relevance of programs to be supported by a new phase of the competitive fund which will form the basis for a new Bank-financed operation. 16 3. ASSESSMENT OF OUTCOMES 3.1 Relevance of Objectives, Design and Implementation The project development objective of HEDP is coherent with the Bank's current CAS which supports Jordan's National Agenda to promote knowledge-based private investment, in order to support job creation and poverty reduction. Improving the quality of labor is seen as key to raising employment opportunities by supporting enhanced quality of education and skills of the Jordanian population at all levels. Improving linkages between the education system and the labor market is a main priority of Bank support. The HEDP represented a significant effort to align Jordan's post-secondary education system to the needs of a knowledge-based economy and greater integration into global markets. On the one hand, the project design was relevant since it aimed to initiate reforms at the sector level and institutional reforms at the university level. By introducing formula funding, the competitive fund and accreditation mechanisms, the project rightly aimed to create incentives and accountability mechanisms that would encourage faculty and administrators to focus on quality, efficiency and relevance. However, the project design was overambitious as it did not include a government endorsed sector strategy and a specific plan of actions and activities to ensure that those policy reforms were introduced. Unfortunately the competitive fund was designed as a stand-alone entity and did not adequately link the allocation of investment funds to government priorities such as quality assurance and accreditation mechanisms, faculty development activities and labor market needs. The decision not to modify the scope of the project following the decision to downsize the project financing undermined the feasibility of attaining project objectives, specifically with regard to the restructuring of community colleges, accreditation systems and the MIS. The weak monitoring framework contributed to the lack of clarity on what the project was trying to do. The implementation strategy relied too much on non-government structures by setting up a project implementation unit in the NCHRD without adequate links to universities and the HEC. The project implementation unit did not have enough staff with the in-depth knowledge and experience to guide reforms, implement a competitive fund and build university management capacity. Project management was focused mostly on outputs rather than outcomes. The project was not adapted to the decision taken at negotiations to down-size the project and thus from the beginning the project lacked the strategic leadership and resources to achieve its objectives. 3.2 Achievement of Project Development Objectives The main development objective of HEDP was to initiate improvements in the quality, relevance and efficiency of Jordan's higher education system and to support the government program to reform and modernize the administration and governance of the sector. To improve quality and relevance, the Government's policy framework aimed to establish an accreditation system for both public and private universities, introduce a competitive fund to improve the relevance of study programs, set up faculty development centers to improve teaching, and install an inter- library system and an IT network. The HEDP has made significant contributions to the sector but it has only partially met its development objectives as outlined below under the headings of quality and relevance, efficiency and community colleges. Improving Quality and Relevance · Satisfactory Initiating an accreditation system. The HEAC was charged with establishing an accreditation system for public and private universities. All 12 private universities have 17 undergone accreditation processes and been accredited. This has allowed the private sector to develop in a judicious way, providing some quality assurance to students and families, unlike in many countries where dubious institutions often operate. Consequently, Jordan is a leader in the region and provides assistance to other countries. While at least 3 public universities underwent accreditation processes, no public university was accredited during the project, in large part because rising student enrollments and declining public financing made it very difficult for them to reach international standards on student teacher ratios which rose of 21 in 2000 to 38 in 2005. The accreditation system has focused on inputs and with the creation of the Agency for Quality Assurance and Accreditation in 2006 with greater autonomy, which has more autonomy and is rightly focusing more on internal and external quality assurance processes in public and private universities. · Satisfactory Increased use of new teaching methods and technology, and improved ratings of faculty teaching by students: (i) A national library consisting of a consortium of university libraries and called the Jordanian Center of Excellence for public universities Library Services serves 10 universities. Access to documents has been improved as the system provides for shared cataloging, information retrieval and document delivery systems, a national database of bibliographic records from all universities (over 1 million documents), and over 20,000 online journals for all students and faculty at all universities; (ii) the project established system-wide area and local networks providing all university faculty and students with interconnectivity and high-speed internet access; 70 percent of students use the internet and email for coursework and assignments; 30 percent of students exchange emails with the academic staff and 75 percent of students use standard desktop applications; all of faculty members have PC access and all students have access to a PC for 5 hours a week. The ICT study concluded that students and faculty are increasingly using computers and ICT, but this technology is not fully utilized in teaching and learning to a degree that matches the investment; and (iii) Faculty Development Centers were set up in all universities, and faculty training programs and activities were organized to enhance faculty teaching performance and use of innovative instructional methods. Approximately 150 faculty development activities were carried out benefiting 20 percent of faculty, and 12,500 teacher evaluations were carried out between 2004 and 2006 in all universities. An assessment of the FDCs concluded that their impact would be improved by having full-time directors and full time technical staff and focusing on developing programs responding to the needs of faculty. · Partially Satisfactory Increased enrollments in new programs supported by the HEDF. The Higher Education Development Fund (HEDF) supported 33 sub-projects and resulted in the creation of 11 new undergraduate programs, revised programs for 13 other undergraduate programs and purchased equipment for over 70 laboratories. Approximately 2,150 students enrolled in the first year of new programs and about 5,200 benefited from revised and upgraded programs. The nine entrepreneurial projects were instrumental in opening many university programs to the private and non-governmental sectors including design, applied engineering, pharmaceutical, environment and commerce. The fund created 10 new institutions such as the Multi-disciplinary Institute for Water Research and Environmental Studies and the Jordanian Design Centre and purchased equipment for 70 laboratories in various fields. However a competitive selection process was not used in the first two of three rounds had been planned. As a result, the modified selection process for the first two rounds worked more like a matching fund facility and did not ensure that the selected sub-projects were the most relevant from the country perspective or meet the main objectives of the HEDF; over 20 percent of sub-projects were focused on PhD programs and research which was not the intent of the HEDF. The third round selection of sub-projects followed a competitive process yet the manual of procedures was never modified and updated. The impacts remained in individual sub-projects and there were no demonstration effects within or between universities. 18 Improving Efficiency · Unsatisfactory Efficiency of Public Resource Use: Share of Expenditures for Academic Programs Increases from 1999 Baseline to Target. The Government policy towards the financing of higher education changed during the project, and between 2000 and 2005, the share of public financing in university operations dropped from 50 percent to about 20 percent. Universities generated revenues by introducing parallel fee-paying programs. As well, the Government introduced targeted and subsidized student loan programs which assisted about 14 percent of students in 2006. Data does not exist to show how public resources have been used. With the compression of expenditures and despite the increase in self-generated resources, universities have accumulated arrears and increased student teacher ratios from 23 in 2000 to 38 in 2005. No-teaching to teaching ratios did not improve, going from 2.7:1 in 2000 to 2.8:1 over the same period. Universities and colleges were not able to establish budget policies and allocate resources to improve content, classroom delivery, and associated learning resource material. FDCs were not staffed with full time directors or technical staff and thus were unable to completely fulfill their mandate. Universities ability to finance quality was adversely affected by increased enrollments and reduced funding from the Government. The Management Information System would have been one of the most important mechanisms to improve informed decision making, but unfortunately this was not completed as planned. · Unsatisfactory Management and Governance: Transparent Mechanisms used by HEC for the Allocation of Recurrent Resources (formula funding) and New Program Investments (HEDF). The formula funding mechanism was abandoned in large part because a consensus to agree on the formula could not be reached. As a result, operating budgets remained negotiated and did not follow a transparent strategic plan. The competitive mechanism for the HEDF was used only for the third round. Reform of Community Colleges · Unsatisfactory Employer Satisfaction with skills of graduates improved over baseline. A Tracer study and employer attitude study were carried out in 2004 under the SETVET Project financed by CIDA. The employer attitude study concluded that: (i) employers felt that graduates lacked practical skills and training was too theoretical; (ii) employers would prefer to hire university graduates; (iii) skill shortages existed in nursing, insurance, accounting and construction engineering and colleges were not addressing these training areas; and (iv) students need better skills in English, attitude to work and team skills. Under SETVET, four associate degree programs in four colleges were revised and introduced in 2007. However, it is too early to know the impact of these programs. · Unsatisfactory Relevance: increased enrollments in new associate degree programs (from 10,000 in 1999 to 15,000 in 2005). Under the SEVET Project, four new associate degree programs were developed in collaboration with the private sector and introduced in 2006. The number of students enrolled in public community colleges has remained stagnant at about 10,000 per year over the past 5 years. Enrollment in all community colleges has fallen from approximately 29,000 to 25,500. About 20 percent of students are enrolled in programs for which there are very weak links with labor markets. Community colleges are introducing 4 year bachelor programs. · Unsatisfactory Management and Governance: Organizational structure and operational policy and procedures manual is completed by 2003. Computer and language labs and an MIS system have been put in place for the network of community colleges. The Dean's Council was created, thereby allowing the heads of community colleges to participate in management meetings and decisions at Al-Balqa University. The revision of programs is in an embryonic stage and, with 19 the exception of a few pilot programs, most programs remain theoretical, faculties have little experience in the private sector and the governance and management systems have not evolved to allow greater interaction with enterprises. Weak results are due to the lack of leadership from BAU to make reforms happen. Options for the organizational and management structure were outlined in a 2005 White Paper and a recent evaluation of technical programs offers clear guidance on policy and procedural choices. 3.3 Efficiency This section summarizes the cost-effectiveness analysis that is presented in Annex 5. Cost Benefit Analysis: The analysis calculated internal rates of return (IRR) based on the assumption that graduates would have higher productivity and higher wages with the project. The analysis showed that with a small wage increase of US$5 per month, the internal rate of return would be 10 percent. It was argued that IRR would be higher for community colleges as employers are expected to offer substantially more job opportunities for the skills of students coming out of community colleges. Given that the project activities were aiming to improve the quality and relevance of education for students, the costs were relatively low when compared to the economy wide benefits. Financial sustainability and fiscal impact: The analysis based its projections on a 5 percent annual growth rate of enrollments in public universities. Policy scenarios were generated showing that if enrollments were to increase by 5 percent per year, the financing needs would be substantial, requiring a 10 per cent increase in public financing, 25 percent real increase in tuition and efficiency gains of 10 percent. Enrollments in fact grew on average at 10 percent per annum and the 5 percent projections were probably unrealistic. In this sense the economic analysis did not adequately guide the project design. The risks associated with higher than 5 percent annual enrollment increases and a tight budget situation were not introduced in the risks and mitigation strategies for the project. The analysis assumed that enrollments would follow funding, but in fact the institutional set-up was one where admissions policies for universities were independent of funding decisions. The project also aimed to establish a transparent funding mechanism to allocate recurrent resources to university but the economic analysis did not provide guidance on the level of financing needed, and how this funding mechanism would be managed. The fiscal and financial sustainability analysis should have included an analysis of the trends in financial transfers and tuition fees to university, and the expectations for financing for the medium term. This would have permitted an assessment of the government's capacity to ensure that government financing would increase in real terms on a per student basis, that there was political commitment to increase student fees and agree on measures for efficiency gains. 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory The project was relevant to the needs of Jordan and introduced mechanisms and incentives to change the culture of higher education to one of improving the relevance and quality of programs and clarifying relations between universities and the HEC and HEAC. The project objective was realistic in that it aimed to initiate reforms. Project indicators, however, were quite ambitious and difficult to measure. Inappropriate implementation arrangements and the lack of a solid sector strategy combined with the weak project design undermined the project's capacity from the beginning to fully attain project development objectives. The overall outcome has been rated as moderately unsatisfactory because, in addition to the above, while most output indicators were achieved, progress on key indicators related to components two and three were unsatisfactory. 20 Satisfactory progress was achieved in initiating improvements in quality and relevance of university programs. · An accreditation system was set up and operational. Its focus on private universities was due to the Government's decision to increase student enrollments in public universities and cut funding making it impossible for public universities to meet accreditation standards. Nevertheless, Jordan is a leader in accreditation in the region · FDCs were established in each university but their impact, while positive, was limited as centers did not have full-time directors or technical staff. · A system-wide modern IT infrastructure was established and project objectives for connectivity and computer ratios and utilization were surpassed. Student charges for computer use were introduced to ensure sustainability of investments and ITC is beginning to be used in teaching. The initial target that ITC would be used in 60 percent of classes was not met and probably unrealistic. A world class state of the art inter-library system is operational and considered best practice in the region. · The HEDF financed the creation of 11 new undergraduate programs for 2,150 students in the first year of programs and revised programs for 13 undergraduate programs for over 5,000 students in the first year of the revised programs, equipped 70 laboratories and helped set up 10 new institutions. Moderately satisfactory progress was made to improve the efficiency of resource allocation. The formula funding mechanism to improve the efficiency of resource allocation was abandoned and the competitive process to allocate HEDF resources partially implemented. Government intends to continue with the competitive funding mechanism which will continue to improve the relevance of programs. During the project, the Government broadened its policy to focus on equity and instituted a subsidized student loan program which provided assistance to 14 percent of students, of which 60 percent came from very poor families. Moderately unsatisfactory progress was made in attempts to improve the relevance and governance of community colleges programs which have not evolved as envisaged under the program. Establishing the Dean's Council and implementing the MIS as well as equipping labs were positive developments. Developing pilot college programs under the SETVET Project will provide valuable lessons. Tracer studies as well as other studies on labor markets, employer and student attitudes were carried out and discussed with stakeholders and policy makers and provided guidance for the future. Enrollments in community colleges dropped however from approximately 29,000 to 25,500 and yet there is demand for technical skills which community colleges are not yet addressing systematically. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development (b) Institutional Change/Strengthening The project contributed to a change in the Jordanian higher education system by introducing new practices to promote a culture of quality through the introduction of the competitive fund, faculty development centers and the accreditation system. The development of the inter-library system and the system-wide ITC infrastructure also strengthened the institutional capacity for collaboration between institutions. This collaboration was based on joint committees, the use of memorandums of cooperation and joint financing of activities. Evaluations of the ITC, the competitive fund, faculty development as well as studies on governance, financing and community colleges strengthened the capacity to assess and adapt policies and programs introduced under the project. The National Forum held in February 2007 21 bringing stakeholders together to discuss the studies, and policy recommendations has helped to forge a consensus and formulate a national strategy which will provide the framework for the next project. (c) Other Unintended Outcomes and Impacts (positive or negative, if any) 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME Rating: Moderate The overall risk to attaining the development outcome is moderate. The Government has created the institutions that will allow it to continue to pursue reforms. It has recently expanded the autonomy and mandate of the accreditation council to include quality assurance. Faculty Development Centers will be upgraded to focus on quality assurance and have full-time staff. The Government is in the process of putting together a new operational strategy and is strengthening the capacity of the Ministry of Higher Education and Scientific Research to implement policy reforms and intends to upgrade the competitive fund under the next Bank-financed project to make it effectively competitive. The main risk to sustaining the achievement of development objectives remains the lack of a political consensus on a reform program. The project involved significant investment in ITC and specialized laboratory equipment. Special student computer fees provide maintenance for upkeep and improvement of computers and ITC. University budgets include contracts for maintenance, spare parts and technical support. 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory The Bank's performance to ensure quality at entry is rated moderately unsatisfactory. Introducing a major system-wide higher education reform was a relatively new area of policy reform for the Bank in the MENA region. The Bank team was highly skilled, experienced and had the necessary sector and country knowledge to provide solid policy advice to government on quality reforms and this is reflected in the introduction of competitive funds and other incentive mechanisms. The project was coherent with government policy and priorities to provide a quality education geared towards a knowledge economy and improving competencies and skills of students. It was also aligned with the CAS. The project, however, lacked a solid consensus by stakeholders on the strategic and policy framework. The project was not ready for implementation as was shown by the use of dated covenants as the means to implement reforms related to quality assurance and formula funding and the weak design of the competitive fund even though good examples existed in other countries. It would have been preferable to have planned out prior to appraisal how these reforms were to have been implemented. The Bank team did not take the opportunity to adjust the scope of the project when the Government decided to reduce the budget envelope by 20 percent and indicators were also not revised to reflect the new reality. The risks of lack of technical assistance were underestimated as was the importance of sufficient funding for the sector. In addition, using a project implementation unit located in an autonomous agency such as the NCHRD was not the most effective choice for a project intent on implementing major reforms. 22 (b) Quality of Supervision Rating: Moderately Satisfactory Project supervision by the Bank team is rated moderately satisfactory. Supervision missions were conducted on a regular basis with support from the country office in Lebanon and satisfactory support to strengthening procurement capacity and project management during supervision missions was extremely useful. The Bank team provided good policy support by providing technical assistance with international experience in accreditation, labor market issues and community colleges and project management for the HEDF as a means to make up for the Government's inability to find resources to replace the funds cut at negotiations. At the MTR actions were taken to adapt the project to institutional changes and to ensure that studies and evaluations were carried out to assess program and policy initiatives. Bank supervision contributed to the success of the ITC component, strengthening HEDF implementation and assisting the Government in undertaking the National Forum. Project status reports (PSRs) and implementation status reports (ISRs) were regularly updated. However, the Bank team did not focus enough on policy reforms and project outcomes. Indicators were not collected in the first part of the project. There were missed opportunities by the supervision team. The project was for the most part rated as Satisfactory in ISRs with regard to project objectives although progress was not being made in terms of accreditation of public universities, the introducing the formula funding mechanism, use of a competitive mechanism for the first two rounds of HEDF, and reforming community colleges. During the mid-tem review, an opportunity was lost to restructure the project to respond to these difficulties and to formally change project indicators. Indicators were revised informally but these, in the end, are not what the project is assessed against. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The overall performance of the Bank is rated moderately satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory The Government performance is rated as moderately satisfactory as the inability to develop a consensus around reforms to improve the transparency and efficiency in resource management, in particular for the formula funding mechanism, accreditation system and community colleges meant that the project could not attain its objectives. The formula funding mechanism was abandoned and the accreditation system was only partially implemented. The decision to place the PIU within the HEC or the MEHSR would have built the capacity of these institutions to manage the policy agenda. Frequent changes in institutional arrangements, as well as ministers and senior government staff, did not help consolidate the reform agenda for the sector. The Government did not take adequate measures to ensure that the PIU could be located so that it would have greater institutional clout to steer a complex higher education reform or that there was sufficient leadership to direct the reforms needed for the community college sector. 23 (b) Implementing Agency or Agencies Performance Rating: Satisfactory The implementation unit did an excellent job of managing procurement, project activities, financial management and monitoring once indicators had been agreed on. The staffing of the unit was modest given the volume of ICT and other procurement needed to establish the network, the IT Fund, FDC Fund and the library as well the coordination between institutions. The financial monitoring system was well managed and kept. Annual reports were of excellent quality and useful for supervision mission. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory Considering the satisfactory rating of the PIU and Government performance, the overall performance of the Borrower is rated moderately satisfactory. 6. LESSONS LEARNED Reforms were only partially implemented under the project. It is helpful to understand the factors that impeded the effective implementation of reforms. · From the Bank side, during the preparation of the project, the specific responsibilities and technical expertise needed to implement reforms were not clarified. Rather, the project used dated covenants which were not understood and never complied with. As well, the reforms for community college lacked a dynamic leader to tackle the many problems that existed and undermined the feasibility of implementing a comprehensive reform. Between 2000 and 2006, the higher education sector in Jordan was rapidly changing because of changing demographic trends, increasing demand, the emergence of a private sector, declining public resources for higher education and institutional changes. It was difficult for the Bank team to keep up with policy changes and keep reforms on track. This would have meant restructuring the project and simplifying it to focus on areas where there was a consensus to reform and formally revising indicators. Without adequate incentives to devote to intensive supervision and restructuring, it was easier to introduce cosmetic remedies. · From the Government's side, specific responsibilities and technical expertise were not assigned to implement reforms, for example to institute the formula funding mechanism and restructure community colleges and the technical expertise was not available. Decision-making in higher education on specific issues regarding admissions, financing, student fees and appointments were made by various entities including the Royal Court, the prime minister and the ministers of finance, plan and higher education and yet there was little formal coordination among these decision-makers. Such disparate centers of decision-making meant that it was difficult to maintain a consensus on the reform program. In this project a strong sector strategy endorsed by Government was missing and thus decision-makers lacked the common shared vision necessary to implement a sector strategy and related reforms. Frequent changes in ministers, the institutional set-up and senior appointments also undermined adherence to a sector strategy. Regarding implementation, the main issues that seemed to have undermined efficient implementation are as follows: 24 · Frequently in the MENA, for various reasons, Governments decide to downsize project financing at the time of negotiations. Task teams need then to carefully reexamine the project to decide if the scope of the project, development objectives, risks and indicators need to be reassessed. As well, during supervision, if the project is not making adequate progress or if indicators are found to be irrelevant, teams should not be reluctant to restructure or change indicators and obtain resources from management to carry out quality supervision. Management needs to provide incentives so that task teams will feel empowered to respond to changing circumstances. · The design of competitive funds needs extensive participation to succeed. The guidelines for the competitive fund were prepared by someone external to Jordan universities and they were never validated and disseminated widely. In order for such a vast reform aiming to create a culture of quality within all universities to be successful, the procedures manual needs to be designed, and validated by faculty and administrative staff within universities in addition to extensive training for academics to write proposals accordingly. This requires a lot of time and well coordinated efforts but this preparation will ensure a better buy-in which is a critical factor for success of the competitive fund. · For a competitive fund to add value it needs to be streamlined with government policies for development in higher education and broader economic priorities. Secondly, to be most effective a competitive fund needs to be headed by a well-known and respected academic who can manage the competitive process both technically and administratively in a transparent and efficient way. Political support to this reform champion is essential to implement the competitive scheme in a professional, transparent and unbiased manner. 7. COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING AGENCIES/PARTNERS (a) Borrower/implementing agencies No comments were received by the Borrower (b) Co-financiers There were no co-financiers. (c) Other partners and stakeholders No comments were received from UNICEF or the EU. 25 ANNEX 1: PROJECT COSTS AND FINANCING (a) Project Cost by Component (in USD Million equivalent) Appraisal Estimate Actual/Latest Components Percentage of (USD millions) Estimate (USD millions) Appraisal Component 1: Initiate Improvements in Quality, Relevance and Efficiency with support for 17.926 19.285 107.6 essential infrastructure Component 1: Higher Education Development Fund (HEDF) 34.066 17.139 50.3 Component 2: Initiate Improvements in Governance 1.911 0.004 0.2 Component 3: Reform of the Community College System 5.622 4.082 72.6 Component 4: Project Implementation Capacity 1.509 1.293 85.7 Total Baseline Cost 61.034 41.803 68.5 Physical Contingencies 2.555 0.00 0.00 Price Contingencies 2.187 0.00 0.00 Total Project Costs 65.776 41.803 Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.347 0.347 .00 Total Financing Required 66.123 42.150 63.7 (b) Financing Appraisal Actual/Latest Source of Funds Type of Co- Percentage of financing Estimate Estimate (USD millions)(USD millions) Appraisal Borrower 31.422 9.294 29.60 International Bank for Reconstruction and Development 34.70 32.957 95.0 26 ANNEX 2: OUTPUTS BY COMPONENT The project was to support the first phase of a long-term reform process. Its objective was to put in place the building bocks for improving quality, relevance and efficiency and to initiate governance reform. The project had four main components which aimed to: (i) initiate improvements in quality, relevance and efficiency through two sub-components: (a) system-wide support for essential education infrastructure (IT networks, MIS, libraries establishing FDCs in all universities and faculty training); and (b) a competitive Fund to finance university-specific sub-projects; (ii) initiate improvements in governance; (iii) support reform of the community college system to increase efficiency and relevance; and (iv) support project implementation capacity. Component 1: Initiate Improvements in Quality, Relevance and Efficiency Sub-component 1.1: System-wide support. The object of this sub-component aimed to improve the quality, relevance and efficiency of teaching and learning and administration in universities. The project was to establish an inter-university IT network infrastructure, enhanced computer proficiency, an integrated library system (ILS), an integrated higher education sector management information system linking universities to the Higher Education Council which was to improve decision-making and the establishment of FDCs and support an in-country system-wide training for faculty developers to foster quality teaching and learning. The project was to finance the installation of a robust, standards-based information technology infrastructure, a high speed network linking universities (and library and MIS systems), computer proficiency courses and equipment for computer classrooms, training of trainers and faculty training in teaching. The ITF was initially to have been included under the competitive fund but was transferred to this sub-component. As well, in 2002 it was decided to include activities to equip a GDLN. During negotiations, the TA that was proposed in the PAD was eliminated from the design of the management information systems (MIS), the Library, and the faculty development activities (Faculty Development Centers). At the time of the mid-term review, reallocation of resources from the unallocated category was reinstated some technical assistance for the MIS and library. This sub-component produced the following outputs. IT network. The project executed all planned activities for eight universities by 2005 and then expanded its scope to include two new universities. The project provided the installation of a full secure internal networking at all universities, inter-connectivity of all universities by a high-speed fiber-optic wide-area network (1 Gigabi/sec WAN), professional room-sized vide-conferencing facility in10 universities which are utilizing the high-speed WAN, installation of state of the art security equipment at all universities to secure LANs and to protect the inter- university WAN. Information Technology Fund (ITF) was established to finance student and faculty computers, locally networking to printers, files and internet services, educational technologies and end-user training necessary to operate and maintain the new ICT investments, once the IT network was set functional. Each university was to be allocated a budget envelop to for the ITF prepared a Strategic Investment Plan as part of its Annual Work which was approved by the NSC. The project provided and installed 70 work-group servers, 3090 personal computers (PCs) for students and 2451 PCs for faculty, printers, relevant software, and associated equipment for 125 computer laboratories in all universities. In addition to the above, computer centers at Tafeelah and Al Hussein universities were also equipped workgroup servers, cabinets, monitors and switches. An evaluation of the ITC was carried out showing that the utilization dimension is good; 70 percent of students use internet and email for course work and assignments; only 30 percent email the professor; about 75% of students use standard desktop applications. Public universities are progressing in building the information and communication technologies but the modern education al climate in the presence of IT is still not formed. It concluded saying that ICT is not yet fully utilized in education to a degree that matches the investment. 27 Inter-university library system A national library consisting of a consortium of university libraries and called the Jordanian Center of Excellence for public universities Library Services was set up at Yarmouk University. All planned activities were implemented and extended to the two new universities. The project financed an international library specialist, the design for the installation of the inter- university library system, four study tours for library officials to other Arab countries and one to England, training of more than 120 staff in ILS management, the installation and testing of equipment and software of the centralized library management systems which is serving the 10 universities including the design and installation of systems for shared cataloging, information retrieval and document delivery systems, the connectivity for all university libraries to external information systems, the development of a national database of bibliographic records from all universities (over 1 million documents) and subscriptions to online library content from three main providers (IEEE, ELSEVIER and EBSCO) making available over 20,000 online journals to all students and faculty to all universities. Global Development Learning Network Center (GDLN) The project financed the addition of the GDLN which is hosted by the University of Jordan. An inter-university agreement established a management committee, the organization and financing for the center. The project financed the equipment, servers and cabling for the center. The Australian Agency for International Development financed a technical assistance to develop a business plan and content for the center for course delivery. Management information systems (MIS) This component was not completed because its set up was not accomplished within the project timeframe but designs for the systems were completed under the project. The higher education development project financed the hiring of three international consultants who designed the functional requirements and technical specifications and bidding documents for MIS modules for the MHESR, and the university administrative systems: Student Information Systems Finance/Human Resources, and Assets Management. Faculty Development Center (FDC) Fund and Faculty training FDCs were established in all universities to modernize teaching, curricula development and research in improving learning. The FDC Fund was set up to finance professional development activities for faculty to enhance their academic performance, improve teaching, course and curricula research skills and promote the use of new and innovative teaching methods. Each university FDC prepared an annual work plan to implement faculty development activities. This work plan was then approved by the NSC and the purchasing of goods, materials and consultancies was centrally managed by the PIU. The project financed the purchase of 102 computers and 35 printers, scanners, projectors, optical readers and photocopiers for 8 FDCs. A European Union project financed two international consultants who designed training modules on needs assessment, experiential learning and staff assessments and delivered 20 days of training to 54 faculty development specialists. Universities financed 146 faculty development activities carried out by FDCS for 3443 staff primarily on classroom instruction, methods of teaching, and instructional media, and evaluation and 12,500 teacher evaluations were carried out between 2004 and 2006 in all universities. An assessment of the FDCs indicated that about 20 percent of faculty has participated in development activities and that there was still resistance to training and to student evaluations. The effectiveness of FDCs had been undermined by the lack of a full-time director, frequent changes in director, no full time technical staff and a focus on new equipment and buildings rather than course development. Sub-component 1.2 Higher Education Development Fund (HEDF): The objective of the HEDF was to support improvements in the quality and relevance of higher education and to align higher education given by Jordanian public universities with the needs of a market driven economy. The Fund was initially designed to provide grants through three windows: (i) competitive grants; (ii) a Faculty Development Fund; and (iii) an Information Technology Fund. At the time of negotiations, it was decided to include the FDC and IT Funds under sub-component 1.1. Competitive grants to universities were meant to support the development or introduction of reformed curricula, entrepreneurial projects 28 with private sector enterprises, interdisciplinary programs, phasing out obsolete programs and consolidating programs across universities. The project planned three rounds of the competitive grants. The selection process was detailed in a procedures manual which outlined how proposals were designed, selected, implemented and monitored. The project planned to finance advisory and technical services, contract research, comprehensive service centers, training, incubator services, and multi-disciplinary centers During the course of the project, three rounds of funding occurred. The first round allocated funds to 7 public universities and in the second round universities were invited to submit proposals that were evaluated and awarded on an intra-university basis. Building on an external evaluation of the first two rounds, project proposal guidelines were revised during the third round and training on proposal writing given to 30 submitters. The third round was the most competitive round. The project financed in total 33 grants creating new or upgrading undergraduate and graduate programs and 7 entrepreneurial projects with private sector industries. The project provided the following inputs: · Revised and upgraded 13 undergraduate programs and developed 11 new undergraduate, 13 Masters programs and 2 PhD programs in fields relevant to the development needs of Jordan. Programs fields included engineering (chemical, mechanical, materials), applied sciences, water resources and environment, economics, applied geography, cultural geography, occupational therapy and neuroscience, veterinarian medicine, pharmaceutical discovery, medical skills, biotechnology and biomedical physics. Approximately 5,840 undergraduate students benefited from improved and more relevant courses. Approximately 2150 students entered new undergraduate programs. In addition about 200 students entered MSc programs. · Created 10 new institutes or departments including the Multi-disciplinary Institute for Water Research and Environmental Studies, the Centre for experiential and non-destructive evaluation, a Computing Centre, the Medical Skills Centre, the Academic Entrepreneurship Centre of Excellence, the Jordanian Design Centre and the Jordanian Society for Medical Physics. · Purchased equipment for 70 laboratories in various fields including mechanical engineering, materials, chemical engineering, biotechnology, soil mechanic, concrete technology, immunology, medical lab, water management, water testing, neuroscience, archeological , clinical attesting labs, micro elemental, parallel pressing lab, medical skills, immunology, molecular genetics, microbiology, product design, various simulation labs, immunology, product design, medical imagery, biomedical spectroscopy lab, disease investigation and computer labs. For the vast majority of laboratories highly technical equipment was acquired as well as over 200 personal computers, scanners, printers, cables, servers, photocopiers, audio visual equipment and software (SSPS, CERIUS, MOODLE, management software) · Regarding training, 500 persons received short term training, 25 workshops were organized in Jordan, longer term training or study tours outside of Jordan benefited 87 faculty and technicians and 8 international conferences were organized on semi conductors and polymers, material science, tourism, ordered statistical data, preserving and managing Jordan's cultural history and design management. · Other fund outputs included the publication of four books, 36 articles published, the purchase of books, CD ROMS for libraries, subscriptions to databases, eight international consultancies to assist in program design and other highly technical expertise to provide technical training and knowledge sharing, involvement and stronger links with the private sector and other organizations, particularly in the pharmaceutical, medical, design, cultural heritage, stock markets, handicap and charity organizations. 29 Two evaluations of the HEDF were conducted. It was found that the competitive process was not used in the first two rounds, most of effort was in securing equipment, market and feasibility studies were not carried out, a large proportion (over 30 percent) of resources was used for research and post graduate programs which were not the program's priority. In addition, the selected projects did not respond to clear priorities of quality and relevance in undergraduate teaching as there was a lack of clarity about the strategic objectives of the fund as reflected in the operational manual. Over 30 percent of sub-projects were for research and graduate programs which were not the stated priority of the HEDF. The first two rounds of the Higher Education Development Fund were allocated on a non-competitive basis. Component 2: Initiate Improvements in Governance (Project US$2 million of which Bank portion was US$0.3 million) This component aimed to strengthen the Government's institutional capacity to implement the needed governance reform, and the universities' planning and management capacity. The project was to provide technical assistance, equipment and training. During negotiations, the TA support was removed from this component. At the time of the MTR, it was decided to reallocate un-disbursed funds to provide technical assistance and to finance studies and activities to assist the Ministry to develop a sector strategy. Subcomponent 2.1 Higher Education Council Secretariat (HEC): Initially, the HEC secretariat had no technical mandate or skill to inform and advise the HEC on policy matters. This component was to support the development of the technical capacity of the HEC in policy formulation by providing TA to: (i) help set up the analytical functions of the HEC Secretariat, including the Funding Unit, a Policy Advisory Unit, and a Statistical Unit; (ii) provide equipment to support these functions; (iii) design student loan and scholarship schemes to ensure the financial sustainability of the sector, while improving access and equity; and (iv) provide local and overseas training to staff . Early implementation was to focus on setting-up the Higher Education Funding Unit within the HEC Secretariat. The Unit would be responsible for maintaining the recurrent funding formula, collecting the agreed information required for the model, identifying and evaluating various funding arrangements and their effects on universities, and managing the consultative and evaluative process with the universities themselves. The impact of the new allocation arrangements on the efficiency of universities will be a key feature of process evaluation. This component was not implemented as the technical assistance planned at appraisal was eliminated at negotiations. Additional activities introduced following the mid-term review After the MTR, the Government decided to develop a strategy to reform the higher education system in the following areas: (i) encourage a greater role for the private sector; (ii) sustainable financing; (iii) introduce quality assurance and accreditation measures into the various levels of higher education; and (iv) encourage scientific research. The project financed eight studies and evaluations on the major policy issues and to evaluate the specific initiatives sponsored by the project (IT, FDCs, HEDF). These studies are as follows: (i) Evaluating Higher Education Development Fund; (ii) Community Colleges Assessment Study - Further Development of the Community College System in Jordan; (iii) Evaluation of the Faculty Development Centers at the Eight Public Universities; (iv) Formative Monitoring Evaluation of the Governance in Higher Education; 30 (v) Evaluation of ICT in Higher Education at the Eight Public Universities; (vi) Higher Education Management Information Systems; (vii) Student Information Systems, Human Resources and Asset Management; and (viii) An Evaluation and Assessment of Rounds 2-3 sub-projects implemented under the Higher Education Development Fund. A two-day Forum bringing together over 100 policymakers and stakeholders to discuss these reports and others on financing and governance was organized in February 2007. The results of the forum provided material for the development of the new strategy and the next Bank project. Sub-component 2.2 Higher Education Accreditation Council (HEAC) was created in 1996 and approved in 1998 by Parliament to promote the process of accreditation and evaluation of public and private universities and of their programs of study. The project was to provide technical assistance set up the operational aspects of the HEAC, develop methodologies for external and self-evaluation based on international experience in university evaluation and accreditation of study programs. Other activities to be carried out included: (i) the designation of the members of the HEA; (ii) the establishment of disciplinary committees; (iii) the publication and the circulation of procedures to universities and professors; and (iv) the preparation of an evaluation calendar for universities and programs. The HEAC became operational in 2001; accreditation standards were developed and documented in 2002; 185 accreditation assessors were trained and 59 accreditation committees were formed and implementing accreditation procedures. All new programs for seventeen private universities were assessed, underwent accreditation process and applied recommendations. Assessment processes for public universities were not completed and accreditation standards were limited to inputs. WHY can the HEAC not implement its mandate in line with international best practice in accreditation? A consultant was financed from a Bank Trust Fund to help the HEAC to set up its management and finance systems, a higher education institutions monitoring and control system and institution accreditation standards for private universities and colleges, community colleges, higher studies, distant learning and evening studies. Four workshops were held to design the accreditation formats and approaches, policy directions, and guidelines for accreditation of undergraduate programs in engineering and on the implementation and application of specialized accreditation standards. Sub component 2.3 Improved Planning and Management Capacity at the University Level. The project was to finance training, TA and equipment to bring all planning and management staff and process in public universities in line with the upgraded practices at the national level, and consistent with the new accountability of these institutions. Training of university managers, administrators and selected faculty will include the development of management and financial management, and the strengthening of relations with the private sector. UNESCO organized training for 30 staff and short-term workshops were provided in management and financial management. A five-day workshop was organized on improving the utilization of resources and the capacity to mobilize additional resources in higher education. 31 Component 3: Reform of the Community College System/Al-Balqa Applied University The GOJ established Al-Balqa Applied University to be responsible for the academic, management and administrative affairs of all community colleges. This project was designed to support reforms grouped in four sub-components: (i) governance and management; (ii) program development; (iii) human resources development; and (iv) facility and equipment upgrading. At the time of negotiations, the Government that the Bank project would finance sub-component (iv) and that sub-components (i), (ii) and (iii) would be financed from the Canadian-Jordanian Project (SETVET). Sub-component 3.1 Governance and management: This sub-component was to support: (i) TA for the office of the BAU President by providing a seasoned education administrator to assist in developing the organizational structure, working practices, planning procedures and preparing a policy and procedures manual; (ii) TA for a systems analyst to assist BAU with an assessment of management information products necessary for an efficient management of BAU and its associated community college system; and (iii) acquisition of hardware and software to develop a financial, human resources and student management information system. Fellowships were to be provided for the BAU System Analyst and local training for MIS users. CIDA financed an organizational study in 2003 and recommendations were implemented including establishing planning and program development units at BAU and setting up a Dean's Council. Subsequently, Tracer studies were conducted in 2005 as well as a labor market analysis, college-based interview reports and employer-based interview report. The management information needs study was financed by the CIDA project. The organizational structure design and policies and procedure were prepared but not adopted or used. Sub-component 3.2 Program Development: BAU has committed itself to a process of program renewal eliminating existing terminal academic programs or entry level vocational programs and introduce new employment driven streams at the Associate Degree level in the fields of technology, business and finance, natural resources and applied social sciences. The sub-component was to finance: (i) fellowships for staff in the Program Development Unit (PDU) to manage the program renewal process; (ii) five PDU staff were to be given fellowships to establish competency based curriculum; and (iii) TA to establish procedures for the unit and guide the development process for the first year of the new Associate Degree Programs. Four pilot programs were developed for four community colleges beginning in school year 2006/7. Sub-component 3.3 Human Resources Development (HRD): Reform of the community college system was to involve recruiting and training teaching and non-teaching staff at all levels. This sub-component was to support three mechanisms set up by BAU to strengthen its teaching staff: (i) adoption of lead instructors for sector and sub-sector programs making up the Associate Degree program framework. Fellowships were to be provided to lead instructors for professional and pedagogic upgrading and short term secondments to business or industry; (ii) in-country professional development workshops for staff in topics relevant to their area of instruction; (iii) technical assistance for expert counterpart instructors in specialized areas relevant to the new Associate Degree programs. Activities were carried out by CIDA. 3.4 Faculty and Equipment Upgrade: The planned BAU Associate Degree program was to build technical skills and competencies on a foundation of numeracy, literacy, communications, computer applications, and English comprehension. The sub-component was to finance computer, science and language and specialized labs as well as equipment, books and audio visual equipment. The project financed equipment for computer, language and science labs and MIS infrastructure (hardware and software) for the development of the community colleges management information system. 32 Technical specifications for the procurement of the required MIS originally were prepared under SETVET technical assistance and were redesigned by the HEDP. HEDP financed the equipment included: 90 database and applications servers, switches, cabinets, 145 computers, 15 laser printers, network cabling, 40 days of training for staff and software. The implementation of the MIS for the college system was completed in 2006 and operational in 2007. Component 4: Project Implementation Capacity The component was to support project implementation capacity and financed the necessary expenditures including from government resources: · staffing of the PIU (Project Manager, Fund Coordinator, IT coordinator, Procurement Office, Financial Officer; · administrative costs (peer reviewers, two evaluation conferences and expenses related to evaluation activities); · incremental staff at university UIU (Remuneration for UIU's was the responsibility of the universities); and · Support to NSC, IT/MIS committee, Library Committee and ad hoc advisory committees. 33 ANNEX 3: ECONOMIC AND FINANCIAL ANALYSIS The economic analysis conducted for the project looked at: (i) rationale for a project supporting the public sector; (ii) rationale for supporting the proposed project; (iii) economic efficiency gains of public sector spending and cost-effectiveness analysis for the different investment choices made in the project; and (iv) a financial sustainability and fiscal impact analysis. This annex assesses the economic analysis that was undertaken for the project. Background documents for the internal rate of return analysis were not available as these documents were locked in buildings in Lebanon due to the civil strife in this country. Recent social and economic development In terms of human development indicators, Jordan ranks above average with respect to lower middle-income countries. The incidence of poverty declined from 14.0 percent in 2000 to 7.0 percent in 2006. Education indicators have improved consistently since the mid-nineties; the illiteracy rate is 8.9 percent, the third lowest illiteracy rate in the Arab world; the primary gross enrolment ratio has increased from 71 percent in 1994 to the 98.2 percent in 2006; the transition rate to secondary school has increased from 63 percent to 97 percent over the same period; and transition rates to higher education has varied between 79 percent and 85 percent of secondary school graduates over the past five years. The university gross enrolment rate for the population between 18 and 25 years for university has increased from 18.5 percent in 2001 to 22 percent in 2006, among the highest in the Arab world. In 2005, Jordan allocated about 15 percent of public expenditures to education, representing about 4.5 percent of GDP. Higher Education receives 14.7 percent of education expenditures or equivalent to 0.65 of GDP which is low when compared to the OECD average of 1.6 percent of GDP or to other middle-income countries. Public financing for higher education has been declining over the past five years. Transfers to universities have declined from JD 60.4 million in 2004 to JD 52.6 million in 2007 representing a decline of 14 percent. Rational for supporting public higher education sector: The project was to support investments to transform the role of the state in higher education to one of policy setting. The introduction of an accreditation system would provide transparent information to students and families on both the private and public universities thereby creating incentives to improve the efficiency of the system. The analysis argued that working with the private sector would be costly at the community college level because the kinds of programs that were needed by enterprises were offered in public community colleges only and financing these technical programs in private community colleges would be expensive and unaffordable by students. At the university level, it was argued that due to the autonomy of public universities, resource allocation reforms would involve considerable negotiations between universities and the Higher Education Council and that including private universities would complicate matters. As well most students went to public universities and thus the project would benefit over 70 percent of students. The rational for supporting universities was valid; however, it assumed that the incentives for private and public universities to adhere to accreditation. It was assumed that the Government would proceed to apply accreditation standards to both public and private schools. This was not the case, and only private universities adhered to accreditation standards. There were no financial or other incentives for public universities as overall public funding was declining and they could not meet the same standards as private universities in terms of teacher student and equipment ratios. Retrospectively, the economic analysis should have examined the implications of rapidly growing demand and financing for the private sector including: (i) the incentives for public universities to meet accreditation standards; (ii) the ability of public universities to raise fees; and (iii) proposed linking the competitive fund to meeting accreditation standards. Least Cost analysis: The analysis examined the implementation options for each of the main inter- university and system wide investments (information technology, management information systems, library system, faculty development centers and the Higher Education Development Fund). This analysis adequately outlines the different options for each investment indicated the critical issues that needed to be addressed to 34 ensure success. The exception was regarding the MIS, where the different options were outlined but there was no conclusion or preferred option and by the end of the project the MIS was never installed. Implementing these system innovations required a lot of coordination between universities and commitment of staff in universities to design and implement the components. The analysis did not include the costs associated with coordinating each component amongst universities. At the end of the project, the introduction of the inter-university library system was the most innovative and successful component as it had the best coordination and leadership which resulted in cost saving of over US$1 million a year and more journals had each university developed its own electronic library system. The MIS had the least effective coordination and could not overcome resistance to change in universities. Cost Benefit Analysis The analysis calculated internal rates of return (IRR) based on the assumption that graduates under the project (beginning 4 years after the project) would have higher productivity and higher wages. The analysis showed that with increased wages of only US$5 per month, the internal rate of return would be 10 percent. It was argued that IRR would be higher for community colleges as employers are expected to offer substantially more job opportunities for the skills of students coming out of community colleges. Given that the project activities were aiming to improve the quality and relevance of education for students, the costs were relatively low when compared to the economy wide benefits. Financial sustainability and fiscal impact: The economic analysis posed the question whether limited public resource could sustain the projected levels of enrollment growth. The analysis based its projections on a 5% annual growth rate of enrollments in public universities. The analysis questioned whether these assumptions were realistic. Policy scenarios were generated showing that if enrollments were to increase by 5% per year, the financing needs would be substantial requiring a 10 per cent increase in public financing, 25 percent real increase in tuition and efficiency gains of 10 percent. Figure 1 : The Tawjeehi Exam 80,000 70,000 60,000 50,000 40,000 e 30,000 20,000 Students sitting the Tawjihi exam Students passed the Tawjihi exam 10,000 0 2002 2003 2004 2005 2006 Enrollments in fact grew rapidly at an average annual rate of 14 per cent and the 5 percent projections were probably unrealistic as the questions posed imply. Enrollments between 2000 and 2005 almost doubled between 2000 and 2005 from 77,841 to 152,400 students and the number of public universities has increased from 7 to 10 (see Figure 2). Over the same period, enrollments in 12 private universities grew by about 18 percent annually from 36,642 to 55,744. Given that primary and secondary school completion rates had been increasing, the numbers of students taking the Tawjeehi (end of secondary exam) increased sharply between 1998 and 2000 and success rates in the Tawjeehi were improving, as is shown in Figure 1, it would have been 35 more realistic to useful to look at various financing scenarios other than the 5 percent increase in enrollments. As well, the implications for financing needs for universities and the impact on quality if financing could have been drawn out. In this sense, the economic analysis did not adequately guide the project design. The risks associated with a higher than 5 percent annual enrollment increases and tight budget situation were not introduced in the risks and mitigation strategies for the project. The analysis assumed that enrollments would follow funding, but in fact the institutional set-up was one where admissions policies for universities were independent of funding decisions. The project also aimed to establish a transparent funding mechanism to allocate recurrent resources to university but the economic analysis did not provide guidance on the level of financing needed, and how this funding mechanism would be managed. Expenditure analysis and efficiency In 2005, Jordan allocated about 15 percent of public expenditures to education, representing about 4.5 percent of GDP. Higher Education receives 14.7 percent of education expenditures or equivalent to 0.65 of GDP which is low when compared to the OECD average of 1.6 percent of GDP or to other middle-income countries. Public financing for higher education has been declining over the past five years. Transfers to universities have declined from JD 60.4 million in 2004 to JD 52.6 million in 2007 representing a decline of 14 percent. Figure 2 Total Student Body Universities Total enrollments in Universities and Public Private Total colleges Universities Universities Undergraduate 138,941 53,101 192,042 Graduate 13,489 2,643 16,132 Total HEIs 152,430 55,744 208,174 Colleges 12,500 14,200 27,700 As a result of declining Government transfers, the rapidly increasing student body and a cap on raising student fees, universities introduced parallel fee-paying programs in the early 2000's. Regular tuition fees, for students admitted based solely on grades in the high school exit exam (Tawjeehi), are on average JD 1,000 for tuition in public universities. Students admitted under the parallel system pay on average JD 2500 per year. In 2001, student fees accounted for about 64 percent of university operating costs and government transfers provided 36 percent. By 2005, student fees for both regular and parallel programs covered 78.2 percent of operating costs. Universities financial situation during this period has worsened as numerous have accumulated arrears on loans that they are unable to pay (totaling to date over US$100 million). Student faculty ratios have worsened from 23 in 2000 to 31 in 2004 and 38 in 2005 in public universities. The same trend is occurring in private universities with ratios going from 23 to 25 between 2000 and 2005. Universities have not reduced its non-teaching and administrative staff to absorb expenditure cuts as the administration staff to faculty ratio in public universities has worsened from 2.7:1 to 2.8:1 over the same period indicating a poor track record in improving the allocation of resources to improve the quality of education. The fiscal and financial sustainability analysis should have included an analysis of the trends in financial transfers and tuition fees to university, and the expectations for financing for the medium term. This would have permitted an assessment of the Government's capacity to ensure that government financing would increase in real terms on per student basis, that there was political commitment to increase student fees and agree on measures for efficiency gains. 36 ANNEX 4: BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES (a) Task Team members Names Title Unit Responsibility/ Specialty Lending Francis Steier Senior Education Economist MNSHD Task Team Leader Bassam Ramadan Senior Operations Officer MNSHD Institutional Arrangements Christian Rey Procurement Specialist MNSHD Procurement Nabil Shehadeh Financial Management Specialist MNAVP Financial Management Ghassan N. Alkhoja Senior Information Officer SDNIS ITC Al-Sheikh Counsel LEGAF Lawyer Dumg-Lim Pham ̉perations Officer MNSHD Costing Ambrose Gardiner Disbursement Officer LOALEF Disbursement Christine Djemmal Operations Analyst MNSHD Operations Alenoush Saroyan Consultant MNSHD Faculty Development Centers Robert Davies Consultant MNSHD Library System Arthur Hauptman Consultant MNSHD Financing Higher Education Johan Aitkins Consultant MNSHD Budget Management John Bislec Consultant MNSHD University Management Jean-Francois Desnos Consultant MNSHD Management Information Systems Roger Pearson Consultant MNSHD Community Colleges Quentin Thompson Consultant MNSHD Competitive Funds Supervision/ICR Afifa Alia Achsien Senior Program Assistant MNSHD Administration Ghassan N. Alkhoja Senior Information Officer SDNIS ITC Mouna Couzi Senior Program Assistant MNCLB-LVL Administration Ramadan Basam Senior Operations Officer MNSHD Task Team Leader Robert Bouo Jaoude Senior Procurement Officer MNSHD Procurement Christina W. Djemmal Operations Analyst MNSHD Operations Diana C. El Masri Financial Management Specialist MNAFM FMS Adriana Jaramillo Senior Education Specialist MNSHD Task Team Leader Mona El-Chami Financial Management Specialist MNAFM FMS Haithan Omar Procurement Officer MNAPR Procurement Lina Fares Procurement Specialist MNAPR Procurement J. Roger Pearson Consultant MNSHD Community Colleges Imad Saleh Lead Procurement Specialist EAPCO Procurement Haneen Ismail Sayed Senior Operations Officer MNSHD Task Team Leader Mona Taji Consultant MNSHD Higher Education Higher Education Jamil Salmi Lead Higher Educaiton Specialist HDNP Management and Financing 37 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of staff USD Thousands (including weeks travel and consultant costs) Lending FY00 13 37.87 FY01 7 13.85 FY02 6.18 FY03 0.00 FY04 0.00 FY05 0.38 FY06 0.00 FY07 0.00 FY08 0.00 Total: 20 58.28 Supervision/ICR FY00 0.00 FY01 7 39.39 FY02 8 69.01 FY03 9 67.14 FY04 13 69.11 FY05 22 105.60 FY06 24 99.90 FY07 31 152.27 FY08 7 32.70 Total: 121 635.12 38 ANNEX 5: BENEFICIARY SURVEY RESULTS NA 39 ANNEX 6: STAKEHOLDER WORKSHOP REPORT AND RESULTS NA 40 Annex 7: Summary of Borrower's ICR and/or Comments on Draft ICR The Government prepared a project completion report. This Annex provides a summary of the completion report and focuses on the success of the HEDP: the challenges and limitations and an assessment of the Bank's and Government's performance. Successes HEDP has met its development objectives to initiate improvements in the quality, relevance and efficiency of the higher education, and to support Jordan's program to reform sector governance. During the life time of the project, considerable achievements in reforming sector governance were reached including: (i) reestablishing the Ministry of Higher Education and Scientific Research; (ii) defining the mandate of the Higher Education Council; (iii) making the Higher Education Accreditation a functioning body by establishing the procedures and operations for accreditation and then creating an expanded and more autonomous Higher Education Accreditation and Quality Assurance Commission; (iv) Legislative approval of a Research Fund; and (v) holding a National Higher Education Development Forum for policy-makers and stakeholders. . HEDP contributions to the development of the sector included: · Completed the establishment of a wide area of local networks for all universities (including Tafeelah and German Jordan Universities not included in project plans) which provided all university faculty and students with interconnectivity and high speed internet access. State of the art security equipment was installed, provided to all universities a secured LANs and to protect the inter- university WAN. By this, the information security goal was achieved. · Provision and installation of work group servers, personal computers, printers, relevant software, and other associated equipment for public universities was implemented in two phases and completed in 2005; 3,090 students PCs and 2,451 faculty PCs were provided and equipment to establish 125 computer labs. The universities have exceeded planned student and faculty/ computer ratios. As well, computer centers at Tafeelah and Al Hussein universities were also equipped. · All public universities were provided with video-conferencing facilities. The video conference rooms were also provided with the furniture suitable for the equipment. · A consortium for managing the network of public Jordanian universities was established and staffed April 2004 under the name of the Jordanian Center of Excellence for public universities Library Services located at Yarmouk University. The project financed the equipment and software for the integrated library system allowing for a consolidated library catalogue available at all universities and in Jordanian public school libraries. Bibliographic records from all universities were migrated to the new database (1 million +) and subscriptions to online libraries making over 20,000 online journals available to all students and faculty to all universities. · Established and equipped faculty development centers (FDCs) in all public universities. Universities appointed and trained faculty development specialists, implemented activities to improve the quality of faculty teaching and research in universities. Training undertaking a series of workshops for faculty developers were conducted, workshops focused. A study has been conducted to evaluate the experience of those FDCs. · Created a Global Development Learning Network Center at the University of Jordan. The center is positioning itself as a focal point for dissemination of development knowledge into Jordan, also revenue generations, through marketing of services and seminars. 41 · Introduced and implemented a competitive funding mechanism called the Higher Education Development Fund (HEDF) which supported improvements in the quality and relevance of programs at Jordanian public universities. Three rounds of funding have taken place. During the first round, one project was awarded to each public university on a non-competitive basis. During the second round universities were invited to submit multiple proposals that were evaluated and awarded on an intra-university basis. Building on lessons learned from the foregoing rounds, new proposal criteria were developed, workshops delivered, and guidance given to proponents regarding project design. · During the course of the 3 rounds, contracts were awarded for a total of 33 program development sub-projects and 8 entrepreneurial sub-projects. All projects have been completed. The objective of the HEDF has been met to support improvements in the quality and relevance of higher education. · Strengthened the technical, statistical and analytic capacity of the Higher Education Council (HEC) Secretariat and established the Directorate of Computers, Information, and Statistics (DCIS) in the Ministry of Higher Education and Scientific Research. · Established the Higher Education Accreditation Council by putting in place the organizational framework, management and finance systems, a higher education institutions monitoring and control system and the accreditation standards for private universities and colleges, community colleges, higher studies, distant learning and evening studies. Committees were formed and trained to implement the institutional accreditation standards at universities and committees to follow up applying of the specialized accreditation standards. · Improved the planning and management capacities at universities through training and the purchase of equipment and software applications · Established the planning and program development units, the Dean's Council for Community Colleges, and an MIS at BAU. · The equipment and the software for the community college's MIS has been installed and completed early Jan 2007. · Equipment for computer, language and science labs for community colleges and in particular the necessary equipment to teach the revised instructional programs. Difficulties and challenges: Some key performance indicators were not met. Among these are the establishment of a Management Information System (MIS) and improved relevance of community college programs. Despite the many efforts and investments on IT infrastructure, and the attempts to build the MIS, its set up was not accomplished within the project timeframe but designs for the system have been completed under the project. The following is a brief chronology for the MIS: · At the start of the project, the universities convened a committee of IT specialists to draft requirements and specifications for the MIS. This committee spent about 3 years trying to design the system, but did not reach an effective solution. · Subsequently, the project attempted to recruit an international firm to design the system. The recruitment process took about 1 year (and was impacted by the Iraq conflict in March/April 2003). Negotiations failed with the one responsive firm. · Another year was spent in analyzing and building another plan. The new plan involved recruiting individual consultants to design separate modules of the MIS (one consultant for the MoHESR 42 system, one consultant for student information system, and one consultant for the other administrative systems). · In the final stage, the consultants produced full design for each of the systems; however, there wasn't enough time to implement the systems. MOHESR and the universities now have full design (functional requirements and technical specifications) for all systems, and are ready to issue tenders should the ministry/universities be able to make funds available. Although HEDP made important contributions to increase the relevance, and quality of university programs and funded projects that contributed to link universities with industry and labor market needs, there are important challenges ahead for the sector. The unemployment rates for university graduates increased during the last 7 years, from 13 percent to 21 percent, many externalities are to be considered regarding this trend in unemployment, such as the overall economic growth factors, private sector investment and other important macroeconomic factors. This issue can be investigated later on. It is worth noting that unemployment rate for community colleges graduates decreased from 13 percent to 11 percent. Both universities and private sector enterprises are trying to build institutional mechanisms to enhance the relevancy of higher education to the needs of the labor market, but still this issue needs more effort to reach good results. Concerns regarding the directions being taken by the community college system under BAU mandate have been expressed at senior levels of government. The Bank provided the services of a consultant to prepare a white paper for consideration by HEC and the cabinet addressing this issue, the paper was received in March 2005 and presented to a workshop chaired by H.E. the Minister of HESR. Representatives from MHESR, MOP, NCHRD, BAU, SETVET, HEDP and the Bank discussed the findings and expressed their appreciation and support. While there are valuable assets upon which to build, there remain a number of systemic gaps in the development of an effective community college system in Jordan. These include: (i) a need for increased specificity of policy objectives; (ii) a need for greater clarity and accountability in the governance model; (iii) a need to implement a prioritized strategic development plan; (iv) A need to establish much stronger relationships with the employer community; (v) A need to devolve a greater level of autonomy to the individual college level; (vi) A need to realign the program framework more closely with the structure of the labor market in the country and the region; and (vii) To progressively realign the human resources profile of the instructional staff to include greater depth of professional practitioner experience. The institutional arrangements provided by NCHRD, WB, and the universities ensured greater transparency in the allocation of resources and better accountability in the delivery of project outputs, and made certain that all stakeholders work in harmony to fulfill their complimentary roles. Committees were established to ensure that universities were involved. These included: · The Project Implementation Advisory Committee (PIAC) which included Vice-presidents of beneficiary universities. This Committee in coordination with the National Steering Committee, ensured greater transparency in the allocation of resources and better accountability in the delivery of project outputs, and the work of stakeholders was in harmony for the fulfillment of their complimentary role. · Library Committee composed of the Heads of the Libraries of the universities, is charged with follow-up and coordination for the library system, and was assisted by the PIU IT Coordinator. The Library Committee developed the initial implementation schedule, defined the required systems for the integrated library system and the generic functional specifications for a common library system. The committee was assisted by a international library specialist to finalize the options analysis, functional specifications, and the final detailed implementation plan. 43 · Jordan Information Technology Committee (JITC). The Jordan Information Technology Committee was formed as an official body entrusted with the successful implementation of the IT and MIS sub-components of the project. The JITC was composed of the computer centre directors of each public university. Its work was to: assure the day-to- day responsibility for the successful implementation of the IT and MIS sub-component, Set the overall technical direction for the IT and MIS sub-component, Prepare and validate, for final approval by the NSC, the annual work plans for each university. Prepare progress reports and all sub-component procurement documents according to national and/or World Bank guidelines, and Liaise with NSC and commission ad-hoc committees to ensure successful inclusion of sub-component beneficiaries. Several evaluation studies were conducted to measure the achievements of the project, and to what extent the project activities were relevant, efficient, and effective. These studies indicated positive results and will be taken into account for future projects. Government and World Bank Performance Government Performance and project implementation: Overall, the Government performance was very good, cooperated with the visiting WB supervision missions, provided competent PIU staff on time, proposed, approved and implemented major reform activities in the field of higher education. Performance of the NSC was satisfactory. The following are observations and comments on the GOJ performance: · The project was given full and strong support from his Majesty king Abdullah II who directed the government through two royal messages to the Prime-Minister in order to act efficiently for the improvement of higher education. The Government encouraged all stakeholders to conduct studies and surveys for the evaluation of the project. The government established and strengthened the institutions given the mandate of setting up HE policies, strategies, quality assurance, and accreditation. The degree of commitment was very high. The National Centre for Human Resources Development was a good relevant institution to host and manage the project. · The project preparation was done by a national counterpart team (NCT) established by the MOP and assisted by a policy and human resources development grant (PHRD). Senior representatives from all universities and the MOP participated in the preparation process assisted by NCHRD staff. This representation was one of the advantages of having a demand-driven project. The objectives of this project became familiar to all stakeholders, and the sub-components were selected in accordance with the real needs of higher education. · The Project Implementation Unit (PIU) was provided with qualified, competent and dedicated staff. They were able to apply the World Bank and GOJ regulations, procedures and contributed to the success of the project. Progress reports, financial reports were submitted regularly as requested. Areas to strengthen: The Government did not include the private higher education sector to benefit from the project, even on the basis of assuming the costs by the private sector. Excluding this sector has negative effects that need to be evaluated. One factor affecting implementation of the project is the modest capacity at the university level to implement the sub-projects of the HEDF, and especially in dealing with Bank Procurement Guidelines. Regarding procurement management at universities, technical, delivery and acceptance committees were not solely devoted to evaluating and awarding bids and approving goods as committee members worked 44 on ad-hoc basis. An incentive system for these committees might help accelerate the lengthy tendering process. A database detailing the previous conduct of contractors and equipment providers including history pertinent to liquidated damages has to be up-dated. Only qualified contractors with good performance history should be invited to bids. World Bank performance: Overall, Bank performance was considered satisfactory. The identification, preparation and appraisal took sufficient time and resulted in a reasonable set of activities with low implementation risks. During project preparation, the Bank team held a dialogue with the Government and its preparation team (NCT) on possible elements of a long-term vision for the sector, and the role of the project in that vision. The complexity of the project and presence of more than one institution governed by different regulations were taken into consideration by the WB when preparing the project. The number of WB missions was adequate the stable structure of the supervisory team was considered effective and positively affected the implementation of the project. The WB missions were of great support to the project, their advice to the PIU, NCHRD, and stakeholders were very important in implementing project activities. Value added of Bank support: Bank support made value added in the following areas: (i) analysis of cross-sector labor market issues; (ii) world-wide perspective on analysis of higher education issues; (iii) Specific analysis of Jordanian higher education issues, as demonstrated in the 1996 sector report; (iv) Comprehensive sector perspective resulting from 10 projects in Jordan's education and training sectors since 1975; (v) Translating policy and technical analysis into operational recommendations; and (vi) the major donor involved in HE reform in Jordan. The Bank's monitoring supervisory missions provided great help and guidance towards successful implementation. Training of PIU staff members as well as other stakeholders' staff on WB procedures and regulations on management, procurement, accounting and financial management was highly appreciated by all concerned parties. Training had been considered crucial to all successful project achievements. The project included various studies to be conducted, those studies gave excellent analysis about the activities done, and can be used for many improvements to be introduced for future projects. Establishment of special tendering committee, liberated from the routine bureaucratic procedures, reduced much of the encountered delays. Bank policies and guidelines have been disseminated, and relevant information continued to be disseminated during supervision missions. Bank's standard bidding documents and/or simplified documents were agreed upon during project negotiations for use in the procurement of goods and services. One factor that has affected implementation is the high and underestimated volume of process steps required by the bank to take action. The Bank needs to enhance its response rate for requests for non-objections. 45 ANNEX 8: COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 46 ANNEX 9: LIST OF SUPPORTING DOCUMENTS Brian Stoddart, Evaluation and assessment of Rounds 2-3 sub-projects implemented Under the Higher Education Development Fund, June 2007. Community Colleges Assessment Study, (SETVET) July 10, 2005 Dr. Abdullah H. Z. Kaylani Evaluation of the Faculty Development Centers at the Eight Public Universities, July 2006 Dr. Bahram Bekhradnia, Formative Monitoring Evaluation of the Governance in Higher Education, 2007 Dr. Husein Abdul-Hamid,, Evaluation of ICT in Higher Education at the Eight Public Universities in Jordan, 2006 Dr. Nabil Boulos, Background Papers on Higher Education Management Information Systems, Student Information Systems, and Finance, Human Resources and Asset Management Systems, 2007 Roger Pearson, A White Paper Relating To Further Development Of The Community College System In Jordan, 2006 Roger Pearson, Further Development of the Community College System in Jordan; A Discussion Paper, , March 2005 Sachi Hatakenaka and Quentin Thompson, Evaluating Higher Education Development Fund. June, 2005. T. M. Abu-Sharar, Ali A. Yoghi, Sahar Al Yousef, Enhancing quality and relevance of higher education in jordan, A Position Paper Submitted to The National Forum on Jordan's Competitiveness in Higher Education For Building a Knowledge Economy, February 2006 World Bank, Resolving Jordan's Labor Market Paradox of Concurrent Econmic Growth and Highe Unemployment, March 2007 47 48