Document of The World Bank FOR OFFICIAL USE ONLY Report No: 79486-PG RESTRUCTURING PAPER ON A PROPOSED PROJECT RESTRUCTURING OF THE PAPUA NEW GUINEA: EDUCATION FOR ALL – FAST TRACK INITIATIVE (EFA-FTI) CATALYTIC TRUST FUND “READING EDUCATION (READ PNG)� PROJECT GRANT NO. TF098729 MARCH 2, 2011 TO THE INDEPENDENT STATE OF PAPUA NEW GUINEA MARCH 21, 2014 Human Development Sector Department/Education East Asia and Pacific region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. ABBREVIATIONS AND ACRONYMS ASPBAE Asia South Pacific Association for Basic and Adult Education BoM Board of Management CDAD Curriculum Development and Assessment Division EGRA Early Grade Reading Assessment EPDF Education Program Development Fund FM Financial Management GoPNG Government of Papua New Guinea GPE Global Partnership for Education ICB International Competitive Bidding IP Implementation Progress LEG Local Education Group MS Moderately Satisfactory NDoE National Department of Education NGO Non Government Organisation OBE Output Based Education OP Operations Policy PAD Project Appraisal Document PDO Project Development Objective PEA Provincial Education Administration PEB Primary Education Board PNG Papua New Guinea POM Port Moresby SSM Secretary’s Staff Meeting WB World Bank Regional Vice President: Axel van Trotsenburg Country Director: Franz R. Drees-Gross Sector Manager / Director: Luis Benveniste Task Team Leader: Binh Thanh Vu 2 INDEPENDENT STATE OF PAPUA NEW GUINEA READING EDUCATION (READ PNG) PROJECT (P105897) CONTENTS ABBREVIATIONS AND ACRONYMS ......................................................................... 2 A. SUMMARY ............................................................................................................... 4 B. PROJECT STATUS ................................................................................................. 4 C. PROPOSED CHANGES .......................................................................................... 5 D. APPRAISAL SUMMARY ....................................................................................... 9 Annex 1: Results framework and Monitoring Annex 2: Economic analysis Annex 3: Summary of the EGRA findings 3 INDEPENDENT STATE OF PAPUA NEW GUINEA READING EDUCATION (READ PNG) PROJECT (P105897) RESTRUCTURING PAPER A. SUMMARY 1. This restructuring paper aims to: (i) revise the project development objectives (PDO); (ii) scale down Component 3 “Strengthening Early Grade Reading Assessment - EGRA� activities, in particular related to the collection of baseline data on student reading levels using EGRA tools; (iii) implement post-EGRA assessment interventions in selected provinces; (iv) adjust the results framework in line with the revised PDO and project scope; and (v) extend the project closing date by one year to June 30, 2015. 2. The Reading Education (READ PNG) project was approved by the regional Vice President on March 2, 2011 for a total amount of US$19.2 million financed by a Grant (TF098729) from the Education for All – Fast Track Initiative (now “Global Partnership for Education� GPE). The proposed restructuring is necessary to: (i) ensure that the chosen objectives can realistically be achieved within the life of the project; (ii) adjust the scope of the EGRA interventions from 15 to four provinces to reflect the limited capacity on the ground and time needed to collect baseline data; and (iii) align the results framework with the activities being financed by the project. Therefore, the scaled-down project would allow for: (i) a focused effort on completing the EGRA diagnostic surveys and reporting in all four provinces, increasing the likelihood of collecting representative and good quality data; and (ii) the introduction of post-EGRA pilot interventions in select provinces to improve learning outcome monitoring and help government make informed policy decisions. To complete all project activities and accommodate for initial delays in project start up, a one-year extension is necessary. 3. The proposed restructuring will not require any changes to the safeguards category, and will not result in a material change to the application, which was approved by the EFA-FTI Board of Directors on November 10, 2010. The proposed restructuring has been endorsed by the Local Education Group (LEG) as a prerequisite for GPE’s financed project. B. PROJECT STATUS 4. Initial project implementation has been slow due in part to delays by the government to hire the required financial management and procurement expert, which led to an eight month effectiveness delay. However, once the project implementation team was in place, project implementation progress has been positive across all components. Under Component 1, all 16 contracts for the procurement of supplementary English readers through International Competitive Bidding have been signed, amounting to almost 50 percent of project funds (US$8.5 million). As of December 2013, all 16 suppliers have delivered the reading materials to Port Moresby. Furthermore, the national roll-out of supplementary readers is expected to be accomplished during the new school year in 2014. 5. Other activities under component 1 are scheduled to be implemented as originally designed but are slightly behind schedule: (i) contract for warehouse management and distribution of reading materials, which encountered procurement delays but is now in the process of being resolved; (ii) the procurement of vernacular books; and (iii) procurement of learning materials for elementary schools. 4 Under Component 2, the teacher training materials for the teacher professional training on the use of supplementary readers was finalized and endorsed by the National Department of Education (NDoE) in August, 2013. All 22 provinces have submitted provincial plans for teacher training and distribution of supplementary readers, which were reviewed by NDoE. A consolidated strategy for the national roll-out was prepared by the project team but is currently on hold to coincide with the distribution of supplemental readers under Component 1. The project also drafted the necessary support tools, including facilitator’s guides and teacher’s resource books to facilitate the roll-out. In terms of Component 3 achievements, as approved by the GPE Secretariat on two stages of restructuring1, the diagnostic assessments of early grade reading have been completed in four provinces, the first post- EGRA reading intervention pilot in Madang province is progressing well and on schedule, with midline results expected to be available prior to the beginning of the next school year. Finally, under Component 4, project management has provided overall guidance and support to facilitate component activities. As implementation progress continues to accelerate, a strategy for the project monitoring and evaluation has been drafted to help the project management team report on the different achievements. The activity is scheduled to get started in 2014 to coincide with the teacher professional training roll out at the provincial level and delivery of books to schools. Weaknesses in terms of financial management and procurement, which are causing delays to timely and accurate reporting of information, have been identified and measures taken, and early signs of improvement have been noted. To continue accelerating project implementation, the revised action plan was discussed and agreed upon during the November 2013 mission. Solid progress has been observed as majority of the agreed actions have been completed and few outstanding actions remain in progress. 6. Overall project implementation (IP) and progress towards meeting the projects objectives is progressing well (moderately satisfactory), but as mentioned previously, the current objective cannot be achieved as it is beyond the project scope and activities. Current disbursements are at 51.4 percent, and expected to increase to approximately US$10.7 million, or 55.7 percent of total project funds by the end of February 2014 once all 16 contracts under Component 1 have been fulfilled and disbursed. C. PROPOSED CHANGES 7. The Ministry of Treasury has sent a formal letter dated January 13, 2013 requesting the following changes in the scope of the project: (i) to reduce the scope of Component 3 by reducing the number of provinces from 15 to four provinces, where EGRA diagnostic surveys will be carried out during the life of the project; and (ii) introducing post-EGRA interventions to improve teaching practice in a small number of selected provinces, which will be used to monitor improvements in learning outcomes against the PDO. As mentioned previously, these changes also require the modification of the PDO to ensure its alignment with the changed project scope. Furthermore, these changes are necessary to strengthen the project design and are expected to contribute to the achievement of the PDO by the new project closing date. The Project Development Objective: 1 In March 2013, the project received approval from the GPE’s Secretariat to implement the pilot of first post -EGRA interventions in Madang province. 5 8. As previously mentioned, the current PDO2 is overly ambitious given the limited funds and implementation period, and the scope of activities is too broad to be fully achieved by the current project closing date. Furthermore, the project design and allocation of funds place greater emphasis on increased availability of supplementary readers in classroom libraries rather than activities more closely related to improving student-level outcomes, such as new pedagogical approaches and/or new modalities of in-service teacher training on reading development and close analysis of implementation successes and failures in the proposed pilot provinces to have a proven "good practices" approach that is practically and culturally appropriate and ready for scale-up in the PNG context. Also, there exist differences in the PDO wording between the various project documents (i.e. ISR, PAD and Grant Agreement), which needs to be rectified as part of this restructuring. 9. To bring the objective in line with the activities to be financed and the remaining implementation period, the following PDO is being proposed: “to promote better teaching and learning of reading skills in elementary and primary education.� The amended objective also matches the project’s aim to: (i) help establish baseline assessments of student reading abilities using the EGRA tool in four provinces, selected by the NDoE, given their degree of representativeness at the regional level; and (ii) improve teaching practices and student reading skills through the rigorous piloting of post-assessment interventions in a small number of provinces to be used as case studies to inform policy changes. Components 10. The project comprises four components: Component 1 “Increasing the Availability of Books and Learning Materials by Establishing Classroom Libraries (US$14.3 million)�; Component 2 “Promoting Reading through Professional Development, Professional Teacher Networks and Public Reading Campaigns (US$1.7 million)�; Component 3 “Strengthening Early Grade Reading Assessment (US$2.6 million); and Component 4 “Project Management and Monitoring (US$0.6 million). Around US$60,000 shall be redistributed from Component 1 to Component 4 to accommodate the request to procure a service vehicle for the project, as discussed under “Financing� below. Components 1, 2 and 4 remain unchanged. 11. The proposed changes are limited to Component 3 “Strengthening Early Grade Reading Assessment� and include: i. Reduction in the number of provinces, from 15 to four provinces, where EGRA diagnostic surveys will be carried out during the life of the project; and ii. Introduction of the implementation of post-EGRA interventions to improve teaching practice in a small number of selected provinces, which will be used to monitor improvements in learning outcomes against the PDO. 12. The agreement reached to complete diagnostic assessments using the EGRA instrument in four provinces (one per region of the country) was made mainly in response to capacity limitations and staff availability. The rationale behind the selection was guided by: (i) the provincial education administration (PEA) capacity to support diagnostic assessments and follow-up interventions; (ii) the degree of the language policy implementation in elementary and primary grades; (iii) the average levels of literacy in 2 The current objective of the project is to: (i) improve the reading skills of elementary and primary education students, and (ii) to build the policy, monitoring and evaluation capacity of the Curriculum Development and Assessment Division in the Recipient’s National Department of Education. 6 the provinces as measured by the “Asia South Pacific Association for Basic and Adult Education� (ASPBAE) to proxy for literacy environments at home; and (iv) the presence of a teacher training institution with whom partnerships could be established to administer diagnostic assessment and reading interventions for long term capacity sustainability. 13. The proposed pilot of post-EGRA interventions would allow for testing different instructional pedagogy in the classroom in different literacy environments. This can provide valuable insights to inform policy actions to improve reading instruction in elementary and lower primary grades in the country in order to improve students’ reading skills. Focusing on completion of diagnostic assessment and reporting in four (4) provinces and piloting post-EGRA interventions in selected provinces are expected to be achievable within the life of the project and allow for the establishment of more rigorous reading performance baselines. It also aims to build up capacity of the government in planning and managing the whole process of EGRA assessment and implementation of interventions. Results/indicators 14. The results framework of the original Project Appraisal Document (PAD) will need to be revised to: (i) include relevant PDO-level indicators to properly measure progress towards objective achievement; (ii) dropping indicators no longer relevant given the revised objective; (iii) refining the wording of some indicators to become clearer and easier to track; and (iv) adding new indicators to measure progress of revised component activities and/or be compliant with IDA core indicator requirements. There is now a logical chain between interventions, proposed outputs and expected outcomes, including specific targets. Details of the proposed changes to the results framework can be found in Annex 1. Specifically: Outcome indicators: i. Outcome indicator (No. 1): modified to be aligned with the new PDO. ii. Outcome indicator (No. 2): new indicator added to be aligned with the new PDO and focus on measuring results. Outputs and output indicators iii. Output (No. 1) is kept as original. However, output indicator (No. 1.c) is removed as it cannot be measured within the project implementation period. iv. Output (No. 2): modified to be aligned with the new PDO and with the new introduced reading intervention. Respectively, the output indicator (No. 2.a) has been modified to be directly linked to output. Intermediate outputs and intermediate output indicators v. All intermediate outputs are kept as original. However, the intermediate output indicators are proposed to be modified (1.a; 1.c; 2.a; 3.a) to be realistic and achievable within the project implementation period. Other intermediate output indicators are proposed to be added (1.b; 2.b; 3.b) to be directly linked to the intermediate outputs. Implementation arrangements 15. In 2010, this project was being prepared at the same time as another education projects, the “Flexible Open and Distance Education� - FODE project (P116521). Both are supervised by the Bank, 7 and being executed by the same implementing agency, the NDoE. In order to promote efficient management of donors’ projects by the government and to build government capacity for long term sustainability, it was agreed at the government’s request that two international advisors be hired: one for procurement and one for financial management. These advisors would work under the supervision of the Corporate Service Division of NDoE, and will assist with the implementation of both education projects. After initial delays, both international advisors are on board. The cost associated with hiring these advisors is being borne by the respective projects, i.e. the procurement advisor was hired by READ PNG, while the FM advisor by FODE. They will be supported by the national consultants of each project. Financing 16. Given the security issues in Port Moresby (POM), it was agreed to include a vehicle under project financing to facilitate implementation, communications and transportation for project staff within POM. Funding for the vehicle is not expected to exceed U$50,000 and funding for annual operation and maintenance is not expected to exceed US$10,000. Funding will be reallocated from the savings of procurement of reading materials for primary schools using international competitive bidding process. 17. Financing of activities under component 3 will remain mostly unchanged as specified in the PAD, with no change to the overall estimated expenditure of US$2.6 million (K7.0 million). READ PNG will finance survey logistics costs (transport, per diems and accommodation) for the teams conducting school visits. The project will also cover logistical costs at the central level (training, printing and travel) and consulting fees for consultants who will be responsible for training including development of training materials, delivery, data processing, report writing, dissemination and a reading information campaign. 18. The overall financial envelope of the project will not be affected by these changes and the updated budget plans confirm that the restructured project can be completed within the existing financial envelope. Furthermore, the reduction of the scope of activities under Component 3 does not require a reduction of funds allocated for that component due to the following: (i) at the time of project preparation, the cost of Component 3 activities were significantly under-estimated. This is apparent by the fact that carrying out the baseline surveys in the 4 proposed provinces alone cost US$932,000 out of the US$2.6 million component budget. Therefore, reducing the number of provinces is more in line with the component’s financial allocation; (ii) some losses due to the exchange rate further reduced the availability of overall project funds (approx. PGK 6 million less), which could be absorbed more easily by restructuring Component 3 and reducing the overall disruption to project implementation and proposed activities; and (iii) the post-EGRA interventions’ costs could be covered with the remaining funds while capitalizing on the opportunity to provide the Government of Papua New Guinea (GoPNG) with a rigorous baseline of reading performance and ways to support NDoE's capacity to provide better quality elementary and primary education. Extension of the project closing date 19. Although the government requested an 18 month extension, subsequent agreement was reached to process an extension of 12 months instead and modify the implementation and procurement plans accordingly. The Bank, in principle, supports this request for a one year extension of the project closing date from June 30, 2014 to June 30, 2015. The success of the post-assessment interventions included in 8 the proposed restructuring is critically dependent on the period of time for implementation. Justification for supporting the government’s request include: i. the need to account for delays encountered during project start up; ii. evidence from post-assessment interventions in Africa and Latin America showing that students in the similar pilot program should be taught under the new pedagogical approach for a full school year in order to achieve significant improvements of foundation reading skills; iii. a credible procurement and implementation plan having been developed by the project, which ensure that the project targets and its PDO will be achieved within the new project implementation period; and iv. the extension of the closing date satisfying the relevant Operational Policy (OP) 10.00 that: (a) the project is not subject to suspension of disbursements; or (b) there are no outstanding audit reports or audit reports are not satisfactory to the Bank. D. APPRAISAL SUMMARY Appraisal of the revised component three: 20. Rationale for the proposed changes: between the project’s approval date (March 2011) and its official declaration of effectiveness (November 3, 2011), the World Bank education team for Papua New Guinea (PNG) carried out analytical work on early reading as part of the activities. During that period, the NDoE and Bank teams had gathered additional information about the learning context in elementary and primary grades in the country. The data collected and analysis improved the team’s understanding of the level of complexity of the PNG education sector, primarily in two areas: (i) given the gap in practical knowledge of teachers to carry out classroom reading instruction, focusing project activities mainly on the provision of resource would not significantly improve student reading outcomes; and (ii) given the remoteness and poor accessibility to most schools in the country, school-level diagnostic assessments such as EGRA proved to be more labor intensive, much more expensive than estimated and logistically complex. 21. Therefore, the NDoE request to reduce the number of EGRA baseline assessments from 15 to four provinces, each representing a region of the country, was deemed logical, practical and viable. As previously mentioned, focusing on four provinces is expected to be achievable and allow for the establishment of rigorous baselines of reading performance and implementation mechanisms geared at building capacity at NDoE for sustainability. 22. Based on the criteria mentioned above, the four provinces identified to carry out EGRA assessments are: (i) the National Capital District (NCD - Papua Region), (ii) East New Britain (Islands Region), (iii) Madang (Momase Region), and (iv) Western Highlands (Highlands Region). 23. Progress to date. The baseline assessments have been completed in four provinces. The Madang repot has been endorsed by the NDoE. The reports of the National Capital District and East New Britain 9 results are currently undergoing peer reviewing. Data entry for the Western Highlands has been scheduled for the last quarter of 2013. Each of the provincial assessments represents a case study opportunity for the NDoE to collect provincial-level data about reading levels in different learning environments in elementary and primary grades in the country. 24. Main findings. Each of the provincial assessment completed to date has provided significant information about the average reading abilities of children at the end of elementary school (E2) and two years into primary school (P3 and P4). Findings from the three baseline data results report confirm global evidence on the importance of developing reading foundation in a language students are most familiar with, i.e. their mother tongue. Results from Madang province and the National Capital District where instruction is carried out in English show modest progression in some basic skills such as the sounds of letters and reading of familiar words but critical stagnant levels in reading and listening comprehension. By contrast, students in East New Britain province most of which attend elementary schools where Kuanua language - the predominant spoken vernacular language in the province - is used as the language of instruction, show dramatically positive results in all skills even after they transition into English (see summary results in Annex 3). 25. Dissemination to date. The NDoE has taken the lead in disseminating the research results from the EGRA surveys completed to date. Results have been shared and discussed during two provincial- level workshops (Madang and NCD), one national meeting attended by Provincial Education Advisors from all provinces and two senior-level meetings with policy makers from the NDoE (SSM & OBE task force). In all of these meetings, there was wide recognition of the uniqueness of the data collected and its relevance to inform sector policies. The results have fed into several current sector discussions both among different units within the NDoE and development partners. The proposed evaluations of the post- EGRA interventions under READ PNG will also be instrumental to inform and feed into current discussions on proposed curriculum changes - what the government calls "OBE (Output-Based Education) exit strategy." 26. Post-assessment interventions: The NDoE discussed with the World Bank team their interest in piloting interventions that could be rolled out in the country. Various models have been explored that could provide empirical evidence to inform policy reform and to guide future investment to promote better quality education for all children in elementary and primary schools in PNG. Given logistical and capacity constraints and to ensure the feasibility of successful implementation and evaluation during the life of the project, the NDoE has chosen to pilot a ‘reading booster’ program. This program is intended to support teachers to provide reading instruction in English language in lower primary levels in a systematic - yet simple - way that allows for the ‘big 5’ skills of reading development to be covered daily and expectedly improved. 27. Location and timing for the implementation of pilots: the NDoE selected Madang and Western Highlands to trial the ‘reading booster’ program. The selection of the two provinces responds to the NDoE’s interest to learn if the proposed approach would have similar effects in two different literacy environments: Madang, which has historically received more technical assistance to support its teachers college and various schools, a medium literacy levels in the adult population (as proxy of literacy environments at home), and is representative of various provinces in the country. In contrast, NDoE selected the Western Highlands, which has lower levels of adult literacy and received no technical assistance in piloting interventions for improvement of reading in earlier grade of primary school. 10 28. Following the approval by the GPE secretariat in March 2013 to allow NDoE to start the pilot in Madang province ahead of an official restructuring, the LEG also provided its endorsement and activities commenced in June 2013. The intervention will run for 8 months. The average impact of the intervention will be measured against the 2011 baseline with an end-line assessment scheduled for October, 2014. Findings from the trial are expected to provide average impacts of this approach in a medium-literacy environment. In the Western Highlands, the same approach will be used and is expected to run during the 2014 school year in a much lower literacy context. The average impact of the Western Highlands intervention will be measured in November, 2014 against the 2013 baseline. 29. Economic analysis: Annex 2 contains the detailed economic and financial analysis for the project. While PNG is classified as a middle income country by The World Bank, its human development indicators are very low. In 2009, life expectancy was 57 and the adult literacy rate was only 58 percent3. Education interventions are needed in order to make rapid progress toward achieving the MDG goal of universal basic education by 2015. The proposed interventions in the READ PNG project are similar to others that have shown positive impacts on early literacy and reading skills. Similar investments in the Philippines have been shown to yield net positive returns by increasing the earning potential of students later in life. 30. In the original project design, physical capital was the primary avenue in which to raise the production of reading education. It is shown in annex 2 that the approach of physical capital only educational investments will always be sub-optimal and be pareto improved by the addition of human capital. To incorporate the efficiency gain by investing in both physical and human capital of reading education, the restructured project proposal now incorporates a Madang Province Randomized Controlled Trial (RCT) intervention that aims to develop human capital through a teacher training intervention. At the completion of the trial an assessment of the marginal benefits (impact) and marginal costs will be conducted to inform recommendations on further wide-scale rollout. A financial analysis is also contained in Annex 2 including a depreciation schedule for the book stocks supplied to classrooms as part of component 1. 31. Safeguards: The proposed restructuring does not affect the safeguard category or trigger any new safeguards policies. The project did not require a separate safeguards instrument but instead prepared and disclosed an explanatory note on how the project design addresses aspects of an Indigenous Peoples Plan. This explanatory note remains valid and will continue to be applied. 32. Legal Covenants and financial management (FM): The project continues to comply with all legal covenants, meets the relevant fiduciary requirements and has no outstanding audits or financial reports. However, a recent FM review of the implementation arrangements of the project has identified a number of issues, which led to the FM rating being downgraded to Moderately Unsatisfactory. These issues were discussed with the project management team as well as the NDoE, and an action plan was developed by the project and endorsed by the World Bank. To date, a number of identified actions have already been completed, and a letter from the Secretary of Education confirmed the government’s commitment to the action plan and to meet all outstanding actions. Therefore, no changes to the FM arrangements are currently being proposed as part of the restructuring. 3 World Development Indicators, 2009 11 12 Annex 1: Results Framework and Monitoring PAPUA NEW GUINEA: READING EDUCATION (READ PNG) PROJECT Results Framework | Use of Project Outcome PDO Project Outcome Indicators Information Revised: “To promote better teaching 1. Proportion of primary school teachers using supplementary Reading results from interventions in and learning of reading skills in readers during reading instruction in a sample of 1800 pilot provinces will inform ongoing elementary and primary education�. classrooms with classroom libraries, as defined by classroom policy dialogue on curriculum observation checklists. [modified to be aligned with the new development, learning outcomes and PDO] teacher training. 2. Improvement in students’ average scores in 3 EGRA sub- tests (phonemic awareness, letter-sound identification and familiar word reading) in pilot provinces (reported by gender). [New - added to be aligned with the new PDO and focus on results] Outputs Output indicators Use of output monitoring 1.a: Number of students in elementary and primary schools Monitoring of project progress. with access to learning and reading materials in their classrooms, provided by the project.[original] Timely identification of project areas that may need special attention, Provide elementary and primary schools 1.b: Number of teachers in elementary and primary schools revisions and/or support. 1 with learning and reading materials and who have participated in professional development activities, professional development of teachers. provided by the project. [original] 1.c: Classroom teaching practices reflect enhanced knowledge in reading instruction and use of learning materials [removed this output indicator as it cannot be measured] Build capacity in the National 2.a: At least 2 post-assessment reading intervention pilots Monitoring of project progress. Department of Education to plan and designed and completed; including their evaluation reports for pilot reading interventions to improve dissemination. [modified to be aligned with the new PDO and Timely identification of project areas 2 classroom instruction.[modified to be directly linked to the output] that may need special attention, aligned with the new PDO and focus on revisions and/or support. results] 13 Use of intermediate output Intermediate outputs Intermediate output indicators monitoring 1.a: Classroom libraries established in approximately 18,000 Inform project progress reports. grade 3 – 8 classrooms (each classroom library containing 60 titles of supplementary readers)[modified to be more precise Inform mission supervision reports. and correct] Timely identification of areas where Greater availability of learning project faces risks and difficulties and 1.b: Learning material kits and teacher guidelines on the use of 1 materials by establishing classroom requires additional support. kits in approximately 15,000 elementary classrooms (grades libraries. EP- E2)[new - added to be linked to the output] 1.c: Vernacular reading materials in 2,500 elementary classrooms (grades EP- E2)[modified to be more realistic and achievable within the project period] 2.a: Approximately 4500 elementary teacher and 3600 Inform project progress reports. primary teachers received professional development Promoting reading through professional training.[modified to be realistic and feasible within the project Inform mission supervision reports. 2 development, professional teacher implementation period] networks and public reading campaigns. Timely identification of areas where 2.b: At least one national public reading event during the the project faces risks and difficulties and project.[added to be linked to the output] requires additional support. 3.a: Up to 4 EGRA provincial surveys and results reports Inform project progress reports. completed from baseline assessments in selected provinces.[modified to be aligned with the new PDO, more Inform mission supervision reports. Build capacity to plan and assess early realistic and achievable within the project implementation 3 grade reading in elementary and period] Timely identification of areas where primary grades project faces risks and difficulties and 3.b: Up to 4 events to disseminate results and findings from requires additional support. provincial EGRA surveys to NDOE, provincial officers and other stakeholders.[added to be directly linked to the output] 14 Annex 1 continued: Arrangements for results monitoring Responsibility Project Frequency Data Collection Baseline 2011 2012 2013 2014 2015 for Data Indicators and Reports Instruments Collection Proportion of primary school teachers using supplementary readers Consolidation during reading CLMU in of 10% 60% instruction in a sample ICB process in 100% (1800 Annually, by Classroom Libraries coordination with the 0 supplementary (180 (1080 of 1800 classrooms place classrooms) province Observation Checklist DoE's Standard and readers and classrooms) classrooms) receiving classroom Guidance Officers distribution libraries, as defined by classroom observation checklists. Sub-test 2: Sub-test 2: Phonemic Phonemic Awareness Awareness Boys Boys G3 = 46% G3 = 50% G4 = 53% G4 = 58% Girls Girls G3 = 44% G3 = 49% G4 = 49% G4 = 54% Sub-test 3: Sub-test 3: Improvement in Letter-sound Letter-sound students’ average identification Pilot identification scores in 3 EGRA sub- Planning and intervention Endline tests (phonemic Boys preparation implemented Boys assessment after 1 CLMU in awareness, letter- Dissemination Madang G3 = 26.1 for pilot and endline G3 = 28.7 year of EGRA assessment coordination with sound identification of results G4 = 33.0 intervention assessment at G4 =36.3 implementation, in teacher colleges and familiar word in Madang end of school selected provinces reading) in pilot Girls year Girls provinces (reported G3 = 24.0 G3 = 27.0 by gender). G4 = 29.9 G4 = 33.0 Sub-test 4: Sub-test 4: Familiar Familiar word reading word reading Boys Boys G3 = 28.7 G3 = 32 G4 = 41.0 G4 =45 Girls Girls G3 = 29.7 G3 = 33 G4 = 46.7 G4 =50 15 Responsibility Project Frequency Data Collection Baseline 2011 2012 2013 2014 2015 for Data Indicators and Reports Instruments Collection Sub-test 2: Sub-test 2: Phonemic Phonemic Awareness Awareness Boys Boys G3 = G3 = G4 = G4 = Girls Girls G3 = G3 = G4 = G4 = Sub-test 3: Sub-test 3: Letter-sound Letter-sound identification Pilot identification intervention Boys implemented Boys Western N/A N/A G3 = and endline G3 = Highlands4 G4 = assessment at G4 = end of school Girls year Girls G3 = G3 = G4 = G4 = Sub-test 4: Sub-test 4: Familiar Familiar word word reading reading Boys Boys G3 = G3 = G4 = G4 = Girls Girls G3 = G3 = G4 = G4 = Consolidation Number of students in of elementary ICB process in elementary (E) and E=0 E=0 learning kits 262,500 375,000 place primary (P) schools and CLMU records of CLMU and Provincial with access to learning distribution Annually, by delivery of books to Project Steering and reading materials P=0 ICB process in Consolidation 72,000 504,000 720,000 province schools Committees (PPSCs) in their classrooms, as place of provided by the supplementary project. readers and distribution 4 Assessment completed on May, 2013; Data to be coded and entered in Sept/Oct 2013. 16 Responsibility Project Frequency Data Collection Baseline 2011 2012 2013 2014 2015 for Data Indicators and Reports Instruments Collection Consolidation Number of teachers in of elementary Number of teachers in ICB process in elementary (E) and E=0 E=0 learning kits 10,500 4,500 elementary (E) and place primary (P) schools and primary (P) schools CLMU records of who have participated distribution Annually, by who have participated teachers professional in professional Consolidation province in professional development development activities, of development ICB process in as provided by the P=0 supplementary 1,800 12,600 3,600 activities, as provided place project. readers and by the project. distribution 8-month implementatio Planning for Preparation n of Analysis of scale up in Baseline and planning At least 2 post- 0 intervention results and provinces (Madang) of Madang assessment reading and endline dissemination requesting the intervention Teacher School Visit intervention pilots assessment program Ongoing, during Questionnaire, designed and (Madang) pilot CLMU Classroom Observations, completed; including 8-month implementation CLMU Records their evaluation reports implementatio Baseline Baseline Analysis of for dissemination. n of 0 0 (ENB and (Western results and intervention NCD) Highlands) dissemination and endline assessment Classroom libraries established in Consolidation approximately 18,000 of CLMU records of CLMU and Provincial ICB process in Annually, by grade 3 – 8 classrooms 0 supplementary 3,600 12,600 1,800 delivery of books to Project Steering place province (each classroom library readers and schools Committees (PPSCs) in a box containing 6o distribution supplementary readers) Learning material kits Consolidation and teacher guidelines of elementary CLMU records of CLMU and Provincial to use kits in ICB process in Annually, by 0 0 learning kits 10,500 4,500 delivery of books to Project Steering approximately 15,000 place province and schools Committees (PPSCs) elementary classrooms distribution (grades EP- E2) Vernacular reading Provincial materials in 2,500 survey of CLMU records of CLMU and Provincial elementary languages to 1,500 out of 2500 out of Annually, by 0 0 0 delivery of books to Project Steering classrooms (grades be covered, 2500 2500 province schools Committees (PPSCs) EP- E2) including distribution 17 Responsibility Project Frequency Data Collection Baseline 2011 2012 2013 2014 2015 for Data Indicators and Reports Instruments Collection 2 teams of coaches and 22 teams of provincial classroom library 2 teams of trainers trained by the CLMU records of CLMU and Provincial coaches and 6 14 teams of 2 teams of Annually, by project to facilitate 0 0 0 delivery of books to Project Steering teams of trainers (ToT) trainers (ToT) province classroom library schools Committees (PPSCs) trainers (ToT) training to teachers at the district level in each province. Demonstration Support to activities on Provincial teaching and Education At least one national learning Officers to public reading event strategies and Annually, by 0 0 0 0 carry out CLMU Records CLMU during the life of the materials to province similar events project. coincide with in their the 2014 provinces National Book (upon request) Week Madang, ENB, and NCD Baseline EGRA results assessment reports Up to 4 EGRA Western EGRA instruments in in East New endorsed by provincial surveys Highlands English (Madang, NCD Baseline Britain (ENB) the and results reports Results and Western 0 assessment and the Secretary's N/A As scheduled CLMU completed from Report Highlands), and Kuanua in Madang National Staff Meeting baseline assessments endorsed by & English (East New Capital (SSM), plus a in selected provinces. SSM Britain) District baseline in (NCD) the Western Highlands (WHP) 1 Workshop for Up to 4 events to education 1 Wrap-up disseminate results and stakeholders policy CLMU Records, findings from 1 1 to disseminate workshop to including clips from provincial EGRA 0 0 (Madang (NCD As scheduled CLMU results from present the newspapers showing surveys to NDOE, Province) Province) all 4 project's final coverage provincial officers and provincial results other stakeholders. EGRA surveys 18 Annex 2: Economic and Financial Analysis PAPUA NEW GUINEA: READING EDUCATION (READ PNG) PROJECT I. CONTEXT 1. Table 1 presents key economic indicators for Papua New Guinea. This country is classified by the World Bank as a lower middle income country and with a GDP per capita of 2255 (current USD) above Kiribati (1,631) and slightly below Vanuatu (3,039). The industrial sector is the largest contributor to GDP, followed by services and agriculture. According to the latest Barro and Lee estimates, the average number of years of schooling for adults is 4.3 years. Table 1. Key Economic Indicators 2007 2008 2009 2010 2011 2012a GDP (US$bn) (current prices): 6.3 8.0 8.1 9.9 12.7 15.4 GDP per capita (US$): 1,047 1,291 1,279 1,521 1,900 2,255 Real GDP growth (% change yoy): 7.2 6.6 6.1 7.6 8.9 7.7 GDP - composition by sector Industry 37.7% Services 32.1% Agriculture 30.3% Current account balance (% GDP): 4.0 8.4 -16.4 -25.6 -36.4 -28.4 Inflation (% change yoy): 0.9 10.8 6.9 6.0 8.4 6.8 Average years of schooling 4.1 4.2 4.2 4.3 Source: IMF, DFAT, Human Development Report UNDP. (a) all recent data subject to revision. 2. Table 2 presents key data for the education sector in Papua New Guinea. In 2008 (the latest available data), gross intake into the first year of elementary school (elementary prep) was quite high at 98.7 percent; however, due to low progression rates throughout primary the gross enrolment rate of the last year of primary was only 49.1 percent and only 7.6 percent for the final year of secondary. 59.3 percent of grade 8 students continued on to lower secondary and only 38.5 percent of grade 10 students progressed to upper secondary. Total expenditure by the Government of Papua New Guinea on the education system was 964,590,000 Kina which translates to 741.98 Kina per student. However, these figures exclude spending by provincial and district governments, as well as private expenditure, which is higher for secondary students due to tuitions (see World Bank 2004). 19 3. Education features in the second pillar of the 2013 – 2016 medium term goals of the PNG Country Partnership Strategy. Gender-equitable improvements in lives and livelihoods are among the strategy’s three strategic priorities and basic education is listed as one of the niche expenditure priorities. Education interventions are needed in order to make rapid progress toward achieving the MDG goal of universal basic education by 2015 (ADB, AUSAID, WB 2007). The Bank’s Country Assistance Strategy is closely aligned with the government’s strategy. 4. The primary purpose of this investment is to contribute to the improvement of learning and teaching of reading for elementary and primary school students in Papua New Guinea. This will be achieved by creating classroom libraries consisting of books and other reading materials. Books will be primarily in English and supplied to primary school classrooms where English is the language of instruction, and reading materials adaptable to local vernacular will be provided to elementary school classrooms where the local vernacular is the language of instruction. Proper use of books and reading materials are crucial to effectiveness of this investment, and teacher training will be provided. Other components of this investment include an Early Grade Reading Assessment to be conducted in each province, as well as capacity building for managing school subsidies and procurement. Table 3 presents the estimated project costs for each activity. Table 3. Estimated Project Costs by Component (US$ million) Increasing the Availability of Books and Learning Materials by Establishing 14.3 Classroom Libraries in Elementary and Primary Schools Promoting Reading Through Professional Development, Professional Teacher 1.7 Networks and Public Reading Campaigns Strengthening Early Grade Reading Assessment 2.6 Project Management and Monitoring 0.6 20 II. ECONOMIC ANALYSIS 5. The purpose of this analysis is to assess: (i) the cost effectiveness of the project; (ii) whether its benefits exceed its costs; (iii) the equity impact of the project; and (iv) its alignment with the country assistance strategy. A. Cost Effectiveness Analysis 6. Evaluations of interventions similar to this investment have shown positive impacts on early literacy and reading skills. For example, Elly, Cutting, Mangubhai, and Hugo (1996) review three interventions in Fiji, Singapore, and Sri Lanka. The first, Fiji Book Flood project from 1980 to 1981, provides an interesting example since students there studied in their own vernacular until grade 3 and then subsequently English, as is currently the policy in Papua New Guinea. This intervention exposed year 4 and 5 students in 8 rural schools to an “abundant supply of high interest books;� half of these students used the books in an interactive manner, or under a shared reading method, while the other half used them individually, under a silent reading method. Year 4 students exposed to the books exhibited an increase in reading performance equivalent to 15 months while those in the control group exhibited an increase of only 6.5 months. Year 5 students exposed to the books under the shared reading method improved by 15 months; those under the silent reading method improved by 9 months, and those in the control group only by 3 months. Additionally, tests in the second year of the program revealed an effect size for the intervention of 0.76 to 1.18 standard deviations, and in the year 6 external exams, those who were exposed to the books were twice more likely to pass than before. A similar program in Sri Lanka from 1995 to 1996 revealed an increase in scores from pre-test to post-test of 11 percent for year 4 students exposed to books while only 4 percent for the control group; for year 5 students the increase was 9 percent and 3 percent, respectively. Finally, in Singapore in the late 1980s, students exposed to classroom libraries performed better than students in the control group on 53 out of 65 reading tests performed over three years. 7. Time spent reading is crucial to improving literacy and reading skills. Numerous studies have established this link. For example, Anderson, Wilson and Fielding (1988) found that time spent reading was the best predictor for gains in reading achievement between 2nd and 5th grade. Neuman’s (1999) randomized study provided economically disadvantaged preschools with books and teacher training and found that the amount of reading increased by students increased and that this improved early reading outcomes as well as later learning outcomes in kindergarten. 8. This investment is cost-effective because the only alternative feasible intervention to increase the availability of reading material that may be less expensive to creating classroom libraries would be to build school libraries. The presence of school libraries has been shown to be positively associated with cognitive achievement; Fuller and Clark (1994) reviewed cognitive production function research and noted that 16 studies found a statistically significant correlation out of 18 studies which included some type of variable on the presence of school libraries. However, these are not causal only correlations. Additionally, school libraries do not guarantee students will actually read. For example, Anderson, Wilson and Fielding (1988) found that on most days outside of school, children spent little time actually reading. What seems to be effective in improving early reading skills is how students interact with the books. Each of the interventions reviewed by Elly et al. (1996) were described as “book floods�, but they all 21 included aspects of teacher training to ensure their students actively used the books and used them effectively. Even differences in how books were used lead to differences in outcomes. In Fiji, for example, year 5 students which followed the shared reading method performed better than those that followed the silent reading method. While the procurement of books may represent the largest cost to this investment, the value of the investment stems from the interaction between students, teachers, and books within the classroom setting: Neuman (1999) concludes that her study (mentioned above) provides “powerful support for the physical proximity of books and the psychological support to child care staff on children’s early literacy development.� 9. Economic Cost Minimization. To analyze the efficiency of the inputs into Education, its production function and relative input costs must be examined. Educational productivity is characterized by two factors of production, human capital and physical capital with respective input costs . Human capital refers to a teacher’s (the teacher for the purpose of this analysis can also be a parent, peer, or community member) skills and knowledge of the topic area and physical capital refers to physical items that assist learning such as books, desks, computers and classrooms. 10. Human capital and physical capital in education are partially but not perfectly substitutable. For example, text books can replace some face-to-face teaching time and vice versa but neither can perfectly take the place of the other. Some combination of both typically produces a better outcome. This implies an input relationship for education that is strictly convex, and therefore appropriately represented by the Cobb-Douglas production function (see Gyimah-Brempong and Gyapong 1991, Pritchett and Filmer 1999, Strauss and Sawyer 1986, for examples of Cobb-Douglas education production functions). 11. In the context of the READ PNG project, the output is reading education and the inputs include books, reading resources and teacher training. In the original project design, physical capital was the primary avenue in which to raise the production of reading education. The effect of the increased output through the provision of books for the establishment of classroom libraries on educational output is illustrated in Fig 1. The initial amount of human and physical capital lies at point A. A vertical shift of physical capital from to results in a new input bundle, point B. This input bundle lies on a new isoquant curve f(x1,y2) = y to the right of the previous isoquant curve which has greater output. However the cost minimizing bundle for the same level of output on the isoquant curve is the point where the diagonal isocost line is tangential to the isoquant curve which is at point C. More precisely, the cost minimization problem is: Such that . This problem has an interior solution with positive input prices and a,b>0. Deriving the minimized cost function shows that costs are minimized (or conversely output is maximized) when: ( ) ( ) 22 12. The above derivation demonstrates that when both input prices are positive (ie not free) and production is convex ie. a,b>0, the optimal amount of and are both positive, meaning that an optimal bundle cannot have a positive amount of one input and zero amount of the other. A change in relative input prices will shift the optimal input bundle towards more of the less expensive input. However, as shown mathematically above, the approach of physical capital only investments will always be sub-optimal and be pareto improved by the addition of human capital. 13. To incorporate the efficiency gain by investing in both physical and human capital of reading education, the restructured project proposal now incorporates a Madang Province Randomized Controlled Trial (RCT) intervention that aims to develop human capital through a teacher training intervention. While there was and still remains a training strategy for teachers to train them on how to set up a classroom library in the project, the additional teacher training proposed for the RCT in component 3 will target improved teaching instruction of reading in order to improve the efficiency of reading education investment. Physical Capital x2’’ B Greater output (y) D C A x2’ f(x1,y2) = yi Human Capital x1’ Figure 1: choice of input factors that minimize cost of output 14. Madang Province Randomized Controlled Trial for Evidence-Based Policy Making. Given that human capital inputs will result in an efficiency gain in reading education, the question then becomes which human capital investments and quantities will be most effective for PNG? This question is important because any investment involves an opportunity cost. The impact or return on investment, can vary greatly depending on the choice of intervention. Banerjee et. al. (2011) give many examples of human development interventions in developing countries that were assumed to have a positive impact and the subsequently found to have no or little impact when rigorously tested by a RCT. Trialing interventions before their wide-scale rollout and evaluating their effectiveness can potentially save millions of Kina being spent on ineffective programs. Component 3 of the READ PNG project has been restructured to 23 incorporate a RCT of a reading instruction intervention in Madang Province in order to provide valuable information for policy makers to optimize reading education investments. 15. The proposed RCT in Component 3 enables the calculation of the marginal costs and benefits of an intervention so that the efficient quantity of intervention can be analyzed prior to any wide-scale roll-out. Overshooting or undershooting will result in inefficiencies. Economically, the efficient quantity is the quantity where marginal costs equal marginal benefits. The first step in calculating the efficient quantity of input is to construct a Total Cost Function (TC). Using quotations and estimates of the fixed and variable costs of developing and implementing the proposed reading instruction intervention in 15 treatment schools in Madang province the following TC function has been constructed in Kina (K): 16. The fixed costs of the proposed reading intervention are estimated at approximately 57000 K represented by the intercept in the TC function. The variable costs include costs such as venue hire, printing of instructional manuals, per diems and in-service facilitation and are represented by the last two terms in the TC function. The TC function is linear in q because the suppliers approached for quotes did not offer any bulk discounts on printing, accommodation, venue hire or facilitation and charge on a constant per unit rate. The TC can be used to estimate the cost of a full-scale province-wide rollout. There are 41 schools in Madang province with approximately 168 grade 3 and 4 teachers. Using the TC function, the total cost of a province- wide rollout is estimated to be 835 277 K. The marginal cost (MC) can be derived from the TC function by taking the first derivative, resulting in a MC approximately equal to 4 633 K. The marginal benefits of the intervention are represented by the difference in outcome measures in the RCT between the Control group and Treatment group. The marginal benefits of the proposed reading intervention are only able to be measured ex-ante when the intervention is complete. At the completion of the trial an assessment of the marginal benefits (impact) and marginal costs will be conducted to inform recommendations on further wide-scale rollout. B. Cost Benefit Analysis 17. The monetized net benefit of the project will hinge primarily on how it affects schooling attainment and the marginal product of schooling. The link between schooling attainment and the return to schooling has been the subject of hundreds of studies: for example, the private return to primary education on average is 26.6 percent while the social return, which accounts for the expenditure by the government but excludes external benefits such as reduced crime or increased health, is 18.9 percent (Psacharopoulos and Patrinos 2004). Under the assumption that earnings equal the marginal product of labor for a given level of schooling, the social return to schooling is a measure of its monetized benefit to society excluding the external benefits (Psacharopoulos 1995). 18. Use of classroom libraries should increase progression rates and schooling attainment. A major determinant of dropout rates for young children in low and middle income countries is their ability to succeed at school. For example, in the Demographic Health Survey for Benin 2006, 21 percent of females aged 15 to 25 reported that the reason they stopped school was due 24 to failure; this was second to the 25 percent who were unable to pay school dues and 23 percent who reported that they did not like school. In the Burkina Faso 2003 Demographic and Health Survey, a similar pattern emerges with 20 percent of females reported failure as the reason why they left school (author’s calculations). In an analysis of the Egyptian Demographic and Household Survey data, failure at school was listed as a reason for dropping out for 40.8 percent of boys 34.5 percent of girls (Siliman and El-Kogali 2002). Increases in the time spent reading and early reading ability has been linked to future cognitive ability and learning (Cunningham and Stanovich 1998); consequently, this investment is likely to improve student performance in school and decrease dropouts. 19. Use of classroom libraries should increase the marginal product of schooling. There is an emerging body of research establishing the link between cognitive ability (typically measured by test scores) and earnings for a given number of years of schooling (Patrinos and Sakellariou 2007; Moll 1998; Glewwe 1996; Murnane et al. 1995). For example, in Green and Riddell’s (2001) analysis of earnings and adult literacy in Canada, when the magnitude of literacy skills as measured by data from the International Adult Literacy Survey is included as an explanatory factor in wage determination, the number of years of schooling is shown to have little or no association to earnings; this suggests that literacy, and cognitive ability in general, is the key determinant of earnings and likely the marginal product of schooling, and not the number of years of schooling. As a result, even if the installation and use of classroom libraries has no effect on the number of years of schooling, it is possible that for a given number of years of schooling, the marginal product of schooling will increase since, as described above, similar interventions have had positive impacts on literacy and reading skills. 20. Estimating accurately an internal rate of return for this project is not possible due to a lack of required data in Papua New Guinea. Ideally, estimates of the effect of this project on both schooling attainment and the marginal product of schooling attainment would be combined with an estimated earnings function (relating years of schooling and experience to gross earnings) in order to estimate the future stream of additional output generated by the project. Combined with estimates of costs, this could then be used to calculate an internal rate of return to measure the profitability of the project. However, the magnitude of the effect of this project on schooling attainment and marginal product of schooling attainment are unknown since these effects from similar projects in other countries have not been evaluated. Additionally, the current lack of recent individual earnings data for Papua New Guinea prevents estimating an earnings function and consequently the current marginal product of schooling. 21. However, the relationship between the internal rate of return and the effect size of the project on schooling attainment, as well as on marginal product of schooling, can be analyzed under various earnings function scenarios. In other words, by adapting various Mincerian earnings functions from comparator countries to fit the Papua New Guinea context, how the monetized net benefit of the project interacts with its effect on progression rates can be analyzed. This helps reveal the conditions required for this project to have a positive net benefit and the circumstances which will prevent profitability. In order to analyze the relationship between the project’s effect on early progression rates and its internal rate of return (and, analogously, for its effect on returns to schooling) for a particular earnings function scenario, five steps are required. First, the earnings function needs to be scaled to fit the Papua New Guinea context. To do this given the current data limitation, the constant is adjusted so that the model predicts GDP per 25 capita using the average years of schooling and experience presented previously. Second, for the next 55 years, the adjusted model is used to predict the total annual earnings of students enrolled in the first year of primary or elementary under the current profile of progression rates. Third, for a given effect of the project on early grade progression rates, the total annual earnings are recalculated using the new profile of progression rates. The difference in earnings each year is the stream of benefits net of opportunity cost produced by the project (which are negative in the short term because of fewer labor market entrants). Forth, the costs of the investment as well as the costs due to the additional enrolments per year are calculated, and subtracted from the stream of benefits producing a stream of net benefits. Finally, an internal rate of return is calculated. In order to estimate the relationship between the project’s effect on early grade progression rates and its internal rate of return, this is repeated for a range of effect sizes. A similar process is conducted for estimating the relationship between internal rate of return and effect on marginal product of schooling. Table 4 lists the specific assumptions of this modeling exercise. 22. Earnings functions scenarios for this analysis are drawn from two studies with typical returns to schooling, from two studies with atypically low returns to schooling, and from one study of informal sector earnings. Table 5 lists the studies and the parameters for returns to schooling, experience and experience square. The earnings function estimated for urban, formal sector Papua New Guinea in 1983 to 1985 (Gibson and Fatai 2005) and for Filipino men in 1998 (Schady 2000) exhibit a return to schooling close to the cross country average of 10.8 percent (Psacharopoulos and Patrinos 2004). Ideally, the gross earnings function from a country very similar to Papua New Guinea would be adapted. But countries similar even in single dimensions such as GDP per capita or consisting of many ethno-linguistic groups did not have earnings function estimates. As a result, these two earnings function scenarios were chosen for this analysis as representing typical earnings functions because of the closeness of the return to schooling to the international average, as well as because one is from the country in question and the other is from a country in the region which also consists of islands and isolated regions. In fact, these two earnings functions have almost identical parameters for schooling and experience. To conduct a sensitivity analysis, earnings functions from two countries with exceptionally low returns to schooling were also included in this analysis: these are Ghana in 1995 (Jones 2001) and the 1992 to 1993 transition period in Vietnam (Moock et al. 2003). Finally, since 83 percent 26 of the population in Papua New Guinea is supported by the informal sector, a recent study of Rwanda’s informal study (Lassiblle and Tan 2005) is also included. 23. The relationship between this project’s impact on early grade progression rates and its internal rate of return is concave; consequently, even small effects on progression rates will produce a modest internal rate of returns under typical earning scenarios. This finding is demonstrated in Figure 1 which plots this relationship for each earnings scenario. The marginal effect of early grade progression on the internal rate of return is initially very high and diminishing. For example, a 1 percent increase in progression rates from grades 1 to 5 (e.g. for grade 1: an increase from 94.6 percent to 95.5 percent, etc.) will produce a 12 percent internal rate of return under this earnings function scenario. In fact, a 0.47 percent effect on progression rates will yield a 10 percent internal rate of return. This concavity stems from non-increasing returns to schooling and the fact that increasing the progression rate defers the project’s benefits into the future, but does not defer its investment cost. A property of the definition of present value is that each year a payment is deferred into the future at a positive non-increasing growth rate, the discount rate required to maintain the same present value increases at a diminishing rate. This means that each year labor market entry is deferred (as is the consequence of higher progression rates), the discount rate needed for the present value of future earnings to remain equal to the project cost (in other words, the internal rate of return) increases at a diminishing rate. If returns to schooling were increasing over time or if there were no up-front project investment cost, then the relationship between this project’s effect on progression rates and its internal rate of return would not be concave. 27 Figure 1. Project’s effect on early grade progression rates and its internal rate of return by earnings scenario 24. Due to the concavity of the relationship between the project’s effect size on progression rates and the internal rate of return, this project will be profitable even at low effect sizes, as long as the gains to going to school net of the opportunity cost are positive. This result emerges from the sensitivity analysis conducted by examining this relationship for the atypical earnings function scenarios presented in Figure 1. Under the Ghana earnings function scenario, with a low return schooling at 7.1 percent, the project would still be profitable even with small effect sizes due to the concavity of the relationship. However, under the extreme example of Vietnam, the project would not be profitable under any effect size. This is because in the parameters of the earnings function, the returns to experience exceed that of schooling by a high enough margin so that deferring labor market entry would not be beneficial. As long as the economic rate of return of schooling is positive, or in other words as long as it is productive for Papua New Guinea children to increase their years of schooling even if its earnings function is atypical, then this project will be beneficial at very low effect sizes. The question is whether or not schooling is beneficial in Papua New Guinea. The informal sector example of Rwanda, which is particularly relevant, exhibits the same concavity although lower internal rates of return for each given effect size as compared to the typical earnings function scenario; although the return to schooling in the informal sector of Rwanda is higher compared to that of Papua New Guinea 1983 to 1985 and Filipino males in 1998, the return to experience is much higher. While returns to schooling were shown to be high in Rwanda, other studies of the informal sector have shown varying and lower returns in Africa (see Keswell and Poswell 2004; Kuepie et al. 2009). 25. A very small positive effect on marginal product of schooling will increase the internal rate of return. This occurs since increasing the marginal product of schooling while holding attainment constant will not produce any additional costs either to the government or household. 28 If this investment produces no impact on attainment, then a very small impact on marginal product of schooling is required to offset the cost of the investment; the minimum effect size for all earnings function scenarios considered here to be profitable is 0.24 percent. Figure 2 displays the relationship between the effect of this project on returns to schooling and the corresponding internal rate of return holding attainment rates constant for each of the various earnings function examples. Similar to the relationship between internal rate of return and years of schooling, the relationship is concave. Figure 2. Project’s effect on return to schooling and its internal rate of return by earnings function scenario 26. In addition to the monetized benefits discussed so far, education has numerous social benefits including improving health outcomes, reducing unemployment, civic participation, and a decrease in crime rates. For example, in theory, education attainment reduces crime rates by increasing wages and thus the opportunity cost of crime. It also changes preferences for risk- taking, and alters the individual’s peer group (see Lochner 2010 for a recent summary of the literature). While these social benefits should be included in the cost benefit analysis of this project, quantifying them is challenging, especially in Papua New Guinea where data is scarce. C. Equity 27. Ethno-linguistic, urban-rural and socio-economic inequities generated by this project will depend on the dependence of early grade reading material on language. Since the language of instruction in elementary schools is in the local vernacular according to policy, those elementary students who do not have reading materials in their local vernacular will be unable to benefit fully from the project. However, the creation of reading materials which are independent of language, for example those which are produced by the instructor or at the school, will offset the dependence of reading material on language and help offset this potential inequity. 29 28. Ethno-linguistic, urban-rural, socio-economic and gender inequities generated by this project will depend on enrolment differences among these sub-populations. Table 4 presents attendance rates for aged 6 to 11 girls and boys in urban and rural areas from a preliminary report of the 2006 Demographic and Household Survey (National Statistical Office 2009)5. Gender differences in attendance were relatively low while urban and rural differences in attendance rates are significant; for all genders they were 60.3 percent and 38.6 percent, respectively. Entrance rates probably vary similarly by these sub-populations. Since this project benefits only those who have entered school, this project will benefit urban areas more than rural areas. Additionally, while attendance rates by region were not reported in the DHS report, schooling attainment varied considerably by region. If a similar pattern holds for school entrance rates, the spatial distribution of ethno-linguistic groups in Papua New Guinea implies that entrance rates would differ by ethno-linguistic group and that consequently some groups would benefit more from the project than others. However, in all cases, this project is not expected to adversely affect specific sub-populations, but rather benefit some more than others. D. Alignment with the Country Assistance Strategy 29. The principle development objective of READ PNG, to improve the teaching and learning of early grade reading. Further, (1) it is expected to increase schooling attainment by reducing dropout rates, and (2) it is expected to increase the productivity (and return) for a given number of years of schooling. Reducing dropout rates is crucial to achieving universal basic education, as is improving literacy to human resource development. Consequently, this project is closely aligned with the Medium Term Development Strategy and Country Assistance Strategy. III. FINANCIAL ANALYSIS 30. While this project itself will involve few direct recurrent costs, scheduled depreciation of capital stock is essential to the long term sustainability of benefits of Component 1. The majority of the funds allocated to this project will be used to provide basic classroom libraries. Once the libraries are established, the books printed under this project will enter the local market for direct purchases by schools and parents. This will ensure the availability of reading material in the future without imposing direct additional recurrent costs related to book design. Similarly, once teacher training networks are established, material, knowledge and experiences can be shared using the existing infrastructure and other already established channels of exchange among teachers. Training could be replicated without generating many additional recurrent costs. Finally, the recurrent costs of the Early Grade Reading Assessment will be in the order of $60,000 per province, on average, every two years. Total recurrent spending requirements for 5 Data from this survey was not available for use for this economic analysis. 30 EGRA would be about $300,000 per year if 5 of PNG’s 17 provinces undertake an EGRA survey in any given year. 31. However, the books procured for the classroom libraries represent approximately 12 million US dollars’ worth of physical assets. As with most capital, its value will depreciate over time as the books get used and damaged and eventually decommissioned. In order to sustain the benefits of the classroom libraries beyond the life of this project, the project should provide the PNG Government with an asset management strategy and depreciation schedule of capital stock. The strategy should involve the creation of a capital account by the PNG Government to hold scheduled deposits according to the depreciation schedule. All deposits should be fully budgeted in annual fiscal planning. A procurement schedule should also be developed in accordance with the depreciation schedule to replenish book stocks as they become fully depreciated. Table 7 provides a depreciation schedule using the straight line depreciation method. The U.S. Department of Treasury IRS specifies an asset life of 11 years for printed books. Table 7. Depreciation schedule of classroom libraries book stocks Year Value beginning of Depreciation Cumulative Value at the end the year expense in year depreciation of the year (m USD) (m USD) (m USD) (m USD) 2013 12.5 1.136 1.136 11.364 2014 11.364 1.136 2.272 10.228 2015 10.228 1.136 3.408 9.092 2016 9.092 1.136 4.544 7.956 2017 7.956 1.136 5.68 6.82 2018 6.82 1.136 6.816 5.684 2019 5.684 1.136 7.952 4.548 2020 4.548 1.136 9.088 3.412 2021 3.412 1.136 10.224 2.276 2021 2.276 1.136 11.36 1.14 2022 1.14 1.136 12.5 0 32. Inflation and the appreciation of the Kina. PNG’s rising inflation and the appreciation of the Kina put pressure on the future financial sustainability of project activities. In 2012, the inflation rate in PNG was around 7 percent (table 1). High inflation erodes purchasing power and poses a risk to the project and sustainability of continuting investment by the GoPNG. However if the depreciation schedule is appropriately indexed to inflation, financial sustainability will be insensitive to rises in inflation. The procured basket of books will include both locally published books and internationally sourced books. To offset the additional costs of a rapidly appreciating currency the mix of international and locally procured books can be adjusted to reduce the impact of the shocks caused by changes currency prices and futures options for the purchase of books might also be considered. 31