Document of The World Bank Report No: ICR00002821 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-437401) ON A CREDIT IN THE AMOUNT OF SDR17.7 MILLION (US$27.5 MILLION EQUIVALENT) TO THE INDEPENDENT STATE OF PAPUA NEW GUINEA FOR A SMALLHOLDER AGRICULTURE DEVELOPMENT PROJECT June 29, 2014 Timor-Leste, Papua New Guinea & the Pacific Islands Country Department Sustainable Development Department East Asia and Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective as of May 31, 2014) Currency Unit = PGK (Kina) US$1.00 = PGK2.20 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS AusAID Australian Agency for International Development BA Beneficiaries Assessment BPF Beneficiaries Participation Framework CBO Community-based Organization CDD Community Driven Development CELCOR Center for Environmental Law and Community Rights CLUA Clan Land Users Agreement CPS Country Partnership Strategy CSTB Central Supply and Tender Board DOW Department of Works EA Environmental Assessment EMP Environmental Management Plan FCP Forestry Conservation Project FFB Fresh Fruit Bunch FM Financial Management GDP Growth Domestic Product GoPNG Government of Papua New Guinea GRM Grievance Redress Mechanism ha hectare HIV/AIDS Human Immunodeficiency Virus and Acquired Immunodeficiency Syndrome IBRD International Bank for Reconstruction and Development IDA International Development Association ISR Implementation Status Report IT Information Technology LCT Local Coordination Teams (LCT) LLG Local-level Government LLI Local-level Institution LPC Local Planning Committee LSS Land Settlement Schemes M&E Monitoring and Evaluation MIS Management Information System MOU Memorandum of Understanding MTDS Medium-Term Development Strategy (2005–2010) MTR Mid Term Review NGO Non-governmental Organization OPIC Oil Palm Industry Corporation ii OPID Oil Palm Infrastructure Development Unit OSOPDP Oro Smallholder Oil Palm Development Project PAF Plantation Approval Form PDO Project Development Objective PIM Project Implementation Manual PNG Papua New Guinea PNGOPRA PNG Oil Palm Research Association PNGSDP PNG Sustainable Development Program PPII Provincial Performance Improvement Initiative PSC Project Steering Committee REU Road Engineers Unit RMTF Road Maintenance Trust Fund RPF Resettlement Policy Framework RSPO Roundtable for Sustainable Palm Oil SA Social Assessment SADP Smallholder Agriculture Development Project VOP Village Oil Palm WNB West New Britain Vice President: Axel van Trotsenburg Country Director: Franz Drees-Gross Sector Director: John A. Roome Country Manager Laura E. Bailey Sector Manager: Michel Kerf Project Team Leader: Kofi Nouve ICR Team Leader: Kofi Nouve ICR Primary Author: Sati Achath iii INDEPENDENT STATE OF PAPUA NEW GUINEA SMALLHOLDER AGRICULTURE DEVELOPMENT PROJECT TABLE OF CONTENTS A. Basic Information....................................................................................................................... v B. Key Dates ................................................................................................................................... v C. Ratings Summary ....................................................................................................................... v D. Sector and Theme Codes........................................................................................................... vi E. Bank Staff .................................................................................................................................. vi F. Results Framework Analysis ..................................................................................................... vi G. Ratings of Project Performance in ISRs ................................................................................. viii H. Restructuring ........................................................................................................................... viii I. Disbursement Profile ................................................................................................................. ix 1. Project Context, Development Objectives and Design ............................................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................................. 5 3. Assessment of Outcomes .......................................................................................................... 17 4. Assessment of Risk to Development Outcome ......................................................................... 25 5. Assessment of Bank and Borrower Performance ..................................................................... 25 6. Lessons Learned........................................................................................................................ 28 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners........................... 30 Annex 1. Project Costs and Financing .......................................................................................... 31 Annex 2. Outputs by Component.................................................................................................. 34 Annex 3. Economic and Financial Analysis ................................................................................. 40 Annex 4. Bank Lending and Implementation Support/Supervision Processes............................. 49 Annex 5. Beneficiary Survey Results ........................................................................................... 51 Annex 6. Stakeholder Workshop Report and Results................................................................... 52 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................................... 55 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................................... 63 Annex 9. List of Supporting Documents ...................................................................................... 64 MAP IBRD 35472 ........................................................................................................................ 65 iv A. Basic Information PNG-Smallholder Country: Papua New Guinea Project Name: Agriculture Development Project ID: P079140 L/C/TF Number(s): IDA-43740 ICR Date: 06/30/2014 ICR Type: Core ICR Lending Instrument: SIL Borrower: GOPNG Original Total XDR 17.70M Disbursed Amount: XDR 15.23M Commitment: Revised Amount: XDR 15.23M Environmental Category: B Implementing Agencies: The Oil Palm Industry Corporation Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 06/25/2003 Effectiveness: 01/28/2009 01/28/2009 06/13/2012 Appraisal: 02/23/2007 Restructuring(s): 11/14/2013 Approval: 12/18/2007 Mid-term Review: 09/20/2010 10/07/2010 Closing: 12/31/2012 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: High Bank Performance: Moderately Satisfactory Borrower Performance: Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Unsatisfactory Implementing Moderately Quality of Supervision: Satisfactory Agency/Agencies: Unsatisfactory Overall Bank Overall Borrower Moderately Satisfactory Unsatisfactory Performance: Performance: v C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision Yes None (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Agricultural extension and research 8 8 Crops 21 21 Other social services 3 3 Rural and Inter-Urban Roads and Highways 66 66 Sub-national government administration 2 2 Theme Code (as % of total Bank financing) Participation and civic engagement 33 33 Rural services and infrastructure 67 67 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg James W. Adams Country Director: Franz R. Drees-Gross Nigel Roberts Sector Manager: Michel Kerf Rahul Raturi Project Team Leader: Kofi Nouve Oliver Braedt ICR Team Leader: Kofi Nouve ICR Primary Author: Sati Achath F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objective of the project was to increase, in a sustainable manner, the level of involvement of targeted communities in their local development through measures aimed at increasing oil palm revenue and local participation. vi Revised Project Development Objectives (as approved by original approving authority) The PDO was not revised. (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Increase in smallholder income from oil palm production (Kina) K287.5 million in Value 2011, K193.0 million quantitative or K75.1 million K96 million in 2012, and K125.4 Qualitative) million in 2013 Date achieved 07/14/2006 12/31/2012 12/31/2013 Comments The result is achieved at more than 100% for all three years. Even in 2013 when total (incl. % small holder income from oil palm production was lower, the target was surpassed at achievement) 131%. Increase in the level of funds and resources invested by local communities to their local Indicator 2 : development Value To be determined Baseline was not Component was quantitative or after baseline results established dropped Qualitative) are available Date achieved 05/24/2010 12/31/2012 06/13/2012 Comments (incl. % Component was dropped achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Indicator 1 : Increase in smallholder yields (Tons/hectare) Value (quantitative 15.2t/ha 15-19%increase None 23.37t/ha or Qualitative) Date achieved 07/14/2006 12/31/2012 09/28/2012 12/31/2013 Comments The target is surpassed with 54% yield increase with respect to the baseline, against an (incl. % expected 15-19% increase. achievement) Indicator 2 : Kilometers of road rehabilitated/upgraded (Kilometers) Value (quantitative 0 550 km 190.0 km 223.1 km or Qualitative) Date achieved 11/15/2007 12/31/2012 09/28/2012 12/31/2013 Comments Upgrading of roads serving oil palm catchment areas. While the achievement represents vii (incl. % only 40% of the original target (550 km), it has surpassed the revised, post- achievement) restructuring, target by 117% Indicator 3 : Social and Environmental Audits Completed (Audit Report) Value 2nd Audit Report 2nd and 3rd Audit (quantitative 0 Completed Report Completed or Qualitative) Date achieved 07/14/2006 12/31/2013 12/31/2013 Comments (incl. % 3 audit reports completed, against the target of 2. Percentage achieved is 150%. achievement) Indicator 4 : Number of growers trained in sustainable production practices (Numbers) Value (quantitative 0 1500 1500 or Qualitative) Date achieved 07/14/2006 09/28/2012 12/31/2013 Comments There was no original target as this indicator was only introduced at the time of (incl. % restructuring in order to incorporate a core sector indicator. The target of 1,500 is achievement) achieved at 100%. G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 03/01/2009 Satisfactory Satisfactory 0.00 2 05/14/2009 Satisfactory Satisfactory 0.00 3 11/12/2009 Moderately Unsatisfactory Moderately Unsatisfactory 0.74 4 06/28/2010 Moderately Unsatisfactory Unsatisfactory 0.89 5 06/22/2011 Moderately Unsatisfactory Unsatisfactory 1.05 6 01/03/2012 Moderately Unsatisfactory Unsatisfactory 3.65 7 09/28/2012 Moderately Unsatisfactory Unsatisfactory 8.37 8 05/15/2013 Moderately Unsatisfactory Moderately Unsatisfactory 14.22 9 12/30/2013 Moderately Unsatisfactory Moderately Unsatisfactory 22.26 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions The restructuring (i) modified a number of key performance indicators in the results framework 06/13/2012 MU U 4.61 to reflect the changes to the scope of the project; (ii) revised total project costs; (iii) reallocated credit proceeds among viii ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions disbursement categories; and (iv) extended project closing date by one year to December 31, 2013. The restructuring reallocated credit 11/14/2013 MU MU 20.77 proceeds among disbursement categories I. Disbursement Profile ix 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Country and Sector Background: At the time of project appraisal in 2007, the agricultural sector of Papua New Guinea (PNG) contributed one-quarter of the country’s gross domestic product (GDP). In the rural sector, incomes and living standards were generally low. Although incomes were higher in the oil palm growing areas compared to other rural areas in the rest of the country, smallholder oil palm productivity was below potential. The oil palm sub-sector operated under a nucleus estate system, and was characterized by a close relationship among the following four entities: (a) palm oil milling companies with oil palm plantations and palm oil production and export marketing facilities; (b) smallholder oil palm producers who produced on 2–6 hectare (ha) oil palm blocks; (c) Oil Palm Industry Corporation 1 (OPIC) which provided extension and technical advice to smallholders; and (d) PNG Oil Palm Research Association (PNGOPRA). A significant part of total production (38 percent) was provided by smallholder oil palm growers. A major issue in rural communities was the low level of services. While communities directly located in oil palm growing areas could count on some support from OPIC and the palm oil milling companies, rural communities in general were not adequately supported by services from their local-level institutions (LLIs). Major problems existed largely because of: (a) the centralized supply-driven approach to local development, which inhibited community development programs in providing proper services; and (b) a lack of locally generated revenues to provide public services at the local level. Increasing both the production and productivity of oil palm and an improvement in community services delivery from local level governments was required to increase the living standards of rural communities in oil palm-growing provinces. Project Background: The Smallholder Agriculture Development Project (SADP) intended to make operational the Government of PNG’s Medium Term Development Strategy (MTDS) by: (a) supporting interventions that would increase and sustain agricultural output and productivity in the smallholder oil palm sector, thereby enhancing smallholder incomes, palm oil production and palm oil exports; and (b) supporting the development and demonstration of sustainable mechanisms to support local governance, promote participatory planning, and promote local accountability at the community level in both oil palm and non-oil palm communities in oil palm producing provinces. To address the needs of rural communities in the project areas, the project aimed to support a multi-sectoral component based on the community-driven development (CDD) approach that would work through the decentralized local governance framework. This approach would enable the project to demonstrate efficient mechanisms to tackle some priority issues affecting rural areas, particularly those related to past weaknesses associated with a centralized, supply-driven approach to local development. 1 The Oil Palm Industry Corporation (OPIC) is a statutory body established under the Oil Palm Industry Act 1992 to provide extension services to the Oil Palm smallholders of PNG. 1 Rationale for Bank Involvement: Project design and implementation would benefit from the long association of the World Bank Group with the oil palm sub-sector and its stakeholders, which started in the late 1960s and continued until the closing of the Oro Smallholder Oil Palm Development Project (OSOPDP) in 2001. OSOPDP successfully demonstrated that there was a strong demand by smallholders for participation in oil palm production because of the assured regular cash income and enhanced road access. It also confirmed that smallholder output and productivity could be substantially increased and that industry- and grower-funded support service organizations set up under the project, such as OPIC, can be both effective and sustainable. The involvement of the Bank in the project would ensure that smallholders’ interests were fully integrated into the design of the project that would take into consideration key lessons learned from previous projects. Further, the Bank’s involvement in the project would ensure that the design of the project drew on the lessons from numerous Bank-funded CDD projects globally and the review of specific CDD experience in PNG. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The development objective of the project was to increase, in a sustainable manner, the level of involvement of targeted communities in their local development through measures aimed at increasing oil palm revenue and local participation. The key indicators were: • Increase in smallholder income from oil palm production • Increase in the level of funds and resources invested by local communities in their local development 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The objective was not revised. However, project indicators were revised following a project restructuring which was carried out in June 2012 and became effective in September 2012 (see details in Section 1.7). 1.4 Main Beneficiaries The main beneficiaries of SADP were: • Oil palm smallholder households: Village Oil Palm (VOP) and Land Settlement Scheme (LSS). • Non-oil palm communities directly affected by the project. • Local government institutions. • Community-based organizations (CBOs). • Non-government organizations (NGOs). • Church groups. • Oil palm organizations (e.g., OPIC, PNG OPRA and the oil palm grower associations). 2 1.5 Original Components (as approved) The project consisted of three components: Component 1: Smallholder Productivity Enhancement (Total cost US$59.2 million). This component was to support: (a) smallholder oil palm development; (b) road works; and (c) agricultural extension. Some 9,000 ha of oil palm were to be established on vacant blocks of village land along existing access roads (in-fill planting) within the area already covered by existing oil palm infrastructure. The project was to finance land preparation, purchase of seedlings, fertilizers, basic tools and other inputs provided to smallholders by the mills as suppliers’ credit. About 550 km of existing provincial access roads serving the oil palm catchment area were to be upgraded (reconstruction). The project was to finance construction and maintenance contracts and the purchase of non-routine maintenance equipment. A Road Maintenance Trust Fund (RMTF) was to be established in each smallholder oil palm scheme, financed by smallholders, palm oil milling companies and provincial governments. The existing oil palm extension and research services were to be upgraded to generate and disseminate productivity improvements. The project was also to finance technical assistance and training to support the management of oil palm smallholdings during and after the project duration. Component 2: Local Governance and Community Participation (Total cost US$3.1 million). This pilot component was to support the improved provision of local services and infrastructure in the two project provinces of Oro and WNB through participatory processes (CDD). While increased oil palm income has boosted the demand for infrastructure and public services, Local Level Governments (LLGs) have been unable to respond adequately not only because of insufficient funding from government but also because of the poor allocation and management of public expenditure. Community participation in the allocation process has been found to improve the efficiency of local expenditures, and this component was to demonstrate proven but innovative mechanisms for community participation in local development. The component was to finance small community grants, technical assistance and training to LLGs and communities. The grants were also to finance small- scale community infrastructure and services through sub-projects. Component 3: Project Management and Institutional Support (Total cost US$6.5 million). OPIC was to take overall responsibility for project management, coordination and performance of the project components. Component 1 implementation was to be undertaken through OPIC and its field offices with the support of the palm oil milling companies and PNGOPRA. Component 2 was to be implemented by a management agency contracted by OPIC: (a) to set up Local Coordination Teams (LCT) at provincial level; (b) to select and contract consultants and service providers to carry out all capacity-building activities; (c) to carry out M&E activities for the component; and (d) to transfer the grants and manage the grant accounts at the LLG levels, and to transfer funds for sub-project activities to the Sub-project Implementation Teams’ (SITs) accounts at the LLGs’ request. Component 3 was to strengthen OPIC’s capacity to manage the project and to provide extension to growers. OPIC was also to provide support to and coordinate with existing HIV/AIDS awareness/prevention campaigns in the project areas. Environmental monitoring was to be supported in close coordination with the Department of Environment and Conservation (DEC). 3 1.6 Revised Components At the time of project restructuring in September 2012, Component 2 was dropped. Several reasons explained the decision to drop Component 2 on local governance and community participation. Chief among those are OPIC’s capacity limitations and the delays in the other key activities, namely road rehabilitation and infill planting. OPIC capacity issues have undermined program implementation in many ways, but the main impact was the substantial delays in project implementation. At the time of the first project restructuring (September 2012), it was deemed that even if OPIC were to recruit the management agency to implement Component 2, it would have already been too late to obtain a meaningful result under this Component before the project’s revised closing date of December 31, 2013. Given the delays in other critical project activities—road rehabilitation and infill planting—a decision was made that all project efforts would be concentrated on these activities and on improving extension service delivery. Hence, the upgrading of basic community infrastructure, to be achieved under Component 2, was not included in the restructured Project. 1.7 Other significant changes Project Restructuring. The project was restructured twice during implementation as summarized below: (i) First restructuring: A Level 2 restructuring of the project was carried out in June 2012 2 and became effective in September 2012. It involved the following changes: (a) Results/Indicators: A number of key performance indicators in the results framework were modified to reflect changes to the scope of the project. (b) Components: The scope of several activities under Component 1 was scaled back due to a shortfall in funds arising from: (i) appreciation 3 of the PNG Kina vis-à-vis the US$; (ii) a sharp increase in local costs 4 ; and (iii) revision in the Bank financing share due to insufficient counterpart funding. Some activities such as infill planting were also scaled back because of the delayed start of these activities and the possibility of not implementing the full infill program originally planned for during the remaining project implementation period. Accordingly, the revised target for infill planting by the revised project closing date was changed to 2,500 hectares. The scope of road rehabilitation/upgrading was also 2 Between the Midterm Review in September 2010 and the first restructuring in June 2012 was marked by delays getting the request from Treasury to Bank. The processing was further delayed because of non-compliance with loan covenants (overdue OPIC audits). 3 Based on the World Development Indicators, the PGK/$US exchange rate was 2.97 when the project was appraised in 2007; at the time of project structuring in 2012, the exchange rate appreciated by 30% to 2.08. At the same time, cumulative inflation was 34.4% between 2008 and 2012, with an annual average of 6.9%. This means that on average, the available amount in Kina was 30% less, whereas prices increased by 34%, compared to appraisal estimates. 4 See previous footnote. 4 significantly scaled back from the target of 550 km to 190 km of existing provincial access roads serving the oil palm catchment area (including spot repairs and drainage improvements) plus 14 km of incomplete roads in Oro. In addition, Component 2 on Local Governance and Community Participation, aimed to support the improved provision of local services and infrastructure, using a pilot CCD program in the provinces of Oro and West New Britain. This component represented a mere 4.5% (US$3.1 million) of the total project cost (US$68.8 million), and the project has continued to support local participation through other components. As a consequence, the drop of Component 2 did not require a change in the PDO. (c) Financing: The revised project cost was estimated to be US$40.2 million. The reduction in total project cost reflected the fact that the Road Maintenance Trust Fund (RMTF) concept would not be implemented within the remaining project timeframe. Under the original project design, the West New Britain and Oro Provincial Governments, together with the palm oil milling companies and project area smallholders, were to contribute US$ 23.7 million to the RMTFs. Under the revised scope of the project, it was recognized that while the RMTF study would be completed by the extended closing date, the recommendations of the study, i.e., the establishment of RMTFs, could only be implemented after the project period. The PNG Sustainable Development Program (PNGSDP) contribution to project was also reduced by about US$3 million, reflecting the reduced scope of the infill planting program. (d) Reallocations: The credit amount was reallocated among disbursement categories. (e) Project Schedule: The closing date of the project was extended by one year from December 31, 2012 to December 31, 2013 to allow sufficient time for completion of planned road works and implementation of the infill planting program and extension activities. (ii) Second restructuring. A second Level 2 restructuring was carried out in November 2013 for a reallocation of the credit among disbursement categories to strengthen road maintenance capacities in the project areas through the procurement of three graders, one for each project area. The graders were intended to be used to maintain, on an annual basis, the parts of the smallholder road networks that are in fair or good condition. The purchase of graders increased expenditures of the ‘Goods’ category. This increase was financed by reallocating expenditures from the ‘Consulting Services and Training’ category, which had unspent funds. All three graders have been purchased and received at the three project sites. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Project processing was delayed because of the long suspension of the Bank-funded Forestry Conservation Project (FCP) in PNG (suspended from September 2003 until its subsequent cancellation in June 2005). From August 2003 to September 2005, a number of Bank technical visits took place for SADP and some technical background studies were commissioned. 5 However, the main preparation activities funded under the project’s PHRD project preparation grant approved in February 2004 could not proceed as all project preparation activities in PNG were put on hold during the suspension of FCP. Supervision and technical missions were allowed to proceed. Full preparation effort resumed only after the 2005 Annual Meetings when the official restart of project preparation was agreed with the Government of Papua New Guinea (GoPNG). Design and Quality at Entry • The project’s overall design was fairly simple and straightforward, except for the design and operation of a sustainable RMTF and Component 2 (Local Governance and Community Participation). • A Social Assessment (SA) report was prepared as part of the project preparation. This included a Beneficiaries Assessment (BA), a Resettlement Policy Framework (RPF) and a Beneficiaries Participation Framework (BPF). • In 2007, as part of the development of SADP, an Environmental Assessment (EA) report was prepared, which the Environment Management Plan (EMP). These documents identified and helped manage potential environmental impacts of SADP. • The design included adequate measures to minimize and manage risks of deforestation, consistent with Bank policy requirements. The project also incorporated measures to avoid risks to critical habitats. • A thorough screening mechanism was put in place for infill planting under the project. • A number of alternatives were considered in arriving at the chosen design. On the other hand: • For a country like PNG where existing communication and transportation facilities are weak, implementation was challenging. Project design overestimated OPIC’s capacity, which had declined during the years after the end of OSOPDP. More resources should have been allocated to strengthen OPIC management and implementation capacity. • There was inadequate due diligence on the palm oil mill effluent issue. Trying to resolve this issue during implementation was very complex as the milling companies felt that the Bank had changed the goal posts. Even though the Bank might have placed a bit too much faith in voluntary certification under the Roundtable for Sustainable Palm Oil (RSPO) initiative, OPIC and the milling companies could have made more efforts to meet the Bank’s environmental safeguards requirements. Quality Assurance Group (QAG) assessment. In 2008 QAG conducted a learning review for PNG to help address outstanding quality concerns and to improve portfolio management. The overall QAG findings were that the project reflected the type of operation that was appropriate to, and essential for, fragile states of the Pacific Region. Specifically, the review highlighted that: (a) there was good sector knowledge and prior implementation experience that the Bank’s Independent Evaluation Group (IEG) had rated as Satisfactory; (b) the project contained important innovations in the form of the RMTF and a small but important CDD-type activity for non-oil palm communities in the project area; and (c) project design also addressed three short- 6 comings noted by the IEG review of the previous project, i.e., (i) road maintenance funding had not been assured, (ii) fertilizer use was low and productivity was sub-optimal, and (iii) monitoring and evaluation (M&E) systems under the project were weak. Soundness of background analysis. As part of project preparation, the sector background was studied, main sector issues were analyzed in depth, and government strategies to deal with these issues were considered. Project design took sector issues into account. Lessons of previous project taken into account. The following lessons were incorporated into the project design, which were derived from experience in OSOPDP and in the AusAID-funded road maintenance program in Bougainville: • Strong, effective project management is necessary. To reduce the extent of coordination required, project design should minimize the number of executing units. • To maximize economic benefits and to maintain the confidence of smallholder growers, regular Fresh Fruit Bunch (FFB) collection from smallholders must be guaranteed. Provincial access roads will need to receive regular maintenance. In order to reduce the unit transportation cost of FFB supplied, oil palm development should be promoted and maximized in the areas that have practical access to the road network. • Well-coordinated and continuous infrastructure contracts and funding are required to encourage the continued presence of competent contractors in the provinces concerned. • For the CDD component, parallel structures, such as social funds should be considered only if the goal is to accelerate delivery of investment funds at the local level. To address systemic challenges of local governance, including accountability, transfer equity and local participation, direct support to local level institutions is paramount for durable change. • For local groups to develop a sense of ownership of their activities and as a pre-condition of sustainability, they should participate at both the planning stage and during implementation. • Project design must be flexible. Phased implementation should be preferred to permit learning and/or redesign. Risk Assessment. The overall risk of the project was rated as Substantial in the Project Appraisal Document (PAD). The ICR team agrees with this assessment. The PAD had identified several potential risks and appropriate mitigation measures for them. At project completion, many of these identified risks had materialized. These include, but are not limited to: (i) a decline in commitment to the traditional PNG/OPIC smallholder oil palm model, although commitment remained high overall for oil palm development in the country; (ii) no provision of resources to capitalize the road maintenance trust fund; and (iii) delays in award of contracts through the Central Suppliers and Tenders Board (CSTB). One important risk, the limited capacity of OPIC to effectively implement project activities, was not adequately assessed, and this turned out to be very costly for the project. In hindsight, the Bank should have included some conditionality regarding the appointment of the OPIC Board. The other risk that was overlooked, and which was equally detrimental to the project, was the withdrawal of support for the project by private 7 milling companies. Mitigation measures should have been considered as management often changes in private companies and there is no guarantee of continuing support. 5 Adequacy of participatory processes. Free, prior and informed consultations were undertaken during project preparation and broad community support for the project existed in the three oil palm growing areas targeted under the project. More than 550 people were directly consulted, including oil palm smallholder households, non-oil palm households directly affected by the project, local government representatives, community-based organizations (including women’s and youth groups), NGOs, church groups and oil palm organizations (including oil palm growers associations). Participants in the consultations were provided information on the project verbally in Tok Pisin, other local languages and/or English, depending on the audience. On the other hand, documentation of the consultation process in the Social Assessment should have been more detailed and complete. Adequacy of Government Commitment. While the project was approved by the Board on December 18, 2007, the Legal Documents were only signed on July 9, 2008, and the Credit was declared effective on January 28, 2009. The long delays between Board approval and Credit effectiveness (about 14 months) were largely due to the difficult relationship between GoPNG and the Bank following the cancellation of FCP. The slow clearance mechanisms within GoPNG and the weak communication channels between the different government departments were also factors of delay. Subsequent to credit effectiveness, the relationship between GoPNG and the Bank continued to be difficult, hindering the timely resolution of implementation start-up issues, such as the amendment of the Subsidiary Agreement between GoPNG and OPIC, and the appointment of OPIC Board members. Restructuring. As discussed earlier, the project was restructured, and project components scaled down to reflect slow implementation progress and reduced project resources, due to strong appreciation (by 30%) of the Kina and an escalation in costs between appraisal and the time of the restructuring. Component 2 on local participation was dropped because of OPIC’s inability to recruit a management agency to implement that component, and because the remaining Project implementation time was too short to launch Component 2, which involved some CDD elements. 2.2 Implementation As mentioned below, the project had to face several challenges during implementation. The Bank conducted a Midterm Review (MTR) in September-October 2010 and assessed progress of project components, implementation issues and the actions to be taken to ensure successful completion of the project. Based on these discussions, the task team and OPIC revised the implementation schedule for the period until the original closing date of December 31, 2012, taking into account the long delay in project implementation start up. Factors affecting implementation 5 During project implementation there was a change in ownership of the milling operations in Oro province 8 The MTR and regular implementation review missions identified the following main factors that had affected project implementation: (i) Inadequate counterpart funding and delays in provision of counterpart funding from the inception of the project contributed to implementation delays and the reduction in the scope of the project. (ii) After the project became effective, there were significant delays in recruiting consultants for key project management positions, purchasing key assets to support project management operations, as well as getting implementation arrangements in place. (iii) Project management capacity was weak mainly due to: (a) lack of adequate staff capacity at the OPIC head office; (b) delayed recruitment of procurement advisors and several other positions, including the extension specialist and road engineers due to procurement delays and long delays in getting work permits and visas issued for consultants. Consequently, Project Coordinator was forced to focus more on procurement and financial management rather than overall coordination of project implementation; (c) prolonged delay in the recruitment of the OPIC General Secretary by GoPNG. This was due to the OPIC Board not being operational during the first few years of project implementation and resulted eventually in the OPIC Financial Controller fulfilling the role of Acting General Secretary; (d) OPIC management’s lack of ownership of the project and weak oversight by the OPIC Acting General Secretary and the Project Coordinator; and (e) low motivation of OPIC field staff, partly due to low compensation rates, which had not been adjusted for 10 years. This situation was exacerbated by high staff turnover, death of the procurement advisor in 2012, and several SADP officers having to be fired for misappropriating project assets or for non-performance. (iv) Project implementation progress was extremely slow due to OPIC’s inability to procure required consulting services and goods (e.g., vehicles and IT equipment) in a timely manner. Procurement delays were mainly due to the following factors: • Inordinate slowness in preparing bidding documents, short lists and requests for proposals for consulting services. • Very poor quality of various procurement documents that were submitted to the Bank for review, which did not enable the Bank to issue no objections rapidly. • Lack of prioritization of procurement activities to align them with the requirements of the implementation schedule. • Internal differences amongst OPIC staff that further delayed procurement activities. • All contracts beyond a certain amount had to be approved by the Central Supplies and Tenders Board (CSTB). The Board’s lengthy procedure for approval contributed substantially to implementation delays. • There were frequent changes in the procurement specialist position. The performance and competence of the five procurement specialists during the implementation period were not consistent, and ranged from unsatisfactory to satisfactory. (v) The Project Steering Committee (PSC), tasked with overall strategic planning and the resolution of implementation issues to ensure satisfactory progress, had never met at the time of MTR. Lack of oversight by PSC was detrimental to implementation progress. 9 (vi) Owing to the high appreciation of the Kina vis-à-vis the US$, and the high rate of inflation in PNG, funds available for implementation were inadequate. (vii) Severe damage to road infrastructure network in Oro caused by Guba Cyclone in November 2007, and extensive flooding in Oro province from November 2012 which continued into the early part of 2013, disrupted road works. At times during the 2012 PNG general election campaigning it became difficult for contractors to progress their works and for OPIC to adequately supervise contractor’s work. Implementation of the infill program was also disrupted during the election period. Also, poor communication systems (email and landline) and continued power outages at each of the three project areas made it extremely difficult to communicate between project sites and between project sites and OPIC headquarters thus detrimentally impinging on the efficiency of implementation. (viii) The OPIC Board was not operational during the first few years of project implementation because GoPNG did not appoint the Board members. Once in place, the Board was divided about the role of OPIC in delivering extension services to growers. The rates paid to OPIC extension staff had not been adjusted since 2004 and there was no provision in OPIC’s budget for the retrenchment of a number of OPIC staff who were approaching, or were already past, retirement age. This situation had a negative impact on the effectiveness of OPIC and was used as a justification by the palm oil milling companies to propose that they should take over the delivery of extension services, leaving OPIC to play a limited advisory role in the sector. (ix) Unavailability of seedlings to meet growers’ demand, the unavailability of transport to deliver seedlings, delays in land preparation for planting and delays with processing infill loans for growers contributed to extensive delays in the infilling program, particularly in the Oro project area. One of the challenges for OPIC for infilling was that it was entirely reliant on the milling companies for the provision of seedlings. Poor communication between OPIC and the companies regarding OPIC’s estimates for the future uptake of seedlings had resulted in the milling companies preparing an excess number of seedlings in their nursery. They had to destroy the seedlings when OPIC failed to use the seedlings due to delays in the start-up of the infill program. This resulted in the milling companies reserving insufficient numbers of seedlings for OPIC and led to a seedling shortage when the infill program accelerated in 2012. A compounding factor was that the milling companies were very aggrieved due to the Inspection Panel process and the Effluent Audits and, as a result, were less than supportive of SADP activities. The infill program in the Hoskins project area was suspended in May, 2013 due to irregularities in the implementation of the infill program. (x) During implementation there was only limited ownership of project activities by the milling companies, outside of road rehabilitation. (xi) Work on the design of the Road Maintenance Trust Fund (RMTF) fell significantly behind schedule, mainly due to slow progress and poor quality outputs of the consultant. 10 (xii) Intervention by the Independent Consumer and Completion Commission (ICCC) delayed the start of the review of the FFB price formula. (xiii) The Inspection Panel process had a negative impact on relationship between the World Bank, OPIC, GoPNG and milling companies, which adversely affected project implementation. Inspection Panel On December 17, 2009, the Inspection Panel registered a Request for Inspection. The Request for Inspection was submitted by the Center for Environmental Law and Community Rights (CELCOR), acting as a representative of the Ahora/Kakandetta Pressure Group, other claimants from the Oro Province and affected smallholders within the three project areas. The Request contained claims that the Panel indicated may constitute violations by the Bank of various provisions of its policies and procedures, including the following: • OP 1.00, Poverty Reduction • OP/BP 4.01, Environmental Assessment • OP/BP 4.10, Indigenous Peoples • OP/BP 4.36, Forests • OP/BP 4.04, Natural Habitats • OP/BP 10.04, Economic Evaluation of Investment Operations • OP/BP 13.05, Project Supervision • OMS 2.20, Project Appraisal. Bank Management responded to the Request as follows: • The Bank made diligent efforts to apply its policies and procedures and to pursue its mission statement concretely in the context of the Project. The Bank had in all material respects followed guidelines, policies and procedures applicable to the matters raised by the Request. As a result, Requesters’ rights or interests had not been adversely affected by a failure of the Bank to implement its policies and procedures. • Management recognized several areas for improvement: (a) No documents were translated into local languages. The Bank committed to ensure that key documents (including a summary of the environmental assessment) were translated and made available by OPIC in the Project areas in ‘user friendly’ formats and that OPIC radio programs communicated key aspects of the Project to smallholders. (b) Documentation of the consultation process in the Social Assessment should have been more detailed and complete. (c) There was insufficient detail in the EA on the matter of effluents. Based on the review, it was agreed to undertake an analysis of the impact of increased effluents due to Project activities. In light of the concerns raised in the Request, Management also agreed to take the following actions: 11 • Ensuring that inconsistencies between the Project Implementation Manual (PIM) and the Environmental Management Plan (EMP) were addressed. Some inconsistencies between the EMP and the PIM in describing the division of labor between OPIC extension officers, the environment officers and the land officers for completing the Oil Palm Planting Approval Form (PAF) were identified, and the Bank committed to discuss with OPIC and agree on making necessary amendments to the Project documents. • To further ensure that all environmental and land-related provisions were strictly adhered to, the Bank committed to discussing with OPIC and ensuring an explicit sign-off by environment and land officers on the Oil Palm Planting Approval Form. • Discussing with OPIC and agreeing on measures to further strengthen the consultation process for major activities during implementation. These included measures to ensure that the processes for community involvement and obtaining and documenting community support for the demand driven components of the Project were more explicit. • Providing inputs to OPIC to ensure that: (i) design of RMTFs was done in a consultative way with the objective of ensuring sustainability; (ii) the process of collecting and analyzing data and revising the FFB pricing formula would involve smallholders, through their representatives, and OPIC, as well as the milling companies; (iii) provisions in the Road Reconstruction Sub-Manual, the Environmental Management Plan and the Resettlement Policy Framework would be re-examined taking into account that IDA had agreed to finance some of the incomplete roads in Oro province (which were previously to be financed by PNGSDP); (iv) adequate provisions would be made for independent social and environmental audits; and (v) grievance mechanisms under the Project would be strengthened. The Inspection Panel process was completed in December 2011 and Management issued its first progress report to the Bank’s Board of Directors in January 2014. The second report is being finalized for submission to the Board. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization M&E design. The project’s Outcome Indicators and Results Indicators for each component were relevant and adequate to monitor progress and reflect the project’s achievements although it could have been useful to further develop the framework to measure project’s impacts in incomes 6. Baseline data and target values at the PDO and Component 1 levels were preliminary and were planned to be adjusted following the baseline survey to be carried out by OPIC within the first six months of project implementation. Baseline data and target values for Component 2 were not available and were planned to be established following the baseline survey. Activities envisaged under Component 2 were new and there was little information available on the communities in the project area. Collecting detailed baseline data and household information was part of the design of this component and was viewed as being an integral aspect for its success. 6 For example, a baseline survey/evaluation methodology including a treatment area and a control group (to enable a difference-in-difference evaluation) would have been necessary to attribute outcomes to project activities. The baseline conducted by the project was of limited use. 12 M&E implementation. During restructuring, as some indicators were dropped as they were ambitious, and some targets were scaled back. OPIC had its own monitoring system for indicators and the quality of data was reasonably good. However, the data were not updated regularly. Two baselines were conducted: first for Component 2, and second to cover Components 1 & 3. Both were of unsatisfactory quality. So the Project heavily relied on OPIS M&E system, which provides basic but adequate information for capturing the main achievements of the Project. M&E utilization. Appropriate data collected by OPIC, such as a detailed asset management database containing the inventory, condition and indicative repair cost of the entire network of smallholder access roads in the 3 project areas comprising approximately 2,000km and data on the roads rehabilitated under the project and the infill areas planted, were used for planning and decision making purposes. The satisfactory baseline survey would have been useful in ensuring better utilization of the MI&E, particularly with respect to making more robust assessment of project’s impacts. 2.4 Safeguard and Fiduciary Compliance Social Safeguards. While some limitations have been identified regarding social safeguard issues, as reported above under the Inspection Panel (IP) section, compliance with social safeguards requirements was moderately satisfactory overall, although the post-IP performance has been satisfactory. Except for those that were dropped as part of Component 2 under the restructured project, social mitigation measures integrated into the project design were all implemented. These included: (a) hiring of Extension Officers and Land Officers to help build OPIC’s capacity to address land issues and promote greater inclusiveness; (b) support to PNG Oil Palm Research Agency’s research to formulate a smallholder engagement strategy; and, (c) support to growers associations to enhance their capacities to effectively represent their members through the provision of office equipment and training, and getting them involved in information dissemination. Social safeguards in the Standard Operating Procedures (SOPs) were generally complied with during implementation. Project activities did not cause any physical or economic displacement (i.e., loss of homes or income sources). There were only a few crop damage claims due to road construction in Oro and these were settled in accordance with the process provided in the SADP Resettlement Policy Framework. All road upgrading projects had undergone consultations with the affected communities and Community Consent Agreements were secured prior to their implementation. Communities along rehabilitated roads also had undergone HIV/AIDS awareness seminars in collaboration with Provincial AIDS Councils. All infill block owners voluntarily applied for the program and had undergone prior training or orientation about the program. None of the infill blocks had shown signs that garden food production was disrupted. Consultations were held on the RMTF and FFB price reviews and these consultations have been documented. One of the positive outcomes of the Inspection Panel process was a substantial improvement in the documentation process. Environmental Safeguards. Independent Environment and Social Audits of SADP indicated that compliance was moderately satisfactory. Two independent environmental and social audits were conducted for the project, the first in August 2012 and the second in February 2013. 13 Performance was measured in term of the percentage of compliance with respect to a set of standard practices. Verifications were carried out through questionnaires and field visits. The audits showed slight performance improvement in overall environment management over time. Stream Health Monitoring. As part of SADP, the PNG Oil Palm Research Association (OPRA) established an arthropod indicator survey to help track changes in water quality over time. The arthropod survey for stream health monitoring commenced in February 2012 in West New Britain and was then expanded to Oro. A range of laboratory apparatus and equipment were purchased for the laboratory and the field survey. Protocols for stream assessment have been prepared and related training was conducted in the Hoskins, Bialla and Poponedetta project areas. The monitoring methodology used water quality metrics computed from survey data on aquatic invertebrate assemblages on selected sites. Land Use and Forest Cover Change in Project Areas. As part of SADP’s efforts to monitor the project’s environmental impacts, the Bank facilitated work to monitor land use and forest cover changes in project areas. This work was undertaken under a Bank partnership with the European Space Agency. In addition to objectively confirming that there was no destruction of primary forests due to oil palm cultivation in Oro since 2005, the maps produced were subsequently used in the final environmental and social audit to identify and inspect high risk blocks in the project area. This work was innovative and helped demonstrate the usefulness of satellite based monitoring to manage risks associated with oil palm cultivation. Details of this work is available online. 7 Roads. The February 2013 audit reflected a modest improvement in the categories for Road Reconstruction (from 50% to 56%) and Gravel Extraction (from 54% to 59%), although the audit revealed several areas of non-compliance and/or declining performance in both these categories. This audit also included construction site closure, which scored a low 40%, primarily due to a lack of gravel pit rehabilitation. Recommendations from the audit included: improvements to road and gravel pit monitoring, integration of Environmental Officer and Land Officer roles into the OPIC structure, improving documentation and availability of SOPs and other project documentation, introducing a formal site screening assessment, standardizing the use of PAFs and CLUAs, improved interaction with non-government organizations and government, and encouraging youth or community group employment in infill block activities. Infill Planting. A number of issues needed to be addressed by OPIC: • Inadequate documentation - The three projects used different versions of the Clan Land Use Agreement (CLUA) and, in some instances, old versions of the Planting Approval Form (PAF) instead of the final approved PAF version dated May, 2011. By end of project a legally reviewed standard CLUA and standard PAF had been implemented at all projects. It was agreed that going forward the SADP PAF would be the basis for all planting approvals, as has been the case since Project closed on December 31, 2013. 7 Details available online at: http://esamultimedia.esa.int/multimedia/publications/ESA_WB_Partnership_Report_2013_complete/ 14 • No systematic training of new growers before planting - The PAF required evidence that applicants have attended a grower workshop as one of the criteria for grower readiness. However, there were some instances when new growers had not been trained prior to the planting of seedlings on their blocks. Compliance with planting and training procedures were much improved in the second half of 2013. • Poor cultivation practices - Growers needed to improve the upkeep of their block, with the early establishment of cover crop and installation of drains where required. A replacement Extension Specialist was recruited for the last half of 2013 and his emphasis was on planting standards and block maintenance, including planting of alternative cover crops (e.g., groundnuts) for improved food security. OPIC officers were encouraged to do regular block inspections. • Inadequate buffer establishment - There was a lack of a required buffer in one of the infill blocks. This instance of non-compliance with the approved infill screening procedures under the project would make the block ineligible for RSPO certification. This particular problem was resolved and the buffer area has been returned to natural vegetation. Palm Oil Mill Effluent Action Plan. Concerns were raised during project implementation over the establishment of effective safeguards to manage potential environmental impacts associated with additional Palm Oil Mill Effluent (POME) produced as a result of SADP. Due to the delays in the recruitment of the Social and Environmental Audit consultants, work on the effluent baseline started late. The effluent baseline report was finalized in 2011 and the follow-up effluent audits were completed in February 2012 and October 2013. The February 2013 audit concluded that “overall, the palm oil mill effluent systems across all mills were operating adequately to ensure PNG regulatory compliance is achieved at each mill most of the time.” A number of actions and mitigation measures were agreed and are being implemented by the relevant milling companies as a result of the study and audit. Most actions were completed as planned, although some of the actions are expected to be completed in 2014. The Department of Environment and Conservation has updated the PNG Code of Practice for Palm Oil Processing, and the Bank has reviewed the Terms of Reference and draft reports, and participated in stakeholder consultations held in May 2012. Electronic filing of data has been achieved so that OPIC can use these data to monitor smallholder oil palm productivity as well as the condition of the road network in the long term. Fiduciary Compliance. As most of the procurement done by OPIC ended-up being prior reviewed by the Bank, no serious compliance issues were detected. Likewise, adequate financial management system and records were maintained. However, the Bank’s entity audit requirements were not complied with by OPIC. An audit action plan dating back to 2009 was agreed with OPIC as part of the 2012 Project restructuring. At the time the project closed on December 31, 2013, OPIC had received the audit certificates for 2009 and 2010, while the 2011 and 2012 OPIC audits remain outstanding. Moreover, OPIC has not adhered to the timetables agreed with IDA in the action plan. 2.5 Post-completion Operation/Next Phase (a) Transition arrangements. 15 (i) Institutional. OPIC’s capacity has been strengthened on several fronts, e.g., a well-structured reporting, documentation and archiving system that includes the establishment of a Management Information System (MIS) in Oro and a detailed road asset management database covering the entire smallholder network in the three project areas. OPIC also benefitted in terms of advice on project implementation and services of road engineers, tree crop experts, social and environmental specialists, and extension specialists who provided training to OPIC officers. OPIC has internalized PAF and other environmental and social procedures. On the other hand, OPIC has not come up with a retrenchment plan for staff, and international consultants and some staff specially recruited for the project may not be retained after project closure. Due to OPIC’s weak performance, the project area milling companies are considering the possibility of taking over OPIC’s extension role. In this context, the Bank recommended that GoPNG/OPIC carry out a strategic review of the future delivery of oil palm extension services, with the aim of seeking and reaching a consensus among the main stakeholders (milling companies, growers, GoPNG, provincial governments, OPIC) on the way oil palm extension services should be delivered, and in that context define the future role of OPIC, if any. This has not yet been followed-up by GoPNG/OPIC. Road Engineers Unit (REU): The road engineering unit had experienced and competent staff. If they are retained by OPIC, they will be able to maintain the roads. The REU has already been scaled back, and the remaining engineers are being funded through GoPNG financing to OPIC. The sustainability of this financing (possibly through the RMTF) would be critical in the continued delivery of REU services for road construction and maintenance in the oil palm producing areas. (ii) Technical. OPIC had requested financing in 2014 government budget for completing the rehabilitation of the smallholder road network. An allocation of PGK10.2 million has been made in the 2014 budget. One grader has been bought with project funds. In addition, 10% of the purchase price of the two graders has also been paid from the project prior to closing. The balance was expected to be paid by OPIC. The remaining two graders have been received and the balance payment made by the Project. All three project sites have now each one grader, to be used for road maintenance work. Infill Planting. The PAF has been accepted as a standard format as is being used for all new planting. Training has been given to growers. An awareness raising and consultation process was introduced for the infill planting program and these processes are expected to be maintained in the future (iii) Budget. OPIC is financed from a levy paid by the growers and matched by the milling companies based on production. OPIC is also trying to get budget from the government. (iv) Staffing and Management. The OPIC Board has been functioning since 2010. Currently there is no General Secretary and the Financial Controller is acting as General Secretary, which is diverting his regular work as financial controller, including financial management and taking care of audit issues. 16 (b) Follow-on project. No follow on project is anticipated in the near future. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of objective: The objective is still relevant, timely and appropriate to the current needs of Papua New Guinea’s social and economic development. It is also timely and appropriate to the needs of the country’s agriculture sector, especially in terms of improving the income of smallholders. The objective is consistent with the current Country Partnership Strategy 8 (CPS). For example, during the CPS period of 2013-2016, the Bank has envisaged support for improved productivity and profitability of smallholders growing cash crops, productivity gains for rural households growing food crops, improved sustainability and resilience to price and weather volatility, and improved market chain infrastructure. Relevance of design: The core design remains relevant and retains the potential for increasing oil palm revenue and the involvement of communities in local development. However, project design overestimated OPIC’s capacity, which had declined during the years after the end of the earlier Oro Smallholder Oil Palm Development Project (OSOPDP). While this could not be necessarily anticipated during project preparation, more resources could have been allocated to strengthen OPIC management and implementation capacity, perhaps through a project preparation fund. In hindsight, OPIC’s weak implementation capacity and the challenges posed by operating in an environment with limited private sector competition and a weak regulatory and governance environment, have been major impediments to fully achieving Project’s outcomes. In addition, more adequate due diligence could have been paid to the effluent issues, as well as to documenting consultations during the Project’s preparation phase. Relevance of implementation arrangement: The implementation arrangement remains largely relevant. However, considering the weak capacity of OPIC during implementation, it might in hindsight have been better to hire a firm to manage the project 9 , as was done for the Road Maintenance and Rehabilitation Project (RMRP) in PNG or place a stronger project management unit in the field, rather than in the capital. This said, OPIC capacity issues could not necessarily have been anticipated during project preparation, and OPIC weaknesses were exacerbated during implementation, as a result of weak government support. Moreover OPIC faced considerable staff turnover and it was difficult to replace staff. If a firm had been recruited, it would have been responsible for replacing staff. Within OPIC, hiring of counterpart funded staff for SADP was delegated to the field managers. In many cases, the caliber of individuals hired was questionable. 8 Report No. 71440-PG; dated November 8, 2012 9 These weaknesses mainly reflected the weak commitment of GoPNG which did not intervene to ensure the appointment of the OPIC Board, the provision of oversight by the PSC or timely counterpart funding. 17 Rating: Considering these factors, the relevance of objectives and design remains substantial, although the relevance of implementation of modest. Therefore, the overall relevance of the objectives, design and implementation is rated Modest. 3.2 Achievement of Project Development Objectives Moderately Unsatisfactory 10 The achievement of the development objectives is rated moderately unsatisfactory, for both the periods before and after the June 2012 restructuring, which became effective in September 2012. Before project restructuring, implementation progress had been slow, mainly due to OPIC’s weak capacity, and the limited government oversight. However, some progress was achieved in two areas: (i) undertaking preparatory works, compiling the road asset management database and scoping the roads to be rehabilitated; and (ii) supporting improved farm management and productivity increases among smallholder farmers through OPIC’s extension efforts and the significant involvement of milling companies. OPIC also undertook a major reorganization exercise for its extension services in the second half of 2011. OPIC failed to recruit a management agency to implement Component 2 on local governance and community participation. After project restructuring, project implementation progress improved, and results were achieved under the road rehabilitation and reconstruction component, with the revised target of 190km for rehabilitation and 13km for reconstruction slightly surpassed. While notable, achievements on the road program represent 43% of the appraised target. Progress was also made on the infill program, with approximately 40% of the revised target of 2,500ha in-fill planting having been achieved (11% of the appraised target). In addition, the target of the number of farmers trained in improved management practices by OPIC (1,500) was fully achieved (the 1,500 target was set during project restructuring to include a core-indicator in project monitoring system). The post- restructuring achievements were important, but they came too late and were too little with respect to the original targets. OPIC’s capacity limitations, the limited oversight, and long procurement delays, the Kina appreciation and the long protracted investigation of the Project, all contributed to the weak overall performance of the project. While the results obtained have helped growers to improve their farm management capacities, increase yields and to have improved access to roads, and while the country has also benefitted from stronger instruments for managing compliance with respect to environmental and social safeguards (improved consultation frameworks, adherence to RSPO certification guidelines and Code of Practice for oil palm processing), PNG could have gained much more from the Project. In light of the results achieved and given the significant shortcomings outlined above, the overall achievement of the development objectives is rated moderately unsatisfactory. 10 As the project scope was downsized and output indicators were revised during the September 2012 restructuring> Project outcomes have been assessed against achievements before and after restructuring. To assist in arriving at an overall outcome rating, separate outcome ratings have been weighted in proportion to the share of disbursements made in the periods before and after restructuring. Based on the two ratings, the overall achievement of the project is rated Moderately Unsatisfactory. 18 I. Project outcome before restructuring The project’s achievements before restructuring were as follows: A. Increasing oil palm revenue • Rural roads rehabilitation: Before restructuring, OPIC and REU completed the inventory and condition survey of the entire smallholder road network in all three Project areas in 2011. The condition survey covered 2,000km, which was the entire smallholder network. The original project design was to rebuild the entire pavement and drainage structure of 550km of roads. In 2011, OPIC/REU prepared an initial program of spot improvements, comprising repairs of severe defects and reinstatement of drainage along selected priority roads, based on agreed initial prioritization criteria. The bidding package included: (i) a total of 40 roads in all five divisions with a combined length of 54 km, with spot repairs over a total equivalent length of 23 km in Hoskins; (ii) a total of 26 roads in all three divisions with a combined length of 54 km, with spot repairs over a total equivalent length of 15 km in Bialla; and (iii) a total of 19 roads in all four divisions with a combined length of 81 km, with spot repairs over a total equivalent length of 11 km in Oro. The total roughly corresponded to the revised road improvement program target of 190 km. • Increase in smallholder income from oil palm production: Smallholder income increased from K75.1 million in July 2006 to K204 million in September 2012, compared to the end target of K96 million. The increase in income was primarily due to the increase in palm oil prices, but also due to OPIC and the milling companies’ efforts in increasing smallholder productivity, facilitating fertilizer distribution and application and ensuring regular FFB pick-up where road access was still reasonable. • Increase in smallholder yields: Smallholder yield increased from 15.2 t/ha in July 2006 to 19.4 t/ha in September 2012, which is a contributor (albeit not the most important factor) to the increase in income. As indicated above, yield improvement may be attributed to more frequent harvests due to the milling companies’ efforts in ensuring regular FFB pick-up and to improvements in extension advice that facilitated the use of improved farm management practices by smallholders. Significant investments were made to upgrade OPIC’s extension delivery by the time of the restructuring including increasing the number of extension officers (28 new extension staff were hired in addition to the internationally recruited Extension Specialist), improving the mobility of extension officers (through the purchase of new vehicles and motorcycles), extensive training for OPIC staff and growers associations, and communications upgrades (for example through the mobile phone closed user group setup in Hoskins). B. Increasing local participation. As no activities were conducted under Component 2 on Local Governance and Community Participation, there was no increase in local participation as it was originally envisaged under the CDD framework. However, participation has been supported in various ways as part of the activities of the other components. 19 C. Overall Assessment Considering that the project has achieved some results, but performed less than anticipated, the rating during the period prior to restructuring is rated as Moderately Unsatisfactory. II. Project outcome after restructuring A. Increasing oil palm revenue • Road Rehabilitation: Due to cost escalation, as of January 2014, 223.7 km of “spot repairs” were done rather than full rehabilitation. Under spot repairs, 43 km was graveled (full rehabilitation would gravel the entire road.) In addition, as of January 2014, 15.44 km of incomplete road has been completed, exceeding the revised target of 14 km. The project contributed to the rehabilitation of a total 238.54 km of roads, exceeding the revised target of 190 km (for details see the Table below). Road Contracts Hoskins Bialla Popondetta Total Main Contracts 54.13 54.56 81.12 189.81 2014 Surplus Funds - 1.81 31.48 33.29 Incomplete Roads - - 14.84 14.84 Incomplete (Surplus funds) - - 0.60 0.60 Total 54.13 56.37 128.04 238.54 The original design was to rebuild the entire pavement and drainage structure of 550km. But it is critical to understand why a decision was made at Midterm Review to shift the focus from road reconstruction to spot repairs, with the exception of Oro incomplete roads. Based on the scoping work outlined about, it was decided that the entire first year implementation works were to be limited to spot reconstruction (primarily longitudinal table drains, turn out drains and gravel patching). The primary objective of these repairs was to restore wet weather access for impassable roads. The rationale for this was that the targeting of resources enabled a larger number of roads to be covered across the network and thus benefiting a greater number of smallholders earlier, and generating more FFB production in a shorter time. This decision was taken on the basis of the first package tendered for the Oro incomplete roads where it became apparent that a revised strategy was needed because of the significant increase in costs for road works. 11 The road inventory also indicated that, for most roads, the conditions varied along the length of the road ranging from good to bad, therefore it made sense to adopt a spot patching approach. Community roadside maintenance was piloted during the project to ensure that rehabilitated roads were maintained. Rehabilitated roads will help improve access to critical social services, including health and education. Roads and other transport infrastructure would also 11 Preparation cost estimates were around K79,000/km for the incomplete roads (US$28,000/km). The first package tendered by PNG SDP for Oro incomplete road came in at K620,000/km, which was nearly an eight-fold increase. 20 give households better access to markets, allowing them to engage in a wider range of income earning activities and diversify their incomes. • Infill planting: The original target of 9,000 ha was scaled down to 2,500 ha under the restructuring. By November 2013, 1,006 hectares had been planted. The details are as follows : Project Area Revised targets after Achievement 2012 restructuring (ha) (ha) Hoskins 1,130 144 Bialla 520 276 Popondetta 850 586 Total 2,500 1,006 • Increase in smallholder income from oil palm production: Smallholder income increased from K75.1 million in July 2006 to K287.5 million in 2011, K193 million in 2012, and K125.4 million in 2013. The end of project agreed target at restructuring was K96.1 million. This target was surpassed in all years, including in the terminal year by more than 30 percent. However, the increase in income is largely attributable to higher palm oil prices, and OPIC and the milling companies’ efforts in increasing smallholder productivity, facilitating fertilizer distribution and application and ensuring regular FFB pick-up. It is also partly due to the rehabilitation of the smallholder road network carried out under the project from mid-2012. • Number of growers trained in sustainable production practices. As of December 2013, 1,500 growers were trained as envisaged under the target. This indicator was included in the results framework in the 2012 project restructuring, in response to the need to include a core indicator in the project’s monitoring framework. B. Increasing local participation. As Component 2 was dropped at restructuring, the project could not contribute to any increase in local participation. C. Overall Assessment in the period post-restructuring While Project’s achievements improved after restructuring, limitations have remained significant, as discussed in previous sections. Therefore, the rating during the period after restructuring is Moderately Unsatisfactory. III. Overall project outcome Since the project rating before and after restructuring is Moderately Unsatisfactory, the weighted calculation of project rating, taking into account pre- and post-restructuring disbursement rates, 21 will not change the overall rating. The rating value remains at 3, and the overall project rating is Moderately Unsatisfactory. Before After Overall Restructuring Restructuring 1 Rating Moderately Moderately Unsatisfactory Unsatisfactory 2 Rating value 3 3 3 Weight (% disbursed 19.7% 80.3% before/after PDO change) 4 Weighted value (2 x 3) 0.59 2.41 3.00 5 Final rating (rounded) Moderately Unsatisfactory * Value for each rating: Highly Satisfactory=6, Satisfactory=5, Moderately Satisfactory=4, Moderately Unsatisfactory=3, Unsatisfactory=2, and Highly Unsatisfactory=1 3.3 Efficiency Economic analysis shows good returns to investment under the Project. The economic rate of return from the project is 17.5 percent and the benefit: cost ratio was 1.35. Even under conservative assumptions of falling yields and prices, the project showed a very good rate of returns suggesting potential for smallholder oil palm development in the country in the established areas. These results are in agreement with the expected results presented in the Project Appraisal Document. For example, the results indicate relatively high returns to the smallholder oil palm plantation for the smallholder (Annex 3). The Internal Rates of Return (IRR) range from 27.9% in Hoskins to 22.6% in Oro with an average of 24.6%, which are in line with the appraised levels of returns in the PAD, estimated at 27%, 24% and 22% for the three projects, respectively. Sensitivity analyses show that these returns remain robust with drought- induced fall in yields and market-induced fall in prices. Overall economic returns of the project also remain strong. Benefits are expected from additional income from the newly planted oil palm trees, additional yields from more systematic harvesting, better block management and FFB collection activities, savings from on transport costs stemming from more efficient road operations, including reduction in maintenance costs on mills’ fleet. 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory Relevance. As explained in Section 3.1, the relevance of objectives, design, and implementation is rated as Substantial. Achievement of PDOs. As explained in Section 3.2, the achievement of the PDOs is rated as Moderately Unsatisfactory. 22 Efficiency. As discussed in Section 3.3, the Project, despite the partial implementation, continues to display good return on investments. The concept of smallholder oil palm sector development in existing oil palm areas proved to be an efficient concept that is expected to yield high return on investment, improve the provision of public infrastructure (roads), build the capacity of smallholder farmers to better manage their farms, and strengthen institutional capacity for improved management of environmental and social safeguards. Based on the above factors and the discussion in sections 3.1, 3.2 and 3.3, the overall outcome is rated Moderately Unsatisfactory. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Poverty Impacts. It is difficult to measure the project’s impact on poverty alleviation, as the roads were completed only towards the end of the project. However, these rehabilitated roads will have long term impact on poverty alleviation, because, with improved roads a large number of smallholders would be in a position to increase the sales of their oil palm fruit (FFB), thereby increasing their income. OPIC’s database indicated that as of September 2013, 2,055 families had improved access to roads. Improved roads would also improve access to schools and health facilities, as well as to markets for other products, thus enabling smallholders to improve the overall welfare of their household. In addition, in order to strengthen households’ livelihoods, workshops on income generation opportunities were conducted by OPIC under the Project. 12 Gender Aspects. Women are benefitting from improved roads and from their participation in road maintenance work, which enable them to sell their agricultural products, such as fruits and vegetables. Another positive feature of the project was that it ensured equal participation of women in the consultation processes. Social. Two major points are noteworthy: • The project was instrumental in strengthening the use of consultations with communities and beneficiaries in the palm oil sector with regard to oil palm cultivation (particularly planting), and road construction and maintenance. Extensive consultations were also held on the FFB price formula. • The project increased awareness of HIV/AIDs among smallholders, particularly those living along roads that were rehabilitated. (b) Institutional Change/Strengthening There was substantial institutional strengthening as a result of the project, as demonstrated by the following achievements: 12 For example, an Income Generating Projects Committee was established in Oro Province with representation from the growers, the provincial government, financial institutions, community organizations, churches and NGOS. One of the objectives of the committee was to help link growers with other opportunities available in the province. 23 • Safeguard measures adopted for this project in order to comply with OP4.10 (IP Policy) and OP 4.12 (Involuntary Resettlement) have improved oil palm communities’ involvement in OPIC’s development efforts, particularly in infilling and road repair and maintenance. • Implementation of the PAF ensures compliance with environmental and social safeguards and CLUA requirements, which are mandatory for screening and qualifying infill blocks. This process has not only greatly benefitted growers and OPIC to meet RSPO certification requirements, but has also created awareness in communities regarding environmental and social safeguards. • A Grievance Redress Mechanism (GRM) has been set up at OPIC’s field offices in Oro, Hoskins and Bialla. Although still quite crude, it has proved to be a useful management tool for these offices, effectively providing a system for tracking and monitoring growers’ complaints and improving response time. Growers’ awareness of OPIC’s GRM also appears to be high, with the number of grievances registered in 2012 numbering more than 900 (114 in Oro, 714 in Hoskins, and about a hundred in Bialla). Most of the grievances registered pertain to normal OPIC operations and are not project-related. • The roads component was managed through REU and the Oil Palm Infrastructure Department (OPID), the technical department of OPIC established under SADP. The REU was staffed by a team of consultants. OPID engineers worked alongside the REU team and their capacity has been significantly strengthened through this process. • The capacity of OPIC has been strengthened considerably. For example, many OPIC staff in field offices have become knowledgeable about budgeting, finance and accounting, and MIS (for details see Section 2.5). Extension officers have become more informed about environmental issues, RSPO and better block management. • SADP contributed to strengthening the capacity of extension staff because of the training programs financed by the project. • Purchase of vehicles, computers, Satellite mobile communications equipment (Vsat) and motorbikes for field visits, has greatly improved communication between growers and OPIC staff. • The review of the FFB pricing formula started in 2013, and is expected to be completed in 2014. The last review of the formula dated back to 2001, making the SADP-financed review particularly important to adjust the formula taking into account changes in the cost structure and efficiencies of smallholder production and milling. • Palm Oil Mill Effluent (POME) Action Plan. An effluent baseline was established for all mills in the project area and a follow-up effluent audit was completed. Implementation of mitigation measures identified in the baseline and subsequent audits will ensure compliance with national effluent regulations and will need to be monitored by the Department of Environment and Conservation. • A standardized oil palm training module was also developed and implemented under the project. • The capacities of the Growers Associations were strengthened. All three Associations received support from SADP in the form of office equipment (a computer and printer), MYOB accounting software was installed, and grower representatives received training on leadership, accounting and budgeting. The Bialla Oil Palm Growers Association (BOPGA) produced financial statements for 2010 and 2011. This was the first time that any of the Growers Associations had produced such statements. 24 (c) Other Unintended Outcomes and Impacts (positive or negative) • An increase in roads/civil contracting capacity in Oro could fairly be attributed to SADP. A good West New Britain (WNB) contractor has established itself in Oro as a result of winning two SADP contracts and is apparently successfully bidding for other work in the province. • By the end of the project, cultivation of peanuts was introduced as an alternative income generation product and for enhancing soil fertility. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N/A 4. Assessment of Risk to Development Outcome Rating: High The ICR team considers that the risk to development outcome would be high based on the following factors: • The RMTF study was completed and accepted by OPIC in November 2013. If the RMTF is not properly institutionalized, there is potential risk that the pickup and transport of FFB to the mills will be affected as there will be no regular maintenance. However, to partly mitigate the risk of a delay in implementing the outcomes of the RMTF study, OPIC has procured three graders, one for each project area. The purpose of the grader acquisition is to provide a reliable if limited resource for carrying out routine maintenance on smallholder roads that are in good or fair condition, until such time as the RMTF becomes operational. • Since Bank funds are no longer available after project completion on December 31, 2013, and OPIC does not receive funds from GoPNG for its recurrent budget, OPIC’s resources will likely only consist of the levies paid by growers and the milling companies. This may be insufficient to absorb the incremental staff hired under the project and finance OPIC staff salaries and field operations, let alone retrench staff due for retirement and upgrade the staff compensation package. It is likely that OPIC may face a serious financial crisis 2015, which could jeopardize the project’s achievements as well as earlier investments in the delivery of extension services to oil palm growers. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory The Bank's performance in the identification, preparation, and appraisal of the project was moderately satisfactory. During preparation and appraisal, the Bank took into account the adequacy of project design and all major relevant aspects, such as technical, financial, economic, and institutional, including procurement and financial management. A number of alternatives 25 were considered for the project design. In addition, major risk factors and lessons learned from other earlier projects in the agriculture sector were considered and incorporated into the project design. Project preparation was carried out with an adequate number of specialists who provided the technical skill mix necessary to address sector concerns and develop a good project design. The project was consistent with the CPS and government priorities in the sector at the time. However, the Bank overestimated OPIC’s capacity, which had declined during the years after the end of the Oro project. Were this capacity not overestimated, more resources could have been allocated to strengthen OPIC management capacity under SADP. (b) Quality of Supervision Rating: Satisfactory The Bank's performance during the implementation of the project was satisfactory. Sufficient budget and staff resources were allocated, and the project was adequately and intensively supervised, and closely monitored. The task team prepared Aide-Memoires regularly and alerted the government and OPIC to problems with project execution and facilitated remedies in a timely manner, in conformity with Bank procedures. Implementation Status Reports (ISRs) realistically rated the performance of the project both in terms of achievement of development objective, and project implementation. The task team also monitored fiduciary and safeguard compliance, including the efficient management of the Inspection Panel Review process and the ensuing action plan. The Bank provided necessary support and training on fiduciary and safeguards aspects to OPIC staff to ensure compliance. The task team carried out a Mid-Term Review in September 2010. Based on MTR discussions, the execution of project activities was rescheduled. The project was then restructured in June 2012 (with effectiveness in September 2012) with the project scope scaled back and the closing date extended by one year. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory With a Moderately Satisfactory rating for quality at entry and a Satisfactory rating for quality of supervision, overall Bank performance is rated as Moderately Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Unsatisfactory As mentioned in Section 2.1, there was lack of ownership and commitment on the part of GoPNG. For example, inadequate counterpart funding and delays in provision of counterpart funding since the inception of the project contributed to implementation delays and the reduction in the scope of the project. Also, the government did not provide guidance to OPIC on resolving project implementation issues. The failure to appoint a General Secretary to OPIC, and substantial delays by CSTB in approving contracts, weakened delivery capacity and ultimately resulted in the moderately unsatisfactory outcome of the project. 26 The participation of provincial governments in the SADP project was limited, 13 although this could have been more significant were Component 2 on local participation implemented. In addition communication between OPIC and government was inadequate. There were internal government problems with staff resources management, and the Liquefied Natural Gas project/extractive sector projects, drew significant attention away from SADP at the critical stage where bold government decisions to address issues could have made the difference in final project’s outcomes. Finally, the provision of counterpart funding was insufficient and the provision of funds was usually not timely, which slowed Project implementation. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory Project management continued to be weak throughout implementation mainly due to: (i) lack of adequate staff capacity at the OPIC office in Port Moresby; (ii) delayed recruitment and mobilization of the Procurement Advisor, which resulted in the Project Coordinator having to continue to focus on Procurement; (iii) prolonged delay in the recruitment of the OPIC General Secretary by GoPNG; and (iv) poor coordination of field activities, with the exception of road works which were under the supervision of REU. Lack of management oversight at all levels within OPIC had led to serious deficiencies in the infill program in the project area, particularly in the first years. This has weakened milling companies’ confidence in OPIC’s delivery capacity, and ultimately led to worsened relationships between OPIC and these companies. For example, milling companies lost seedlings in 2012 to be used by the then ill-performing program; when the program turned around and needed seedlings, they were provided in insufficient quantities, limiting the overall achievements of the program. Management of the road rehabilitation program by REU consultants was moderately satisfactory. Overall, establishment of Unit made significant contributions to SADP road construction and rehabilitation programs. Unfortunately, sustainability of the Unit will remain critical so long as there is no secure mechanism for funding and retaining the engineers on the Unit. OPIC Monitoring and Evaluation was basis, but it put in place arrangements to monitor progress toward the attainment of key performance indicators. To a large extent, OPIC had failed to provide regular monthly reports to the Bank. Procurement: OPIC’s procurement performance was unsatisfactory. Project implementation progress was extremely slow due to OPIC’s inability to procure required consulting services and goods (e.g., vehicles and Information Technology equipment) in a timely manner. Procurement performance slightly improved overtime, especially when the Procurement Advisor was recruited. However, overall procurement performance has remained low due to CSTB delays. 13 The envisaged role of provincial governments was in terms of their participation in the Project Steering Committee, as well as involvement in Local Planning Committees. 27 Financial Management: Financial management performance was moderately satisfactory. The Project Agreement required OPIC to maintain a project office with dedicated staff including two project accountants. While OPIC had one project accountant on staff, the Financial Controller had been Acting General Secretary for OPIC since 2012 and was not performing the role of the Financial Controller. In late 2012 OPIC recruited a staff member, paid by SADP, to fill the position of the Financial Controller, however his work focused on OPIC general accounts work and getting OPIC financial statements in order. As such, the Project Coordinator continued to perform the accounting and financial functions for the project instead of the Financial Controller. There were also delays in audit reports, although all project audits have now been received by the Bank. Progress has been made in clearing part of the backlog of OPIC audits, with the 2009 and 2010 audits being received by the Bank. However, entity audits of the 2011, 2012 and 2013 financial statements are yet to be completed. (c) Justification of Rating for Overall Borrower Performance Rating: Unsatisfactory In light of the Government and OPIC performance as discussed above, the overall performance of the Borrower is rated Unsatisfactory. 6. Lessons Learned Project Preparation and Implementation • SADP confirmed the well-known lesson that Project relevance alone does not guarantee implementation success in an environment where commitment and ownership are lacking on the part of the government and stakeholders, particularly the milling companies and OPIC. In addition, good relationships between the government, implementing agencies, and the Bank are also critical factors for the success of a project. Nominating members of OPIC’s Board could have been made a condition of effectiveness so that the Board would have been operational from the beginning of project implementation. However, strong government oversight to ensure that implementation issues are tackled in real time in order to keep a project on track is even more important. • The project also confirmed that adequate consultation with all stakeholders is a very important pre-requisite during project preparation and implementation. Equally important is the proper documentation of the consultation process and feedback, not only for administrative reasons, but also to record consent and smooth the path of implementation, especially in environments where land rights are highly contested. • Lessons from both project preparation and implementation demonstrate that development of an effective communication and outreach strategy can help generate awareness and participation at the community level, which would help farmers understand the benefits of a project such as SADP and not be misguided by external influences. In addition, better 28 communication and coordination between the implementing agency, milling companies and the credit provider would have benefitted the implementation of the infill program. • SADP has offered a great opportunity to learn a series of lessons on preparing and implementing projects in a weak capacity environment. OPIC has severe governance issues that greatly impacted on the ownership and implementation of the project. OPIC’s Board is dysfunctional and the post of General Secretary has been filled on an “acting” basis for years. Considering the weak capacity of OPIC, more emphasis should have been placed on capacity building at the headquarters level and the project level. Also, in order to make up for OPIC’s lack of capacity, it is essential for OPIC to seek the services of relevant subject matter specialists for evaluations and technical reviews. It is important to carefully evaluate the capacity and staffing needs of the implementing agency during project preparation, so that adequate capacity is in place for implementation. Issues of salary differentials between project and non-project staff within an organization are potential source of conflicts that may cripple the organization if they are not effectively handled, and OPIC fell a victim of that. • In addition, in a weak capacity environment, there could be a role for third parties to help manage critical sector studies, especially when the implementing agencies have limited technical and managerial capacities to effectively oversee these studies. This was required by the Bank for the FFB Review, based on OPIC’s inability to manage previous studies, and given that the study will not be completed prior the project’s closing date. • In terms of procurement lessons, the use of post-qualification requirements in 2011 resulted in local contractors in both Oro and West New Britain either not bidding, failing to qualify, or being restricted to single lots. The ‘national’ (larger) contractors, perhaps seeing little competition, generally submitted significantly more costly bids. As a result, contract prices were higher than expected, although some of these variations may also be due to base estimates being inaccurate. The new framework introduced in 2013 with lower multipliers for qualification criteria and higher thresholds for shopping should go a long way to overcoming this problem. • The poor performance by OPIC calls for more realism in assessing implementation capacity. Implementation readiness is critical for the success of project, and Projects should build in stronger measures to address lack of counterpart performance. Sector specific • Delay-induced cost increase The average cost of road rehabilitation in 2012 was approximately three times greater than the 2007 estimate in Kina and four times greater than the 2007 estimate in US$ This resulted in the scope of reconstruction being drastically reduced, and a high post-project cost remaining to restore the balance of the network to a passable condition. While this drastic reduction may not have been entirely foreseen during project preparation, exchange rate risks could have been more vigorously factored into price contingency estimates, were the LNG 29 boom adequately anticipated. 14 For extreme fluctuations, mitigation measures could be as simple as properly calibrating communication around project activities so as to effectively manage expectations. • Prioritization process Where roads within a network are prioritized for improvement, the prioritization process needs to be very carefully managed to maintain transparency and objectivity. This becomes even more important in situations where one is forced to make drastic reductions in the original scope, as was the case of SADP where two-thirds of the originally planned roads were not rehabilitated due to increased costs and shortage of funds. • Providing good side drainage, road shape and adequate cross drainage Oil palm areas are generally very flat and the roads are usually constructed at ground level, so there is little opportunity to divert surface water away from the road reserve, and in these circumstances shallow longitudinal side drains quickly silt. It is therefore critical to provide a good cross-sectional shape with a minimum 5% camber, and the road pavement sufficiently elevated above the side drains to avoid waterlogging. It also appears that when some smallholder roads were originally constructed, insufficient attention was given to cross drainage, and this caused major problems with wash-outs rendering roads impassable. • Use of screened or crushed river gravel Material available for constructing road pavements in project areas is almost entirely sourced from river gravel, either screened or crushed. While crushed gravel is clearly far superior to screened gravel, its cost is generally prohibitive. Screened gravel has to be used but should meet the specification and be within an acceptable ‘grading envelope’, which requires mechanical screening. Gravel extraction should also ensure compliance with environment requirements. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The team has reviewed the Borrower’s assessment of Project performance and the lessons learned. It appears that the assessment of SADP outcomes is incomplete, many results as discussed in Annex 2 being omitted. The Borrower rightly stressed the importance of road improvement for smallholder oil palm production and incomes. It also clearly identified key factors that have affected the implementation of this component, including procurement delays, lack of counterpart funding and significant appreciation of Kina and cost escalation in the project areas. The team also takes note of the Borrower appreciation of the Bank performance, including repeated remarks on procurement procedures. It should be noted that while some of the delays may be due to the Bank, most Bank-related delays are due to poor quality documents which required several rounds of corrections. Major delays are caused by lengthy turnaround at national level, and this proved to be very costly to the 14 The boom led to rapid influx of foreign investments that provoked a Dutch Disease in an environment of relatively rigid absorptive capacity, which led to the rapid Kina appreciation and the significant increases in prices. 30 project. The capacities built through SADP, the infill plantings, the arrangements to continue some of Project activities after closing (e.g. the FFB price review), and the purchase of one grader for each of the project sites are some of the achievements that are expected to continue to deliver benefits beyond the project implementation period. (b) Cofinanciers The Papua New Guinea Sustainable Development Program (PNGSDP) was the main co- financier for SADP. PNGSDP indicated that the roads reconstruction and maintenance program was fully justified, but the approach could have been more flexible. According to PNGSDP, SADP could have financed more roads, if the specifications for the roads were less stringent (for example, using straight screen river gravel in lieu of crushed aggregate gravel). In addition, PNGSDP also considered the rationale for the infill program was also solid. Filling the gaps along constructed and rehabilitated roads was a cost-effective proposition for palm oil operations, both for growers and milling companies. The infill program also fully justified future road maintenance work, helping to ensure that project investments in roads will be durable. While the program was slow to commence, PNDSDP ensured that the agreed interest rate with PML was maintained at 8% for the growers. Regarding the use of aggregates for road works, Bank missions advised while crushed gravel is of a far superior quality, screened gravel is a good alternative. However, that screened gravel should meet the specification and be within an acceptable ‘grading envelope’, which requires mechanical screening. PNGSDP’s support to the infill program through the provision of loans is highly appreciated. It is unfortunate the program has been stopped. It could also have been useful for SADP to have a more systematic access to PNGSDP contributions to the project. (c) Other partners and stakeholders N/A 31 Annex 1. Project Costs and Financing 15 (a) Project Cost by Component (in US$ million equivalent) Appraisal Actual /Latest Percentage of Revised Components Estimate Estimate Appraised Estimate (US$ million) (US$ million) Estimates* 1: Smallholder Productivity 55.5 28.8 15.72 28.3% Enhancement 2: Local Governance and 3.1 0.00 0.00 Community Participation 0.0% 3: Project Management and 6.2 11.4 9.69 Institutional Support 156.3% Total Baseline Cost 64.8 40.2 25.41 39.2% Physical Contingencies 1.8 - - Price Contingencies 2.2 - - Total Project Costs 68.8 40.2 25.41 36.9% Project Preparation Facility (PPF) - - - Front-end fee (IBRD only) - - - Total Financing Required 68.8 40.2 25.41 36.9% * With respect to revised estimates, percentage achieved are 63.2% overall, with 54.6% for component 1 and 85% for component 3. These figures do not include PNGSDP’s contributions. 15 These costs do not include contributions from PNG Sustainable Development Program (PNGSDP), which was to provide US$2.9 million to finance the upgrading of “incomplete roads” in Oro Province. The Program also supported PML in providing loans to smallholder growers. In their comment on the ICR, PNGSDP indicated they contributed US$7-8 million to support SADP (both roads and loan programs). It was not possible to independently verify this contribution, and OPIC financial information management system does not show records of PNGSDP’s contribution to SADP. 32 (b) Financing 16 Appraisal Revised Actual/Latest Type of Percentage of Estimate Estimates Estimate Source of Funds Cofinan- Revised (USD (Restruc- (USD cing Estimate millions) turing) millions) (iii) Borrower (ii) Parallel 7.40 5.50 2.15 39.1% Local Communities Parallel (i) 7.30 0.00 N/A N/A PNG Sustainable Parallel (i) 10.2 7.20 N/A(iv) N/A(iv) Development Program International Development Parallel 27.50 27.50 23.26 84.6% Association (IDA) (v) Local Sources of N/A(i) 16.40 0.00 N.A N/A Borrowing Country Total - 68.8 40.2 25.41 63.2% Notes: (i) The financial reporting system does not record information on all the other financiers identified at appraisal, namely the milling companies, smallholder farmers, PNG Sustainable Development and local governments in Oro and West New Britain provinces. (ii) Project accounts also show that total PNG contribution to the project amounted to about US$6.46 equivalent. Since an equivalent amount of US$2.15 has been disbursed under the project, the statements as of May 2014 shows a balance of about US$4.31 million, to be used to pay outstanding liabilities, which may include taxes. (iii) Total project costs were estimated at nearly PGK55.96 million, which is equivalent to US$25.41 million, with 91.5% (US$23.26 million) representing IDA contribution, and the remaining 8.5% (US$2.15 million) representing contributions from the GoPNG. (iv) PNG Sustainable Development Program was to provide US$2.9 million to finance the upgrading of “incomplete roads in Oro Province. Some of this funding has been released, but (v) Compared to the approved US$ equivalent amount of 27.5 million, total disbursement rate prior to project closing was 86.1%. However, out of the total IDA amount disbursed, about 1.7% or approximately US$0.4 million has been refunded back to the Bank, implying effective amount disbursed was approximately US$23.26; the corresponding effective disbursement rate is therefore 84.6%. 16 A total of PGK52.11 million was received in the designated account over the life of the project. This corresponds to a Client Connection’s record of approximately US$23.67 million as of May 14, 2014, implying an average PGK/US$ exchange rate of 2.20. 33 Annex 2. Outputs by Component Component 1: Smallholder Productivity Enhancement Infill planting: The original target of 9,000 ha was scaled down to 2,500 ha under the restructuring. By May 2013, 1,006 hectares had been planted, representing 40.2% of the revised target. The details are as follows: Project Number of Total Area Blocks (hectares) Hoskins 72 144 Bialla 127 276 Popondetta 290 586 Total 489 1,006 In addition the infill planting program helped build a robust screening and planting approval platform. The process and systems established are based on best practice procedures in terms of environmental and social safeguards screening in oil palm planting. They rely on an integrated and coordinated approach among OPIC, PML (microfinance institution) and the milling companies. Under the infill loan program, loans were extended to approved growers by PML. Loans repayments were collected by the milling company as a deduction from payments to the grower for oil palm fruit delivered to the milling companies. A Memorandum of Understanding was agreed between OPIC, PML and the respective milling companies laying out the terms and conditions of the infill program. Interest rates were capped at 8 percent and loans included a three year grace period before loan repayments became due to accommodate the long gestation period planting and the first harvest. Interest will accrue during the grace period and be capitalized on a semi-annual basis. In addition, the capacities of OPIC to systematically improve the documentation and filing of consultation records were enhanced as part of the infill program. However, this process was gradual and was only firmly grounded towards the end of 2013, with the acceleration of the pace of new plantings. Road Rehabilitation Due to cost escalation, as of December 2013, a reduced length of 205.4 km of roads were upgraded through spot repairs and drainage. Under this framework, a total length of 43 km was graveled, which could be considered to be fully rehabilitated. In addition, the revised target of completing 13 km incomplete roads in Oro Province has also been met. The demonstration of 50 community contracts highlighted the potential for using rural communities to undertake small maintenance works, to help prolong the life of the road infrastructure. Community road maintenance not only helps in providing extra income to the community group, it also helps strengthen ownership, hence improving the chances that the infrastructure will continue to benefit from the community maintenance over time. 34 The REU established, maintained and handed over to OPIC a detailed asset management database containing the inventory, condition and indicative repair cost of the entire networks of smallholder access roads in the 3 project areas comprising approximately 2,000 km. The database also provided the means of prioritizing annual maintenance and rehab interventions through a simple comparative cost/benefit analysis. The first Oro incomplete road package had to be completed by OPID using force account after the contract had to be terminated. The successful completion of this work demonstrated that force account can be successfully applied in the right conditions. To secure consistent funding for oil palm road maintenance, a Road Maintenance Trust Fund (RMTF) study was completed. The study allowed for an element of user pay mechanism, where stakeholders take ownership of the program. A part of the study, extensive consultations were held in the three project areas. The report made recommendations for smallholder farmers, milling companies and the Government of PNG (GoPNG) to contribute to the RMTF. However, the proposed levels of contributions to the fund are still to be endorsed by all parties involved. The study proposed that GoPNG contribute the full cost of the one-time road rehabilitation (K 60 million) and 80 percent of the annual maintenance cost of K 50 million. The remaining 20 percent would be covered by the mills and the growers in the ratio of 15 percent for the mills and 5 percent for the growers. The RMTF has not yet been implemented as designed, although some road maintenance activities continued after the Project closed. As part of transitory road maintenance arrangements, SADP facilitated the purchase of three graders for each project site to facilitate road maintenance until the RMTF is institutionalized. Some of the oil palm stakeholders, particularly the milling companies, have recommended the use of in-house maintenance equipment for road maintenance in the project areas because they are expected to be generally more economical over tendering for maintenance contracts. While final arrangements are still to be made, it is likely that a combination of in-house road maintenance with outsourcing of some specialized maintenance work would deliver superior results, both in terms of cost-effectiveness and speed/quality of the works. The graders are owned by OPIC and will be operated by the milling companies under an MOA. The RMTF provided the basis for a Public Investment Project (PIP) submission for K70 million in the 2014 budget of which only K10.2 million was allocated. Support to emergency road repairs prior to effectiveness. SADP helped repair damaged roads in the aftermath of the 2007 cyclone Guba in Oro province. About 740 smallholder blocks regained access for their FFB collection as a result of the repairs undertaken. As the project was not effective, the co-financier (PNGSDP) advanced the funds for the emergency works (repairs of culverts, etc.) from the budget allocated for the Oro incomplete roads and these were later adjusted by IDA paying for one of the packages of the Oro incomplete roads. 35 Extension Service: Significant investments were made to upgrade OPIC’s extension delivery by increasing the number of extension officers (28 new extension staff plus an internationally recruited Extension Specialist were hired under the project), by improving the mobility of extension officers (through the purchase of new vehicles and motorcycles), through extensive training for OPIC staff and growers associations, and through communications upgrades (for example through the mobile phone closed user group setup in Hoskins). A major reorganization of OPIC’s extension services commenced in 2011 and the reorganization implementation began in early 2012. However, the reorganization was not fully supported by the milling companies as it was seen as an expansion of OPIC’s role and was to a large extent abandoned by late 2012. A number of frameworks and processes were put in place to support the extension efforts, which includes a consultation framework and a communication strategy, both completed and operationalized as part of project activities. The project has also established an oil palm grower curriculum (planting approval process, financial literacy and oil palm agronomy), and smallholder block inspection protocols have been put in place, which helped enhance communication between growers and OPIC staff. Component 2: Local Governance and Community Participation This Component was dropped. Component 3: Project Management and Institutional Support Building OPIC’s Capacity: The project financed training of OPIC staff in project management, financial management and procurement. In addition, relevant staffs were trained on Environmental and Social safeguards to meet RSPO guidelines and to facilitate the planting approval and road maintenance works. The project also financed purchase of vehicles, office and telecommunication equipment, and the MIS to encompass smallholder database. Improvements were made to housing for extension staff in OPIC villages; two new houses were constructed and four houses were repaired in Hoskins. Nine (9) were repaired in Bialla, and another three houses were repaired in Popondetta (Oro) project sites. Some furniture and white good were also purchased for OPIC villages and offices. Several critical studies were completed under the project including the environmental and social audits; the effluent baseline study and follow-up audit; a smallholder engagement strategy and the stream arthropod indicator study undertaken by PNGOPRA; and the RMTF design study. The capacity of the Growers Associations was strengthened and an MIS system established. SADP supported the FFB price formula. This review was necessary because the previous review dated back to 2001 and followed earlier reviews in 1990, 1996, and 1998. Because of the milling companies’ monopsony relation to the growers, the price review is a very sensitive issue. While the growers generally felt that the review was well overdue and that the review should result in an increase in the price they received for their FFB, the milling companies saw the formula as a fair, were worry about the privacy of their operations cost, 36 and considered that the review would only generate a limited value added. OPIC management was successful in reconciling the positions and the review was finally agreed. The terms of reference for this consultancy were approved the Department of Agriculture in October 2010 and expressions of interest were advertised. However, in late 2010, an objection was raised by the Independent Consumer and Competition Commission (ICCC) on the allegation that OPIC/SADP will participating in a price fixing scheme through the review. The review was then put on hold on ICCC demand, and was only allowed to proceed after a conference was held in September 2012 on the issues between ICCC, OPIC and smallholder growers. Expressions of Interest were called in November and received in January 2013, but the contract was only approved on August 27, 2013, mainly due to delayed CSTB approval. Due to the delays, it was not possible to complete all activities scheduled under the contract before the project closed. With the Bank advice, a separate agreement was entered into between OPIC and the Institute of National Affairs (INA) whereby INA had agreed to oversee the FFB Price Formula Review and Audit after SADP closed. The agreement was successfully executed, and INA has been working to ensure the implementation of the revised formula, including the revival for commodity review committee for regular monitoring and audit of the formula. Appendix 1 presents the project’s results framework. 37 Appendix 1: Results Framework and Monitoring (Post-Restructuring) Papua New Guinea: PNG Smallholder Development Project Project Development Objective (PDO): The Project Development Objective is to increase, in a sustainable manner, the level of involvement of targeted communities in their local development through measures aimed at increasing oil palm revenue and local participation. Revised Project Development Objective (PDO): The PDO was not revised D=Dropped Core C=Continue Unit of End Results % PDO Level Results Indicators* N= New Baseline Measure Target Achieved Achieved R=Revised Indicator One: C Kina K75.1 K96 K287.5 131% for Increase in smallholder income from million million million in 2013. oil palm production 2011, K193.0 million in 2012, and K125.4 million in 2013. Indicator Two: D Increase in the level of funds and resources invested by local communities to their local development INTERMEDIATE RESULTS Intermediate Result (Component One): Increased smallholder revenues from oil palm production in an environmentally sound manner Revised Intermediate Result (Component One): Intermediate Result indicator One: Tons/hectare 15.2 15%-19% 23.7 t/ha 154% Increase in smallholder yields C increase (54%) Intermediate Result indicator Two: Kilometers 0 190.0 km 223.1 km 117% Kilometers of road N rehabilitated/upgraded Intermediate Result indicator Three: Audit 0 2 3 150% Social and Environmental Audits N Report completed Intermediate Result indicator Four: Numbers 0 1500 1500 100% Number of growers trained in C sustainable production practices Intermediate Result indicator: Decrease in FFB losses due to road- D related collection problems Intermediate Result indicator: Decrease in the transport costs along D rehabilitated roads Intermediate Result indicator Four: Decrease in number of non- compliance incidents from D environmental audit Intermediate Result (Component Two): Improved community participation and local governance in the project areas Revised Intermediate Result (Component Two): Component was dropped 38 D=Dropped Core C=Continue Unit of End Results % PDO Level Results Indicators* N= New Baseline Measure Target Achieved Achieved R=Revised Intermediate Result indicator: Percentage of people in targeted villages satisfied with their level of D participation in local decision making processes at ward and LLG level Intermediate Result indicator: Number of sub-projects commenced, successfully completed and D maintained Intermediate Result indicator: Level of contributions to sub-projects D mobilized from local sources *Please indicate whether the indicator is a Core Sector Indicator (see further http://coreindicators) 39 Annex 3. Economic and Financial Analysis 17 1. Introduction The development objective of the project was to increase, in a sustainable manner, the level of involvement of targeted communities in their local development through measures aimed at increasing oil palm revenue and local participation. The project had three main components: (i) Improvement in smallholder productivity; (ii) Enhancing capacity for local governance and community participation; and (iii) Project management and institutional support. With the aim smallholder productivity, the project envisaged two investment activities: in-fill planting of smallholder oil palm and construction and maintenance of rural roads for collecting Fresh Fruit Bunches (FFB) and promoting timely harvesting and processing of FFB. The original project objectives under the first component were: (i) infill planting of 9,000 ha along existing access roads; (ii) upgrading and reconstruction of about 550 km of provincial access roads; and (iii) improving extension and research services for oil palm development. The project was later restructured and the original target of infill planting was scaled down to 2,500 ha, which represents only 28% of the original target. By December 2013, 978 hectares (39.1% of the revised target) had been planted (11% of the original target). The revised targets and achievements for infill planting at project completion are as follows: Table 1: Project targets for infill planting and achievements Project Area Revised targets Achievement Percentage (ha) (ha) 18 achieved Hoskins 1130 144 12.7% Bialla 520 254 48.8% Popondetta 850 580 68.2% Total 2500 978 39.1% The new infill plantings as they become mature will have higher yields and will thus increase the overall productivity and incomes from the plantation. Oil palm has a gestation period of 3-4 years before the new plantings begin to yield; it reaches stable yield levels in about 6-7 years. The original target for road construction and maintenance was also scaled down at restructuring, from 550 km to a revised target of 190 km. As of December 2013, 205.4 km of road update was completed as spot repairing; 43 km of the upgraded roads were fully graveled. The road construction took place in 2012 and 2013, following significant initial delays in road rehabilitation works and overall project implementation. The road rehabilitation facilitated 17 Figures used in the economic analysis are slightly less than the final figures reported in Annex 2. For example, infill planting area in this section amounted to 978ha, representing 39.1% of the revised target; final achievement is 1006 ha, which is 40.2% of this target. Revising the economic analysis with the new figures would marginally change estimates (upward) but this would not alter the main conclusions of the analysis. 18 This corresponds to results achieved as of December 2013. The latest figures, as of May 2014 are slightly higher, 1,006ha, but the difference does not significantly affect the results of the economic and financial analyses. 40 timely harvest and thus higher output, as well as timely processing and higher oil quality; it also reduced transport costs. Due to poor implementation progress, the second component to improve local governance and community participation was dropped during project restructuring. 2. Financial analysis of smallholder oil palm development The financial analysis provides evidence as to whether the financial investment in the project is recovered. It provides an indication of the financial sustainability of the project. The financial analysis is done for one-hectare oil palm plantation using the data available at the time of project completion. Oil palm is a perennial crop with a gestation period 19 of 3-4 years, with full and stable yield reached in 7-8 years. The economic life of the plantation is about 27 years. Since the project got delayed the initial plantings were done after May, 2011 and these new plantings have not yet reached the mature age and stable yields. Hence, it is too early to estimate the yields from the in- fill plantings done under the project. 2.1 Assumptions Assumptions made in the analysis refer to (a) smallholder yields, (b) fertilizer application, (c) labor input, (d) material costs, (e) the extent of intercropping food crops around young palms, and (f) the farm gate price. a. Smallholder yields: New plantings under projects are still in gestation, so data on actual yields could not be obtained. Projected yields from the new plantings based on data provided by OPIC in Hoskins, Bialla and Popondeta project areas. These yield projections assume that smallholders will follow fairly average management practices including fertilizer application following the recommendations of extension agents provided under the project. b. Fertilizer application: The project has encouraged smallholders to apply fertilizers and follow average management levels. The smallholders are assumed to continue to use average levels of fertilizer application and plantation management. The fertilizer inputs are priced at the farm gate price. c. Labor input: Labor costs for planting, plantation management and harvesting are computed using technical coefficients on labor requirements provided by OPIC based on data collected from smallholders. Labor is then valued at the minimum wage rate for unskilled labor in PNG. Although there is high rate of unemployment, it is assumed that the agricultural wage rate in the local labor markets is around the minimum wage rate for unskilled labor. d. Material costs: Other major costs for planting and maintenance are the costs of implements and tools used. The costs of those material inputs are computed based on required inputs’ data provided by the OPIC. 19 Gestation period refers the period from planting to first harvest. 41 e. Intercropping/raising food crops around young palms: When newly planted palms are young before the full canopy is established, farmers plant annual food crops as intercrops around the young palms. These intercrops generate additional revenues for the smallholders, which are considered as additional benefits from the project. f. Farm gate price: Smallholders sell FFB to the milling company after harvest at the farm gate by the roadside. The milling company pays the farmers for the FFBs at a price derived based on the palm oil price in the international market. In order to do the financial analysis, we have used the actual prices received by the farmers for the FFBs in years 2012 and 2013. For the future years we have used the FFB prices tied to the World Bank’s commodity price forecasts for palm oil. In general, prices have increased significantly since appraisal. 20 2.2 Financial measures of project worth The Financial rate of return to infill planting is based on a standardized project costs and benefits of one hectare under smallholder plantation management practices (details shown in Appendix 1). The costs of infill planting establishment and maintenance are derived from the PAD and updated with OPIC costs data as well as data current available price. The returns to infill planting include the returns from FFB harvested and the revenues from planting annuals around young palms in the first three years until the full canopy is established. The opportunity cost of land for alternate competing uses is also included as costs. The results indicate relatively high returns to the smallholder oil palm plantation in the smallholder (Table 3). The Internal Rates of Return (IRR) ranges from 27.9% in Hoskins to 22.6% in Oro with an average of 24.6%. The high returns are in line with the appraised levels of returns in the PAD, estimated at 27%, 24% and 22% for the three projects, respectively. The Food and Agriculture Organization also reports similar returns as high as 25 percent and above for oil palm development projects in Hoskins and other regions in Papua New Guinea. With increasing demands for palm oil from India and China, the prospects for oil-palm production and the industry continues to be good. These returns remain robust with drought-induced fall in yields and market-induced fall in prices. The rates of return show that oil palm plantation in the smallholder sector yield very good returns on the investment under average management conditions. The Internal Rates of Return (IRR) in Table 3 ranges from 27.9% in Hoskins to 22.6% in Oro with an average of 24.6%. Such high returns are in agreement with the expected rates of return presented in the PAD at the time of project development. Food and Agriculture Organization also reports similar returns as high as 25 percent and above for oil palm development projects in Hoskins and other regions in Papua New Guinea. With increasing demands for palm oil from India and China, the prospects for oil- palm production and the industry continues to be good. 20 Compared to 2007 when the project were appraised, world price for crude palm increased by 4% in 2011, 28% in 2012 and 10% in 2013. 42 In order to analyze the returns under alternative scenarios that may arise from yield and price risks, we have also presented sensitivity analysis under two scenarios. The motivation for this analysis is that, in addition to timely harvesting and transportation, returns from oil palm in PNG depend on the yield and prices for FFB. The yields in turn depend on timely and adequate rainfall. The smallholders in PNG are price takers and the farm-gate prices are determined by prices in the international market. The first scenario considers the case when there are variations in annual rainfall resulting in fall in yields by 50 percent in every five years 21 . Under this scenario, the rates of return ranged from 19.8 in Oro to 24.4 in Hoskins. The benefit/cost ratios dropped marginally but the benefits covered costs even under conditions of falling FFB yields. The second scenario combines the above scenario with an annual decline in farm gate price by 5 percent. In this scenario, falling yield and prices, the rates of return drops to 14.2 percent in Oro to 20.3 percent in Hoskins with an average of 17.7 percent. Thus, the above analysis shows that even under moderate shocks in yields and prices the investment in oil palm would continue to yield robust returns. Table 3: Financial analysis: Returns to investment in infill (one hectare) Net Present worth IRR B/C ratio (Kina/hectare) Standard one hectare model Hoskin 27.9% 2.03 16,249 Bialla 23.0% 1.69 10.823 Oro 22.6% 1.68 10.694 Average 24.6% 1.80 12,589 Sensitivity analysis: Reduction in yield due to poor rain Hoskin 24.6% 1.77 12,123 Bialla 20.2% 1.48 7,508 Oro 19.8% 1.47 7,373 Average 21.6% 1.57 9,001 Sensitivity analysis: Reduction in yield (as above) and prices falling at an annual rate of 5% Hoskin 20.3% 1.35 5,578 Bialla 18.3% 1.28 4,350 Oro 14.2% 1.09 1,391 Average 17.7% 1.24 3,773 21 It is assumed that there could be shortfall in rainfall in every five years resulting in yields falling by about 50 percent. 43 3. Economic analysis The economic analysis is for the whole project, in the absence of reliable information on specific benefits and overhead costs per project areas. 3.1 Project benefits and costs Benefits. Project benefits are estimated using the incremental benefits approach by comparing the returns with and without the project. The incremental benefits included the following: incomes from newly planted infill areas (978 ha are described earlier); increased harvests and yields of FFBs from existing palm oil stands; savings in FFB mill collection fleet as a result of road improvements; and additional collection from Oro scheme as a result of road improvements. a. Incomes from the newly planted oil palm trees: Total benefits derive from extrapolating the net benefits of the one-hectare standard model the total area planted (978 ha); b. Additional returns from more systematic harvesting and FFB collection activities as a result of the project: The rehabilitated road network in project areas (from mid-2012) is expected to increase timely harvest and collection of FFB from smallholders, thereby increasing returns to the smallholder farmers in the project areas. More importantly, use of improved management practices, including fertilizer use (facilitated by milling companies), is also expected to be a major driver of the increase overall harvests of FFB from smallholders. Data on returns of smallholders in the project areas were collected for the years before the project and after the project. The smallholder’s return in year 2007 is used as the baseline. The smallholder returns in years 2011 through 2013 were compared to the above baseline and the incremental/additional returns are assumed to accrue to the project activities 22. c. Saving in transport costs and savings to the mill collection-fleet: Improved roads are expected to reduce transportation costs of the mill collection-fleet and to facilitate timely transport of FFB to processing centers. From mid-2012 onwards, the improved roads also reduced the maintenance costs of the collection fleet. Both savings (transport and maintenance costs) may be considered as additional benefits from the project investments. d. Additional assumptions. Many exogenous factors have contributed to the increased in smallholder oil palm incomes. Appendix 2 is an attempt to separate the impact of the exogenous factors (namely price) from the impact of the project related factor (production). Based on the analysis, approximately 40% of the impacts on incomes can be attributed to the Project. Hence this analysis assumes that only 40 percent of the increase in smallholder incomes observed could be attributed to the project. 22 Since we do not have similar data from another area comparable to the project area without similar interventions we could not do a difference-in-difference analysis to confirm increase in smallholder revenues as a result of the project. 44 Project costs. The actual project expenditures obtained from the implementing agency are used in this analysis. The total project costs were $23.28 million over a period of five years. 3.2. Measures of project worth The analysis showed that the investment in smallholder oil palm development in Papua New Guinea is a very lucrative investment. The project’s Economic Rate of Return was 18.4% and the Benefit: Cost ratio was 1.35. The Net Present Value of project investments worked out to US$12.7 million. These results suggest that smallholder oil palm development is a lucrative business in the country under favorable output prices and weather conditions that result in favorable yields. The results justify the project strategy to develop smallholder oil palm through infill planting, development of rural roads, improved management through extension activities and timely harvesting and collection of FFBs. Such investments could raise smallholder incomes and could have significant impacts on poverty. 3.3 Sensitivity analysis Two scenarios are considered for the sensitivity analysis: (i) yields assumed to fall every five years by 50 percent below the average levels; and (ii) FFB prices fall at an annual rate of by 5 percent in addition to the fall in yields by 50 percent in every five years. The results, shown in Table 4, reinforce earlier conclusion that smallholder oil palm undertaking is a lucrative business, even under stressful weather and price conditions. The analysis further shows the significant impacts of access to rural roads and timely harvest and collection of FFBs on returns to oil palm production in smallholder sector in Papua New Guinea. Table 4: Summary measures of project worth Economic Rate Benefit/Cost Net Present of Return ratio Worth (%) (Million Kina) Base case 17.3 1.35 12.69 Sensitivity analysis Scenario 1 14.3 1.14 4.11 Scenario 2 26.6 1.04 3.42 These results are in agreement with the ex-ante analysis presented in the Project Appraisal Document. It may be noted that there were significant delays in project implementation in the beginning and consequent restructuring. When the project was restructured, the investment was scaled down and the second component was dropped. However, the results of the economic analysis shows robust returns even when the project was scaled down which strengthens the case for investment in smallholder oil palm sector in the country. Potential poverty reduction impacts. The robust returns from oil palm development in the smallholder sector shows opportunities for increasing smallholder incomes and poverty reduction in the country. Proper support to the existing smallholder oil palm establishments, following RSPO guidelines, is expected to raise incomes and reduce poverty. 45 Appendix 1: Smallholder oil palm plantation: Financial model (Kina per hectare) Production years Year 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Costs Labor costs 618 641 412 204 242 278 314 350 386 424 462 497 532 552 561 561 561 561 561 561 561 561 523 Equipment & M aterial 1657 114 500 294 294 306 294 419 463 294 319 313 294 321 456 444 321 306 294 294 333 294 444 costs Total costs 2275 756 912 498 536 584 608 769 848 718 782 810 826 874 1017 1005 882 867 855 855 894 855 967 Yield profile (FFB, tones/ha) Hoskins 0 0 0 3 5 8 10 12 14 16 18 20 22 24 24 24 24 24 24 24 24 24 22 Bialla & Oro 0 0 0 3 5 5 7 9 10 12 14 16 18 20 21 23 23 23 23 23 23 23 23 Gross income (K/ha) Hoskins 0 0 0 1097 1253 1389 2003 2434 2870 3328 3765 4222 4638 4888 4992 4992 4992 4992 4992 4992 4992 4992 4532 Bialla 0 0 0 1097 1114 920 1555 1768 2142 2538 2912 3286 3682 4056 4430 4680 4784 4784 4784 4784 4784 4784 4784 Oro 0 0 0 1006 1012 859 1413 1793 2172 2573 2953 3333 3733 4113 4493 4746 4851 4851 4851 4851 4851 4851 4851 Net income (including from intercrops) Hoskins -2275 -256 -412 1099 217 305 895 1164 1522 2110 2483 2913 3312 3514 3475 3487 3610 3625 3637 3637 3598 3637 3065 Bialla -2275 -256 -412 1099 78 -163 447 499 794 1320 1630 1977 2355 2682 2914 3175 3402 3417 3429 3429 3390 3429 3317 Oro -2275 -256 -412 1008 -24 -224 305 -14 824 1355 1671 2023 1287 2739 2976 3240 3469 2029 3496 3496 3457 3496 1929 All regions -2275 -256 -412 1068 91 -27 549 729 1047 1595 1928 2304 2692 2978 3121 3301 3493 3509 3520 3520 3482 3520 3255 46 Appendix 2: Estimated Average Contribution of Price and Production to Monthly Changes in Smallholder Oil Palm Incomes in Oro, Hoskin and Bialla Project Areas 23 Data. Smallholder oil palm income depends on both price and production. Both price and production are further determined by other factors, such as exchange rate for prices, and weather or road conditions for production. Trends in Oil palm Fresh Fruit Bunch (FFB) prices and productions show a steady decline since 2011, with only slight reversal in price trends in 2013. Details for the three project areas are shown below. Compared to the baseline, smallholder oil palm growers’ incomes have increased, as reported in the report. These increases are driven by the combined effects of production and price changes. In order to understand how smallholder oil palm incomes change as productions and prices change, the analyses uses OPIC monthly data on the three variables from 2011 to 2013. Methodology. A simple regression technique using a robust, zero constant, ordinary least square estimation was used to compute the average contributions of price and production changes to observed average changes in incomes. The dependent variable in this regression is the percentage change in monthly oil palm incomes of each Project area, and the two independent variables are 23 Data and regression estimates available upon request. 47 the corresponding percentage changes of monthly FFB prices and productions. The data on monthly smallholder oil palm incomes, FFB prices and productions were obtained from OPIC. All estimates are highly significant at 1% level, and the use of robust estimates on differenced data minimizes biases associated with autocorrelation. Results of the Analysis Oro Project Area Coefficient Mean Estimated Contribution to total Variables Estimate value value change % Change in monthly revenue -0.04702 -0.04557 100% % Change in monthly FFB price 1.02514** -0.02281 -0.02339 51% % Change in monthly production 1.04413** -0.02125 -0.02218 49% Notes: Number of observations: 33, from January 2011 to October 2013 Hoskin Project Area Coefficient Mean Estimated Contribution to total Variables Estimate value value change % Change in monthly revenue -0.04302 -0.04458 100% % Change in monthly FFB price 0.99896** -0.02426 -0.02423 54% % Change in monthly production 0.99176** -0.02052 -0.02035 46% Notes: Number of observations: 33, from January 2011 to October 2013 Bialla Project Area Coefficient Mean Estimated Contribution to total Variables Estimate value value change % Change in monthly revenue -0.04026 -0.04409 100% % Change in monthly FFB price 1.01895** -0.03079 -0.03137 71% % Change in monthly production 0.98415** -0.01292 -0.01272 29% Notes: Number of observations: 29, from January 2011 to June 2013 Summary Contribution of Price and Production Changes in the 3 Project Areas Average contribution of price change (3 projects areas) 59% Average contribution of production change (3 projects areas) 41% Results and limitations. The analysis only provides an approximation of the average contribution of production and price changes to average monthly change in smallholder oil palm incomes. On average, 41% of such change can be attributed to changes in production. The analysis does not provide direct evidence of the project’s contribution to increased incomes. This contribution is inferred from the knowledge that: (a) production increase does contribute to income increase, in the magnitude of 4 Kina for each 10 Kina increase in income; and (b) a number of project’s achievements, in terms of extensions support and road rehabilitation are strongly correlated with increase in production. An analysis based on difference-in-difference of impacts, based on a more rigorous collection of project data, would have yielded a much robust estimates of the impacts of the project, but this approximation is a first step in the right direction. 48 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Mona Sur Lead Agriculture Economist/TTL ECSAR Kofi Nouve Senior Rural Development Specialist EASNS Allan Tobalbal Oliver Operations Officer EASNS James Orehmie Senior Environmental SASDI Monday Engineer John R. Butler Lead Social Development Specialist ECSOQ Oliver Braedt Sector Leader AFTSN Christopher Robert Fabling Senior Financial Management Specialist EASFM Stephen Paul Hartung Financial Management Specialist EASFM Olga Hoxon Rizo Program Assistant EASER Erik Caldwell Johnson Senior Operations Officer EASNS Angela Nyawira Khaminwa Senior Social Development Specialist AFTCS William Casmir Mandui Operations Officer EASNS Cristiano Costa e Silva Nunes Senior Procurement Specialist EASR1 Nicolas Perrin Senior Social Development Specialist ECSSO Michelle Lisa Chen Program Assistant SASDO Gitanjali Ponnambalam Country Program Assistant EACNF Barbara Verardo Senior Rural Development Specialist SASDL Philippe Fernand Boyer Consultant EASNS John Lowsby Consultant, Infrastructure Engineer - Tom Vigus Consultant Environmental Specialist - Tim Armitage Consultant, Roads Engineer - Joe Nagy Agriculture Economist - Christiphe Ribes-Ros Consultant, Social Development Specialist - Nuno Santos Consultant, Economist - Consultant, Social & Environmental Safeguards Jonas Bautista - Specialist Robert Crooks Consultant - Guzman Garcia-Rivero Consultant - Lars Lund Consultant - 49 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending 23.23 125.22 FY03 35.11 160.65 FY04 24.65 119.58 FY05 27.20 225.69 FY06 36.70 257.95 FY07 19.24 67.50 FY08 23.23 125.22 Total: 166.13 956.58 Supervision/ICR FY08 0.00 19.84 FY09 12.91 132.20 FY10 22.32 293.71 FY11 13.83 143.00 FY12 20.94 225.60 FY13 10.44 234.09 FY14 9.64 101.18 Total: 90.08 1,149.61 Grand Total: 256.21 2,106.19 50 Annex 5. Beneficiary Survey Results N/A 51 Annex 6. Stakeholder Workshop Report and Results Consultations on the draft ICR were carried out in the project areas and among representatives of project stakeholders in Port Moresby on December 12, 2013. This annex summarizes some of the key perceptions, issues identified, and recommendations as presented and discussed with stakeholders. Some of the statements may not be accurate, and may be exaggerated, and one should refer to the main ICR for a factual treatment of these issues. 1. Perception on Major Issues The overall perception from various stakeholders—including beneficiary farmers, OPIC, PNG Oil Palm Research Association, Milling Companies, and various departments in charge of Environment and Conservation, Agriculture and Livestock, National Planning and Monitoring as well as Treasury—was that the project had a good design, and the project objectives were relevant and well defined during project preparation. The Project’s focus on roads reconstruction and maintenance, in order to stimulate increased FFB production through more frequent harvests (and higher yields), adoption of improved farm management practices, was equally relevant. The ultimate design goals were to optimize the use of space, by increasing yield per hectare, production per kilometer road, and greater production of mills in each Project area. Stakeholders have also recognized that the Project concept drew from past experiences, building especially on the remnants from previous ORO project by WB from 1990s. While recognized the relevance of the design, stakeholders also highlighted some of the main issues that have crippled Project implementation. The first oft-mentioned issue was the long implementation delays, mostly linked to procurement challenges, as well as difficulties in getting timely responses from the Government and the Bank on critical project implementation issues. The second issue was the low budget of the project, particularly following the strong appreciation of Kina which diminished the resources available (in Kina) to support project activities. The Kina appreciation effect has been exacerbated by substantial increases in costs (largely linked to limited competition), thereby significantly reducing the scope of the road construction and rehabilitation work. The suspension of Bank missions, following the issues around the FCPF Project, was also perceived to be untimely, since it deprived SADP of the opportunity to proactively tackle and address issues with full Bank support. 2. Institutional Capacity and Coordination The private sector demonstrated strong engagement during the concept and inception phases of the Project, and was supportive of the process. However, the support greatly phased away during project implementation due many reasons, including disagreements around the perceived role of OPIC and the scaled down scope of road rehabilitation, especially in Oro Province. A result, the Project’s relationship with the private sector has not been fully effective, with a private sector stakeholders feeling most of time left out of major discussions and decisions, and using in turn their power to undermine these decisions. Weak Oversight and Institutional Support. OPIC as an implementing agency was weak, and staff were not technically equipped to manage projects the size of SADP. OPIC had no proper Board, and National Government was largely disengaged. The poor performance of the public sector service delivery (extension and roads services), the inordinate delays from Government 52 combined with numerous internal issues on OPIC side, have all contributed to the overall poor Project performance. The lack of Government counterpart funding at critical junctures of Project life, and the limited commitment of the Departments in charge of overseeing project implementation resulted in OPIC being orphaned. Systems and Processes. OPIC found the World Bank procurement processes difficult. Lack of procurement expertise within OPIC and difficulties in recruiting procurement specialists caused delay and mismanagement of procurement processes, including contracting individuals and firms to undertake Project activities. Lack of coordination between PNG Microfinance Ltd, OPIC, the Milling Companies and the growers resulted in a very slow loan processing. A number of systems and processes were put in place, but it took quite a while for OPIC staff to learn and make them work. The major one was following the Planting Approval Form (PAF), which involved the CLUA and the loan processing approvals. These required well-coordinated teams from all parties involved to ensure that the proposed blocks for infills met the requirements. Communication. There was lack of clarity on roles and responsibilities of government at provincial and national levels, and at OPIC, which resulted in communication and coordination difficulties. The implementation also suffered from lack of proper operational oversight and management of consultants by OPIC. There was no communication between OPIC Headquarters and OPIC field officers were considered a major drawback on the project. OPIC field officers were not kept updated with regards to mission aide-memoires for actions to be taken on the issues identified during the missions. Prevalence of distrust and dissatisfaction amongst the OPIC and SADP staff – issues of pay and benefits, and work load were among the issues that contributed to the poor implementation progress. 3. Suggestions and Recommendations from Stakeholders Many suggestions and recommendations were noted during consultations and the stakeholder workshop. The recurrent ones include the following: • Some of the positive outcomes of the Project included: (i) the establishment of management information system (MIS), which despite its limitations, was useful in tracking project’s results; (ii) adherence to RSPO certification guidelines, through the use of SADP PAF as environmental and social safeguards requirements; (iii) Part of SADP legacy was to contribute to terms of reference for updating the Code of Practice for oil palm processing led by Department of Environment and Conservation. This update was required under the Effluent Action Plan. • Areas in which improvements could have been made included: (i) It could have been more effective to use an independent project management unit apart from OPIC to manage the Project; and (ii) Project should have started the infill program only after OPIC had qualified staff, which would ensure better and more timely project implementation. 24 24 The infill program started only after the first extension specialist was hired. There issue may be more with the competence as well as the motivation of extension officers. 53 • Other suggestions included: (i) PML loan process took too long delay; it needs to be approved at Project site level, for speedier implementation and procurement of seedlings. The process could have been speedier, according to PML, were it given direct responsibility of loan disbursement, after PAF and other regulatory standards were met; and (ii) “When World Bank projects come in and when they go out, they leave untreated wounds”. There is a need for more consultations to understand field level problems and frustrations with expectations that were never met. 54 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 25 1. Project Objective The development objective of the Project is to increase, in a sustainable manner, the level of involvement of targeted communities in their local development through measures aimed at increasing oil palm revenue and local participation. Good roads are crucial for oil palm. The last World Bank project in the PNG smallholder oil palm sector, prior to SADP, was the Oro Smallholder Oil Palm Development (OSOPDP) Project 1993-2001. This project more than doubled the planted hectares of smallholder oil palm and vastly increased the road network in Northern Province. Starting from the end of OSOPDP consultations held with the sector stakeholders, the overwhelming feedback received from smallholder farmers and from the companies was – establish sustainable road maintenance. Road maintenance has been and continues to be the number one constraint to smallholder production; i.e. the worst problem for farmers preventing them from earning more money. This is where SADP started. The objective was clear to the stakeholders and can be simply stated, to establish sustainable road maintenance arrangements. 2. Project Design and Effectiveness The project had a very lengthy and very costly design phase. During this phase there were consultations held with the stakeholders. However, it appears that the meetings were not well documented and this ultimately led to an Inspection Panel investigation. The design encompassed components and subcomponents that were very worthwhile. In hindsight there was an overestimation of OPIC’s capabilities to implement all the elements of the project. The unforeseen PNG Kina appreciation, which led to reducing what could have been achieved with the same budget, would turn out to be very costly, especially for road construction and rehabilitation works. Considerable time and effort was wasted trying to obtain the necessary GoPNG and World Bank approvals. The project and financing agreements were approved by the Bank in June 2007 but not made effective until January 2009. Getting the Ministers signature was the main delay. 3. Implementation 3.1. Responsiveness of GoPNG and the World Bank Project Implementation was rated unsatisfactory or moderately unsatisfactory for its entire four year life. This has created difficulties and rigidities in proactively managing project 25 “Smallholder Agriculture Development: Project Investment Completion Report”, Prepared by OPIC on behalf of the Government of PNG. Original version: 13 November 2013; Revised version endorsed by GoPNG on 03 June 2014. 55 implementation. An extension of the project was sought as early as February 2010. This was never acted on by the Bank because of the unsatisfactory rating of the project, which was mainly attributable to lack of Government counterpart funding and poor progress on procurement matters. In addition, delays in GoPNG and the World Bank response to request have further compounded these difficulties. For example, project restructuring was agreed in October 2011 but did not become effective until September 2012. Getting the request from Treasury to World Bank was difficult, and the processing was further delayed because of non-compliance with loan covenants (overdue OPIC audits). Even a simple reallocation of funds between categories took Treasury and at the Bank four months to process. A major problem for SADP has been obtaining audit certificates. Timely presentation of audit certificates for both the SADP project accounts and for the accounts of OPIC, the implementing agency, are conditions of the Project Agreement. Compounding the problem of OPIC’s incapacity has been a staggering lack of urgency at the Auditor General’s Office. 3.2. Procurement and Financial Management Procurement Specialist: Recruiting and keeping procurement specialists has proved to be challenging during the project’s first years. The first Procurement Specialist was terminated for non-performance in October 2010. A short term PS-2 was contracted fairly quickly and the same candidate was chosen for the long term position. PS-2 signed his contract in February 2011 and CSTB signed in the same month. By April it became apparent that PS-2 would not be coming to PNG due to health and cultural reasons. A second short term specialist, PS-3, was contracted. PS-2 and PS-3 handled most of the procurement activities during 2011 but not having a full time PS in Moresby was a definite hindrance to implementation. PS-4 recruitment got underway in May 2011 and the consultant came on board in March 2012. PS-4 helped the project enormously. Our final PS, the fifth, was also very helpful and because no CSTB involvement was necessary she was recruited quickly. Lessons: (1) If it is nearly impossible to recruit PNG staff capable of managing Bank procurement procedures then perhaps the procedures are too stringent 26; and (2) Projects are to comply with two independent and sometimes conflicting procurement structures, the Bank’s and CSTB’s. We perceive that the Bank is not going to be flexible, i.e. submit to CSTB procedures only. Meanwhile, CSTB is overworked. Procurement of Goods: The project was declared effective in January 2009 and a road show conducted in March of that year. Procurement of goods was a major impediment to the project. The bulk of the materials were not actually received until late in the third quarter of 2011. Lesson: In a commercial environment a vehicle or computer can be purchased and delivered within a month. Three quotes, a purchase order, and deliver the goods. If it takes two years to source critical goods for a project then something is not right with the system. 26 The problem of ineffective system at OPIC is less linked to the Bank procedures, since other projects faced less problem at the same time. The problem is closely link to weak capacity at OPIC and a track record in producing sub- standard procurement documents for review. 56 The procurement of road fleet to support rehabilitation and maintenance was fundamental to the project. In late 2011 the Bank acquiesced to the purchase of one grader for each project area, after having objected earlier to grader purchaser. It is OPIC’s perception that the Bank objects in principle to financing equipment that is then under the control of a private entity (the milling companies). It would be useful to reassess stance, particularly in environments where support could be delivered to smallholders through linkages with the private sector. Recruitment of Project Staff and Consultants: The project design called for the addition of some fifty staff to support implementation from year one. The first consultant recruited was the Procurement Specialist in October 2009 followed by the Project Coordinator in November 2009. While the loan funds were in place the project could not drawdown due to non-Government counterpart funding available to disburse the loan funds. There was very little implementation done by OPIC during 2009. • Project Coordinator: The contract for the Project Coordinator was submitted to CSTB in August 2009, and was on short term contracts renewed every three months until finally signing his contract by CSTB in April 2010. • Project Engineer: Recognizing that the Project Engineer position was critical to the project, recruitment of a short term engineer started in September 2009. The consultant started work in July 2010. • Extension Specialist: Refining of the Extension Specialist TORs started in December 2009 and contents of the TORs were agreed with the Bank by the time the position was advertised in July 2010. The position was readvertised in September 2010. The evaluation and contract was completed by February 2011 and submitted to CSTB. The contract was signed in May 2011. Towards the end of 2012 the Extension Specialist decided that he would not extend his contract through 2013. A new Extension Specialist was recruited short term so that the contract didn’t have to go to CSTB for approval. • Management Agency for Component 2: OPIC was tasked to recruit a suitable firm to manage the implementation of a support program for local level government and community groups. By October 2010, there were concerns whether enough time was available before the project closure scheduled for December 2012 and the component was dropped through project restructuring that became effective in September 2012. • Consulting Engineer: As per the Project Agreement no work could commence on infrastructure development until a consulting engineering firm was contracted. While the selection process for the consulting firm commenced in December 2009, the contracted consultant (KBR) started work in late January 2011, more than a year later. • Environmental and Social Auditors (ESA): the ESA consultancy should have commenced in 2010. The contract was executed on the 4th of June 2012. This recruitment was complicated by the late addition to the TOR of Palm Oil Mill Effluent sampling and reporting arising from the Inspection Panel process. The work performed under this contract was generally to a high standard. • Monitoring and Evaluation: The contract was signed in November 2011. This consultancy was changed from the planned individual consultant to a firm. The original budget for M&E was US$150,000. The final contract value was approximately K1.1million, which is 2-3 times the original amount. OPIC believes that this should have 57 been a fixed price contract rather than a time based contract. After paying 90% of the overall contract amount, none of the outputs from the contract were useable. • Road Maintenance Trust Fund: Again, this consultancy was not a quick process. The consultancy was changed from an individual consultant to a firm. The revised TORs were done by July 2010. Request for Proposals were issued in April 2011. The contract was awarded in April 2012 for just over K1 million (approx. US$0.4 million) versus the original budget of US$66,000. • FFB Price Review: This consultancy recruitment was probably the most difficult. First the Bank wanted it changed from Individual to Firm. The TORs had to be revised and agreed to by DAL, the Growers and Milling Companies. The companies were not satisfied with the TORs and felt they were forced to accept. Insufficient qualified EOI’s were received (no less than six, no more than two from one country). At the same time ICCC considered legal action against OPIC which it perceived as violating the ICCC Act by supporting a pricing mechanism for FFB. A new EOI’s were called in early December and evaluated by February 2013. The proposal was received in May 2013 and the contract negotiations concluded in early July 2013. It then took two months for CSTB to sign the contract for the negotiated price of K0.8 million (approx. US$0.3 million). The original budget was US$66,000. Lessons: (1) TORs for each of the consulting positions were prepared in the design phase but in every instance these TOR’s were discarded and re-done at the Banks direction. More care should be taken in the design phase or the Bank should stick to the design concept, or at least ensure speedy review of TORs; and (2) OPIC struggled to manage the procurement process on just about every contract. Procurement capacity must be built from an early stage recognizing that there is very limited experience in-country. Financial Management: The OPIC financial controller (FC) was nominated as the Chief Financial Officer (CFO) for SADP in the project design. Although the FC made some contribution in the early stages, his performance deteriorated steadily until his termination in 2011. At one point the Bank threatened to close the designated account, which would have crippled the project for at least six months. The Project Coordinator assumed the CFO duties from mid-2011. Financial management was generally satisfactory from that point with the exception of the “separation of duties” control, and the problems with audits as noted above. 3.3. Implementation of Project’s Components Component 1: Smallholder Productivity Enhancement Road Rehabilitation: Road rehabilitation was the most critical and most costly element of SADP. It was critical because in the absence of a major rehabilitation of the road networks there was no possibility of implementing sustainable road maintenance arrangements. Too much maintenance money would need to be spent on repairing roads that are beyond repair. The rehabilitation suffered from three main problems. • (i) Procurement delays. The Project Engineer was recruited in July 2010 and he proceeded to do road surveys and scoping. Without adequate staff, however, he could only do so much. When the consulting engineers finally came on board, much of the scoping had to be redone due to the wet season. Bidding documents for the first tranche 58 of road works were prepared and submitted to the Bank in mid-March 2011. The bidding documents were passed and tendered at the end of May 2011 and the bid evaluation period ran until November 10, 2011. The contract was then quickly submitted to CSTB for approval and signature, but was delayed until it was finally signed on the February 24, 2012. The procurement period was almost one year. • (ii) Lack of counterpart funds. The second two tranches for Bialla and Oro rehabilitation were delayed in 2011 because insufficient counterpart funding had been allocated to cover the GST component of the contracts. • (iii) Insufficient budget. The 2007 project budget allowed K51.4m to rehabilitate 442 kilometres. Due mainly to the appreciation of the Kina between 2006 and 2011 the available funds for rehabilitation were reduced to K24.4m. In addition, because costs had risen so dramatically, the project was only able to achieve 190km of “spot repairs” rather than full rehabilitation. (The difference being that under spot repairs only 43 kilometres of the 190 was gravelled; full rehabilitation would gravel the entire road.) Aside from those three issues the road rehabilitation was actually quite successful. The 190 kilometres that was started in 2011/2012 was augmented by additional funding in the last half of 2013. About 24.9km of road was refurbished in Oro and 1.9km with some additional gravelling was accomplished in Bialla. Overall the work has been very beneficial to a lot of growers. Measurable results are expected after year end FFB production figures are tallied. Oro Incomplete Roads: In its significance to a particular subset of growers, the Oro Incomplete Road element would be the saddest failure of the project. At the end of the Oro Smallholder Oil Palm Development Project (1993-2001), 105km of road were left unbuilt with the blocks planted. The growers have suffered for twenty years. Under SADP, PNG Sustainable Development Program Ltd was to fund the completion of these roads. As with road rehabilitation, this funding was insufficient. A total of 15.5km was done with the allocated funds. There are approximately 42km of unconstructed road left to build plus another 43km that have some access but are in need of major work. There are currently no solutions on offer for completing these roads. Road Maintenance Trust Fund: When the RMTF overcame its procurement problems it soon became apparent that the consultants were inadequate. The original team that was assessed were all replaced, two due to the long delay between EOI and contract signature, and one after the start of the study because he did not produce results. The first round of consultations went well but the Initial Design and Options Report was deeply lacking in quality. Under the contract it was deliverable by week eleven. It was finalized after ten months. Much of the report was in fact produced by Bank and Project staff. Towards the end of the process it became clear that the Milling Companies were not willing to participate in any PPP arrangements unless the road rehabilitation was completed as designed. With the facts and figures from the consultants reports it was possible for OPIC to make a case to GoPNG to continue funding for the remaining rehabilitation work. The study provided a possible sustainable way to keep the road maintained. To that extent it was a success. The study also helped the diffusion of a range of information into the public domain through consultations, thereby helping to achieve some consensus among the stakeholders talking. 59 Infill Planting: Infill planting was an appropriate element of the project even though it was not directly applicable to the objective of sustainable road maintenance. It was a way to get more FFB from each kilometer of road and, in theory, more money from the users for road maintenance. It was especially applicable in Oro where the blocks-per-kilometer of road ratio is the lowest. There was also a lot of demand from growers. But the infill planting program suffered from delays. The first hurdle was getting PNG Microfinance to agree that the loans would be made at 8% interest. PNGSDP had signed up to this in 2006 when they held a controlling interest in the lender. The second hurdle was the issue of seedling availability where priority was given to replanting programs, as opposed to new infill blocks. By the time the milling companies accepted the agreement to provide seedling, the Inspection Panel had commenced its investigations and World Bank instructed OPIC not to sign agreements. From September 2011 until April 2012 very little planting was accomplished by OPIC. OPIC did eventually build sufficient capacity to carry out infill planting but then seedling availability became an issue. The milling companies, who were never much in favour of the program, and who were skeptical after the stop-start nature of the planting, did not prioritize seedling deliveries for SADP. However the situation in Hoskins was different of a different nature. Blocks were approved for planting that were clearly not eligible under the environmental guidelines. While the Hoskins management was reporting 162 infill blocks at the end of 2012, thorough investigations showed that only 72 blocks were considered as legitimate infill blocks with PML loans. Lesson: OPIC needs better management if it is going to manage a planting program. Component 2: Local Governance and Community Participation This component was dropped during the September 2012 restructuring. Component 3: Project Management and Institutional Support Strengthening OPIC and Upgrading Its Service Delivery: When the Extension Specialist was recruited in May 2011 he was given specific priorities in addition to his extensive TORs. The priority was to assess OPIC field staff and duties, put performance measurements in place, and remove low performers through retrenchment or retirement. While the Extension Specialist did a lot of work with the staff, in the end there were no significant changes in work methods or attitudes. The “restructuring and reorganization” that the Specialist designed and documented was done with the full participation of OPIC staff. While this was desired by everyone, the actual implementation of the plan became a difficult task. During the reorganization, very little effort was made to engage the milling companies and the LPCs. The reorganization report was rejected by the companies expressed dismay that they hadn’t been consulted, although the company’s claim was not entirely true. The companies felt that they were working to shrink OPIC’s operation and SADP was seeking to expand it. In the end there has been no real improvement in OPIC service delivery. 4. Assessment of SADP Outcomes Some of the main outcomes of the project include the following: 60 • Around one-half of “poor” roads have been upgraded to at least “fair” condition. This will allow higher production and increased incomes for many small farmers. • A plan has been made for achieving sustainable road maintenance. The plan will be put in place if GoPNG supports it. • Around 500 families will benefit from having new oil palm blocks and regular income from their land. • OPIC has received an injection of capital assets including vehicles, housing, furniture, and equipment. • OPIC has built some capacity in the field of civil engineering, environmental monitoring and control, and dealing with land issues. The sustainability of this new capacity is however tenuous. • Research on the social aspects of smallholder oil palm in PNG has been advanced. 5. Assessment of Bank and Borrower Performance Evaluation of the Borrowers Own Performance: • Implementing Agency–OPIC. OPIC’s performance can only be rated as poor. A core competency of an agriculture extension service is planting. When the opportunity arose to actually put trees in the ground and create livelihoods for families in rural districts OPIC was sorely lacking. They struggled with the paperwork, there was little initiative or motivation. Some non-project staff indicated that they would only contribute to SADP if they were remunerated. • Overseeing Department– (1) National Planning and Monitoring. DNPM has generally been engaged and supportive of the project. There have been a number of changes in the Secretary position and in senior management responsible for the project. During transition periods it was not always possible to get timely assistance from DNPM. Lack of counterpart funding was a continuing issue but whether this was due to DNPM, Treasury, or Finance is unknown. One thing is clear; GoPNG signs legally binding agreements to provide counterpart funds and then fails to fulfil their obligation. (2) Agriculture and Livestock (DAL). DAL’s engagement with the project has been minimal. One officer has been involved and supportive. • Treasury. Generally engaged and helpful although getting letters signed by the Minister has been very problematic. • Other members of the Project Steering Committee (PSC): o Department of Environment and Conservation (DEC)–DEC was engaged with the project to the extent of sending their officers on field trips with the E&S auditors. Despite the significant expense incurred by SADP to fund these officers (travels, stationeries), the project never received feedback on their trips, although one officer assisted with some procurement support. o Department of Works (DoW)–There was no engagement from DoW at the PSC level. One officer did provide some help to the RMTF consultant. o Provincial & Local Level Government Affairs–One officer attended one PSC meeting. There was no engagement from Community Affairs. 61 o Provincial Administrators–The Project received good support from both the Northern and West New Britain provincial administrations. The two consecutive Governors of WNBP were also both very supportive. Evaluation of the Bank’s Performance: The World Bank provided a team of talented professionals who consistently supported the project. The Team Leader in particular should be recognized for her tireless efforts on behalf of the project and for the quality of her contribution. OPIC understands that the Bank sets very high standards for the quality of work it expects from the implementing agency. This is good and it helps build capacity. But if every contract, tender, report, and submission has to be re-written many times to the Banks satisfaction and if this causes lengthy delays then perhaps the “perfect” becomes the enemy of the “good enough”. Evaluation of PNG Sustainable Development Program Ltd: PNGSDP was an excellent partner for SADP. There were no problems accessing funding. Their two officers for Agriculture and Infrastructure were wholly engaged with the project from start to finish contributing technical expertise and ideas for better implementation. The only minor problem with PNGSDP was lack of feedback on their expenditure. The accounts for emergency repairs and Oro incomplete roads were not shared with OPIC. 6. Next Steps Proposed Arrangement for future operation of the Project: An application has been lodged with DNPM (on national investment budget) to fund the core elements of SADP that have been left incomplete at the end of 2013. The budget proposal encompasses the completion of the rehabilitation of the three road networks, the purchase of three maintenance fleets one for each location, community participation contracts for manual road maintenance, and salaries/fees for a team of four engineers to oversee the work plus limited on-costs. 62 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders The Papua New Guinea Sustainable Development Program (PNGSDP) was the main co- financier for SADP. Although no financial reporting was shared with SADP Project Management Unit, SADP estimated that it has contributed approximately US$7-8 million to finance project activities through mainly roads and infill program support. Background on PNGSDP The Papua New Guinea Sustainable Development Program (PNGSDP) was established since 2001 as a private nonprofit trust company with majority ownership (63.4%) in Ok Tedi Mine Limited (OTML). PNGSDP was set up to sustain development initiatives for populations in Papua New Guinea and especially along Fry River in the Western Province where Ok Tedi mining activities take place. The Program is also to provide a financial assurance fund for mine closure. PNGSDP was an important partner in co-financing the Smallholder Agriculture Development Program. The Program was involved in the project from design stage up to implementation, and provided a budget of about US$10 million to support the project. Of this amount, US$1.9 was allocated to roads and fully spent. Out of the US$8.1 balance, US$5-6 million was spent to support the infill program through PNG Micro Finance Limited (PML) loans to growers, and also to other project activities. Summary of PNGSDP Comments PNGSDP saw a lot of potential in SADP and were very supportive of the project, which explain which the Program accepted being the main co-financier of the project. The roads reconstruction and maintenance program was fully justified, but the approach could have been more flexible. SADP could have financed more roads, if the specifications for the roads were less stringent. For example, crushed aggregate gravel was the requirement but this was not always necessary for some roads for which straight screen river gravel could have been used. This latter option would have allowed coverage of more kilometers of roads. 27 The rationale for the infill program was also solid. Filling the gaps along constructed and rehabilitated roads was a cost-effective proposition for palm oil operations, both for growers and milling companies. The infill program also fully justified future road maintenance work, helping to ensure that project investments in roads will be durable. While the program was slow to commence, PNDSDP ensured that the agreed interest rate with PML was maintained at 8% for the growers. PML is owned by PNGSDP and would have liked to see this program continue. However, PNGSDP is now close down since October 18, 2013, after the Government of PNG advised of its intentions to take over the Company. 27 Bank missions advised while crushed gravel is of a far superior quality, screened gravel is a good alternative. However, that screened gravel should meet the specification and be within an acceptable ‘grading envelope’, which requires mechanical screening. 63 Annex 9. List of Supporting Documents • Project Implementation Plan • Project Appraisal Document for Independent State of Papua New Guinea Smallholder Agriculture Development Project (SADP) dated November 19, 2007 (Report No: 38558) • Aide Memoires, Back-to-Office Reports, and Implementation Status Reports. • Project Progress Reports. • Borrower's Evaluation Report dated November 2013 *including electronic files 64 MAP IBRD 35472 65