Document of The World Bank Report No: ICR3022 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-74270 IDA-42600) ON A LOAN IN THE AMOUNT OF US$32.8 MILLION AND A CREDIT IN THE AMOUNT OF SDR 39.9 MILLION (US$60.0 MILLION EQUIVALENT) TO THE REPUBLIC OF INDONESIA FOR A FARMER EMPOWERMENT THROUGH AGRICULTURAL TECHNOLOGY AND INFORMATION PROJECT December 24, 2013 Indonesia Sustainable Development Unit Sustainable Development Department East Asia and Pacific Region i CURRENCY EQUIVALENTS (Exchange Rate Effective: June 30, 2013) Currency Unit = Indonesian rupiah US$1 = IDR 9,925 IDR 10,000 = US$1.01 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS AAEHRD Agency for Agricultural Extension and Human Resource Development ACAP Anti-Corruption Action Plan AIAT Assessment Institute for Agricultural Technology AISI Agricultural In-Service Institute APBD District or provincial budget allocation APBN National budget allocation ARMP II Second Agricultural Research Management Project ATC Agricultural Training Center Bakorluh Badan Koordinasi Penyuluhan or Provincial Extension Coordinating Body Bapeluh Badan Pelaksana Penyuluhan or District Extension Institution Bappeda Regional Development Planning Agency Bappenas National Development Planning Agency BIA Beneficiary Impact Assessment BPP Balai Penyuluhan Pertanian or Agricultural Extension Office at sub district CADI Center for Agricultural Data and Information of MOA CAS Country Assistance Strategy for Indonesia CPMU Central Project Management Unit CPS Country Partnership Support for Indonesia CV Comanditaire Venootschapt DEC District Extension Committee DAFEP Decentralized Agricultural and Forestry Extension Project DAK Special Budget Allocation to district governments DPIU District Project Implementing Unit EOP End of project FEATI Farmer Empowerment through Agricultural Technology and Information Project ii FET Field Extension Team FM Financial management FMA Farmer-managed extension activity FO Farmer Organization GoI Government of Indonesia IAARD Indonesian Agency for Agricultural Research and Development of MOA ICATAD Indonesian Center for Assessment of Technology and Development ICR Implementation Completion and Results ICT Information and Communication Technology IDR Indonesian Rupiah KIS Knowledge and Information Systems KUB Kelompok Usaha Bersama or Farmer’s business group MOU Memorandum of Understanding M&E Monitoring and evaluation MoA Ministry of Agriculture MoF Ministry of Finance MTR Mid-Term Review NECB National Extension Coordinating Board or Badan Koordinasi Nasional Penyuluhan Pertanian, Perikanan dan Kehutanan or BKNP3K NGO Non-government organization O&M Operations and Maintenance PAD Project Appraisal Document PCR Project Completion Report PDO Project development objective PEC Provincial Extension Committee Perda Local Government Regulation PPK Holder of Funds (for a project) PPMU Provincial Project Management Unit PPP Public-Private Partnership PT Perseroan Terbatas or Limited Liability Company QAG Quality Assurance Group of the World Bank R&D Research and development RoI Republic of Indonesia RPJMN Rencana Pembangunan Jangka Menengah Nasional or National Medium-Term Plan of GoI RPPK Revitalisasi Pertanian, Perikanan dan Kehutanan or Agenda for Revitalization of the Agriculture, Fisheries and Forestry Sectors Sub-PMU Sub-Project Management Unit TA Technical Assistance TOR Terms of Reference UP-FMA Unit Pengelola – FMA or Village Management Unit for Farmer-managed extension activity iii USD United States Dollar VEC Village Extension Center or Post Penyuluhan Desa or PPD Vice President: Axel van Trotsenburg Country Director: Rodrigo A. Chaves Sector Manager: John Roome Project Team Leader: Fabrizio Bresciani ICR Team Leader: Mariam Rikhana iv INDONESIA FARMER EMPOWERMENT THROUGH AGRICULTURAL TECHNOLOGY AND INFORMATION PROJECT CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................. 1 2. Key Factors Affecting Implementation and Outcomes ............................................. 5 3. Assessment of Outcomes ....................................................................................... 15 4. Assessment of Risk to Development Outcome ....................................................... 21 5. Assessment of Bank and Borrower Performance .................................................... 22 6. Lessons Learned .................................................................................................... 26 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ........... 28 Annex 1. Project Costs and Financing ....................................................................... 29 Annex 2. Outputs by Component ............................................................................... 30 Annex 3. Economic and Financial Analysis ............................................................... 35 Annex 4. Bank Lending and Implementation Support/Supervision Processes............. 40 Annex 5. Beneficiary Survey Results ......................................................................... 43 Annex 6. Stakeholder Workshop Report and Results ................................................. 44 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders....................... 50 Annex 9. List of Supporting Documents .................................................................... 51 MAP v   A. Basic Information Farmer Empowerment through Agricultural Country: Indonesia Project Name: Technology and Information IBRD-74270,IDA- Project ID: P083742 L/C/TF Number(s): 42600 ICR Date: 12/18/2013 ICR Type: Core ICR REPUBLIC OF Lending Instrument: SIL Borrower: INDONESIA Original Total USD 92.80M Disbursed Amount: USD 87.79M Commitment: Revised Amount: USD 88.80M Environmental Category: B Implementing Agencies: Agency for Agricultural Extension and Human Resource Development, Ministry of Agriculture- Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 12/16/2003 Effectiveness: 06/28/2007 06/28/2007 03/27/2012 Appraisal: 04/11/2006 Restructuring(s): 09/24/2012 03/11/2013 Approval: 02/13/2007 Mid-term Review: 12/31/2009 03/01/2010 Closing: 06/30/2012 06/30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Significant Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Moderately Satisfactory Unsatisfactory Quality of Supervision: Moderately Implementing Moderately i Unsatisfactory Agency/Agencies: Unsatisfactory Overall Bank Moderately Overall Borrower Moderately Performance: Unsatisfactory Performance: Unsatisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No Moderately Satisfactory at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Agricultural extension and research 60 60 Central government administration 14 15 General agriculture, fishing and forestry sector 15 15 Sub-national government administration 11 10 Theme Code (as % of total Bank financing) Other Private Sector Development 13 10 Participation and civic engagement 25 30 Rural non-farm income generation 13 10 Rural policies and institutions 25 10 Rural services and infrastructure 24 40 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg James W. Adams Country Director: Rodrigo A. Chaves Andrew D. Steer Sector Manager: Nathan M. Belete Rahul Raturi Project Team Leader: Fabrizio Bresciani Shobha Shetty ICR Team Leader: Mariam Rikhana ICR Primary Author: Dely P. Gapasin ii F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document ) The overall development objective of the project is to develop a demand-driven, market oriented agricultural services system, based on partnerships between farmer groups, public agencies and private sector enterprises at all levels. Revised Project Development Objectives (as approved by original approving authority) Not revised (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Percentage of members of participating farmer organizations (FOs) with Indicator 1 : improved household expenditures. Value quantitative or Not available 70 percent Not assessed Qualitative) Date achieved 11/12/2008 06/30/2013 06/30/2013 Comments Indicator lacked relevance/monitorability for measuring project outcomes. No (incl. % quantitative baseline established to show amount of improvement (versus scope), achievement) results cannot be directly attributed to project activities, and data was not reliable Indicator 2 : Number of technology packages developed by AIATs that are used in FMAs Value quantitative or Not available 60 percent 25 packages 33 packages Qualitative) Date achieved 11/12/2008 06/30/2012 12/31/2012 06/30/2012 Achieved (target exceeded). Percentage vs number clarified in 2012 RP. Both the Comments number of tech developed by AIATs and the percentage or packages used in the (incl. % FMAs increased substantially under the project. Data on extent of use technology achievement) not available Percentage of participating farmers benefitting from e-Petani and other Indicator 3 : information and communication technology (ICT) services. Value ePetani and ICT not quantitative or 60 percent 30 percent 7.8 percent available Qualitative) Date achieved 11/12/2008 06/30/2012 12/31/2012 06/30/2013 Comments Not achieved. Even though the target was formally reduced, achievement was (incl. % very low, mainly because e-Petani was not deployed, as the project only achievement) facilitated ICT services only in 18 pilot districts, despite 68 districts. Number of joint activities/ventures with private sector established by FOs under Indicator 4 : FEATI. Value 2400 join quantitative or NA 70 percent 1,288 activities activities Qualitative) iii Date achieved 11/12/2008 06/30/2012 12/31/2012 06/30/2013 Comments Partially achieved. Measures partnerships on market-oriented activities promoted (incl. % by the project. The value achieved represents number of written agreement achievement) between FOs and buyers regarding quantity and quality of the products. Indicator 5 : Number of services provided to FO members. 10,351 services. NA (baseline survey only FOs have provided Value identified 10 services 5000 services at least one service quantitative or provided by FOs to FO provided to their members, Qualitative) members) implying that the EOP target was met Date achieved 11/12/2008 12/31/2012 06/30/2013 Comments Achieved (target exceeded). The value achieved indicated that the project was (incl. % able to improve the capacity of FOs measured using 10 identified services. achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Percentage of FO members participating in plan and decision-making Indicator 1 : meetings.NEW. Value (quantitative NA 60 percent 66.6 percent or Qualitative) Date achieved 11/12/2008 12/31/2012 06/30/2013 Comments Achieved (target exceeded). This achievement indicates that the project (incl. % strengthened FOs in leadership and managerial skills achievement) Percentage of participating FO members that participate in planning and Indicator 2 : decision-making that are women or poor. 36.6 % for women Value participation; (quantitative 40 percent 30 percent 17.2 % for poor or Qualitative) participation Date achieved 12/31/2012 12/31/2012 06/30/2013 Comments Achieved. This achievement indicated an increased participation of women and (incl. % the poor as promoted by the project achievement) Percentage of FOs assessed to have sound financial and procurement Indicator 3 : management practices. Value (quantitative NA 80 percent 52.7 percent or Qualitative) Date achieved 06/30/2012 06/30/2013 Comments Partially achieved. Although the project fell substantially short of the target, iv (incl. % many FOs showed improvement. Mitigating conflicts were also practiced by achievement) FOs. Percentage of FMA where experts from external providers (universities, research Indicator 4 : organizations, NGOs, farmer experts/experience farmers, and private sector) participate as resource person (NEW) Value (quantitative NA 30 percent 94.3 percent or Qualitative) Date achieved 06/30/2012 06/30/2013 Achieved (target exceeded). The significant increase was a result of FMA Comments implementation in which FOs were given opportunities to identify and select (incl. % their service providers to facilitate their learning activities and improve their achievement) enterprises. Percentage of farmers reporting benefits from skills and services of extension Indicator 5 : staff. Value (quantitative 80 90 percent or Qualitative) Date achieved 06/30/2012 06/30/2013 Comments Achieved (target exceeded). The figures indicated that provision of training and (incl. % facilitation supports to the extension staff improved their skills and facilitation achievement) supports to farmers . Number of collaborative activities in technology development and dissemination Indicator 6 : between AIATs and other research institutions /service providers (private sector, universities, NGO, local government). Value 40 collaborative 107 collaborative (quantitative activities activities or Qualitative) Date achieved 12/31/2012 06/30/2013 Comments Achieved (target exceeded).. The achievement of this indicator indicated that the (incl. % project's support to AIATs resulted in strengthening AIATs to be more active in achievement) establishing collaborative activities to support farmers. Percentage of AIAT technology packages that are adopted in FMAs are fine- Indicator 7 : tuned in collaboration with the participating farmer groups. Value (quantitative 60 percent 65.2 percent or Qualitative) Date achieved 12/31/2012 06/30/2013 Comments Achieved (target exceeded).The achievement of this indicator indicated that (incl. % AIATs were recognized by farmers as a technology provider. achievement) Indicator 8 : Percentage of farmer beneficiaries satisfied with R&D services. Value (quantitative 80 percent 70 percent 61.3 percent or Qualitative) Date achieved 06/30/2012 12/31/2012 06/30/2013 Comments Partially achieved. The target was lowered due to limited capacity (number of (incl. % staff and budget) of AIATs in reaching all project villages. The target could v achievement) have been achieved if the survey conducted in the areas where the technology introduced Percentage of participating FOs with members availing information and services Indicator 9 : of e-Petani and other KIS. Value (quantitative 70 percent 30 percent 7.8 percent or Qualitative) Date achieved 06/30/2012 12/30/2012 06/30/2013 Comments Not achieved. The lower target was to address that the e-Petani was piloted in 18 (incl. % districts, and it was expected by EOP could reach 68 districts. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 07/26/2007 Satisfactory Satisfactory 0.00 2 06/24/2008 Moderately Satisfactory Moderately Satisfactory 10.00 3 05/28/2009 Moderately Satisfactory Moderately Satisfactory 22.95 Moderately Moderately 4 06/29/2010 58.29 Unsatisfactory Unsatisfactory Moderately Moderately 5 04/12/2011 64.29 Unsatisfactory Unsatisfactory Moderately Moderately 6 10/21/2011 74.29 Unsatisfactory Unsatisfactory Moderately 7 03/13/2012 Moderately Satisfactory 85.29 Unsatisfactory Moderately 8 05/25/2012 Moderately Satisfactory 85.29 Unsatisfactory 9 06/28/2012 Moderately Satisfactory Moderately Satisfactory 85.29 10 02/11/2013 Moderately Satisfactory Moderately Satisfactory 87.79 Moderately 11 08/03/2013 Moderately Satisfactory 87.79 Unsatisfactory H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions 1. Simplification of Farmer- Managed Grants; 03/27/2012 N MS MU 85.29 2. Cancellation of Sub- component 2.3 - 3. Cancellation vi ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions of Sub-component 5.1 Policy- support; 4. Decrease number of districts from 71 to 68; 5. Decrease of the number of participating villages from 3,230 to 3,080; 6. Reallocation of the loan within components 7. Extension of closing date from 30 June 2012 to 30 December 2012. 8. Formal revision of indicators. Extension of closing date from 09/24/2012 MS MS 85.29 30 December 2012 to 30 June 2013 03/11/2013 MS MS 87.79 Partial Loan cancellation I. Disbursement Profile vii   1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1.1.1 Sector Background: Agriculture is the main source of income for the vast majority of poor rural households in Indonesia. At the time of the preparation of the FEATI Project it employed 45% of the total labor force and contributed to 17% of growth in GDP. With an average annual rate of growth of 3.6%, the sector performance in 2006-09 was relatively satisfactory. Rice continued to represent a major staple crop and receive a substantial share of the budget spent on agriculture. Nevertheless, the structure of agriculture has been transforming rapidly with the number of farmers engaged in horticulture nearly doubling to 38% between 1996 and 2002. At the same time, export cash-crops such as cocoa and oil palm were rapidly expanding. A slowdown in agricultural productivity growth had been observed since the mid ‘90s, driven mainly by the limited budget allocated to extension and R&D. Only 0.3% of agricultural GDP was on average invested in R&D, one of the lowest efforts in Asia. Furthermore, the Indonesian agricultural R&D system was highly fragmented as linkages with international R&D agencies and local universities and private centers were minimal. The extension system had also entered in a period of declining effectiveness as a result of declining budget and the decentralization of services to local governments, who had become responsible for staff recruitment, training, and funding. In view of such trends, the revitalization of the agricultural sector ( Revitalisasi Pertanian, Perikanan dan Kehutanan, or RPPK) became a high a priority for the GOI. The RPPK emphasized the development of human resources and the empowerment of farmers. This was to be achieved, among other, through the development of extension service capacity and improved service delivery; improved access to information; increased investment in long-term technological research; and stronger linkages between R&E and extension. The enactment of Law 16 2006 reforming the system of extension in the agricultural, forestry and fisheries sector was important in creating an enabling policy framework for the Project. The complementary role of private extension agents was recognized. The participatory process in the prioritization of extension planning at the district and provincial level was supported through the creation of the District and Provincial Extension Committees (DEC and PEC, respectively). The inter-ministerial decree with the related implementing rules and regulations has not been promulgated yet. 1.1.2 Rationale for Bank Assistance: The Bank’s engagement in Indonesia’s research and extension sector has been quite extensive and it included two Agricultural Management Projects (ARMP and ARMP II, 1995-2002), both of which focused on institutional and management reforms to improve research towards a demand-driven and “farmer first” strategy; and the Decentralized Agricultural and Forestry Extension Project (DAFEP, 2000-05), which had piloted a shift from top-down to participatory approaches, consistent with the institutional shift from centrally managed to decentralized extension services and supportive of the reform promoted through Law 16, 2006. The key innovation introduced by DAFEP consisted in broadening the sources of technical providers that farmers could access, mainly at the village level. FEATI offered the opportunity to scale-up the access to technical services at the district level. Furthermore, the Project would also broaden the access to marketing services for the development of small scale agro-enterprises. The Bank could count on its global expertise in the extension sector and its consistent support to participatory and inclusive approaches. 1 FEATI supported the CAS FY 2004-2007 in three key areas: (i) Improving the Investment Climate; (ii) Making Service delivery Responsive to the Needs of the Poor; (iii) Improving Governance. The Project’s focus was in fact on improving the delivery of extension and research services at the district and provincial level. Through the establishment of competitive learning grants (i.e. FMA grants), FEATI supported increased accountability and effectiveness of public provision of extension and research services. Governance would be strengthened by making FOs capable and responsible for the use of the FMA grants provided under the Project and by strengthening in a transparent manner their linkages with private providers of extension and research services. 1.2 Original Project Development Objectives (PDO) and Key Indicators The Development Objective of the Project is to develop a demand-driven, market-oriented agricultural services system, based on partnerships between farmer groups, public agencies and private sector enterprises at all levels. The Project's original key performance indicators were: (i) the number and types of services provided to FO members (to be assessed through beneficiary surveys); (ii) percentage of members of participating FOs with improved household expenditure rates (to be assessed through beneficiary surveys); (iii) number of technology packages developed by AIATs that are used in FMAs; (iv) number of farmers benefiting from e-Petani and other information and communication technology (ICT) services (to be assessed through beneficiary surveys); (v) increased skills and capacity of farmer organizations and agribusiness development (measured by number of joint activities/ventures). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDO was not revised. Following the Mid-Term Review (2010), a number of PDO and intermediate level indicators were revised, but did not formally become effective until approval of the restructuring in March 2012. The details of dropped, revised, and new indicators are shown in the matrix at the end of Annex 2. Key changes to the formally designated KPIs included: Original indicators Revised indicators Justification 1. Number of participating Percentage of participating To be in line with the EOP farmers benefiting from e- farmers benefiting from e- target indicated in percentage Petani and other Petani and other information (Table 3.2, Annex 3, PAD) information and and communication communication technology (ICT) services technology (ICT) services 2. Increased skills and Number of joint This was simplified to capacity of farmer activities/ventures with private improve clarity on data organizations and sector established by FOs collection agribusiness development; under FEATI 3. Number and types of Number of services provided Monitoring was done using a services provided to FO to FO members list of 10 eligible services members. identified by the baseline 2 1.4 Main Beneficiaries The main beneficiaries of the extension and research components were identified as an estimated 380,000 households of the members of Farmer Organizations, who directly benefited from the FMA grants. Indirect beneficiaries were assumed to include those farm households whose members would learn from the direct project beneficiaries. Altogether, it was assumed that the Project would benefit about 760,000 households (equivalent to 3.8 million people). FMA grants were expected to be allocated to around 19,000 FOs in around 3,200 villages (assuming on average 6 benefiting FOs per village and 20 members per FO) across 18 provinces. As discussed below, the adoption by MOA of the one-village-one-product policy limited the number of FOs receiving the FMA grants. The policy, limited the number of FOS that could compete for the FMA grants to those cultivating the crop supported by the application of the policy at the village level. 1.5 Original Components The original project components were described in the PAD, as follows: Component 1: Strengthening Farmer-driven Extension Services (USD 40.3 million). This component was intended to empower farmers to drive the agricultural extension agenda and to improve their capacity to: (a) adopt new technology to better respond to market demand; (b) develop agribusiness management skills; and (c) engage in public-private partnerships (PPP) at the village, district, and provincial levels. Those outcomes would be achieved by: (i) strengthening farmer capacity to become organized and accountable at all levels; (ii) facilitate productive public private partnerships with multiple service providers through the FMA grants; and (iii) develop public and private extension capacity to facilitate farmers’ access to technology, markets, and knowledge. Activities included organizational support, monitoring and training of producer organizations’ leaders and members, at village, district, and national/regional levels. Personnel from extension, research, and private sector would also be trained to help them work with producer groups and their leaders. Component 2: Institutional Strengthening and Capacity Building (USD 33.7 million): This component was intended to facilitate the development of a demand-driven sustainable extension system, with public and private extension service providers (including farmers) cooperating with farmer groups and private enterprises for the mutual benefit of all parties. This would be achieved through the training of extension worker, the improvement of extension facilities at the sub- district level (BPPs), and the strengthening of the Agricultural Training Centers (ATCs) and Agricultural In-Service Institutes (AIS) to make them more responsive to farmers’ needs. Component 3: Enhancing Technology Assessment and Dissemination (USD 9.6 million): This component was intended to facilitate an uptake in the usage of farmer and market demand-driven technologies and to improve the capacities of the Assessment Institutes of Agricultural Technology (AIATs) to enable them to function more effectively. This would be achieved through the formulation of an innovative program to upgrade the AIATs (human resource development, facilities, and mobility improvement) and by improving research quality and relevance through more effective linkages between private sector, NGOs, universities, farmer partners, and traditional extension services. Improved coordination between the AIATs would be supported strengthening the coordinating agency ICATAD. Provision of management and implementation support for this component under ICATAD’s responsibility was also included. 3 Component 4: Provision of Knowledge and Information Services (USD 3.0 million): This component was intended to address the issue of the lack of access to knowledge related to agricultural technologies and the lack of linkages amongst service providers and between service providers and the user community (farmers, traders, and entrepreneurs). FEATI would strengthen: (i) a web-based Advice Online application (e-Petani) for individualized technical and market assistance, which would be queried by users through SMS on their mobile phones and GPRS devices; (ii) local commodity price and market information; (iii) a portal providing buyer-seller matching. The activities supported under this component included training, enabling the posting of project information on-line, including details on the process of award and implementation of the FMA grants, experiment, and the implementation of the project’s M&E system. Provision of management and implementation support for this component under CADI’s responsibility was also included. Component 5: Extension Policy and Project Management Support (USD 24.2 million): This component was intended to support the drafting of the implementing rules and regulations of the proposed Law on decentralized agricultural systems through multiple service providers. Provision of management and implementation support for the Project was also included in this component. 1.6 Revised Components As requested by the Borrower in a letter dated January 5, 2011, after the mid-term, the main activities of Component 2 and Component 5 were revised, as follows: Original components and its Revised components and its main Justification main activities activities Component 2: Institutional Component 2: Institutional The strengthening of Strengthening and Capacity Strengthening and Capacity agribusiness training Building: Building: centers (ATCs) and 2.1. Human resource 2.1. Human resource development Agricultural In-service development (district, (district, province) (US$ 6.6 Institutes (AISI) was no province) (US$ 6.6 million) million) to improve the capacities longer a priority of to improve the capacities and and skills of extension managers AAEHRD skills of extension managers and field extension staff and field extension staff 2.1. Improvement of extension 2.1. Improvement of facilities extension facilities 2.3 Strengthening agribusiness training centers (ATCs) and Agricultural In- service Institutes (AISI) Component 5: Extension Component 5: Extension Policy Another division of Policy and Project and Project Management AAEHRD provided Management Support: Support : funding for the 5.1 Policy-support to assist implementation of the the implementation of the Project management and Extension Law was new Extension Law coordination financed using APBN (to 5.2 Project management and avoid budget duplication) coordination 4 The revised main activities formally became effective on March 27, 2012, though for practical purposes the dropped activities had already been discontinued more than a year earlier. 1.7 Other significant changes In addition to changes to key performance indicators and main activities, as indicated in section 1.2 and 1.5, the Project revised the EOP target for some key performance indicators and intermediate outcome indicators, as indicated in the Data Sheet as part of the first restructuring (second-order). The Project was also extended twice for a total of one year through the first and second restructuring (the latter, also of second order). Finally a total of $4 million was cancelled upon request of the GOI with the third restructuring (also, second order). 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 2.1.1 Soundness of the Background Analysis: Substantial field work was conducted at the initial stages of project preparation, mainly through focus group discussions with farmer communities and other stakeholders. Although ideally the results of this background analysis should have been made available at an earlier stage of the Project's preparation, substantial inputs could be drawn from the experience developed through the satisfactory implementation of the DAFEP, ARMP, and ARMP II Projects. Besides DAFEP, the design of the mechanisms regulating the use of the learning grants by the beneficiary FOs could also draw from the experience developed through the PNPM program, where community organizations could procure directly services and goods responding to their development needs. FEATI’s support to the development of small scale agro-enterprises managed by farmer organizations (FOs) through the scaling-up of the FMAs to the district and provincial level represented a discontinuity with respect to DAFEP. Under DAFEP the FMAs were at the village level and mainly focused on technology transfer for productivity improvement. In designing FMA to better respond to the challenge of developing agro-enterprises the Bank could have relied more explicitly on its global experience in projects supporting the development of small scale agro-enterprises in partnership with the private sector. This would have allowed better appreciating the constraints that the design of the FMA grants under FEATI represented for those FOs wishing to expand the scale of their business activities. The grants in fact had to be annually planned and disbursed and were earmarked exclusively for learning activities rather than for the purchase of productive assets to be owned by the FOs. Finally, their disbursement was not conditioned to the submission of a thoroughly elaborated business plan that could be developed over a period of time covering for instance three or more years. 2.1.2 Assessment of the Project Design: Project Development Objective In its formulation, the PDO addressed an important developmental constraint in the area of food security and inclusive growth that clearly corresponded to the borrower’s relevant sectoral priorities as confirmed by the President’s RPKK and the promulgation of Law n. 16, 2006. It was also consistent with CAS’s objectives (see section 1.1.2 above) in its effort to improve governance, accountability, and service delivery to the poor. Nonetheless, the PDO was ambitious 5 given its focus on transforming the broader system of agricultural services rather than on changing behavioral outcomes of a set of well identified beneficiaries. Only a subset of the Project’s components strictly aimed at producing impacts at the systemic level (i.e. beyond the targeted beneficiaries and/or the scope of targeted districts). These included component 2.iii (strengthening ATCs and AISs: $1.2 million); component 3.iii (coordination and management support of ICATAD: $1.7 million); and component 5.i (policy support: $0.1 million), for a total investment of $3 million, or 2.5% of the overall investment effort. The remaining effort was directed to well-identified villages, district extension services, and provincial AIATs. The Project’s objective was therefore to develop the processes and the capacity of the public sector to provide extension and research services and assist Farmer Organizations in their effort to commercialize, ultimately leading to a paradigm shift that could be translated into a policy reform and mainstreamed programs. Project components Component 1 (strengthening farmer-driven extension) had a prominent role in the Project’s objective to support the development of agribusiness partnerships with the private sector and the adoption of agricultural technologies through farmer-driven and participatory approaches in extension and research service delivery. The FMA grants ($24.8 million) represented 61% of the component’s investment effort and amounted on average to nearly $8,300 per targeted village over the five year implementation period or $137 per beneficiary farm (assuming 180,000 beneficiary farms). This amount seemed reasonable to support FMAs focusing on learning activities. The development of village-level agro-enterprises, let alone their scaled-up versions at the district and provincial level, could not be reasonably supported with that level of funding; loans or a contribution from a private sector counterpart would have been required. With the aim of developing agribusiness farmer managed agro-enterprises and improving FMA proposal selectivity, the design of the FMA grants could have included as a condition for disbursement a requirement to prove the availability of counterpart funding in the form of a loan approved by a bank or a contribution from a private partner. In the absence of such type of restrictions, incentives would have favored FMAs focusing mainly on technology adoption, particularly in consideration of the fact that FMA grants were expected to be awarded each year to a different farmer group within a village, to maximize the number of beneficiaries, particularly among women and poor farmers. At least 20% of the FMA fund was reserved to finance women’s group proposals. Almost 76% of the investment effort under Component 2 (institutional strengthening and capacity building: $33.7 million) was allocated to the improvement of extension facilities (Sub-component 2.ii: $25.9 million). While this was consistent with the goal of improving the extension services’ outreach and capacity at the sub-district level, no funding was provided to support activities that would use such facilities for activities aimed at connecting farmers and private sector, such as demonstration activities, matching of buyers and sellers, etc. The training of extension managers and field extension staff under Sub-component 2.1 ($6.6 million) was included to improve the skills of the extension services involved in the implementation of Component 1. Research-extension linkages would be strengthened under Component 3 (Improved Research/Extension/Agribusiness/Farmer Linkages: $9.6 million). Its design coherently included incentives to improve the outreach of AIATs research and extension staff as well as support to their field work and ICATAD’s capacity development. On the other hand, the allocated 6 investment effort was small relative to critical role this component could have played in achieving the Project’s objective. Component 4 supported the provision of knowledge and information services, either through access to computers located in the agricultural extension centers or through a mobile application. MOA had been investing in developing the e-Petani system, a web-based agricultural information and learning system. Given the high rates of penetration and usage of cellular phones in Indonesia’s rural areas, the inclusion of Component 4 was in one way timely. Yet, it was also to some extent redundant: its exclusion would have not impaired to a significant extent the achievement of the Project’s objectives. Project management and implementation support under Component 5 was allocated $24.2 million, or 20% of the overall investment effort. With an allocation of $24.1 million, project management and coordination appeared as being excessively funded. The remaining 0.1$ million was allocated to policy studies supporting the development of rules and regulations for the implementation of Law n. 16. Implementation arrangements Implementation arrangements were complex, with three separate executing agencies involved at the national level: (i) the Agency for Agricultural Human Resource Development; (ii) ICATAD; and (iii) CADI/Office of the Secretary General. The geographic coverage of the Project was also challenging, involving nearly 3,000 villages in 68 participating districts in 18 provinces. To address the latter concern, the Project design assigned an important role to provincial and district governments, in line with the new institutions introduced by Law n. 16, 2006. The Provincial Extension Commissions (PEC), created under Law n. 16 of 2006 with the participation of governmental and non-governmental institutions, were tasked with the identification and evaluation of provincial-level agricultural priorities and with the management of the allocation of provincial-level FMA grants. The criteria regulating the functioning of the PECs were delegated to the Project Management Manual. Their funding was delegated to the competent provincial governments. No funding was allocated under the Project to develop their capacity. The newly established provincial extension agencies (Bakorluh) were responsible only for the establishment of Provincial Project Implementation Units (PPIUs) tasked with the collection, monitoring, and evaluation of data for the purpose of preparing and submitting the required reports to the CPMU and the sub-PMU at ICATAD. Similar arrangements, and limitations, determined the establishment and development of the District Extension Commissions (DECs) under the Project. District Project Implementation Units were created by the district extension agencies (Bappeluh) with similar tasks to those of the PPIUs. Extension workers at the sub-district (BPP) and village (VEMU) level were responsible for the implementation at the local level. Together with the district extension workers, BPP staff and VEMUs played a particularly important role in the identification, review, and selection of the FMA proposals submitted by the village farmer groups. 2.1.3 Adequacy of the Government's Commitment: During the Project's preparation and design stages, the level of commitment of the central and local governments was variable, resulting in significant delays, with this period lasting for 38 months (2004-07). The length of the preparation period was also partly attributable to the change in the management of the MOA in 2005. Increased emphasis was placed by MOA on investments to strengthen the public extension 7 service. The weight initially given to supporting the development and strengthening of extension- research linkages was contextually reduced, as noted above. As described in the QEA, the negotiations followed the “country management’s admonition not to practice brinkmanship,” leading to a compromise whereby $25.9 million were allocated under Component 2.ii to support the improvement of extension facilities, about 30% of what the Minister had originally wanted, while funding for research outreach under Component 3 amounted to $9.6 million. 2.1.4 Assessment of Risks: The Project’s Overall Risk was rated as substantial. Four risks were identified: (i) a continued centralistic and paternalistic attitude of MOA resulting in distrust of working with the private sector; (ii) weak linkages between research and extension establishment leading to component C being implemented separately from the remaining ones; (iii) elite capture of the resources allocated through the FMA grants; (iv) weak compliance with financial management and procurement procedures leading to misuse of funds. Excluding from the Project’s scope the capacity development of the newly formed PECs and DECs undermined the effectiveness of the identified mitigating measures. PECs and DECs were supposed to play an important role in mitigating risks (i), (ii), and (iii). They were regarded as an important opportunity for the private and public sectors to learn to work together. Under Law n.16, 2006, they were created precisely to strengthen the research-extension linkages at the local level. They were also supposed to ensure that the FMA grants would be awarded to support meritorious proposals. Leaving to district and provincial governments the responsibility to fund the DECs and PECs, respectively, raised the risk that these could not be placed in a condition to function properly, particularly in view of the well-appreciated history of under-funding of extension activities by local governments. 2.1.5 Quality at Entry: The operation underwent a Quality at Entry Assessment (QEA8) on February 13, , 2007 and received an overall rating of Moderately Satisfactory. In particular, Fiduciary Aspects and Risk Assessment were rated as Satisfactory; Strategic Relevance and Approach, Technical, Financial and Economic Aspects, Poverty, Gender and Social Development, Environmental Aspects, Policy and Institutional Aspects, and Implementation Arrangements as Moderately Satisfactory; ICT was rated as Moderately Satisfactory; and TA and Training was rated as Moderately Unsatisfactory; Bank Inputs and Processes were rated as Moderately Satisfactory. 2.2 Implementation A number of factors influenced the implementation of the Project. The most relevant were the following: Rice self-sufficient vs diversification goals The Project was implemented at a time in which the international grains markets had become volatile, particularly the one for rice. While Indonesia had been relatively successful in managing the international rice price shock of 2007-08, it was not able to avert a dramatic increase in the domestic price of rice in 2010, following a drought that affected almost 300,000 hectares of rice fields. Self-sufficiency in rice and the stabilization of domestic rice prices had become a major priority for the GOI and MOA was given the mandate to achieve significant increases in the productivity of food crops, including rice, corn, soybeans, sugarcane, and beef. In such context, the diversification agenda originally underlying the design of the FMA grants had lost somewhat traction, with local extension agencies prioritizing support to rice and the other food crops. 8 Crop prioritization policy (One Village, One Product) versus customized targeting. During the first years of implementation, the policy of one-village-one-product was adopted and increasingly implemented by MOA. The policy objective was to maximize the impact of MOA’s interventions by focusing resources on a single crop rather than dispersing them across several crops. The policy diminished the Project’s ability to benefit more than one farmer group within each village by rotating the FMA grants across different crops. Instead, different activities related to the selected crop were supported each year.. Learning programs became focused on a single crop, although there was an effort to change the thematic focus of each program from year to year. Unfortunately the M&E system did not allow assessing the extent to which the number of beneficiaries fell short of the estimated target of 180,000 farm households. High turnover of project staff. The effectiveness of the DPIUs was affected by a high staff turnover, which undermined continuity and consistency in program implementation at the local level. The CPMU was not in a condition to control the turnover given the limited legal influence that central agencies have over local agencies. Retraining by the CPMU of newly appointed staff could only have a limited effect. On the other hand, no incentives were provided to extension agencies that would successfully minimize the turnover of the staff assigned to the DPIUs. Mid-Term Review / Restructuring. During the Mid-Term Review (MTR) several crucial tasks were identified with the CPMU responsible for the follow-up, among which: (i) development of a sound strategy for the scaling-up of the FMAs; (ii) preparation of an evaluation of the effectiveness of training programs; (iii) preparation of a business plan for the development of a e- Petani mobile application; and (iv) improvement of the M&E tools (i.e. survey design and data management) and arrangements. Failure to follow-up was attributed by the CPMU and CADI to limited availability of staff and budget allocations. Following the MTR, upon request of the Borrower, Sub-component 2.iii (Strengthening of the ATCs and AICs) and Sub-component 5.i (Policy Support) were dropped. Sub-component 2.iii had lost relevance to the Borrower. On the other hand, while the development of the guidelines for the implementation of Law n. 16, 2006 continued to be relevant, the Borrower opted for funding the activity directly. The weakness of local extension staff in marketing and agro-enterprise development had become clearly a constraint to the development of agribusiness partnerships under Component 1, retarding the scaling up of village FMAs. While a consultant firm was hired to work together with the DPIUs to assist qualified FOs, the quality of services rendered was sub-optimal. Given the CPMU’s effort to make progress on the implementation of Component 1 and the availability of a shortfall in the disbursement of FMA funds motivated the request to extend the Project’s closing date by one year. With the first restructuring it was agreed that a one year extension in the closing date would have been granted conditional on progress at the end of the first six months. The CPMU recruited seven specialists in agribusiness to assist the DPIU with the development of district FMAs in thirteen pilot districts. To support the CPMU’s effort the Bank mobilized a consultant to assist in the preparation of a manual on support to agro-enterprise development, which NCAE has subsequently published and adopted. In view of the progress observed in implementation the second six months extension was approved. Further progress in the scaling-up effort however failed to materialize. The specialists hired by the CPMU received only limited coordination and supervision from the DPIUs and the CPMU, particularly during the second extension. 9 Besides the changes in the Project’s components note above and the Project’s first six-month extension, the first restructuring included also adjustments to the results framework (see section 2.3.1 below). The second extension of Project motivated the second restructuring. Finally, a partial cancellation of the loan in the amount of $ 4 million, was approved with the third restructuring of the Project on November 3, 2013. All of the three restructuring required approval at the level of the Country Director. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 2.3.1 M&E Design: The term “agricultural service system” in the PDO was too broad a definition, making it difficult to measure progress towards the achievement of this objective. A clearer results framework would have helped to better monitor the achievement of the PDO. The original PDO indicators were poorly defined: (i) changes in household expenditure rates was not a meaningful indicator given the PDO and the notorious difficulty of measuring it through surveys whose design falls short of the LSMS standards, which, if adopted, would make the survey itself very difficult and costly to implement; (ii) while the use of e-Petani and ICT services among farmers was measured in terms of beneficiary farmers in the results framework, it was measured in percent of farmers in the monitoring arrangements table in the PAD. During the first restructuring of the Project some changes were introduced to the set of PDO indicators, as indicated in Section 1.2. A fifth indicator measuring the number of services received by the beneficiary farmers through their groups was included upon request of the Borrower. End of Project targets were set in the PAD without a clear rationale and intermediate targets were not specified except for the first PDO indicator (increase in household expenditure). The PAD does not clarify which tools would be used to monitor each of the PDO indicators. However, four main tools were used: (i) an annual Beneficiary Impact Assessment (BIA) survey; (ii) village- level FMA data collected across all villages regularly during the year; (iii) the ICATAD M&E system to monitor progress in the implementation of Component 3; and (iv) the survey administered annually by CADI to monitor progress on Component 4. The design of the M&E system suffered from a number of methodological shortcomings, including: (i) the system did not identify the members of the farmer groups receiving the FMA grants. As a result of the one-village-one-product policy, it was often the case that the same groups received the FMA grants year after year, limiting the inclusion of other farm households in the Project; (ii) although an explicit goal of the Project was to ensure the inclusion of poor farmers in the FMAs, no methodology was established to identify these farmers; (iii) there were no incentives for the DPIUs to provide M&E data to the CPMU in a timely manner, making the consolidation of data problematic; (iv) no funds were allocated to facilitate a random control by the CPMU of the quality of the data provided by DPIUs; (v) the survey administered by CADI was carried out among farmers that had received training on the use of e-Petani, resulting in a biased result on the use of the application by farmers in the villages included in the Project. 2.3.2 M&E Implementation The M&E system and the management information system (MIS) were not established from the commencement of the Project, which had a detrimental impact on the measurement of progress towards the achievement of the Project's objectives. The implementation of the M&E system suffered from weak coordination between the central project management units (CPMU and Sub- PMUs), as evidenced by some duplication of M&E activities (e.g. ICATAD’s surveys collecting 10 similar data as the BIA surveys). As determined during the Bank-executed M&E assessment, the FEATI Indonesia database was not accessed by the CPMU due to a lack of coordination with CADI, in spite of the fact that the same database was regularly updated with data sent from the DPIUs through the PPIUs. Hence, for its consolidation, data would be transmitted to the CPMU in excel format, rather than through the MIS established for the Project, raising concerns regarding the integrity of the data used for reporting purposes. The implementation of the Baseline Survey conducted in 2008 deviated substantially from the TORs prepared at the design stage. Similarly, the data collected in 2009, 2010, and 2011 as part of the Beneficiary Impact Assessment (BIA) surveys, mainly used to track progress in terms of the achievement of the Project's key indicators, was of limited and of questionable quality and remained unused. The 2012 BIA survey was conducted in collaboration with local universities at the provincial level rather than being contracted to a consulting firm, and is of much better quality. An Independent External Evaluation of the Project was conducted during the MTR (2010), but this evaluation failed to meet its objective and has remained unused. An Impact Study was conducted in 2013. The final report, not available to the Mission at the time of the ICR mission, is mainly descriptive and with its findings and conclusions not supported by reliable quantitative data and evidence. Given the shortcomings affecting the M&E system the Bank carried out an M&E Assessment in February 2012, which resulted in an M&E Action Plan. As a result of its implementation by the CPMU, the FEATI Indonesia database was unlocked and became accessible. Routines for the analysis of data and automatic preparation of report tables were developed, while data review and consolidation meetings were organized more frequently. Missing entries in the database were correctly identified as such, rather than appearing as zero values, which had distorted averages. Data was checked for consistency before being accepted in the system. The implementation of the 2012 BIA survey was satisfactory. 2.3.2 M&E Utilization With some exceptions, the quality of the reports submitted to the Bank was poor, with many reports being submitted late. Most of the M&E reports focused on fund disbursements and project inputs, rather than on providing meaningful analysis of outcomes, limiting the usefulness of the M&E for project management. Information on disbursement and project inputs was indeed relied upon by the CPMU to manage the Project. On the other hand, neither the outcome indicators monitored through the M&E and nor the compliance of the DPIUs in their monitoring responsibilities were used by the CPMU or AAEHRD to sharpen the Project’s implementation strategy. The Bank’s recommendation to adjust the funding of the FMA grants on the basis of districts’ performance and DPIUs compliance was not given follow-up. At Project closure, NCAE has taken the lead in migrating the FEATI Project databases into the current MoA M&E system (in AAEHRD/NCAE for extension). NCAE also plans to use the BIA survey to track the outputs and outcomes of its institutional activities. In spite of these positive plans, the prospects for the full integration of the FEATI M&E system with the NCAE’s core M&E system remain uncertain. While the NCAE management is fully aware of the benefits of such integration, the overall organizational culture at the MOA continues to be oriented towards 11 the reporting of data related to financial and physical progress, rather than to program performance. 2.4 Safeguard and Fiduciary Compliance (a) Safeguards: The “B” category for environmental assessment risk still applies. Safeguard issues were well articulated in the Project document including safeguard instruments and actions to be taken by the project i.e. (i) environmental screening criteria and negative list; (ii) basic environmental management criteria for pesticide use; (iii) recruitment of Sr. Environmental Specialist; and (iv) Environmental and Social Management Framework (ESMF). Safeguard supervision was started only in the later stages of project implementation i.e the fifth supervision mission. In early 2012 a Sr. Environmental Specialist was recruited by the project with the following outputs : (i) FEATI Project Environmental Guide Book; (ii) negative lists to screen FMA activities; and (iii) an environmental training manual. The implementation of an environmental management and mitigation plan, as recommended in the QAG assessment, began late, as noted by the sixth and seventh supervision missions. During this period, the plan to develop a Pest Management Plan was dropped, with environmental measures such as the provision of training on integrated pest management and the dissemination of the Environmental Guide Book to ensure that pesticides included on a negative list were not utilized by FOs being implemented as an alternative. ESMF was not explicitly defined in the project manual. Nevertheless, the project had efforts on the inclusion of women and the poor in the farmer’s managed extension activities. Despite the delay in the safeguard supervision and shortcomings at the beginning of the project implementation, at ICR the mission found no evidence of major, sensitive or critical issues related to environmental impacts resulting from the implementation of the FEATI Project. The “B” category for environmental assessment risk is consistent throughout the FMA activities observed during implementation missions because all project activities engaged in limited area of impact and the magnitude of the impacts are also not significant and are manageable. Some good practices implemented to ensure environmental safety were included in learning discussions involving FMAs. For example, cattle manure handling, cattle cage location adjacent to water source, used chemical container handling, used syringe disposal, the use of fertilizer/pesticide, housekeeping and occupational and heath safety aspect. However, good and bad practices of environmental management were not well monitored and documented by the project which could generate lessons learned from the FEATI Project, useful in the training of farmers and extension workers. Safeguard instruments used during project implementation such as FEATI environmental guide book, the negative list, environmental training materials and the recruitment of environmental safeguard specialist were available and implemented during the missions. However, the project was only able to train extension staff and farmers in eight out of 68 districts due to limited budget allocation. The rating of Moderately Satisfactory during supervision missions was realistic as some shortcomings at the early project implementation were able to be addressed at the project closure. (b) Procurement: Overall, procurement processes were implemented in compliance with Bank procurement policies and with agreed upon procedures. However, there were some shortcomings including (i) improper budget allocation for BPP construction resulting in delays to the maintenance period beyond the end of the Government's fiscal year ; (ii) inadequate capacity of project staff resulting in poor record keeping; and (iii) a lack of understanding of the procurement committees regarding the application of qualification requirements in the evaluation 12 and disqualification process. To address this issue, the Bank provided in house training attended by project procurement staff at all levels. (c) Financial Management. An assessment at entry point indicated a relatively high level of risk due to the geographical extent of the Project and the high proportion of soft expenditures that it involved. Measures to mitigate these risks were agreed upon and implemented by CPMU, with these measures including the recruitment of financial management consultants, the formulation of a project management manual, which was approved by the Bank, and the implementation of internal and external audits over the Project. In terms of financial management, the performance of the Project improved after 2010, when oversight of this aspect of project implementation was intensified through increased activity by CPMU staff, consultants, DPIU finance staff, and staff of public works agencies, particularly in the area of the monitoring of the construction of facilities for the BPPs. A simple financial management monitoring checklist was developed as a tool to implement oversight at the farmer group level. Reports of audits of the Project's financial statements audits were consistently submitted on time, with all reports being accepted with unqualified opinion. Internal audits of the Project were also conducted annually by IG of MoA. By the end of the Project (EOP), all backlogs had been resolved, with most of the necessary follow-ups to audit findings having been completed. By EOP, some financial management issues had still not been fully resolved, including identified weaknesses in the financial management of agribusiness operations of the FOs. While farmer groups generally had the capacity to manage block grants, requirements became more complex for more advanced FOs and in higher level FMAs implemented by district and provincial FOs that received larger block grants. On the other hand, the extension workers had limited capacity to assist FO members in matters related to financial management. Another outstanding issue related to the turnover of 25 completed BPP facilities to the local governments. The BPP facilities were not listed as local government assets, meaning that local governments are unable to provide budget allocations (APBD) to support the maintenance of the new facilities. (d) Governance and Anti-Corruption: Prior to negotiation, a mitigation plan containing a set of measures to address these risks was agreed upon. The formulation of an Anti-Corruption Action Plan (ACAP) was prepared by the Project covering measures to mitigate fraud and collusion risks in procurement process; information about the Project disseminated through various media channels. A guideline on complaints handling mechanism was prepared in June 2010. Despite this delay, at the beginning of the project implementation, the project implementers at all levels publicized the project activities and budget in the website of Ministry of Agriculture, and local newspaper. Shortlists for major contracts were posted on the website. Most complaints were received through SMS centers and were recorded at the CPMU office. Over the period of the Project implementation the level of awareness and understanding of the ACAP and related issues was high amongst CPMU staff, this was not the case amongst staff of the implementing units at the provincial and district levels. At EOP, success had been achieved in facilitating external oversight through: (i) the participation of representatives of CSOs and media organizations at bid opening sessions; and (ii) the implementation of third-party evaluations by local university and extension committees. 13 2.5 Post-completion Operation/Next Phase 2.5.1. Transition Arrangements: The MOA/AAEHRD did not submit a Plan for Future Operations of the FEATI Project formally, as required in Section 3.03 of the Development Credit Agreement (DCA), which mandates the formulation of a plan to ensure the continued achievement of the objectives of the Project following its completion. However, based on the discussion with the Director of NCAE, the following arrangement has been made: Central level arrangements: The responsibility for continuing the project activities related to strengthening farmer organizations and enterprise development was transferred to a division under NCAE called “Kelembagan Petani dan Usahatani” or Division of Farmer Organization and Enterprise Development which was established in 2011. This division prepared a program and allocated budget for 2014 including (i) strengthening and improving the sustainability of established FOs to become viable commercial enterprises; and (ii) strengthening the capacities of trained farmer facilitators through newly established Village Extension Centers. Monitoring and evaluation was transferred to NCAE's Sub-division for Planning, Monitoring and Evaluation. In addition, technology dissemination approaches developed under Component C were adopted by ICATAD of IAARD under a new program called “Model Pengembangan Pertanian Perdesaan Melalui Innovasi – m-P3MI” or Model of Rural Agricultural development through Innovation intended to strengthen the capacity to conduct participatory research and to develop and to disseminate best practice technologies for improved agricultural productivity, profitability, and sustainability. This model was included as one sub-component of the recently approved Bank-funded Sustainable Management of Agricultural Research and Technology Dissemination (SMARTD) Project. District and provincial level arrangements: At Project closure, the DPIUs ( at district) and PPMUs (at provincee) were disbanded. No staff was appointed within the local extension agencies (Bapeluh and Bakorluh) responsible for the FEATI Project’s follow-on activities. Despite this, in 2013, some FEATI districts allocated funds to support the operations and maintenance (O&M) of BPPs constructed by FEATI. A significant majority of local legislatures and district officials (Bupati and Bappeda Heads) have indicated their ongoing support for the institutional development of FOs and the local extension agencies created under the Project, a sign of the positive reception for the innovative approach to the provision of extension services introduced by the FEATI project. The ICR Mission noted examples of local government allocations in 2013 and 2014 to support follow-on project activities and the limited scaling-up of village-level approaches within and outside the Project areas. 2.5.2 Performance Indicators to Monitor the Project: A number of key indicators developed for the FEATI Project could be used to monitor follow-on activities. The original targets could be used as a benchmark for monitoring the outcomes of these activities. The design of on line FMA databases developed by the Project (www.featiindonesia.com) could be adopted to better enable the tracking of the outputs and outcomes of farmers' activities as well as public extension services. 2.5.3 Recommended Follow-up by the Bank: The Government has shown interest in further scaling-up the Project to additional districts and provinces. During initial discussions, the Bank indicated its support for scaling-up the Project not only in terms of expanded geographical coverage (horizontal), with the inclusion of new districts, but through innovations and improvements to the FEATI design on the basis of lessons learned through the implementation of 14 that project (vertical). A follow-up project would facilitate the completion of the institutional reforms that were initiated under the original Project and scale up the successful approaches developed under this Project to empower farmers and farmer’s organizations. A critical assessment of the approaches and innovations promoted by the Project should be conducted to serve as input for the design of a new follow-on project. Since no reliable impact study was conducted at the close of the FEATI Project, an impact evaluation should be carried out between 2014 and 2015, in synchronization with the preparation for a follow-on project. The results could also be used as a benchmark for this project. Under the new project, the integration of fishery and forestry, which were not included in the original design of the FEATI Project, should be seriously considered. With the inclusion of fishery and forestry, the new project could be a vehicle to initiate the implementation of Law No. 19, 2013 on Farmers Protection and Empowerment, which is complementary to the Extension Law 16, 2006 that was supported by the Project. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The FEATI PDO remains highly relevant to the GOI's National Medium Term Development Plan (RPJMN) for 2010-2014, which envisions to support creative, innovative, and independent farmers who are able to utilize science and technology and local resources to produce highly competitive agricultural products; to promote healthy, honest, and fair business partnerships; and to improve the quality of services and the level of professionalism of government employees in the agricultural sector. FEATI’s farmer-driven approach has been successful also in improving rice productivity, particularly in marginal areas. Hence, MOA’s increased focus on rice self- sufficiency since 2008 did not result in the FEATI objectives becoming irrelevant. The Law on Farmer Empowerment and Protection promulgated in May 2013 in fact further reasserts and supports these objectives. The FY 2013-2015 Country Partnership Strategy (CPS) for Indonesia emphasizes the need to revitalize agriculture as a pro-poor strategy and foresees a greater level of engagement by the Bank in this area. The Bank’s Public Expenditure Review for Indonesia’s agricultural sector (2010) recommended GOI to reallocate its sectoral expenditure from input subsidies to public goods, such as extension, R&D, and irrigation services. It also supported the use of grant mechanism to empower farmers in driving the demand for such services, provided appropriate monitoring of their use is ensured. The SMARTD and WISMP Projects are supportive of the prioritization of linkages between extension and R&D and of farmer organizations’ capacity development and empowerment in agricultural sector development strategy. The Project’s design needs to improve to be made consistent with the objectives, particularly in some key aspects: (a) To support the scaling up of successful FMAs into agro-enterprises beyond the village boundaries, learning plans and capacity building of the managing farmer organization under Component 1 should be allowed to progress into a grant-based support to the implementation of a multi-year business plan that allows for the acquisition of those assets and equipment that are necessary for the agro-enterprise itself. Requiring counterpart equity by the beneficiary in the form of cash and/or by a private sector counterpart would provide strong incentives towards the selection of agro-enterprises with a high potential for sustainability, particularly whenever a bank 15 loan is used to co-finance the investment. The Bank’s experience in a number of Productive Partnership and Rural Alliance models should be considered in this regard; (b) Capacity building and institutional strengthening under Component 2 should not be limited to the public extension agencies, but should embrace a system perspective. Supporting the capacity of DECs and PECs to provide strategic guidance on local extension priorities can be an effective way to promote the development of partnerships between private and public sectors and farmers at the institutional and operational level. Similarly, under Component 3 support should be provided to develop the capacity of the Provincial Technology Commissions (PTCs), which have the function to identify opportunities for strengthening the research-extension linkages involving the AIATs, the private sector, and farmer organizations; (c) A support to the development of ICT for farmers focused exclusively on the public sector does not serve the purpose. A clear role for PPPs should be explicitly envisioned in the design of Component 4; (d) For M&E to become an effective tool to promote governance and efficient program management, the involvement of independent third parties in the collection, analysis, and verification of Project information is essential. This concerns the assessment of the level of satisfaction of farmers with the services received from both public and private sources, as well as to measuring the contribution of extension services to household income and farm productivity. Implementation arrangements need to include a system of incentives that would improve the commitment of district governments to the project, for instance by reducing the staff turnover at the DPIU level and by ensuring a satisfactory and timely collection of data. For example, prioritization of grant fund allocations across districts could be based on a set of clear and transparent evaluation criteria tied to performance. 3.2 Achievement of Project Development Objectives By EOP, only two of the five defined objectives had been fully achieved; two objectives were not achieved, while evidence for a fifth one was inconclusive due to the lack of appropriate data. Despite a failure in achieving the project indicators, the PDO had been in achieved to a significant extent. Through the implementation of FMA, a top – down agricultural service system was gradually transformed to demand – driven agricultural service system. The training provided to farmer group leaders enabled them to better lead their groups in learning activities as well as developing their enterprises. The trained FOs were able to identify, decide, and implement their needs on services to develop their agro-enterprises. The establishment of trained Farmer’s Council under the project helped farmers in consolidating their learning needs, and linking with non-public service providers to be supported by FMA grants. Through the FMA implementation, AIATs could directly link and collaborate with BPPs in providing information and technologies as demanded by farmers. This led to a wider coverage of technology dissemination. Improved capacity of AIATs’ researchers enabled them to better serve field extension workers and farmers in providing demand – driven technologies. In some provinces, AIATs also facilitated FOs to link with market and product registration. With FMA, FOs improved their services to support enterprises development of their members by providing among other establishing partnership with private sectors on marketing, external service providers on technology and information. Monitoring on its progress was continued by NCAE. Diagram 1 and 2 below indicates the changes in agricultural service system before and after the project. 16 Diagram 1. Agricultural service system before the project : Top – down information / communication. AIATs Agricultura Bapeluh l offices Technology dissemination Agric productivity program BPP Coordinator Field Extension Workers FO FO FO Farmers Diagram 2. Agricultural service system after the project : demand – driven infomation/ technology Other supports Agricultural Bapeluh AIATs offices Feedback to FOs BPP Coordinator Field Extension Workers Farmer’s Council ( Pos Penyuluhan Desa) External service provider Partnership Services demanded by FOs Information request from FO FO FO farmers Feed back to FOs/farmers Private sector/ Information dissemination market/ input from FOs to FO members suppliers Farmers 17 The achievement of the PDO was measured by the following indicators : (i) The M&E system did not collect reliable data to measure the indicator “ Percentage of members of participating FOs with improved household expenditures,” for which evidence is therefore inconclusive. As noted above (Section 2.3.1) the QEA noted serious theoretical and empirical issues associated with the use and measurement of such an indicator. (ii) The indicator “Number of technology packages developed by AIATs that are used in the FMAs” was meant to track the extent to which the AIATs were able to develop agricultural technologies meeting the farmers’ needs and interest, given the market and agro-ecological conditions under which they operated. Through the FMAs, participating AIATs were able to better target the dissemination of the technologies as requested by farmers. Technologies were not only used, but also fine-tuned in a participatory way by AIATs researchers and farmers. AIATs, contextually, improved their capacity to collaborate with other research institutions (private sector, universities, NGOs, and local government) in the development and dissemination of technology. This benefitted more farmers previously unreachable by AIATs. While the participating AIATs innovated on their processes, the quality of the research output developed by the Indonesian Agricultural R&D system – and which the AIATs are supposed to assess at the local level - had yet to follow. (iii) The indicator “Percentage of participating farmers benefiting from e-Petani and other information communication technologies” was intended as a part of agricultural services to assist farmers in accessing various information provided by the e-Petani application (managed by MOA) and by other sources through the use of computers and cell-phones. However, the project was not able to develop a mobile platform allowing the majority of farmers access the information through their mobile phones. (iv) The indicator “Number of joint activities/ventures with private sector established by FOs” was meant to assess the Project’s achievement in developing agribusiness partnerships between FOs with the private sector. Only towards the end of the Project it was agreed that joint activities and ventures should be formalized through a contract to count for M&E purposes. Based on that distinction, only 1,288 partnerships involved written agreements, while 4,400 involved unwritten agreements. The EOP target was underachieved by 46%. The indicator from Component 1 “Percentage of FMAs where experts from external providers (universities, research organizations, NGOs, experienced farmers, and private sector) participate as a resource person exceeded EOP by 214%. This suggests that while the FMAs were an effective approach to stimulate a diversification in the access to information and technology by farmers, they were less successful in the case of agribusiness partnerships where the private sector counterpart’s commitment goes beyond that of a resource person. It should also be considered that typically a contract would materialize at the end of a process in which farmers are able to demonstrate their capacity to comply with the requirements of the buyer. Furthermore, within the limits of a village, the size of the market does not justify the use of contracts. (v) The indicator “Number of services provided to FO members by their organization” measured the extent to which farmer organizations and/or groups had become a conduit for the delivery of services requested by their members. Such services could be provided either by the public extension agencies, by the private sector (including NGOs and universities) and/or other farmers (farmer to farmer extension). The range of services included provision of information, technology, counselling, capital, production inputs, and utilization of natural resouces (land and water management). The EOP target was exceeded by slightly more than 100%. The satisfactory achievement in this indicator is also consistent with the measurement of farmer satisfaction with the services received from the extension agents under Component 2 (“Percent farmers report benefiting from improved skills and services of extension staff:” EOP target exceeded by 12.5%) as well as that related to the performance of the AIATs note above. 18 The achievement was on balance moderately satisfactory, as the Project failed to achieve the EOP target in terms of three out of total nine indicators. Among four components, only Component B fully achieved its EOP target. For Component A, the indicator Percentage of FOs with sound financial and procurement management practices under-achieved the EOP by 34%: the majority of FOs under the Project were newly established and therefore had limited management skills. As noted above, the intermediate indicator of Satisfaction levels of beneficiaries regarding improved R&D services under Component C was not fully achieved. Component D failed to achieve its EOP target due to the failure of CADI to develop a mobile platform for the e-Petani application, which would have enabled a far higher level of access by farmers to the information in the database. 3.3 Efficiency At appraisal, an estimate of the economic rate of return (ERR) or net present value (NPV) was not provided, due to the demand-driven nature of the Project. NPV and cost-benefit ratios computed from a small sample of FMAs supported under DAFEP provided an indication of sizeable benefits from implementation of Component A (approximately 38% of the overall investment effort). A recalculation of NPV or cost-benefit ratios was therefore impossible. Unfortunately, data limitations did not allow for the estimation of ex-post indicators of financial and economic performance of the FMAs. Complementary evidence from other sources is also scant and somehow weakened by methodological shortcomings. Two studies, from the University of Brawijaya and PT Intersys (a Consulting Company), used data from a limited sample of FMAs. They both did not account for biases in the selection process driving inclusion and exclusion in/from the Project. The impact study report of PT Intersys hints at an overall positive impact on farmers’ incomes, depending on the crop being considered, with less satisfactory results being recorded for crops including rice and chilies. Based on a non-representative sample of 59 participating and 22 non-participating farmers in three districts in East Java, the Brawijaya University study concludes that revenues of participating farmers have increased throughout the Project period at double the rate of those of non-participating farmers. Overall, these findings are consistent with those from the DAFEP impact evaluation. A more specific analysis of the FMAs that have been scaled-up to district agro-enterprises under FEATI was undertaken during the preparation of the ICR. The analysis, based on a very small sample and therefore only indicative, shows that the learning grants invested in the development of agro-enterprises such as cattle fattening and mushroom and shallot cultivation have increased revenues significantly, ranging between 25 % (rice) and 700 % (goat). The details of the analysis are in Annex 3. The results suggest that as the support to the development of the agro-enterprise started only too late into Project implementation, most of the agro-enterprises were yet suffering from an unclear legal status and that their members had yet to contribute to the development of the enterprise by re-investing a part of their earnings. Few enterprises had borrowed from banks. The most successful enterprises are those that consistently receive FMA learning grants in the same thematic area (i.e. commodity or livelihood), while annual learning grants or changes in learning topics led to less sustainable outcomes. With regard to Component C, a rigorous assessment was carried out by ICATAD in 2011. The estimated rate of return from the adoption of the technology packages disseminated by the participating AIATs and fine-tuned with farmers through the FMAs indicated a rate of return of in the range of 25 % (rice) – 120 % (sheep). 19 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory The PDO was consistent with the GOI’s sectoral priorities and with the Bank's sectoral strategy both at entry and at end of Project. The design and implementation arrangements could have been improved to ensure a satisfactory achievement of the Project’s objectives. Weaknesses in the design and implementation arrangements noted at entry undermined the full and efficient achievement of the Project’s objectives. The Project only partially achieved the PDO of developing a demand-driven, market oriented agricultural services system, based on partnerships between farmer groups, public agencies, and private sector enterprises at all levels. While the Project was successful in moving towards a farmer-driven approach to the delivery of extension services fell short though of achieving the scaling-up of village FMAs into district and provincial-level agro-enterprises capable of linking smallholders to markets in their efforts to commercialize their production. The skill mix of extension services is mainly oriented towards technical and crop-specific subject matters rather than agribusiness and organizational capacity development. Furthermore, the policy environment was overall attaching more importance to achieving increases in farm productivity rather than farmer income. Furthermore, provincial extension agencies had a marginal role in the implementation of the Project; no explicit allocation had been made from the start to grants to support district-wide and/or provincial-wide FMAs. This resulted in the Project successfully disbursing its funds for most of its implementation period to support village-level FMA, mostly oriented towards the transfer of technology or production innovations and with which extension services were most confident given their skill mix. Only a minor share of the funds for FMA grants had been allocated to district and provincial FMAs in support of agro-enterprise development, and then only too late to result into a clearly discernible sustainable impact (see section 3.3 above). 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Poverty Impacts: Although the FEATI Project Results Framework included indicators intended to measure the Project's level of outreach to poor farmers, the relevant indicator under Component 1 mixed 'women and the poor' under a single category. While, the Project’s M&E guidelines did not provide criteria for the identification of poor households at the community level it should be noted that using the BIAs to identify poor households would have met the same complications note for the measurement of the PDO indicator related to changes in household expenditure levels. Gender Aspects: Results of the 2012 BIA survey showed that 36.6% of those participating in the Project's planning and decision-making meetings were women, a level higher than originally targeted. During field visits, it was observed that the level of participation of women varied significantly, being highest in FOs involved in post-harvest food processing activities (often above 80.0%) and in the raising of cattle and milk production (approximately 50%). A particularly unsatisfactory result is that only 4.0% of the total value of grants disbursed under the Project was awarded to women-led FMAs, well below the EOP target of 20.0%. 20 Social Development: The Project was partly successful in empowering farmers to better manage their productive resources, although it fell short of supporting the establishment of agro- enterprises as originally envisioned. The extensive capacity building of FO managers and members supported by the Project benefited large numbers of women and men farmers in the approximately 3,000 target villages, although data shortcoming do not allow to determine whether the effective number of beneficiaries corresponded to the 180,000 originally estimated. As traditional farmers' groups transformed themselves to become productive economic units, members of these FOs have benefited from an improved ability to produce quality products and to better access markets and directly contribute to the local economy. Some FOs also began operating lending and savings schemes. (b) Institutional Change/Strengthening Although the focus of institutional reforms under the FEATI Project was on the establishment of unified extension institutions in 68 districts in 18 provinces, many other provinces and districts across the country also established similar extension institutions, as mandated by Extension Law No. 16, 2006. More provinces and districts have developed such institutions since the start of the Project. The AAEHRD, and with it MOA, has learned from the implementation of the FEATI Project and is intent in up-scaling the Project’s approach based on the FMAs to a national program level. The recent promulgation of the Law on Farmer Empowerment and Protection is a good indicator of the success achieved in this regard. Although the Project design attempted to involve the PECs and DECs through the awarding of the FMA grants, in fact their role proved to be very marginal. No support to developing their capacity was included under the Project as district and provincial governments were assumed to provide their own funding. Moreover, district and provincial-level FMA grants were considered only towards the end of the Project, making the involvement of these institutions inessential from the very start. (c) Other Unintended Outcomes and Impacts (positive or negative) Some basic principles introduced by the project were used as inputs to the formulation of a new Law no 19, 2013 on Farmer’s Protection and Empowerment including strategy on farmer’s empowerment and roles of extension workers in facilitating farmers. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not Applicable 4. Assessment of Risk to Development Outcome Rating: Substantial The FEATI Project was partially successful in achieving its development objective of establishing a demand-driven agricultural services system by using a farmer-managed extension activities (FMA) approach to allow farmers to determine their own extension activities (FMAs), to access the grants and to implement them, and to build partnerships with private sector enterprises. At a higher policy level, a key risk to the PDO is created by the goal of achieving self-sufficiency in all staple crops, rather than resilience in food security. The self-sufficiency approach orients agricultural agencies towards the achievement of quantitative production targets, rather than prioritizing improvements to farmers’ incomes and living standards, which may entail 21 diversification. It de-emphasizes the use of value chain development as an approach to strengthening market linkages and the retooling of extension agencies to promote their role as brokers of services by facilitating linkages and interactions among farmers, private sector firms, and non-public providers of business development services within a systems approach that promotes the development and adoption of innovations. In terms of risks to the achievement of sustainability of the project results and outcomes of the Project, the following aspects need to be considered : 4.1. Sustainability of Farmer Organizations: Of the 872 newly FOs established under the FEATI Project, only a third, those that were established as formal legal enterprises and/or commercial entities, have the potential to become sustainable. These enterprises would require ongoing support and guidance from both public and private service providers. Many of the FOs has already developed partnerships with private sector enterprises to support the FOs' marketing programs. With the enactment of a new Law on Farmer’s Protection and Empowerment, no 19, 2013, the probability of supporting the established FOs is high. 4.2. Continuation of the Extension Institutional Reforms: Integrated provincial and district extension institutions have been successfully established in the Project locations with variations in the structures of these institutions. The pending enactment of the Presidential Decree on Extension Institutions might have an impact on the stability and sustainability of the existing extension institutions at district and provincial level. 4.3. Improving the Capacity of Extension Workers and Farmer Facilitators: As already stated, the level of capacity of the staff of the public agricultural extension system remains weak, mainly due to their lack of specialist knowledge and experience in areas such as agribusiness development, marketing, entrepreneurship, and value chain management. Targeted training programs originally planned under the Project could help address these gaps. Farmer facilitators (one woman and one man per village) were trained by the Project in 3,000 target villagers. Using more effective methods such as learning-by-doing, participative training approaches, internships and coaching could help sustain the Project outcomes. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory The project was prepared based on a solid assessment carried out by international and national consultant team recruited by the government, and evaluation by the Bank’s team on the previous DAFEP. The Bank mobilized a technical assistance to strengthen the project design, particularly on farmer organization and agricultural extension. Prior to the project launch, the Bank ensured the capacity of the project staff at central level on fiduciary related aspects by organizing in house training on procurement and financial management, and providing guidance on preparing ACAP. The implementing agency and the Bank overestimated the capacity of former DAFEP districts in implementing the Project, as the majority of the project staff were new and had no experience in DAFEP. This resulted in a poor quality of the project implementation. The PDO and key performance indicators and targets were also discussed with the implementing agencies. However, a Bank’s support on monitoring and evaluation to assist the project was not effective in ensuring that the project staff understood about the linkages between PDO and the indicators as well as the data collection allowing the project to set up a proper monitoring and evaluation 22 system. Safeguard requirement was not clearly addressed leading to delay in the implementation of Environmental Management Plan. The inclusion of women and the poor in the project activities were not clearly defined leading to multi interpretation among project implementers. The working mechanism between three central management units i.e. CPMU at NCAE, sub- CPMU at CADI, and sub CPMU at ICATAD to ensure proper linkages among components under each central management unit was addressed and agreed on by these three implementing agencies. However, due to its autonomy of each implementing agency, its implementation was not effective. The above assessment was similar with a Quality at Entry Review (QER) conducted by the Bank in February 2007, identifying shortcomings in the Project design that would subsequently prove to hinder the implementation of the Project : (i) an unclear rationale for the selection of the targets in the results framework; (ii) the lack of a TA grant to support the implementation of the Project, which prevented the Project from benefiting from exposure to international best practices; (iii) the lack of clearly identified strategies for the inclusion of women and poor farmers; (iv) the weak arrangements for the development of ICT software and hardware; (v) the need to proceed to a full disclosure of the Environmental Management Plan at the very start of the Project and to prepare a Pest Management Plan; (vi) a lack of analysis of the structure of incentives for stakeholders involved in the Project; and (vii) the inclusion of an indicator related to changes in household expenditures, an indicator notoriously difficult to measure and subject to influences from a multitude of factors outside of the Project's control. (b) Quality of Supervision Rating: Moderately Unsatisfactory Supervision during the first two years of implementation was limited to one mission per year. A Mid-Term Review was conducted in 2010, as a result of which 128 recommendations were agreed upon. As a follow up of Mid-Term Review, AAEHRD as the leading project implementing agency proposed a restructuring the project in October 25, 2010 which was approved in March 27, 2012. This long period of restructuring was due to issues being restructured were very critical in ensuring the project on the right track and this required intensive discussions with the government. The Project experienced three times of changes in the leadership. Despite a high turn of the Bank’s Task Team Leader (TTL), the Task Team followed up commitments agreed by previous TTLs, and new TTLs provided different angles on how to move forward on the project implementation. Some unidentified issues from the previous Task team were able to be addressed by a new TTL including safeguard and the farmer’s owned enterprise establishment. The Task Team was consistently engaged in a number of technical assistance activities to support CPMU in the implementation of the restructured project including the following: (i) the preparation and implementation of an M&E Action Plan; (ii) preparation of a draft operations manual to support the development of farmer-managed agro-enterprises (January 2013); and (iii) preparation the follow up actions to ensure that the recommendations were workable by the CPMU. The Task Team monitored the progress of the agreed action plans by attending several quarterly project consolidation meetings, and discussed critical issues in the project implementation and how to address it. However, the agreed action plans were not able to be fully implemented due to an interpretation gap among the project implementers, particularly at district and provincial levels. The Task Team recommended the CPMU to summarize and translate from English to Bahasa Indonesia critical action plans and followed by a simple guideline to allow common understanding among the project implementers. The Task Team was not able to provide assistance directly to all participating districts and provinces due to huge number of locations as opposed to the availability of the Task Team. 23 Supervision of Fiduciary aspects was regularly facilitated by the Bank, with the Task Team's procurement and FM specialists conducting regular missions at different times of the year, the results of which have been summarized in clear and exhaustive reports. Despite the delay in conducting safeguard supervision which was only started in the forth year of the project, a safeguard team provided intensive guidance to the project and came up with four recommendations which were met by the CPMU during the rest of the project life including formulation and utilization of Project Environmental Guide Book, negative lists to screen FMA activities and the delivery environmental training for extension staff and farmers in eight out 68 districts. The Task Team recommended the CPMU to train staff and farmers in other districts, but no follow action was taken. Unfortunately, there was no such assessment from the Bank whether all supports and recommendations provided by the Task Team addressed the identified issues in the project implementation, so early remedy / improvement could be taken by the Task Team. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory In view of the above ratings regarding the Bank's performance in ensuring quality at entry and quality of supervision, the overall performance of the Bank is rated as Moderately Unsatisfactory. Since one rating is MU and the other MU, the overall rating for Bank performance is considered MU because the project’s overall outcome rating is MU. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory The Government of Indonesia, through the Ministry of Finance (MoF), MoA and Bappenas, supported the preparation and implementation of the FEATI Project and ensured that sufficient government counterpart funds were allocated annually. Prior to the start-up of the Project, the central government ensured that all requirements for project implementation were met, including the production of a project management manual acceptable to the Bank; the establishment of an inter-agency National Steering Committee under MoA’s Secretary General’s Office and a National Technical Committee under AAEHRD to provide policy and technical guidance; and the deployment of designated and trained staff responsible for project management at AAEHRD, ICATAD and CADI. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory Central level project implementation: The AAEHRD and its units were the lead implementing agency, with the participation of the newly established extension institutions at the district and provincial level. Prior to the initiation of the Project, the AAEHRD established the CPMU, while ICATAD and CADI established Sub-PMUs in their respective agencies. CPMU made efforts to set up monitoring and evaluation system using various tools including on line data base. Weaknesses identified at the central project management level included: (i) AAEHRD was not able to set up a systematic and sustainable coordination mechanism between the CPMU and the Sub-PMUs, creating some disconnect in project planning, monitoring and reporting.; (ii) in 24 sufficient managerial training to the CPMU staff; (iii) the delay in replacing some sub-qualified consultants. For example, it took almost a year to decide to discontinue the contract of a sub- standard consulting firm and to replace it with individual consultants; (iv) the Project M&E system was suffered at the design and implementation stages. In spite of the importance of capacity building in the Project and the willingness of AAEHRD to make budget allocations for this purpose, no assessment of training needs was conducted for the participating extension institutions, resulting in the absence of well-defined training plans. Because of decentralization polices, the AAEHRD did not have the authority to make changes in project staffing at the DPIUs, thereby impeding a satisfactory solution to the high rate of turnover of project staff noted during several supervision missions. Despite these shortcomings, the AAEHRD has committed to continuing to implement the approaches introduced by the FEATI Project as part of the implementation of new Extension Law no 16, 2006 leading to the implementation of the new Law No. 19, 2013 on Farmer Protection and Empowerment. The Agency has just published the manual on developing farmer’s own enterprises, with assistance provided by the Bank, and has allocated funds under its budget for FY 2014 to strengthen the FOs established under the FEATI Project. Decentralized level project implementations: At the beginning of the project, the implementing units at the provincial and district levels namely the provincial Extension Coordinating Bodies (Bakorluh) and the district Extension Implementing Units (Bapeluh) committed in fulfilling the project requirement such as establishing project management units, deployed project staff and allocated operational budgets. The construction of new BPPs and renovation of existing BPPs as a home base of field extension workers was completed before the Project closing. Some districts allocated funds for additional BPPs building with similar design, and have begun to replicate successful village FMA in villages not previously included in the Project. The Project staff received training in areas related to project management and were provided with TA support by the Project under the coordination of the CPMU. However, the high rate of rotation of local staff made repeated training necessary and reduced its effectiveness. Coordination between the CPMU and the PPMUs and DPIUs took place mainly through quarterly meetings, which also provided a venue for the sharing of experiences and the discussion of project management issues. The AIAT staff also participated in these meetings, leading to the development of a closer working relationship between researchers and extension workers. However, for some reasons these meetings were not effective to improve the project implementation. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory The government is committed to reform the research and extension agenda through the implementation of new Extension Law no 16, 2006. However, the implementing agencies at various level had no sufficient efforts to improve the project implementation and its impact to the future of the extension reformation. The changes in the leadership of the implementing agencies affected in the high turn of the trained project staff promoted to other offices. 25 6. Lessons Learned The FMA approach continues to represent an innovative approach to the objective of developing farmer-driven and market oriented extension services. It provided FOs with the opportunity to improve their capacity in achieving their goals based on market and agro- ecological conditions within which they operate. Training provided to farmers and the first-hand experience of FO members in participatory planning, proposal preparation, FMA implementation, simple monitoring, and the financial management of grant funds gave them confidence and the incentive to manage their own learning activities and business enterprises more effectively. The scaling-up of FMAs into district and provincial-level agro-enterprises should have been prepared early-on in the Project, and not towards its end. By the time the Bank provided the technical assistance on how to operationalize the scaling-up it was too late. Most of the farmer groups were just beginning to show good progress as revealed during the preparation of this ICR, yet they lacked proper legal business development assistance. With more time and improved assistance from the Project, they could have represented a successful learning experience for MOA. However, the conclusion to be drawn is not that MOA should avoid investing in the development of agro-enterprises managed by FG. Rather, the field work suggests that this is an area where substantial progress can be made provided that the support from local extension agencies is integrated with that of other services in the local agricultural units at the district and provincial level. Benefits could also be generated through the further establishment of partnerships with the private sector at the institutional and business development level, in alignment with the FEATI PDO. In this regard, the failure to support the capacity development of PECs and DECs through the Project represented an important missed opportunity. The design of FMA needs to be refined to achieve the scaling up of FOs into viable agro- enterprises able to achieve economies of scale and strengthen the linkage of smallholders to markets. Extending the planning horizon beyond the current yearly one, scaling-up the grants to FOs and tying them to contributions from other value chain actors (e.g. banks, private commercial sector, etc.) against professionally developed business plans are all ideas that GOI could explore in refining the FMA design as it starts articulating the guidelines of the recently approved Farmer Empowerment and Protection Law, No 19, 2013. The Bank’s experience in linking smallholders to competitive value chains across Latin America, Africa, and Asia could be tapped for the purpose. In such an approach learning and business development should be integrated in the FMAs. MOA could then consider integrating the FEATI program with a redesigned and revitalized PUAP program. Strong linkages between farmers, researchers and extension workers facilitate an effective dissemination and adoption of improved technologies. Under the Project, the successful dissemination of technology depended on strong collaboration between farmers, researchers and field extension workers, using participatory, learning-by-doing dissemination approaches and involving the provision of intensive research support by scientists who developed the technologies. A lesson from the FEATI Project and from experience elsewhere is that farmers will adopt improved technologies faster if they participate in the field assessment and in the fine- tuning of the technologies through farmer-managed demonstrations, farmer research trials, field days, multi-media information campaigns, and other similar activities. In the Project, early technology adopters included the farmer cooperators and technology demonstrators involved in these activities. 26 The development of ICT for farmers needs to be based on PPP. A business development plan that began outlining a PPP strategy was prepared but not followed-up to. Instead, a MOA-focused strategy was pursued without a clear and systematic approach to the inclusion of the private sector being considered. The involvement of ISP and mobile phone companies can only work if a business case can be identified in the first place, followed by a longer-term engagement in which the public sector takes on the responsibility for controlling the quality of the information provided through the system. It could also lead in the preparation of specific applications either at the corporate level (e.g. e-Petani) or in collaboration with the private sector. Finally, an ICT strategy should recognize that the extension worker has still a key role to play in helping farmers understand the content and implications of the information received through the internet. Hence, ICTs cannot completely substitute for an effective extension service at the local level. In this regard, extension workers need to be part of an effective ICT strategy. In the context of extensive decentralization, effective project management requires a system of incentives being established by the implementing agencies to strengthen the CPMU’s coordination role. The high staff turnover at the DPIU was a major challenge for the CPMU to handle and undermined the Project’s effectiveness in building project management capacity at the district level. The training of newly hired DPIU staff could only be a limited solution and a more powered system of incentives is needed. Possible solutions include: (i) inclusion of a core team of consultants at the DPIU level directly hired by the CPMU; (ii) the stronger involvement of Provincial extension agencies, as they are able to interface more effectively with both the CPMU and the DPIUs; or (iii) determining the district allocation of FMA grant funds using a set of parameters that include the performance of the DPIUs. Performance in the collection and consolidation of M&E data could also be included as a parameter. While the sub district extension centers (BPPs) established under the Project improved the extension workers’ outreach, the BPPs could be further developed into centers for agribusiness development. With assistance from the central government, local extension agencies should consider using the BPPs as a space where to favor matching of demand and supply for agricultural services involving bot the private and the public sector. The demonstration plots organized within BPP’s complex could be used by farmers and the private sector to conduct adaptive trials of technologies demanded by markets. It would entail the use of BPPs to organize events through which successful collaborations between the private sector and farmers could be brought to the attention of other farmers. The use of BPPs could be quite important in developing a mindset of local extension agencies in working as “broker of services” in the interest of farmers and in promoting partnerships as an effective vehicle of agribusiness development. Gender-sensitive strategies are needed to ensure a higher participation of women and the poor, and should be encouraged in a follow-up project given the positive results on the ground. The Project’s success in promoting women’s participation as stakeholders, decision- makers and beneficiaries relied on a number of important provisions: (i) the identification of specific training targets for the poor and both mixed men and women and women-only groups, with many of the latter involved in food processing activities; (ii) the requirement that there should be at least one female elected officer for each farmer group or farmer organization involved in the Project; (iii) the requirement that one of the two village facilitators to be trained by the Project is a woman; (iii) the inclusion in the Project’s results framework of the indicator tracking the level of participation of women in project activities; and (iv) a M&E system that collects, analyzes, and reports gender disaggregated data related to project activities. 27 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The GOI recognizes that at Project entry there were diverse interpretations among the Project implementers due to the lack of clarity in the logical framework. In addition, the devolution of agricultural services to local governments had an impact on implementation due to different priorities among districts beyond the control of the central Implementing Agencies (i.e. AAEHRD, ICATAD, and CADI). Nevertheless, the Implementing Agencies indicated a change direction on the agricultural extension services from production oriented to market demand oriented can overall be appreciated in most of the BPPs in 40 of the districts included under FEATI. Field extension workers have facilitated FOs in establishing partnership with private sectors on marketing and technical assistance in developing their own enterprises. However, this was not reported regularly and comprehensively by the DPIUs to the CPMU and it has become evident after Project closure during the preparation of the Project Completion Report. Lessons learned from the Project as stated in the ICR Report would be utilized for the future operation by NCAE, particularly on strengthening existing FOs to be more viable enterprises. For this purpose, Ministry of Agriculture has signed a MoU with the Ministry of Cooperative and Small-Medium Enterprises in January 2011 on strengthening farmer federations towards becoming cooperatives. This was followed-on by NCAE developing an implementation manual on the “Empowerment of Farmer’s Economic.Institutions”. Overall, the GoI claims that the Project has contributed to the transformation of the agricultural extension and research system in the country towards becoming farmer driven and market oriented. The GOI has also continued monitoring the progress of the Project towards the achievement of its targets beyond EOP. The number of written (i.e. formal) partnerships – one of the outcome indicators found to be not achieved in this ICR – measured as of November 2013, amounts to 2,460 against an EOP target of 2,400. This is well above the number of 1,288 recorded at EOP, which in turn was based on data measured during the first quarter of 2013. Hence, in view of the above circumstances the GOI proposes that the achievement of the Project’s outcome to be rated as Moderately Satisfactory. (b) Co-financiers Not applicable (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) Not applicable 28 Annex 1. Project Costs and Financing (a). Project Costs by Component (in USD million) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD millions) Appraisal (USD millions) Strengthening Farmer-Driven 40.3 Extension 39.9 99 Institutional Strengthening and 33.7 Capacity Building 35.3 105 Enhancing Technology 9.6 Assessment and Dissemination 6.5 68 Provision of Knowledge and 3.0 Information Services 1.4 46 Project Management and 24.2 Implementation Support 28.3 117 Total Baseline Costs 110.8 111.4 101 Physical Contingencies 3.3 - - Price Contingencies 8.9 - - Total Project Costs 123.0 112.2 91 (b) Financing Appraisal Actual/Latest Type of Co- Estimate Estimate Percentage of Source of Funds financing (USD (USD Appraisal millions) millions) Borrower 30.20 24.6 82 International Bank for Reconstruction 32.80 and Development 25.8 79 International Development 60.00 Association (IDA) 61.0 102 111.4 91 TOTAL 123.0 29 Annex 2. Outputs by Component The project was designed with five components, and the following are the summary of outputs by components: Component 1: Strengthening Farmer-driven Extension Services. This component was intended to empower farmers to drive the agricultural extension agenda and to improve their capacity to: (a) adopt new technology to better respond to market demand; (b) develop agribusiness management skills; and (c) engage in public-private partnerships (PPP) at the village, district, and provincial levels. The outputs of this component included number of participating farmers in learning activities (FMA) are 623,169, and 344,874 of which or 55.3 percent implementing the technology introduced in FMA. The project also provided training to 3,780 male farmer group leaders from 128 male groups, and 3,810 female group leaders from 125 groups at village level on organizational skills and leadership. At the project closure the number of village level groups organized as farmer’s own enterprises were 872, and 98 of which were registered as legal entity (having notary act). These village level groups were direct beneficiaries of village FMA grants. The scale-up of FMA at district level was begun in 2012 and only piloted in 16 districts and implemented by 45 district level farmer organizations which majority were newly established. While scale-up at provincial level was only pilot in two provinces (Central Java and West Nusa Tenggara) begun in 2013 involving two existing agricultural enterprises. During ICR mission, these provincial FMAs were just started, and no outcome could be seen yet. The implementation of district and provincial FMAs were supposed to be started prior mid-term and started in 13 former DAFEP districts, and five provinces. This target was not able to be achieved due to the weak capacity of the project implementers in a new approach of scaled-up village FMA. The main crops included in the FMA covered (i) food crops – mainly rice, and followed by maize, cassava, and sweet potatoes; (ii) horticultures ( chili and vegetables); (iii) animal husbandry (cattle, sheep/goat, poultry); (iv) estate crops (cocoa, coffee, and rubber). In addition, food processing technology was dominated by women participants. FMA grants also promoted organic farming and zero waste technology in crops and animal husbandry. The establishment of farmer’s own enterprises at village was due to a requirement in marketing resulting in the development of partnership between farmers and buyers. The number partnership established at village level was 1,288 written / signed contract. Some of the contracts were renewed annually, and some were terminated due to some reasons. This data was generated from a monitoring data available in www.featiindonesia.com set up during the project life. The agreement stated in the contract was mainly related to the volume / quantity and quality to be produced, while prices were negotiated at the time of delivery. The allocation of village FMA grants is presented in Table 1. Table 1. Number of proposals approved and budget allocation for village FMA by years in IDR. By managing FMA grants, farmers / farmer organizations were able to determine their own learning needs and identify service providers to address their own learning needs. Hence, field 30 extension workers were no longer as the sole service provider available to farmers. However, based on a BIA survey in conducted in 2012, the existence of field extension workers as facilitators in farmer’s learning was still high (77.93 percent), followed by experienced farmers (66.46 percent). In each learning activity, farmers were facilitated by more than one facilitator. To allow extension workers be able to facilitate farmers in FMA implementation, the project trained 159 field extension workers (permanent/government employees) located in FEATI villages comprising 123 males and 36 females on FMA facilitation methods and process. While contracted/temporary extension workers located in FEATI villages were trained in house by trained permanent field extension workers. However, the project did not monitor the number of trained contracted extension workers. Component 2: Institutional Strengthening and Capacity Building : This component was intended to facilitate the development of a demand-driven sustainable extension system, with public and private extension service providers (including farmers) cooperating with farmer groups and private enterprises for the mutual benefit of all parties. The training provided by the project included : (i) farmer organization development participated by 30 provincial staff from 18 provinces; (ii) value chain workshop attended by 59 extension workers from 16 pilot districts implementing district FMA focusing on the main commodities including animal husbandry, food crops, and horticulture; (iii) agribusiness management for 25 provincial facilitators (TOM) from nine Agricultural Training Centers, who later trained 2040 field extension workers ; (iv) agribusiness practice in house learning at 707 BPPs participated by field extension workers ; (v) overseas comparative studies for FEATI Project Managers and extension managers to India (10 participants); Korea (16 participants), and China (17 participants). In addition, each FEATI district organized specific locally training on various technologies for field extension workers. However, the project had no data on the total number of the trained extension workers. The project improved the extension facilities including construction of 589 new BPPs, and renovated 200 existing BPPs. These BPPs were equipped with office furniture, computers, and soil testers. Each Bapeluh and Bakorluh were also provided a set of computer Component 3: Enhancing Technology Assessment and Dissemination. This component was intended to facilitate an uptake in the usage of farmer and market demand-driven technologies and to improve the capacities of the Assessment Institutes of Agricultural Technology (AIATs) to enable them to function more effectively. The outputs of this component included : a. Strengthening the capacity of AIATs was achieved by provision of trainings to : (i) 37 trained researchers (25 males, and 12 females) on Farming System Analysis (FSA); (ii) 32 trained AIATs’ staff (16 males, and 16 females) from 17 provinces on Value Chain Analysis; (iii) 37 trained AIATs’ staff ( 19 males, and 18 females) on capacity enhancement for farmer empowerment; (iv) 40 trained AIATs’ staff ( 20 males, 20 females) on scientific training; (v) 40 trained AIATs’ staff (36 males, 4 females) on audio visual aids; (vi) 44 trained librarian from 18 AIATs (26 males, 18 females); and (vii) 35 trained staff from 18 AIATs on e-information. b. Improving the linkages between researchers – extension workers – and farmers was achieved by the conduct of (i) 192 FSAs were organized in 18 AIATs to field test 54 technologies involving 218 farmer learning groups consisting 3,359 male and 1,222 female farmers, 298 male and 141 female extension workers, and 177 researchers; (ii) 133 ARF workshop involving 1,374 farmer learning groups consisting 2,606 male and 650 female farmers, 952 male and 33 female extension workers, 170 researchers and 118 extension agents of AIATs, and four research centers; (iii) 230 demonstrations/ field trials involving 378 farmer learning groups consisting 4,031 male and 1,222 female farmers, 323 male and 90 female extension workers; 335 researchers and 181 extension agents of AIATs, and 108 researchers from research centers; (iv) 125 technology workshop involving 1,860 farmer groups comprising 3,040 males and 737 female farmers; 2,070 male and 705 female extension workers, 351 researchers and 232 extension agents of AIATs, and 15 researchers from research centers; 31 c. Developing linkages between researchers, extension workers, farmers and private sectors was achieved by the conduct of : (i) 18 application technology meetings; (ii) 19 meetings on agricultural technology information; (iii) 30 business meetings, and (iv) 85 field days. d. Technology dissemination was improved by an increase in number and improve quality of media produced by AIATs. During the project life, there were 136,610 leaflets/folders, 16,003 posters, 28,215 brochures, 15,937 pocket books, 1,990 newsletters/magazines, 6,218 bulletins, 4,970 VCDs, and 89 other forms of media produced by AIATs and circulated to farmers and field extension workers. Component 4: Provision of Knowledge and Information Services: This component was intended to address the issue of the lack of access to knowledge related to agricultural technologies and the lack of linkages amongst service providers and between service providers and the user community (farmers, traders, and entrepreneurs). The outputs of this component included : (i) establishment of web base e Petani. This portal was utilized mainly by field extension workers to share experiences and information regarding technology as well as consultation. However, this portal was not effectively yet for individualized technical and market assistance. CADI as the implementing agency of this component was failure to bring service providers to collaborate in e- Petani utilization through SMS on mobile phones and GPRS devices; (ii) training on the use of e- Petani was provided to extension workers and farmers representatives in 18 pilot districts ( total number of trained participants was not available). Total users of e Petani as recorded up to 2012 including 1,900 extension workers; 1,350 farmers, and 5,123,558 other visitors which might visit more than once. Component 5: Extension Policy and Project Management Support: This component was intended to support the drafting of the implementing rules and regulations of the proposed Law on decentralized agricultural systems through multiple service providers, and provision of management and implementation support for the project was also included in this component. When the project was commenced, the proposed extension Law No. 16, 2006 was enacted. The government allocated special government budget on drafting of the implementing rules and regulations of the new Extension Law. Hence, no draft implementing rules and regulations supported by the project. To improve the capacity of the project implementers, the project trained 522 project staff including project managers, financial officers, procurement officers, treasurer, and monitoring and evaluation officers from 68 DPIUs, 18 PPMUs, and 18 PPIUs (AIATs). In addition, the project produced project manuals and posters to disseminate information about FEATI at various levels (central up to village). Revised project indicators: The following matrix presents the adjustment of intermediate results indicators which was restructured in 27 March 2012. The adjustment included also the revision of the EOP target. Table: Adjustment of project indicators as in the Restructuring Paper dated 27 March 2012 Unit of Intermediate Results Indicators Original target Revised Target Measure Indicator One: Number of services provided to FO members. New Number No target 5000 Indicator Two: Percentage of members of Continue participating FOs with improved household Percentage 70% expenditure rates Indicator Three: Number of technology Continue packages developed by AIATs that are used in Number 60% 25 FMAs 32 Unit of Intermediate Results Indicators Original target Revised Target Measure Indicator Four: Percentage of participating Percentage. farmers benefitting from e-petani and other Revised 60% 30 information and communication technologies (ICT) Indicator Five: number of joint Number activities/ventures with private sectors Revised No target 2400 estalished by FOs under the project. INTERMEDIATE RESULTS Intermediate Result (Component One): Strengthening Farmer-Driven Extension Intermediate Result indicator: Percentage of Dropped participating FOs implementing their FMAs Percentage 70 and building partnership with various providers Intermediate Result indicator : Satisfaction Dropped Percentage. level from beneficiaries regarding improved 80 extension services Intermediate Result Indicator : Participation Dropped Percentage. rate of FO members in planning and decision- 60 making meetings, of which a min. of 40% are women and poorest FO members Intermediate Result Indicator : Participation Percentage. rate of FO members in planning and decision- New 60 making meetings Intermediate Result Indicator :Participation rate Percentage. of FO members in planning and decision- New 30 making meetings that are women or poor Intermediate Result Indicator :Participating Percentage. FOs assessed to have sound financial and Revised 80 procurement management practices. Intermediate Result Indicator :Percentage of FMA activities where experts from external providers (universities, research organizations, New 30 NGOs, farmer experts /experience farmers, and private sector) participate as resource persons Intermediate Result (Component Two):Institutional Strengthening and Capacity Building Intermediate Result indicator One: Satisfaction Percentage. levels of beneficiaries regarding improved Revised 80 extension services. Intermediate Result (Component Three): Technology Assessment and Dissemination Intermediate Result Indicator: At least 10% of Percentage. the total revenue of AIATs is derived from non- Dropped 10 govt. sources by EOP. Intermediate Result Indicator: percentage of 60 participating FOs engaged in farmer-led Percentage. cooperative research/trials, with >percentage improving their productivity through adoption Dropped of research results. Intermediate Result Indicator: Satisfaction Revised levels of beneficiaries regarding improved 70 R&D services 33 Unit of Intermediate Results Indicators Original target Revised Target Measure Intermediate Result Indicator: Number of collaborative activities in technology development and dissemination between AIATs New Number 40 and other research intititutions/service providers (private sector, unversities, NGO, local government) Intermediate Result Indicator: Percentage of AIAT technology packages that are adopted in New Percentage 60 FMAs are fine-tuned in collaboration with the participating farmer groups Intermediate Result (Component Four): Provision of Knowledge and Information Services Intermediate Result Indicator: Percentage of participating FOs availing information and Percentage 70 30 services of e-petani Revised Intermediate Result Indicator: Percentage of knowledge sources regularly updating Dropped Percentage 70 information in KIS Intermediate Result Indicator: Percentage of Dropped M&E data collection mechanisms facilitated by KIS Intermediate Result (Component Five): Policy Support and Project Management Intermediate Result Indicator : Dropped Regularity/reliability and quality of reports Intermediate Result Indicator : Percentage of M&E evaluation findings used to improve Dropped project performance 34 Annex 3. Economic and Financial Analysis Economic and Financial Appraisal of Farmer Organizations receiving FMA learning grants within Component A 1. Introduction A field mission was conducted as part of the FEATI Project’s economic and financial assessment from October 19 to 22. It collected data from a range of agro-enterprises (referred to as enterprises in the text below) managed by farmer groups (FG) involved in the Project in four districts. Representatives from the FEATI CPUM and MoA participated to the mission together with representatives from the Bapeluh in each of the involved district, sub-district and village. The findings on the current status and future possible success of each enterprise are presented. The two main components of the assessment are firstly the nature of the enterprise and its participation such as production, value addition and marketing in one or more commodities, and secondly a financial analysis of the outcomes from FEATI support. While the enterprises visited are only a small number of all the enterprises supported under FEATI, findings are considered good indications of pros and cons as outcomes from FEATI support, and highlight future considerations useful to the design of a possible follow-up project . 2. Methodology for enterprise appraisal 2.1 Net Present Value calculation A NPV was calculated for four enterprises for which data is available. It is the actual IDR and US dollar (USD) amount by which the enterprise’s current wealth is expected to increase if the enterprise is carried out. The implicit counterfactual scenario is that no enterprise would have developed without the support of a FMA grant from the FEATI project. While some farmer groups (FG) engaged in bank loans, these were mostly via individuals although in either situation the field mission did not have access to all financial information for the enterprise being assessed or the loans or contributions by members, or time to verify all possible scenarios of bank lending and repayment by the FG or individuals, and there were no precise record keeping by the involved FG or FG individual members. The discount rate takes into account the timing of the future cash flows that are available from an investment. The longer it takes to receive a cash flow (often the reality in agribusiness), the lower the value investors place on that cash flow. The greater the risk associated with receiving a future cash flow, the lower the value investors place on that cash flow. Net cash flows of an enterprise are estimated as End-of-Year (EOY). On the other hand, capital outlays are considered to occur during the first and second Start-of-Year (SOY), depending on circumstances specific to each enterprise. The discount rate used is 10.0% per annum, as this reflects the GoI government bond rate average interest during the FEATI interval. The exchange rate used in the analysis is IDR 10,000 to US$1.00, as this more closely represents its commercial value during the life of the Project. 2.2 Gross margin analysis and farmer group gross margin income The field visits to enterprises reflect activities conducted by the FG as outcomes from the FEATI project: 35 (a) Capital outlays for the enterprise were considered to occur during the first and second successive years, or combining the most reliable two years. The FMA learning grants were the main source of capital outlays. The grants included some funding (no more than 30% as per FEATI guidelines) for the purchase of equipment and productive assets. After reviewing possible NPV outcomes, this was often not a reliable tool to assess the enterprise sustainability as the capital outlays were either not quantifiable with precision or where they were actually spent on was not entirely apparent. In some cases, FMA learning grants not entirely aligned with their business objectives, or learning grants were terminated too early and before the enterprise had reached the stage of being confirmed as sustainable. Many enterprises had yet to be legally established; where they were, income is still derived by individual member households as no accumulation of earnings by the legal entity is still taking place. Most of the enterprises are evolving from a FG where farmers come together for sharing learning activities or access inputs through other government programs. This is a consequence of the fact that most enterprises have been established towards the end of the Project. Therefore these were still in their initial development phase at the time the current field work was undertaken. (b) A more useful approach to assess as a ‘barometer’ of success has been a current gross margin analysis. When divided among known FG members this would yield their potential additional monthly or annual income as a result of FEATI activities. The information from all enterprises assessed was generic. Not all income, direct or overhead costs were known. Net cash flows of an enterprise are estimated from the net gross margin on sales from the enterprise being assessed. The net gross margins are taken for the most reliable income years considered as most reliable and representative of the activities performed by the enterprise. This is either for the interval 2010, 2011, and 2012. 2013 was sometimes excluded as the assessment was completed during October and three months of the 2013 calendar year remain. Where data exists and is considered to reflect relatively accurately the enterprise’s business, data from 2011, 2012 and 2013 was included. 3. Analysis of selected FEATI farmer groups and enterprises A summary presentation of the analysis is presented in this section. For more details, a project paper has been prepared and archived in the Project’s database. The field visit was over four days to four differing districts, and included enterprises involved in producing annual crops (rice, chili, shallot onions, seed and ware potato, mushrooms), livestock (cattle, chickens, goats), and in food processing (cassava and mocaf based products), summarized in Table 1 below. Table 1: Enterprise focus, enterprise name, membership, and locations visited October 19 to 22, 2013 Enterprise Farmer group or legal entity name # FG members 2013 District Village orientation West Java. 19 October 2013 Cattle Koperasi Gabungan Kelompok 29 Cirebon Kubang Peternakan Sapi (KOPENAK) “Padusan”. Mushroom Marga Mulya 9 Cirebon Sindanghayu Shallot onions Koperasi Berkah Jaya 36 Cirebon Pabedilan Wetan Central Java. 20 October 2013 Brown rice  FG: Gapoktan Hargosari, Kecamaten  2008: 25 Batang Jambangan Bawang, Kabupaten Batang.  2013: 4 group in 4  Legal entity: documentation sub – villages or 36 Enterprise Farmer group or legal entity name # FG members 2013 District Village orientation submitted to Ministry of ‘dusan’ farmer 85 Cooperatives 27 Sep 2013; Finalization continues. Hybrid chickens – Koperasi Produksi Taruna Tani Berkah 43 Batang Wonosari eggs, DOC, Hibrida broilers Central Java. 21 October 2013 Goat P. E.  “Ngudi Raharjo Koperasi”. 25 Temanggung Kranggan  “Peternakan Kambing Ettawa”. Chilli Asosiasi Petani Cabe “Sido Makmur 105 Temanggung Mangunsari Jaya”. Potato – seed & P. T. Kledung Agro Lestari  P. T. shareholder: 39 Temanggung Kledung ware  F. G. member: 100  Total: 139 Yogyakarta. 22 October Mocaf & cassava “Putri 21” – or “Princess 21” 80 Gunung Kidul Playen based products Chilli Koperasi Citra Agri Cabe Lestari 40 Gunung Kidul Pelembutan (“Ciribel”), Alar Produksi Cabe Chickens – Pembesaran Ayam Buras Jenis Jawa 125 Gunung Kidul Sambirejo Broilers Super The estimated gross revenues generated by the enterprises supported under FEATI that were surveyed during the preparation of the ICR are reported in Table 2 below. Table 2: Gross revenues generated by the FMAs supported under FEATI Potential FMA Value of NPV gross income Name of the enterprise Specialization ROI Grants yearly (USD) / FG Received sales member/year KOPENAK “Padusan” Cattle breeding, $101,965 28% - $73,000 $2,521 fattening and trading Marga Mulya Oyster mushroom $7,900 50% - $13,500 $1,500 production and sales Kooperasi Berkah Jaya Shallot onions, $1,485,000 38% - $598,740 $16,630 production and sales Koperasi Produksi Taruna Fresh eggs, chicks, $ 65,000 - - $52,709 $1,200 Tani Berkah Hibrida and broilers: production and sales Asosiasi Petani Cabe “Sido Fresh chili, - - $9,852 $415,858 $12,300 Makmur Jaya” production and sales Gapoktan Hargosari, Brown rice, - - $15,400 $92,700 $1090 Kecamaten Bawang production, milling, sales P. T. Kledung Agro Lestari Seed and ware - - $27,200 $3,410 $100 potato production “Putri 21” Cassava and mocaf- - - $8,700 - - based products, production and sales Kooperasi Cita Agri Cabe Fresh chili, - - $11,235 $184,000 $4,600 Lestari (“Ciribel”) production and sales Pembesaran Ayam Buras Broilers, production - - $17,500 - - Jenis Jawa Super and sales 37 Overall, the following considerations can be drawn with regard to the sustainability of the enterprises supported with the FMA grants. Table 3: Economic, financial, business management assessment of selected farmer managed small scale on and off farm enterprises supported by FEATI 1 # Small-scale farmer managed on and off farm # Small-scale farmer managed on and off farm enterprises expected to sustain enterprises that may not be sustainable 1 Cattle, Cirebon District. Activities are consolidating. 1 Brown Rice, Batang District. May continue although not The group has an expansion vision. as a mainstream enterprise; scale is small. Other rice varieties and seed production can be added. 2 Mushroom, Cirebon District. Consistent product and 2 Goat, Temanggang District. A sustainable business well managed enterprises. model is to be confirmed, with land ownership or rental addressed. Sale outlets exist for fresh milk, ice cream, candles and soap. Pasteurizing is needed for milk products consumed by humans. 3 Shallots, Cirebon District. Established producers with 3 Potato, Temanggang District. Scale is too narrow and a vision to strengthen the supply chain. small, without a growth vision. 4 Chickens, Batang district. Well-integrated, diverse 4 Mocaf, Gunung Kidul District. While cassava noodles product range, have achieved scale, and have a future are a unique product, enterprise sustainability is vision. questionable until repositioned for growth with strengthened product range, marketing and a profit oriented business plan. The FG is an outlet for women led activities. 5 Chilli, Temanggang District. Needs strengthened enterprise management and growth model, with diversification, intensification and expansion participating during the year in and across target supply chains. 6 Chilli, Gunung Kidul District. The FG has volume and value, although gross margins vary. A closer assessment may show the entire HH enterprises are sustainable, more so with adding other crops. Options to strengthen the supply chain and add value. 7 Chickens, Gunung Kidul. Issues to resolve include expanding beyond the current small scale. 4. Conclusions The analysis conducted on the small number of enterprises reviewed for the ICR does not allow drawing general conclusions. Nevertheless, it enables drawing some preliminary insights to better assess the Project’s outcomes in the specific area of support to agro-enterprise development: (i) Support to the development of the agro-enterprises was provided too late to ensure sustainable outcomes while the legal status of these agro-enterprises was yet to be defined; (ii) Although the grants resulted in additional income to FGs or individual HHs, this is often not reflected back into the enterprise per se, as sales were mainly carried out individually by FG members, indicating that managers do not yet understand how best to use the legal entity; 1 The analysis should be understood as provisional as it based on the field survey conducted during the preparation of the ICR. More in-depth analysis would require sharing the results with involved beneficiaries to further confirm the findings, something that was not allowed by the timeframe within which the ICR had to be prepared. 38 (iii) The most successful enterprises are those that consistently receive FMA learning grants in the same thematic area (i.e. commodity or livelihood), while annual learning grants or changes in learning topics led to less sustainable outcomes; (iv) In one circumstance, FMA grants were provided to a farmer organization that had been established for several years. While the FMA grants helped further sharpening the business model of that farmer organization, it seems that FMA grants would be best allocated to support the development of newly formed organizations rather than well-established ones. Perhaps a higher counterpart equity could be requested from the latter; (v) Some of the learning grants were used as capital inputs, although in combination with learning activities; (vi) Few enterprises borrowed from banks, although some individual farmer group members did take out such loans, mainly for seasonal finance, and some for capital expenditure. However, enterprises showed increasing levels of confidence once they realized that they were able to service loans. Some members of farmer group contributed to enterprises, with these contributions effectively becoming ‘beneficiary matching grants.’ Support to enterprise development should have been conceptualized from the commencement of the Project, and not near its end. Most of the farmer groups were just beginning to show good progress at the time of the survey, making the transition from informal individual farmer groups into FMA learning groups first and then into agro-enterprises established as legal entities. A follow-up project should first conceptualize the grant type and purpose, encouraging matching equity from the beneficiaries using training, enterprise development or value chain grants. A proportional shift from Project grants towards farmer equity before project closure should be encouraged to avoid creating dependency on external grants. Corroborated by the results of field visits during supervision, these findings indicate that Project funds could have been used more efficiently by focusing on the development of agro-enterprises at an early stage of the Project rather than continuing with the less risky approach already piloted under DAFEP. However, the conclusion to be drawn is not that MOA should avoid investing in the development of agro-enterprises managed by FG. Rather, the field work suggests that this is an area where substantial progress can be made provided that the support from local extension agencies is integrated with that of other services in the local agricultural units at the district and provincial level. Benefits could also be generated through by the contracting of private business development service providers or by developing partnerships with private commercial sector involved in the processing, trading, or retailing of the FGs’ production. In that regard clear shortcoming in the design of FEATI’s support to agro-enterprise development is that for a majority of FG there was sole reliance on FMA grants to finance capital or commercial input expenditures. There was no concept of matching grants by beneficiaries (except in the form of labor inputs, which is a rather weak requirement, and then only in the measure of 10% of the value of the grant), or sliding scale to proportionally reduce FMA grants and increase matching grants by beneficiaries during the life of the Project, on what effectively were value chain or training or enterprise development grants. The selection of FGs for support could also be improved by introducing a competitive selection process based on the preparation of business, rather than learning, plans. In this regard, Indonesia can learn from the substantial experience accumulated in the development of productive partnerships and rural alliances in a number of other countries. 39 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Cecilia Belita Senior Program Assistant SASSD Mohamed N. Benali Consultant EASCS R. Cynthia Dharmajaya Program Assistant EASER MNSAR- Pierre Rondot Consultant HIS Shobha Shetty Sector Manager, Rural Developm SASDL Sumaryo Soemardjo Consultant EASIS Francisco Proenza ICT Sp FAO-IC Information & Communication Technologies Rajiv Sondhi Financial Management Sp EAPCO Financial management Yogana Prasta Operation Advisor EACIF Disbursement Syahfirie Manaf ICT Sp EACIF Information & Communication Technologies William Sorrenson Economics FAO-IC Economics and Financial Analysis Andrew Daniel Sembel Environment Sp EASRD Environment Safeguard Isono Sadoko Social Safeguards EASRD Safeguard Raj Soopramanien Lead Counsel LEGEA Legal Melinda Good Senior Counsel LEGEA Legal Frans Doorman Agricultural Extension Sp Consultant Agricultural Extension David Young Private Sector Development Consultant Steven Burgess Anti-corruption EASSD GAC Patrisia Mulita Anti-corruption EASSD GAC Paul McCarthy Governance Specialist EASSD GAC Supervision/ICR Dayu Nirma Amurwanti Operations Officer EACIF Novira Kusdarti Asra Sr Financial Management Specia EASFM Natasha Beschorner Senior ICT Policy Specialist TWICT Richard H. E. Chisholm Sr Agriculturist EASTS Dely P. Gapasin Consultant Consultant Bisma Husen Senior Procurement Specialist EAPCO Patrick Labaste Sector Leader EASER Mariam Rikhana Rural Development Specialist EASIS Sumaryo Soemardjo Consultant EASIS Fabrizio Bresciani Senior Agriculture Economist, EASIS Task Team Leader - Jan2012 up to 2013 40 Yoshiko Ishihara Sociologist FAO - CP Farmer Organization Jeroen Dijkman Livestock and Innovation Sp FAO-CP Rizal Rivai Procurement Sp EAPCO Procurement Sri Hanizar Procurement Sp EASR1 Procurement Enggar Prasetyaningsih Procurement Sp EASR1 Procurement Imad Saleh Procurement Sp EAPCO Procurement I Gusti Wijaya Kusuma Financial Management Sp EASFM Financial Management Krisnan Isomartana Safeguard Sp EASIS Safeguard Juan Martinez Sr. Social and Safeguards Sp EASIS Safeguard Burt - Swanson Extension Sp FAO-CP Agricultural extension Kee-Chai Chong Extension Sp FAO-CP Agricultural extension Ashis Mondal Monitoring and Evaluation Sp Consultant Monitoring and Evaluation Titie Hadiyati Operation Sp EASHD Project Costing A. Fandi Nasution Governance Specialist EACIF GAC Dewi Sutisna Program Assistant EACIF 41 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY04 FY05 16.49 98.6 FY06 35.94 212.2 FY07 14.17 87.3 FY08 0.2 1.2 Total: 66.8 399.3 Supervision/ICR FY 2007 4.74 32.2 FY 2008 11.71 66.8 FY 2009 10.25 87.4 FY 2010 10.7 137.9 FY 2011 28.57 129.4 FY 2012 24.11 165.4 FY 2013 47.13 138.0 Total 137.21 757.1 42 Annex 5. Beneficiary Survey Results Not applicable 43 Annex 6. Stakeholder Workshop Report and Results Not applicable 44 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR A. Summary of Borrower’s ICR The Borrower prepared the project completion report, and was reviewed and provided comments by the Task Team for an improvement of the project completion report and lessons learned for future operation of similar projects. The report was revised and submitted to the Task Team. However, the report did not clearly explain issues as outlined in the Bank’s ICR Guideline. Here is the summary : Assessment of the operation’s objective, design, implementation and operational experience: (a) Agricultural extension has a very strategic position in agricultural development to facilitate farmers to help and organize themselves in accessing market information, technology, capital in improving farm productivity and efficiency in managing their farm business; (b) The Project design was relevant with the new Extension Law no 16, 2006 in building a market agribusiness oriented service system; (c) FMA approach introduced by the Project provided opportunities to farmer groups to be farmer’s own economic institutions which was previously oriented on agricultural production (d) The Project provided a new insight to the project staff on developing farmer’s own enterprises, and scaled-up beyond village level; (e) The Project introduced an integrated approach in addressing issues limiting farmers to access information and development resources. Assessment of the outcome of the operation against the agreed objectives: (a) The implementation of FMA addressed farmer’s empowerment; agribusiness development, strengthening the approach of farmer to farmer extension, and the establishment of farmer’s own enterprises. District and provincial FMA should have led village level FOs to tap wider market beyond village level; (b) The improved skills and services of extension staff should have been followed with a transformation on their roles from production oriented service provider to knowledge broker on market and enterprise development; Evaluation of the borrower’s own performance – focusing on lessons learned for future operation. (a) Incentive system for board members of farmer’s management unit. The selected board members were expected to dedicate their time to manage FMA. As a consequence, they many times left their farm activities to hired farms labors which increased their production cost. Because of this issue, some of FMA management units were not working well or re-elected. This would have been effective if the project allowed to provide incentive to board members as a compensation of their time dedicated to the project; 45 (b) Intensive supports to trained farmer’s facilitators. Despite the training provision from the project, intensive guidance to newly appointed farmer’s facilitators need to be well planned by Bapeluh as a future operation of FEATI; (c) High turn over of the project staff at central, provincial and district had a major impact on the performance of the project implementation. However, the central project management unit had no authority to keep the project staff for the whole project life; (d) The performance of monitoring and evaluation teams at various levels were improved gradually over the project life. The project was able to develop an on line monitoring for FMA implementation. Some districts have utilized this data for decision making in the allocation of FMA grant for each village, while some were ignorant on the important of the monitoring data. (e) The project established a Complaint Handling Unit at DG of AAEHRD recording all complaints about the project activities and was responded by a team set up by the DG AAEHRD. Various channels such as meetings, newspaper, leaflet, posters and web site were utilized to introduce and disseminate about the project activities and achievement at all central, provincial, and district levels. Since 2010 there were an increased in the number of districts receiving complaints as follows in 2010 – 5 districts, 2011 – 15 districts; 2012 – 55 districts. This indicated that the project promoted awareness on governance and transparency to the public surrounding the project area ; (f) Safeguard issues were addressed by the project since the start of the project activities through the promotion of organic farming in the farmer’s learning activities supported by FMA grants. Integrated Pest Management (IPM) had been promoted in the majority of agricultural areas in Indonesia. However, in some horticulture producing areas the application of pesticides was still high, and land conservation was not well managed. Since, the size of the farming activities under the project was small, no significant impact on environment was found. The project with an assistance from an Environmental Specialist produced : (i) Training manuals on environment management; (ii) Environmental Management Manual; (iii) video on technical management for farm impacts; and (iv) leaflets on environmental impact management. Evaluation of the performance of the Bank. (a) The Borrower indicated that clearance from the Bank on the required documents for project effectiveness was provided in timely manner. (b) The project implementers received guidance from the Task Team through regular supervisions. Recommendations provided by the missions were monitored by the Task Team. In addition, Task Team members participate in the project quarterly meeting and provided guidance on technical implementation of the project activities, procurement, financial, and monitoring and evaluation aspects. (c) The Task Team identified weakness of the project implementation, and provided technical supports to the project management : (i) in February 2012 the project was provided international Monitoring and Evaluation Consultant to improve the project data collection ensuring the project indicators were properly measured, and (ii) mobilization of international Agribusiness Consultant to review and improve the implementation of scale-up FMA from village level to district level. In addition, the Task Team assisted CPMU to link with an external service provider – REKADESA, to assess and facilitate early stage of farmer’s owned enterprises established under the project. 46 (d) There were weakness identified by the Borrower : (i) a gap in understanding the project documents by the project implementers and the Task Team leading to a different the evaluation of the project performance; (ii) a communication gap between the project managers and the Task Team Leader resulting in difficulties on the discussions of the project related issues, in particular during Bank’s missions; and (iii) the PAD was not meticulously assessed ensuring its consistency between sections. Hence, revisions of the PAD could have been avoided. Description of the proposed arrangement for future operation of the project : a) The management of e Petani implementation had been handed over to NCAE which was integrated as part of Cyber Extension developed by NCAE. A content provider team was established in NCAE responsible to update the content of e Petani; b) The follow up on strengthening of established farmer’s own enterprises was handed over to an NCAE division of Farmer’s Institution and Enterprise Development. B. Comments on Draft ICR ICR Report Comment Achievement of PDO indicator # 3 : reduction of the target value is due to limited human Reduction of the target value is due to a weak resources allocated to conduct the activities capacity of CADI 2.1.5. Quality at Entry At the Beginning of the project it was not clearly Too broad definition of agricultural services system understood by the project implementer s on the scope in PDO of PDO No clear logical framework Formulation of the result framework and the implementation of monitoring was not consistent MES and MIS were not established from the commencement of the Project 3. Assessment of outcomes 3.2 Achievement of PDO, (iii) : Component D failed to achieve its EOP (iii) Component D failed to achieve its EOP target target due to the failure of CADI to develop a due to the failure of CADI to develop an appropriate mobile platform for the e-Petani application. mobile platform for e-Petani application, (iv)This figure of 1,288 was per September 23, (iv) Number of joint activities/ventures with private 2013. Up to November 2013 achieved 2,460 sector established by FO. It was found that 1,288 ventures with written partnership as supported by the partnerships involved written agreement. The EOP attached empirical data that automatically corrected target was the establishment of 2400 ventures and the bank’s re-computation from joint activities formalized through a contract www.featiindonesia.com and it is proven that the indicator was achieved. Despite failure to achieve the PDO, the project Considering the FEATI project has achieved 3 out of experiences were recognized by the MoA as a 5 key indicators, and given impact to social model for the reform of the system from production development (see para 3.5.) and also recognized by orientation to a market orientation MoA as a model for the reform of the system from production orientation to a market orientation, the achievement of the PDO should be considered as Moderately satisfactory 47 ICR Report Comment 3.4. Justification to overall outcome rating : The project provided training on Agribusiness Management to extension staff. The trained extension worker developed manual to facilitate farmers on marketing and enterprise management. The project promoted brokerage role to extension workers. Some BPPs in 40 districts facilitated FOs to link with private sectors. This led to the achievement of PDO. The followings are BPPs facilitating farmers establishing partnership with private sectors : (i) District of Serang: BPP Cikeusal and BPP Kramat Watu have actively supported the development of partnership between local FOs with private enterprises in vegetables (Cikeusal) and paddy rice seeds (Kramat Watu); (ii) District of Ende: BPP Nangapanda of Ende District have been successfully linked local FOs with PT Mars in marketing of cacao; (iii) District of Tuban: BPP Tambakboyo facilitated channeling local FOs to assess capital from the Bank Syariah Mandiri and PT Holcim; and BPP Plumpang facilitated FOs to access capital from the Bank BRI and PT Petrokimia; (iv) District of Magelang: BPP Pakis supported FOs to develop partnership with PT Bumi Sari Lestari in exporting vegetables to Singapore; BPP Ngablak linked local FOs with Milk Cooperative in the district of Boyolali and with PT Alamanda Bandung in marketing vegetables; (v) District of Kolaka: BPP Latambaga and BPP Lambadia facilitated FOs to develop partnership with PT ADM cocoa in marketing of cocoa; BPP Tinondo facilitated FOs to develop partnership with PT Bintang Perkasa in marketing of nilam oil The project was expected to contribute in policy to increase export, value added and competitiveness as well as farmer welfare through development demand and market driven farm businesses. 3.5. Overreaching Themes, Other Outcome and Development of farmers enterprises that initiated by Impacts FEATI already strengthened by MOA policy to encourage formation and development of farmer’s own enterprises. This reflected in the issuance of Pedoman Pengembangan Kelembagaan Ekonomi Petani (Manual on Developing Farmer’s own Enterprises) in 2012 which is followed up by the issuance of Law No.19/2013 on Farmer Protection and Empowerment. In addition, the MOA and MOCSME signed MoU in January 2011 on strengthening farmers federation toward farmers 48 ICR Report Comment cooperatives To support the schemes the NCAE has allocated funds in 2014 to a value of Rp 6,7 billion to strengthen and improve the sustainability of 771 FOs. Local Government has also provided budget to strengthen and develop FOs. 3.5.(c). Other unintended outcomes The enactment of Law No 19/2013 on Farmer’s Protection and Empowerment is a significant outcome of the FEATI project. 6. Lessons learned. Relevant with the experience project implementation OVERALL COMMENT Based on the above arguments, it is proper and reasonable if we propose that the overall rating of FEATI be Moderately Satisfactory. 49 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not applicable 50 Annex 9. List of Supporting Documents 1. FEATI Project Aide-memoire -- ICR Review, September 03-23, 2013. 2. FEATI Project Stakeholders Workshop, June 19, 2013, Summaries in Power Point Presentations. 3. Food and Agriculture Organization of the United Nations, Assessment of FEATI Monitoring and Evaluation System, Implementation Support Mission, February 2012. 4. Government of Indonesia, National Medium-Term Development Plan for 2010-2014. 5. Ministry of Agriculture, Project Completion Report, Farmer Empowerment through Agricultural Technology and Information Project, October 2013. 6. Ministry of Agriculture, Regulation of the Minister of Agriculture No. 273/2007 on “Guidelines for Empowering Farmer Institutions, April 13, 2007. 7. Quality Assurance Group (World Bank), Eighth Quality at Entry Assessment Panel Report of the Farmer Empowerment through Agricultural Technology and Information Project, May 31, 2007. 8. Republic of Indonesia, Law No. 16 on “Agricultural, Fishery and Forestry Extension System”, November 15, 2006. 9. Republic of Indonesia, Law No. 19 on “Farmer Protection and Empowerment”, 2013. 10. World Bank, Restructuring Paper, Farmer Empowerment through Agricultural Technology and Information Project, March 27, 2012. 11. World Bank, Country Partnership Strategy for Indonesia FY2009-2012: Investing in Indonesia's Institutions for Inclusive and Sustainable Development. 12. World Bank, Project Appraisal Document, Farmer Empowerment through Agricultural Technology and Information Project, January 4, 2007. 13. World Bank, Development Credit Agreement: Farmer Empowerment through Agricultural Technology and Information Project, March 28, 2007. 14. World Bank, Loan Agreement: Farmer Empowerment through Agricultural Technology and Information Project, March 28, 2007. 15. World Bank, Country Assistance Strategy for Indonesia, FY 2004-2007. 51 IBRD 40534 INDONESIA 0 100 200 Kilometers FARMER EMPOWERMENT VIETNAM PHILIPPINES 0 100 200 Miles THROUGH AGRICULTURAL THAILAND Sulu TECHNOLOGY AND INFORMATION Sea PA C I F I C (FEATI) OCEAN XX FEATI PROVINCES PROJECT KABUPATENS PROVINCE CAPITALS Banda Aceh BRUNEI NATIONAL CAPITAL DARUSSALAM Celebes KABUPATEN BOUNDARIES Sea 1 MALAYSIA Talaud PROVINCE BOUNDARIES Natuna Is. Medan Besar INTERNATIONAL BOUNDARIES 3 4 3 2 This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information Simeulue 2 1 19 MALAYSIA 24 Morotai GSDPM shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any SINGAPORE 23 25 Manado 5 Map Design Unit endorsement or acceptance of such boundaries. 1 Tanjung Pinang 4 Nias 2 3 1 Ternate Halmahera 1 2 Pekanbaru 3 2 K A L I M A N TA N 4 Gorontalo Waigeo 1 4 Lingga Pontianak 20 3 Samarinda Palu 26 30 Manokwari Biak Padang Me 2 2 Peleng Siberut 5 Jambi 21 32 nt 4 1 5 Bangka SULAWESI aw Obi Yapen 1 Pangkal Pinang 27 Sula Is. Misool ai Palangkaraya 3 Jayapura SUMATERA 1 9 2 Mamuju 4 Is 1 Belitung . Palembang 7 22 PAPUA Banjarmasin 28 Ceram PAPUA NEW GUINEA Bengkulu 4 3 Kendari Buru Ambon 6 5 29 33 8 Java Sea 3 2 Muna Makassar 2 1 Kai Is. Enggano 11 1 Serang JAKARTA 12 31 Aru Tanjungkarang- 1 8 Semarang Is. Telukbetung 7 6 Banda Sea 4 Madura 5 3 4 3 4 1 2 10 2 1 JAVA Surabaya 16 Wetar Babar Tanimbar Bandung 13 1 2 Bali Sumbawa Is. Alor Moa Yogyakarta 2 3 1 3 15 Lombok 4 6 14 1 2 3 5 5 4 TIMOR-LESTE Denpasar Flores 3 Mataram 17 18 1 Sumba 2 Kupang Timor Arafura Sea INDIAN OCEAN PROVINCES 1 NANGGROE ACEH DARUSSALAM 12 JAWA BARAT 23 KALIMANTAN TIMUR 2 SUMATERA UTARA 13 JAWA TENGAH 24 SULAWESI UTARA 3 RIAU 14 D.I. YOGYAKARTA 25 GORONTALO INDONESIA 4 SUMATERA BARAT 15 JAWA TIMUR 26 SULAWESI TENGAH 5 JAMBI 16 BALI 27 SULAWESI BARAT 6 BENGKULU 17 NUSA TENGGARA BARAT 28 SULAWESI SELATAN 7 SUMATERA SELATAN 18 NUSA TENGGARA TIMUR 29 SULAWESI TENGGARA 8 LAMPUNG 19 RIAU KEPULAUAN 30 MALUKU UTARA 9 BANGKA-BELITUNG 20 KALIMANTAN BARAT 31 MALUKU 10 BANTEN 21 KALIMANTAN TENGAH 32 PAPUA BARAT AUSTRALIA 11 D.K.I. JAKARTA 22 KALIMANTAN SELATAN 33 PAPUA NOVEMBER 2013