Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004949 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-55210 IDA-H9760) ON A CREDIT IN THE AMOUNT OF SDR 5.4 MILLION (US$ 8.25 MILLION EQUIVALENT) AND A GRANT IN THE AMOUNT OF SDR 4.4 MILLION (US$ 6.75 MILLION EQUIVALENT) TO THE KYRGYZ REPUBLIC FOR THE PASTURE AND LIVESTOCK MANAGEMENT IMPROVEMENT PROJECT December 23, 2019 Agriculture And Food Global Practice Europe And Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective {Dec 11, 2019}) Currency Unit = Kyrgyz Som (KGS) KGS 69.99 = US$1 US$ 1.37 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Cyril E Muller Country Director: Lilia Burunciuc Regional Director: Steven N. Schonberger Practice Manager: Frauke Jungbluth Task Team Leader(s): David Tuchschneider, Artavazd Hakobyan ICR Main Contributor: Talimjan Urazov ABBREVIATIONS AND ACRONYMS AHHP Animal Health and Husbandry Plan AHSC Animal Health Sub-Committee AISP Agricultural Investment and Services Project AO Aiyl Okmotu (Local Government) APIU Agricultural Projects Implementation Unit ARIS CDIA – Community Development and Investment Agency CPF Country Partnership Framework CPMUP Community Pasture Management and Use Plan CPS Country Partnership Strategy DPLF Department of Pasture, Livestock, and Fisheries DTC District Technical Consultant EA Environmental Assessment ECA Europe and Central Asia ECAPDEV Europe and Central Asia Capacity Development EFA Economic and Financial Analysis E[S]MP Environmental [and Social] Management Plan [F][E]IRR [Financial] [Economic] Internal Rate of Return FM Financial Management FMD Foot and Mouth Disease [F][E]NPV [Financial] [Economic] Net Present Value FY Financial Year GDP Gross Domestic Product ICR[R] Implementation Completion [and Results] Report IDA International Development Association IFAD International Fund for Agricultural Development IFI International Financial Institution IP Implementation Progress IRI Intermediate Results Indicator KGS Kyrgyz Som KLPRI Kyrgyz Livestock and Pasture Research Institute LMD Livestock and Market Development LSGB Local Self Government Bodies M&E Monitoring and Evaluation MAFIM Ministry of Agriculture, Food Industry and Melioration MTR Mid Term Review PAD Project Appraisal Document PC Pasture Committee PDO Project Development Objective PLMIP Pasture and Livestock Management Improvement Project PUU Pasture User Union RDAD Rayon Department for Agrarian Development RF Results Framework SDR Special Drawing Rights TF Trust Fund US[D] United States [Dollar] WB World Bank TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 6 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ..................................................... 10 A. CONTEXT AT APPRAISAL ....................................................................................................... 10 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 13 II. OUTCOME .................................................................................................................... 14 A. RELEVANCE OF PDOs ............................................................................................................ 14 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 14 C. EFFICIENCY ........................................................................................................................... 18 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 19 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 19 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION ................................................................................... 21 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 22 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 23 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 24 C. BANK PERFORMANCE ........................................................................................................... 25 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 26 V. LESSONS AND RECOMMENDATIONS ............................................................................. 26 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28 APPENDIX A TO ANNEX 1 ..................................................................................................... 40 STANDARD STRUCTURE OF CPMUP ........................................................................................... 40 APPENDIX B TO ANNEX 1 ..................................................................................................... 41 PUU ASSESSMENT TOOL METHODOLOGY. ................................................................................. 41 APPENDIX C TO ANNEX 1 ..................................................................................................... 43 COMMUNICATION MATERIALS ................................................................................................. 43 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 46 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 48 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 49 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 52 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P145162 Pasture and Livestock Management Improvement Project Country Financing Instrument Kyrgyz Republic Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Community Development and Investment Agency, Kyrgyz Republic Ministry of Agriculture and Melioration (MOAM) Project Development Objective (PDO) Original PDO The Project Development Objective is to improve community based pasture and livestock management in the Project area Page 6 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 6,750,000 6,750,000 6,191,587 IDA-H9760 8,250,000 5,458,160 4,721,683 IDA-55210 Total 15,000,000 12,208,160 10,913,270 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 15,000,000 12,208,160 10,913,270 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 15-Jul-2014 27-Aug-2015 23-Aug-2017 31-Mar-2019 28-Jun-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 26-Feb-2019 9.06 Change in Components and Cost Change in Loan Closing Date(s) Cancellation of Financing Reallocation between Disbursement Categories Change in Implementation Schedule KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest Page 7 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 31-Oct-2014 Satisfactory Satisfactory 0 02 09-Apr-2015 Satisfactory Moderately Satisfactory 0 03 27-Jul-2015 Moderately Satisfactory Moderately Unsatisfactory 0 04 26-Jan-2016 Moderately Satisfactory Moderately Satisfactory .50 Moderately 05 24-Jun-2016 Moderately Unsatisfactory .70 Unsatisfactory 06 17-Oct-2016 Moderately Satisfactory Moderately Satisfactory 2.03 Moderately 07 21-Apr-2017 Moderately Unsatisfactory 2.54 Unsatisfactory 08 27-Jun-2017 Moderately Satisfactory Moderately Satisfactory 2.70 09 22-Dec-2017 Moderately Satisfactory Moderately Satisfactory 3.37 10 14-Jun-2018 Moderately Satisfactory Moderately Satisfactory 4.96 11 21-Jan-2019 Moderately Satisfactory Moderately Satisfactory 8.64 12 29-Jun-2019 Moderately Satisfactory Moderately Satisfactory 10.69 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 100 Fisheries 26 Public Administration - Agriculture, Fishing & Forestry 48 Livestock 26 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 8 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Finance 22 Finance for Development 22 Agriculture Finance 22 Urban and Rural Development 78 Rural Development 78 Rural Markets 22 Rural Infrastructure and service delivery 56 ADM STAFF Role At Approval At ICR Regional Vice President: Laura Tuck Cyril E Muller Country Director: Saroj Kumar Jha Lilia Burunciuc Director: Laszlo Lovei Steven N. Schonberger Practice Manager: Dina Umali-Deininger Frauke Jungbluth David Tuchschneider, Artavazd Task Team Leader(s): Peter Goodman Hakobyan ICR Contributing Author: Talimjan Urazov Page 9 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At the time of project appraisal in 2014, the Kyrgyz Republic being one of the poorest countries in the ECA region was recovering from a period of economic shocks, internal conflict and political instability. In the volatile political and economic environment, agriculture, although declining in terms of its contribution to GDP (one fifth) and employment (one-third), was a key priority for government because of its role in poverty reduction, rural employment generation, reduction of rural-urban income inequality and food security. Real GDP growth in agriculture was driven by irrigated agriculture (1.3 million hectares) and pasture-based livestock production (9 million hectares). The sector was severely affected by the 2010 crisis, several droughts and a generally weak business environment. In 2013 growth in the sector was 2.9 percent. 2. Livestock production, accounting for around half of agricultural GDP and dominated by small farms, was important for rural household food security, providing regular income and acting as a social safety net, particularly in times of hardship and economic distress. Prior to the approval of the Pasture Law (2009), fragmentation of responsibility for pasture management between different levels of local government had led to disruption of seasonal grazing routes, non-transparent and inequitable allocation of pasture rights (particularly affecting small livestock owners) and insufficient pasture related investments in pasture tracks, bridges and watering points. In general, winter pastures were over-grazed and needed resting, while some summer pastures were under-grazed, leading to species degeneration. Land tax collection rates on pastures were low. 3. The Pasture Law that was adopted in 2009 had helped to address these issues. Key features of the new pasture management arrangements included transfer of authority for management from Oblast1 and Rayon2 Administrations to Local Self-Government Bodies (LSGB) at Aiyl Okmotu3 (AO) level and subsequent delegation of management authority from LSGB to Pasture User Unions (PUU) and their executives, known as pasture committees. The World Bank Agricultural Investments and Services Project (AISP, 2008-2013) had supported the design and initial implementation of these reforms. The project had funded social mobilization as a basis for the establishment of a PUU for every AO that had pastures (454 in total), including putting in place governance arrangements and demarcating legal boundaries. 4. A follow-on project was needed to consolidate these achievements. PUUs had operated for only three seasons and needed further support to become inclusive, technically competent and financially sustainable organizations. In particular, support was needed to: (i) build inclusive governance arrangements; (ii) embed pasture management skills in advisory services and pasture committees; and (iii) demonstrate the benefits of and raise members' willingness to pay for the advisory services, veterinary services and infrastructure maintenance which are needed to reach potential pasture and livestock productivity. 1 Oblast is a common term used in post-Soviet countries, meaning Province. 2 Rayon is a part of an Oblast, a district. 3 Aiyl Okomotu is Kyrgyz term meaning Government of a rural settlement. Page 10 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) 5. The project was set to contribute to reducing poverty and shared prosperity by increasing access to pastures and animal health services of the poorest livestock holders (which included some of the most vulnerable groups of the population including landless rural poor households and female-headed households) by strengthening the voice of vulnerable groups in PUUs and by helping to ensure that pasture rights were allocated more equitably. 6. The project was also in line with the Country Partnership Strategy (CPS)4 2014-17, which was rooted in a single pillar - support for improved governance. The CPS aimed to reform the relationship between the state and the citizen: that is, to make the state more accountable to its citizens and, at the same time, to strengthen citizens’ voice in the activities of the state. The Pasture and Livestock Management Improvement Project, along with Bank investments in irrigation, which also followed a community-based approach, were two key investments through which improved stewardship of natural resources would be achieved under the CPS. Theory of Change (Results Chain) 7. The logic behind the design of the project - as described in the Project Appraisal Document (PAD) - is illustrated in Figure 1, presenting the links between the activities supported under the main components of the project and the related key outputs, outcomes, and long-term outcomes. Figure 1: Overview of the Project’s Theory of Change Activities Output Outcome Long-term outcome • Pasture management plan A • Community pasture Community-based preparation and implementation management plans pasture management is including training and legal support are prepared and improved: to pasture communities; approved; • Community pasture • Pasture boundary demarcation and • Number of new PUU management plans inventory; B micro-projects are prepared and • Sub-grants to improve pasture implemented in implemented infrastructure; accordance with the satisfactorily C • Improvement in • Provision of equipment and training microproject • Area of utilizable livelihood of to state pasture institutions; framework pastures increased; rural population; • Consultancy and investment for agreements; Community-based • Increase in forestry enterprise pasture • Target clients livestock management productivity in management; satisfied with is improved: the livestock • Consultancy support and training to agricultural services; • Animal health and sector. each PUU’s Animal Health and • Animal health and husbandry plans are Husbandry Group; A husbandry plans are prepared and • Training and equipment provision to prepared and implemented private veterinarians; approved; satisfactorily; • Establishment of rayon associations • Private veterinarians • Percentage of of private veterinarians and trained; pasture users consultancy support to the • Client days of satisfied with the Veterinary Chamber; training. quality of services • Sub-grants for veterinary and provided by PUUs is husbandry. high. 4 Report No. 78500-KG, endorsed by the Board on July 25, 2013 Page 11 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Critical Assumptions A. Pasture management and Animal health and husbandry plans are supported by local governments B. Sub-grants are of sufficient size to cover investment needs C. Better pasture and livestock management will lead to increased productivity and higher incomes in accordance with assumptions made for economic analysis Project Development Objectives (PDOs) 8. The PDO of PLMIP as set out in the Financing Agreement was to improve community-based pasture and livestock management in the Project Area. The wording of the PDO is identical in the Financing Agreement and the PAD. The Project Area encompassed Chui Oblast and Talas Oblast of the Kyrgyz Republic. Key Expected Outcomes and Outcome Indicators 9. Several indicators were set up to assess the achievement of the PDO. Improvement of the community-based pasture management was expected to be seen in terms of: (i) number of community pasture management plans prepared and implemented satisfactorily based on Project PUU performance criteria; (ii) area of utilizable pastures increased; and (iii) percentage of pasture users satisfied with the quality of services provided by PUUs. Improvement of the livestock management was expected to be seen in terms of: (i) number of animal health and husbandry plans prepared and implemented satisfactorily based on based on Project PUU performance criteria; and (ii) percentage of pasture users satisfied with the quality of services provided by private veterinarians. Components 10. According to the PAD the project had three components. Component 1 Community Based Pasture Management (planned US$11.4 million; actual US$8.7 million) consisted of three subcomponents: (i) Community based pasture management and investment, which included activities like strengthening community-based pasture governance and management by supporting Community Pasture Management and Use Plan (CPMUP) preparation and implementation including training, legal support, infrastructure investment subprojects as well as pasture boundary demarcation and inventory and other activities; (ii) Strengthening state pasture institutions; and (iii) Forestry enterprise pasture management and investment. 11. Component 2 Community Based Animal Health and Husbandry Services (planned US$1.9 million; actual US$0.8 million) consisted of two subcomponents: (i) Animal health and husbandry services planning and investment, and (ii) Private veterinarian development. The component planned to contribute to project objectives by (i) building the capacity of each PUU’s Animal Health and Husbandry Group; (ii) equipping and training private veterinarians and facilitating contacts between veterinarians and PUUs; and (iii) demonstrating to pasture users the benefits of veterinary services and improved animal health. 12. Component 3 Project Management (planned US$1.7 million; actual US$1.4 million) was aimed at supporting Agricultural Project Implementation Unit (APIU) and Community Development and Investment Agency (ARIS). The APIU was expected to carry out coordination with the technical agencies Page 12 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) involved in the project including the Pasture Department, Veterinary Chamber, KLPRI, and others. ARIS was responsible for implementation of the sub-grant program and mobilization of communities. 13. Total project cost according to the PAD was US$15 million5, to be financed with an IDA credit equivalent to US$8.3 million6 and an IDA grant equivalent to US$6.8 million7. Actual project cost totaled US$10.9 million. The difference is due to the partial cancellation of the IDA credit of SDR 2 million, explained below in the Significant Changes section, and ten percent appreciation8 of USD to SDR that happened over the course of project implementation. Lower disbursement in Component 2 is related to the less than anticipated amount of community grants allocated to the veterinary activities. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 14. No changes were made to the PDO or PDO indicators. Other Changes 15. In February 2019, the project was restructured, through a second level restructuring, which included a partial cancellation of the IDA credit in the amount of SDR 2,000,000 , and a three-month extension of the project’s closing date until June 28, 2019. Extension of the project was needed to capture the spring season of 2019, and to complete several activities, notably pasture, feeding and livestock improvement micro-projects. The extension also allowed to partially compensate for start-up delays, explained in more detail below in the Key Factors That Affected Implementation section. Rationale for Changes and Their Implication on the Original Theory of Change 16. The cancellation of SDR 2,000,000, which was made at the Government’s request, involved cancellation of Sub Component 1.3 - “Forestry Enterprise Pasture Management and Investment”, which was not implemented because its activities were assumed by the newly-approved Integrated Forest Ecosystem Management Project (P151102) and the decision not to implement the “Pasture boundary demarcation and inventory” activity under Component 1.1 due to delays caused by the unavailability of a suitable provider. It was compensated by a digital demarcation that was completed and did not affect the original theory of change. 5 SDR 9.8 million 6 SDR 5.4 million 7 SDR 4.4 million 8 The SDR/USD exchange rate was 1.38 at project closing, compared to 1.53 at project appraisal, which reduced the available amount of USD. Page 13 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 17. The PDO “to improve community-based pasture and livestock management in the Project Area” remained relevant to the overarching goal of the new CPF for the Kyrgyz Republic for the period FY19- FY229, albeit its shift in focus from governance improvement to diversified, export-oriented, inclusive, and sustainable growth. Still, addressing governance challenges through citizen participation and measuring satisfaction remained a CPF priority. Particularly, the project helped to support sustainable growth through CPF Focus Area 3 “Enhance economic opportunities and resilience”. 18. Pasture management improvement remains high on the public agenda of the Kyrgyz Republic: (i) A draft law to amend the Law on Pastures has been cleared by the Government and is under discussion by Parliament; (ii) a draft Resolution of the Government of the Kyrgyz Republic “On measures to implement the Kyrgyz Republic Pasture Law” has also been cleared and is expecting approval; and (iii) a “Program on Development of Pasture Farming and Livestock Breeding for 2019–2023” has been finalized with inputs from several ministries. 19. The project contributed to agricultural growth and helped to enhance resilience to climate change and disaster risks as illustrated below in the Other Outcomes section, which are stated as CPF objectives. Even though the PDO did not state the livelihood aspect, the “community-based approach” was explicitly stated by the PDO and improved livelihood was identified as a long-term objective in the project’s Theory of Change, thus PDO outcomes and long-term objectives continue to be aligned to the CPF. The somewhat limited ambition of the PDO is explained by the project’s relatively small size and is viewed as a moderate shortcoming in the relevance to the current CPF objective. Pasture management improvement in the Kyrgyz Republic remains vital for agriculture and environmental management. Pasture lands represent about 90 percent of all land allocated to agriculture and their adequate administration is vital for food security, income generation and the sustainable provision of ecosystem services to society. Hence, the relevance of the PDO is rated Substantial. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome Did community-based pasture management improve? 20. Community-based pasture management has improved significantly as a result of the project. The improvement is best demonstrated with the governance system that the project helped to establish for pasture management in the project area. The project supported formulation of Community Based Pasture Management and Use Plans (CPMUPs) which included analysis of pasture conditions, pasture improvement activities based on the analysis, machinery planning and maintenance, grazing plan and timetable as well as rigorous budget planning and related activities that were absent before. As a part of 9 Report No. 130399-KG, endorsed by the Board on November 13, 2018 Page 14 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) CPMUP implementation, the project financed pasture, feeding and livestock improvement micro-projects, ensured supervision of implementation of the planned activities as well as monitoring of results. Support to governance and implementation capacity has triggered behavioral changes like correction of grazing patterns and increased fee collection ratios, as explained below in more detail. 21. Better management system led to more balanced grazing pressure on all pastures in line with their carrying capacity as shown in CPMUPs, which eventually paid off in terms of better pasture condition as confirmed by the monitoring and qualitative assessment of pastures done by the PUUs and APIU. Quantitative assessment of the actual change is not yet available as only 2017 and 2018 were covered by project activities. Over time, a quantitative assessment is expected to confirm increased pasture productivity. More balanced grazing pressure became possible due to the improved access to remote pastures that allowed to increase usable pasture area by 32 thousand ha (3.1 percent of the total pasture area) in two oblasts and respective elaboration of CPMUP that prescribed grazing pressure and animal movement for PUU members. Project implementation reduced the number of PUUs where pastures were overgrazed from 140 (all PUUs in the project area) to 20 and eliminated cases of extreme overgrazing, where the number of animals grazing exceeded the carrying capacity of a pasture plot for twice or more periods. Good pasture conditions were confirmed by beneficiary surveys, project monitoring and were also observed by the Bank team during supervision missions; however, a longer monitoring period is required to make quantitative assessment of changes in pasture conditions. 22. The impact of the CPMUP was also assessed by international experts.10 It supported the organization of a complex pasture management system, animal health, and a veterinary system in 140 PUUs in two important regions of the country. The CPMUP became a central tool for the coordination of pasture management planning processes, management of pasture committees, fee collection, and budgeting. Fee collection of pasture use amounted to KGS 34.7 million in 2018 compared to KGS 23.7 million at the beginning of the project in 2015. This makes for an increase of almost 50 percent and is a strong indication that pasture users widely accept and support the renewed governance system. 23. The project M&E system was designed to measure improvement in community-based pasture management with the following PDO indicators: (i) number of CPMUPs implemented satisfactorily based on Project PUU performance criteria (explained below) which stood at 113 at project closure, slightly exceeding the final target of 112; (ii) percentage increase in utilizable pasture area, which is 3.1 percent of the total pasture area comparing to the final target of 8 percent. The partial achievement of the latter indicator is explained largely by a significantly reduced project actual implementation period due to start- up and effectiveness delays explained in more detail in the “Key Factors That Affected Implementation Section”. However, the likelihood of achievement of the final target in the next two years is considered to be strong as PUUs plan to continue to increase access to pasture areas, using for this purpose machinery and other tools acquired with the project. Fifty-nine percent of PUUs purchased machinery (excavators, bulldozers, loaders) that will be used for road maintenance, rehabilitation and pasture improvement. The strong likelihood of achievement of this target is also confirmed by 5-years CPMUPs, approved in all PUUs, which envisage funds for continued pasture investments. Beneficiaries’ survey indicated the intention to follow investment plans; the enhanced capacity of PUUs should ensure their satisfactory implementation. The third PDO indicator was the beneficiary satisfaction ratio from implementation of CPMUPs, which 10Michael Hale, Flavio Forabosko, Assessment of Community Pasture Management and Use Plan, which was prepared to inform policy makers in MAFIM about CPMUPs’ impact. Page 15 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) stood at 59 percent at project closure according to a random survey of the population in the project area. The number of satisfied beneficiaries was even higher (75 percent) if respondents were direct pasture users. Both figures exceed the project target of 50 percent. 24. By developing a solid and detailed PUU performance assessment tool, which included several criteria based on recommendations of international consultants and the Bank team, the project succeeded in establishing a full-fledged pasture management M&E system. Such system did not exist before and was not an explicit project objective. The tool included both a technical dimension consisting of evaluation of pasture conditions, grazing patterns, feed storage and watering points condition, as well as an institutional assessment. Detailed description of the assessment tool methodology is given in Appendix B to Annex 1. Clear descriptions for each grade in pasture characteristics and straightforward calculations for measuring conditions make the tool user-friendly and useful for monitoring effectiveness and comparison between PUUs. The assessment tool gives PUUs necessary information that helps to identify investment activities for inclusion in the next year’s CPMUP and lays the ground for continuous improvement. It also provides information for the Republic Information System, set up in the Pasture Department of MAFIM, which monitors overall pasture management performance, including the expansion of pasture areas. 25. Achievement of relevant intermediate indicators confirms successful implementation of the project activities. Project reports indicated achievement of the following results: • Number of community pasture management plans prepared by PUUs and approved by the project – 140 (target was 112); all PUUs in the project areas developed and adopted CPMUPs. • Number of new PUU sub-grants (micro-projects) implemented in accordance with sub-grant agreements with ARIS – 475 (target was 280); this target was exceeded by close to 70 percent thanks to community mobilization and proactive administrative actions provided by ARIS; • Target clients satisfied with agricultural services (percentage) – 77.1 (target was 60); survey showed high satisfaction of PUU members with PUU activities; • Direct project beneficiaries, including female beneficiaries – 197,268, of which 49.6% female (target was 190,000 of which 10% female beneficiaries); this number includes number of households’ residents, members of PUU. All households became members of PUUs with high participation of women. • Number of people mobilized to prioritize sub-grants – 110,306 of which 27,343 (or 25%) women; Number of direct beneficiaries of sub-grants – 183,100 of which 68,300 (or 37%) women; these indicators were not envisaged originally but proved to be useful to demonstrate the community involvement and inclusive project coverage in the targeted area. They were included informally during implementation to track beneficiary participation and ensure social inclusiveness particularly with regards to gender. 26. Given satisfactory implementation of the pasture management plans, levels of participation, transparency in management and observed improvements in pasture conditions (a direct result of the better governance at the local and sectoral levels), the extent of achievement of the project’s objective is assessed as Substantial. Page 16 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Did livestock management improve? 27. The robust governance system that was introduced in PUU management positively affected livestock management in the project area. Similar to CPMUPs, elaboration of Animal Health and Husbandry Plans (AHHPs) led to several outcomes that collectively showed improvement in the management of livestock, particularly: (i) more than 100 percent increase in the number of veterinary practices, which became more comprehensive and came to include not only standard vaccination against FMD and brucellosis but also against many other diseases, as well as antiparasitic and anthelminthic treatment; (ii) greater awareness of PUU members of good husbandry practices through training and demonstrations (see Annex 1 Key Outputs Section Outcome 2 for the full list of training delivered); and (iii) considerably improved discipline in the actual implementation of the planned activities. In addition, AHHPs served to coordinate (through a detailed calendar) some complex veterinarian activities such as animal identification, vaccination, and animal control. Particularly important were various project activities that built capacity of local private veterinarians (including young graduates) in the project area. They ensured successful implementation of the planned activities and improved sustainability of results in the longer term. 28. Better management processes resulted in improved epizootic environment as confirmed by the lack of noticeable cases of animal diseases during project implementation and improved productivity of animals as confirmed by the economic assessment shown below. The PUU performance assessment tool for AHHP implementation was set up a similar way as for the pasture management. Description of the methodology is given in Appendix B to Annex 1. 29. The following PDO indicators were included in the project M&E system to measure improvement in livestock management: (i) Number of AHHPs implemented satisfactorily based on PUU performance criteria, reaching 120 at project closure and exceeding the final target of 112; and (ii) Proportion of pasture users satisfied with the quality of services provided by Private Veterinarians, which is, according to the final evaluation survey 96 percent. This achievement shows the close relationship built between the population and veterinarians, who rendered the services not only to direct pasture users but to the whole population in the selected areas. These results were confirmed by the Bank supervision and ICR missions that observed that private veterinarians are generally adequately equipped and skilled to perform essential veterinary services. There is overall good knowledge by PUU members of the private veterinarians, including their location, office hours and mobile phone numbers. 30. Implementation of most of the activities is also confirmed by achievement of intermediate indicators. Project reports indicated achievement of the following results: • Number of animal health and husbandry plans prepared by PUUs and approved by the project – 140 (target was 112); all PUUs in the project area developed and adopted AHHPs. • Number of private veterinarians trained – 743 (target was 420); this training program got high demand and received priority. • Client days of training provided (number) of which for women – 7,885 (of which for women 854), target was 14,500 (of which for women 1,450); This indicator is the only one that was delivered at a lower level than initially planned. In addition to overestimation of training needs in the PAD, the amount of training was optimized due to a shortened project implementation period and cancellation of some activities related to forestry management and demarcation. As such, it is not Page 17 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) viewed as a significant shortcoming. Training programs that included topics most important for the project development objective like pasture management, livestock husbandry, and training for private veterinarians were given priority and were completed in full. 31. The systematic integration of animal husbandry into the PUU planning process remains the only area for improvement. AHHPs have limited number of activities beyond basic veterinary services but the gradual expansion of AHHP scope is expected once the focus is shifted from disease prevention to further increase in productivity. At the current stage disease prevention along with monitoring of livestock keeping has already led to an increase in productivity as shown in para 33; as such, the extent of achievement of this operation objective is assessed as Substantial. Justification of Overall Efficacy Rating 32. Project activities directly contributed to the improvement in community-based pasture and livestock management in multiple dimensions as outlined above, resulting in noticeable behavioral change of project stakeholders. PDO indicators, measured by the performance assessment tools and the satisfaction levels, capture the most important of them. Changes in practices were also confirmed by observation of the current condition of pastures and livestock as explained above and important unintended outcomes and impacts (see below) were measured. Taken together, these results of improved governance and significant investments through confirm that the project achieved the PDO and make it likely that the longer-term outcomes proposed in the Theory of Change will be achieved. Given substantial achievement of PDO results, the overall efficacy of the operation is rated Substantial. This rating recognizes overall success of the project activities but acknowledges a few shortcomings in the Results Framework and partial achievement of some outcome indicators. C. EFFICIENCY Assessment of Efficiency and Rating 33. To make it consistent with the PAD, the same methodological approach was followed in respect to the project benefits. Efficiency was evaluated based on the project benefits that resulted from lower stocking rates in the with-project compared to without-project scenario because more pastures are made accessible due to infrastructure investment and better pastures result from reallocations of animals. The reduced stocking rates in overgrazed pastures result in more available dry matter, higher growth rates in fat-stock and higher milk yields. Higher fertility rates, lower mortality due to a lower parasitic burden and better body condition and higher feed conversion efficiency due to better pasture quality should result in higher livestock productivity, as was observed in the project area. 34. The Economic and Financial Analysis (EFA) showed that the project investments were both economically viable and financially profitable for farmers. The shortened implementation period and the underachievement of the pasture increase target somewhat affected results of EFA but still yields were robust. The financial IRR turned out to be 36 percent compared to 49 percent estimated at appraisal and net incremental benefits with a financial net present value (FNPV) of US$10.5 million against US$29.2 million expected in PAD. The project generated an economic IRR of 46 percent (vs 52 percent expected in PAD) and net incremental benefits with an economic NPV of US$16 million (vs US$32 million expected in Page 18 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) PAD). It should be noted that the results are generally in line with the sensitivity analysis that was done at the project preparation stage. The project also estimated that FIRR for individual micro-projects implemented at PUU level were on average about 42 percent depending on the type of works that were financed, which confirms results obtained at the macro-level. 35. The project area showed a solid increase in livestock productivity, as confirmed by statistical data. Although the project influence can’t be separated from other factors it should be acknowledged that such an increase is higher than what would be expected in the “without the project” scenario. The volume of gross output of livestock production of farms in Chui and Talas regions increased from KGS 14.4 billion to KGS 15.1 billion or by 5 percent. Meat production for all types of farms increased from 68,600 tons to 73,000 tons or by 6.4 percent. The production of raw milk increased from 448,700 tons to 477,000 tons or by 6.3 percent. Further increase in productivity is projected in the EFA. A slight increase in the retail prices for milk and meat also contributed to the better EFA results. 36. Despite start-up delays and other factors explained below in the Key Factors section, project administrative and operational efficiency remained adequate, which allowed achievement of the project objectives thanks to proactive actions by the implementing agency and the bank team. The project was able to complete almost all activities (and significantly surpass the micro-projects target) with just a three months extension despite start-up difficulties, which effectively reduced the implementation period. As explained above, robust and focused CPMUPs including performance assessment tools were completed by the end of the project although additional grant resources were not used. Depreciation of Kyrgyz Som made a bit more funds available; however, the Government sought to cancel SDR 2 million, thus completing project activities with tight efficiency, using less than 75 percent of the original amount. In addition, despite some administrative and M&E staffing issues, solid training and capacity building led to institutionalization of M&E tools that confirmed achievement or exceeding most of the project targets. 37. Despite a shortened implementation period and underachievement of some output indicators, the EFA showed that project efficiency reached a level that would be normally expected in the other projects in the agricultural sector. Administrative inadequacies were overcome proactively to substantially achieve results. As such the project overall efficiency is rated as Substantial. D. JUSTIFICATION OF OVERALL OUTCOME RATING 38. Despite implementation difficulties reviewed in the subsequent sections the project substantially achieved its promised objective with an expectable efficiency level. The project activities also remain relevant to the current CPF aspirations and objectives. Still, considering moderate project implementation shortcomings and difficulties as well as shortcomings in the achievement of some project targets, as explained above, the overall outcome is rated as Moderately Satisfactory. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 39. Although the PAD does not focus on gender issues, it is clear that gender inclusion was a part of the project design. Female inclusion or satisfaction were tracked by two indicators (female percentage of beneficiaries and female participants in training). Number of female beneficiaries well exceeded Page 19 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) expectations, which proved active participation of women in the project activities. Anecdotal evidences also show that women played a notable leadership role in PUU governance as well as substantial capacity and knowledge that was gained from the project training and activities. A high level of women participation (more than 30 percent) was observed at Focus Groups meetings (for micro-project identification) as well as PUU activity assessment meetings. About 20 percent of all respondents of the beneficiary satisfaction survey were female or female-headed households. Their satisfaction with the project results were in line with male respondents, as such, the project didn’t have any noticeable negative impact on gender related matters. Institutional Strengthening 40. In order to facilitate community-based pasture management improvement, the project financed a number of activities to strengthen the institutional environment and build capacity of relevant organizations. This included training sessions, advisory services (including legal), and supply of equipment for the Department of Pasture, Livestock and Fisheries (DPLF), Rayon Departments for Agrarian Development (RDAD), local self-government bodies (LSGB) and Pasture Committees. More importantly, the project prepared guidelines and training material that will continue to be used after the project closure. An important outcome of these efforts is the ongoing legal and regulatory reform in the pasture management area aimed at improvement of tenure rights. This is expected to delegate more authority at the local level, streamline cooperation between government agencies, and provide better incentives for sustainable management. The reform gained traction in the Government of Kyrgyzstan and is currently at the final stages of discussions in Kyrgyz Parliament. 41. Completion of digitizing the registered external boundaries contributed to improved pasture management by enabling better planning and reducing conflicts between pasture committees. Under the project, the external pasture boundaries of all PCs were digitized and installed in Google Earth Pro. Trainings were also conducted on using GoogleEarthPro, GPS and electronic mapping, and on the compilation of cartographic materials for CPMUP (marking the infrastructure, area calculation, distribution of pasture users by plots, conflict resolution, etc.) 42. As mentioned above, an important indicator of better management that was not directly watched by the project is the increase in pasture usage fees that was observed over the course of the project implementation. This was enabled by participation in planning and the users’ perceived improvements in pasture management and veterinary services. It clearly demonstrates the strengthening of PCs that are now able to increase financing of the pasture improvement activities and better plan their own activities. Poverty Reduction and Shared Prosperity 43. While there was no indicator measuring the impacts of the project on the poor, all project activities were pro-poor and contributed to inclusion and more equal distribution of pasture resources. Increased livestock productivity discussed above is expected to contribute to raising the income of community pasture users, which are on average amongst the poor segment of the country’s population. Regular focus group meetings and other awareness raising activities boosted inclusive economic growth by ensuring everybody’s participation in resource allocation, eliminating cases of elite capture and reducing potential conflicts that were observed before. Page 20 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Other Unintended Outcomes and Impacts 44. The project raised awareness of the livestock sector stakeholders to climate risks and change, which is the first step in building resilience. In order to timely alert about inclement weather conditions, the project arranged for information exchanges from the Hydrometeorological Center to pasture users via cellular communication. This includes weather forecasts for the next 10-15 days and storm warnings transmitted 5 days in advance, allowing pasture users to be prepared for sudden changes in weather conditions. This was complemented by training sessions on climate change preparedness and information sharing on adaptation technologies. The trainings were conducted with the participation of DTC, PC, the PUU members, RDAD, and others. Preparedness for climate risks became crucial over the past 2-3 years, which recorded a higher incidence of natural disasters, increase in day and night temperatures, variability of precipitation, instability of seasonal weather (the risk of frosts, heavy rains, droughts), and other effects that call for urgent adaptation of livestock and pasture farms. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 45. The project was designed as a follow-on of the WB Agricultural Investments and Services Project (AISP, P096993) that initiated the reforms in pasture management. It was evident that the technical capacity to manage pastures at the local level was absent and PLMIP’s priority was to advance implementation and ensure sustainability of pasture reforms. The PAD identified a broad set of lessons learned that were used to design project activities. The project was prepared in close collaboration with several Government agencies and local communities which helped to identify needs in pasture management and infrastructure. It was aligned with the CPF priority for improved governance and enjoyed strong Government support. A thorough economic and financial analysis estimated a financial rate of return of 49 percent and net incremental benefit with FNPV of US$29.2 million that included expected gains in livestock productivity. 46. The World Bank was not the only IFI that worked in the pasture management area. Given its high importance for livestock production IFAD was also planning a similar Livestock and Market Development (LMD) project in other regions of Kyrgyzstan and close collaboration between two agencies was needed to maximize the development effect. However, difference in the processing schedule of IFAD and IDA led to PLMIP starting two years later than LMD project, which significantly complicated coordination. In anticipation of the earlier start of the LMD project, the preparation of PLMIP omitted important background activities like development of the PUU assessment tool and detailed guidelines for CPMUP preparation. It was assumed that PLMIP would benefit from the advanced stage of the LMD project, which unfortunately didn’t happen as LMD implementation was also delayed and didn’t produce some instruments that the Bank team counted on. 47. Project preparation was also planned to be supported by the Europe and Central Asia Capacity Development Trust Fund (ECAPDEV TF) grant that was expected to cover early local and international consultancy for PUU assessment and the baseline study for the project. The need for this consultancy became apparent once the Bank team realized that some outputs of the IFAD-funded LMD project could not be used. Regrettably, lapsing of the grant agreement because of absence of the parliamentary Page 21 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) ratification deprived the project team of these funds and had a significant effect on the pace of project implementation as explained below. 48. Considering these factors, the overall PDO of this operation was justified; however, in retrospective, it seems that the project design overly relied on factors beyond the project control and associated risks were not properly considered. It should be acknowledged that the project objective required a lot of supportive activities related to the capacity building and training, awareness raising, physical investment and technical advice to many stakeholders. Incorporating all these activities into the project was challenging, but the project components look well-structured and implementation arrangement well thought through. Still, the large number of planned activities complicated the results framework and not all activities were reflected equally in the project monitoring system. Delays in the associated activities negatively influenced project readiness for implementation. B. KEY FACTORS DURING IMPLEMENTATION 49. Delayed project effectiveness due to long parliamentary ratification and associated clearance procedures in the government was a systemic issue in Kyrgyzstan portfolio and affected this project as well. The project became effective at the end of August 2015, more than one year after project approval. 50. Start-up delays were exacerbated by lapsing of the ECAPDEV grant that led the project to spend the first year of implementation in preparatory activities like development of the PUU assessment tool and CPMUPs. Consequent delays in consultancies impacted negatively the preparation of CPMUPs, whose approval by the Bank was a condition for community micro-projects, leading to delays in disbursement. As a result, the project PDO and IP progress rating had been downgraded to Moderately Unsatisfactory level several times during implementation. 51. Engagement of some government agencies notably Giprozem (State Institute on Land Resources) was not explicitly discussed and agreed beforehand, causing confusion regarding the mode of their engagement. Giprozem sought to be a project contractor, which was supported by MAFIM request to contract them on a sole source basis to provide surveyor services. It turned out to be not compliant with the Bank procurement rules as Giprozem is a state-owned entity, dependent on state funding. This led later to complex discussions over technical parameters over which Giprozem had regulatory authority. Inability to award the contract due to the absence of qualified bidders and lack of time eventually led to the need to cancel the internal inventory of pasture areas, which was an important complementary activity. Luckily this did not have any effect over project core activities. 52. Sudden resignation in APIU director and M&E specialist in the middle of project implementation somewhat disrupted continuity and weakened overall management and monitoring and evaluation. A permanent director was not eventually appointed, although, overall capacity of the implementing agencies including fiduciary remained at adequate levels. Institutional capacity of the implementing agency (ARIS) responsible for micro-projects remained strong during the project lifetime and made it possible to start financing of micro-projects quickly once CPMUPs were approved. 53. A rigorous Mid Term Review (MTR) of the project was done in August-September 2017 that summarized activities accomplished to the moment, identified potential issues and discussed actions to be taken by the project teams. Detailed review of the project progress at that time helped to expedite Page 22 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) project activities; however, it was clear that the effective period of project implementation had been significantly reduced to just 2.5 years and at least one-year extension would be extremely helpful to solidify expected achievements. No restructuring or extension decision was made at that time as the preparatory phase was finally over and there was a need to see actual implementation of CPMUPs and community micro-projects. 54. Community micro-projects implemented by PUUs in accordance with sub-grant agreements with ARIS greatly supported the new pasture governance system and proved to be crucial for PDO achievement. Sub-grant disbursement was initially delayed due to conditionality related to CPMUP development. Lifting that constraint for the second round allowed a quick response from PUUs and ARIS. Eventually, 475 micro-projects were financed, well exceeding the initial target of 280 projects. Micro- projects financed PUU machinery and equipment for pasture infrastructure improvement and fodder preparation as well as community veterinary facilities. Full list of microproject types that were financed are given in the Key Outputs section. Effects of micro-projects were not intended to be measured quantitatively; however, in addition to the economic effect mentioned above, micro-projects had an important demonstration effect. They showed in practice that actual investment in pasture infrastructure supports good governance system by improving public support and fee collection, which in turn allow more investment in pasture infrastructure, generating a virtuous cycle in pasture management. 55. Project implementation pace picked up after the MTR and pasture management plans were successfully implemented for two seasons (2017-18 and 2018-19). An additional year of project implementation could have helped to document project achievements better, see trends in pasture condition changes, advance the achievement of the pasture area indicator by implementing an additional season of community grants for pasture management, and consolidate sustainability of results achieved. However, despite the government informal request, the Bank allowed only a three months extension, which was needed to complete ongoing community micro-projects but not enough to complete all remaining activities; and, as mentioned above, some of them were cancelled at restructuring in February 2019. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 56. The project RF and M&E design were relatively straightforward. This reflected the project’s component structure, yet it did not aim to capture all outputs of the various activities that were implemented within the components, particularly Component I. That seemed to be not feasible anyway, as, in many cases, activities were aimed at strengthening enabling environment and were complimentary to the overall objective. Although the operation’s theory of change is clear, it proved to be challenging to identify sound measurable indicators to assess actual improvement. As such, it was decided to link indicators to the PUU assessment tool, explained above in the Efficacy section. This made project M&E inseparable from the performance of the pasture units, which are project main beneficiaries. This arrangement has a strong advantage of ensured sustainability but has some shortcomings in terms of additional coordination needed between APIU and PUUs and possible increase in maintenance cost. Page 23 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) M&E Implementation 57. The M&E implementation was a responsibility of APIU that faced significant difficulties as PUU assessment tool was not developed at the preparation stage and methodology of what data are to be collected and analyzed was not established. Other indicators including intermediate indicators were tracked to the extent possible but progress toward achieving PDO was difficult to monitor. This was complicated by lack of a Baseline Survey, which was planned as a part of the PUU assessment survey, which was also delayed. As such, the project M&E was rated Unsatisfactory at some points and started to really function only after the MTR. Staff turnover in APIU required some additional time as new M&E staff went through their learning curve. Still, by the project completion M&E achieved the desired level and started to track pasture unions in a comprehensive manner. This is an important project achievement as it allows to incorporate various aspects of pasture management and activities’ impact into a single evaluation system. M&E capacity was also built at the project and PUU level as a number of training and consultancies were provided. M&E Utilization 58. The M&E system established by the project allowed collection of a great deal of technical data related to the pasture and livestock management that will be utilized in designing future activities. Of particular importance are data on pasture monitoring that assess dry mass yields and overall pasture condition, which is used in the PUU assessment tool. All M&E data constitute an integral part of CPMUPs for the next year and critical in determining investment activities at each PUU local level. M&E findings were regularly communicated to the various stakeholders, which raised trust in the system. Justification of Overall Rating of Quality of M&E 59. Overall, the M&E system experienced some difficulties to adequately assess the achievement of the objectives and test the links in the results chain as additional evidence was required to make convincing conclusions. Besides, there were some start-up weaknesses as not all preparatory activities were completed at the design stage. These shortcomings are assessed to be significant and overall M&E quality is rated Modest. High utilization potential as explained above is acknowledged but more time is needed to see the evidence that the system is fully used for decision making. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 60. The project was rated Environmental Category B, because of the potential limited environmental issues associated with the small-scale investments in pasture infrastructure. Three safeguards were triggered under the project: Environmental Assessment (OP/BP 4.01), Natural Habitats (OP/BP 4.04), and Pest Management (OP 4.09), although no large-scale, significant, or irreversible negative impacts were likely. Environmental assessment (EA) and the environmental and social management plan (ESMP) were prepared for the project and discussed at the disclosure workshop. 61. Overall, the project was implemented in compliance with the Bank and national environmental assessment rules and procedures as well as existing environmental requirements. A comprehensive environmental safeguards review was done at MTR that helped to fix outstanding issues related to the environmental sections in pasture management plans and overall reporting. Environmental safeguard Page 24 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) procedures were particularly followed while developing and testing Caragana11 combat plans. Particular use of herbicides was cleared by the State Agency for Environmental Protection and was correspondent to the ESMF requirements about the need for the limited “positive list” of pesticides. Site specific EMP with the elements of safe pest management was developed. It should be noted that improvements in pasture management as well as many subprojects related to pasture rehabilitation are expected to produce a positive environmental effect because of the decreased land degradation and better grazing practices. 62. The procurement performance under the project was generally satisfactory, delays in contracts awarding were due mostly to factors beyond the control of the implementing agencies. The implementing agency had sufficient procurement capacity with two-three procurement specialists with experience in international procurement employed full time depending on the workload. The filing system was generally adequate, some issues were noted related to the storage of copies and timely filing of signatures, which were resolved. Ex post reviews of procurement were conducted regularly. 63. The financial management (FM) arrangements including accounting, budgeting and planning, reporting, internal controls, external audits, and funds flow have been at the satisfactory level. The FM capacity at both implementing agencies APIU and ARIS, including organization and staffing was satisfactory. The level and timeliness of counterpart funding was assessed to be adequate. The internal control systems at APIU and ARIS were acceptable to the Bank and assessed to be capable of providing timely information and reporting. The Financial Management Manuals in both agencies properly documented specific project related internal control procedures, including authorization of expenditures and approval of the payments; bank reconciliation, verification of expenditures eligibility by the Financial Managers; description of financial documents flow/circulation; indication of eligible cash transactions, budgeting procedures, formal reconciliation procedures of project records, safeguards for assets, etc. C. BANK PERFORMANCE Quality at Entry 64. As discussed above, the project was designed as a follow-on to AISP, all aspects were well worked out, and the project was designed to address key pasture management issues that were identified at that time. The project was in line with the previous CPS and Government strategy aimed at improved governance. Implementation arrangements were well thought through and all relevant project agents were engaged in accordance with their capacity and competencies. The PAD is clear and covers all aspects of the project design. The thorough economic analysis was made to quantify potential gains. If the Bank team were more ambitious in setting PDO to cover economic growth agenda, the project would have probably retained high relevancy in the following CPS; however, that would probably change the project scope and would effectively mean a different operation. 65. In retrospect, it is clear that the risk of delays of important preparatory activities was not properly taken into account. It turned out that the PUU assessment tool and pasture planning methodology required more effort than was originally thought. At the same time, support from similar activities of other IFIs and additional resources from TF funds that the team counted on did not 11 The Caragana is a persistent weed on the pastures in Kyrgyzstan that is considered a big problem in the country Page 25 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) materialize. This caused considerable delays and put additional pressure on the Bank team once the project became effective. Quality of Supervision 66. The World Bank team made a significant effort to ensure timely implementation of the project activities. Formal implementation support missions were conducted semiannually and both APIU and ARIS were appreciative of the available technical and fiduciary support from the Bank team as well as the World Bank’s Country Office. Aide Memoires were informative and supportive to decision making. Supervision of safeguards issues including environmental and social aspects and fiduciary reporting was regular and ratings were used effectively to highlight outstanding issues. A comprehensive MTR was carried out in August-September 2017 that mobilized all team members, including environment, social, and fiduciary members, facilitating implementation by mapping out key measures to recover some of the lost time. 67. Still, the initial project delays probably could have been reduced had the Bank more proactively identified the issues and worked more closely with APIU. Quicker turnaround of draft documents including initial pasture management plans and joint review of possible options for the assessment tool could have helped to mitigate the negative impact of start-up delays. More intensive support would have required additional staff-time and other resources on the Bank side. Justification of Overall Rating of Bank Performance 68. The Bank performance proved to be generally sufficient to ensure substantial achievement of the project objectives. Underestimation of the risks related to completion of important preparatory activities is considered a moderate shortcoming in quality at entry, as such Bank performance is rated as Moderately Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 69. The risk to development outcomes is assessed as moderate and mainly related to the factors beyond control of the project stakeholders. Positive effect of pasture management plans is widely noted and accepted by pasture users, so there is strong intention to continue implementation of pasture plans after project completion. Continuous local governments’ support to the established institutional arrangements as well as the growing financial stability of PUUs largely mitigates any risk related to sustainability of achieved outcome. Machinery procured during implementation of community micro- projects is important in helping PUUs generate additional cash, which strengthens their financial status. In the longer run the biggest risk is related to the negative effect of climate change that may cause pasture deterioration and loss of productivity. In this regard, activities initiated by the project like monitoring of pasture conditions, just in time alert on weather changes to farmers, and more studies on the climate change impact should help pasture communities to mitigate expected effects. V. LESSONS AND RECOMMENDATIONS 70. The project confirmed the importance of the implementing agencies’ capacity for the project success. Social mobilization activities were carried out by ARIS, an organization with wide experience and Page 26 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) with great commitment. In short, it was shown that an experienced implementing agency can overcome emerging difficulties and speed up implementation. 71. In case of multi-donor financing, a single project with joint supervision is likely to produce better results than isolated projects supervised separately by different organizations. Cooperation with IFAD LMD project fell short of expectations set during preparation and may warrant a separate analysis. 72. A more realistic implementation schedule should be in place for countries that have extensive legal procedures prior to the project effectiveness. Parliamentary ratification and associated activities were not unexpected and required an interim arrangement that could have helped to retain momentum gained during AISP implementation. Reliance on the ECAPDEV grant was not practical as it also required parliamentary ratification. 73. Projects that rely on technical assistance like this one need consultancy support built into the project design, so that its availability can be assured once the project starts. Development and approval of various methodologies and guidelines are labor intensive and time consuming, which makes it difficult to avoid delays. 74. Initial pasture monitoring results show that climate change is getting an increased importance in natural resources management and may pose a significant risk to development outcome. Incorporating this aspect into a project design was correct and more research is needed to educate communities and help them to prepare better for probable environmental changes. . Page 27 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To improve community based pasture management in the project area Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Community Number 0.00 112.00 113.00 Pasture Management Plans implemented satisfactorily 15-Jul-2014 15-Jul-2014 14-Jun-2019 based on Project PUU performance criteria. Comments (achievements against targets): 100.9% achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Area of utilizable pastures Percentage 0.00 8.00 3.00 Page 28 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) increased 15-Jul-2014 15-Jul-2014 14-Jun-2019 Comments (achievements against targets): 37.5% achieved during project lifetime. The likelihood of achievement of the final target in the next two years is considered to be strong as PUUs plan to continue to increase access to pasture areas, using for this purpose machinery and other tools acquired with the project. Fifty-nine percent of PUUs purchased machinery (excavators, bulldozers, loaders) that will be used for road maintenance, rehabilitation and pasture improvement. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of pasture users Percentage 0.00 50.00 58.60 satisfied with the management of pastures by 15-Jul-2014 15-Jul-2014 14-Jun-2019 PUUs Comments (achievements against targets): 117.2% achieved. Objective/Outcome: To improve community based livestock management in the project area Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Animal Health and Number 0.00 112.00 120.00 Page 29 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Husbandry Plans 15-Jul-2014 15-Jul-2014 14-Jun-2019 implemented satisfactorily based on based on Project PUU performance criteria. Comments (achievements against targets): 107% achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of pasture users Percentage 0.00 50.00 95.50 satisfied with the quality of services provided by Private 15-Jul-2014 15-Jul-2014 14-Jun-2019 Vets Comments (achievements against targets): 191% achieved. A.2 Intermediate Results Indicators Component: Community Based Pasture Management Page 30 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Community Number 0.00 112.00 140.00 Pasture Management Plans prepared by PUUs and 15-Jul-2014 15-Jul-2014 14-Jun-2019 approved by the Project. Comments (achievements against targets): 125% achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of new PUU Number 0.00 280.00 475.00 microprojects implemented in accordance with the 15-Jul-2014 15-Jul-2014 14-Jun-2019 microproject framework agreements with ARIS. Comments (achievements against targets): 169.6% achieved. Page 31 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Component: Community Based Animal Health and Husbandry Services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Animal Health Number 0.00 112.00 140.00 and Husbandry Plans prepared by PUUs and 15-Jul-2014 15-Jul-2014 14-Jun-2019 approved by the Project. Comments (achievements against targets): 125% achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of private vets Number 0.00 420.00 743.00 trained 15-Jul-2014 15-Jul-2014 14-Jun-2019 Comments (achievements against targets): 176.9% achieved. Component: Project Management Page 32 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Targeted clients satisfied Percentage 0.00 60.00 77.10 with agricultural services (percentage) 15-Jul-2014 15-Jul-2014 14-Jun-2019 Targeted clients satisfied Percentage 0.00 0.00 73.00 with agricultural services - male 13-Apr-2018 Targeted clients satisfied Percentage 0.00 0.00 64.00 with agricultural services - female 13-Apr-2018 Comments (achievements against targets): 128.5% achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Client days of training Number 0.00 14500.00 7885.00 provided (number) 15-Jul-2014 15-Jul-2014 14-Jun-2019 Client days of training Number 0.00 1450.00 854.00 provided - Female (number) 15-Jul-2014 15-Jul-2014 14-Jun-2019 Page 33 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Comments (achievements against targets): 54.4% achieved due to shortened period of implementation and optimization of training program. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 190000.00 197268.00 15-Jul-2014 15-Jul-2014 14-Jun-2019 Female beneficiaries Number 0.00 19000.00 98634.00 03-Nov-2017 Comments (achievements against targets): 103.8% achieved. Page 34 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 Improve Community Based Pasture Management in the Project Area 1. A number of CPMUPs implemented satisfactorily based on Project PUU performance criteria – 113 (actual at completion) Outcome Indicators 2. Percentage increase in utilizable pasture area – 3 percent (actual at completion) 3. Beneficiary satisfaction ratio from implementation of CPMUPs – 58.6 percent (actual at completion) 1. Number of community pasture management plans prepared by PUUs and approved by the project – 140 (actual at completion) 2. Number of new PUU sub-grants (micro-projects) implemented in accordance with sub-grant agreements with ARIS – 475 (actual at completion) Intermediate Results 3. Target clients satisfied with agricultural services (percentage) – 77.1 percent (actual at completion) Indicators 4. Client days of training provided (number) (of which female) – 7,885 (854) (actual at completion) 5. Direct project beneficiaries (of which female) – 197,268 (98,634) (actual at completion) 6. # of people mobilized to prioritize micro-projects (of which female) – 110,306 (27,343) (actual at completion) 7. Direct beneficiaries of micro-projects (of which female) – 183,100 (68,300) (actual at completion) 1. A single format of the community pasture management and use plan (CPMUP) has been developed for the PUUs. Structure of CPMUP is given in Appendix A to Annex 1; 2. PUU assessment tool based on implementation of CPMUP has been developed. The assessment tool is explained in Appendix B to Annex 1; Key Outputs by 3. Development of CPMUP in 140 PUUs were supported in 2017-2019 Component 4. Legal awareness and Regulatory Reform: (linked to the • The draft Law “On Amendments to Certain Legislative Acts of the Kyrgyz Republic (to the Land Code of the Kyrgyz achievement of the Republic, the Kyrgyz Republic Law “On Pastures”) has been developed and cleared with Ministry of Finance, the Objective/Outcome 1) Ministry of Economy and the State Tax Inspectorate of the Kyrgyz Republic, State Agency for Local Self-Government and Inter-Ethnic Relations (SALSGIER) of the Kyrgyz Republic, and the Ministry of Justice; • The draft Resolution of the Government of the Kyrgyz Republic “On Amendments to the Resolution of the Government of the Kyrgyz Republic” “On measures to implement the Kyrgyz Republic Pasture Law” dated 19 June 2009 #386 was developed and cleared with relevant ministries; Page 35 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) • The draft Program on Development of Pasture Farming and Livestock Breeding for 2019–2023 has been developed and finalized according to the comments and suggestions of the ministries; 5. Social Mobilization Activities were conducted: # Activity Since the project launching No of No of Of which micro- participants females projects 1 Round table at the regional level 2 130 18 (13,8%) 2 Round table at the district level 12 666 81 (12,2%) 3 Round table at the level of AK 141 4,613 1,194 (25,9%) 4 Sharing information on the Project 145 2,678 721 (26,9%) 5 Introductory meetings at village level 412 19,301 6,300 (32,6%) 6 Focus groups 407 13,460 4,142 (30,8%) 7 PUU/PC activity assessment for focus groups 393 12,891 3,926 (30,4%) 8 PUU/PC activity assessment for PC members 145 1,749 209 (11,9%) 9 PUU/PC activity assessment, study of documents by experts 140 8,565 10 First village meeting (PUU spring meeting) 407 16,574 4,470 (26,9%) 11 PUU general meeting (Staffing of PC, PUU AC, AHSC) 140 8,647 2,006 (23,2%) Filling in pasture profile sections. Completion of IA and 12 140 2,277 360 (15,8%) prioritization of problems at the level of aiyl aimak 13 Development of CPMLDPs and annual plan 140 1,428 219 (15,3%) Meeting of PC on preliminary selection of MPP and 14 registration of MPG in PC. 140 1,691 171 (10,1%) Meeting of PC on finalization of CPMLDPs. Request for final 15 MPP and approval of final MPP versions. 140 1,716 185 (10,8%) 16 Meeting of Aiyl Kenesh 140 17 Annual reporting meeting (autumn meeting of PUU) 371 13,920 3,341(24%) TOTAL 3 589 110 306 27 343 (24,8%) 6. Methodological Guidelines on Demarcation of the Pasture External Boundaries and Establishment of the Pasture Internal Boundaries were developed and approved by order of the MAFIM KR No.418 of December 18, 2017. The Guidelines Page 36 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) consist of the “Plan of inventory work for pilot PUUs” and the “Plan of informing the public during inventory works in pilot aimaks of Chui and Talas regions of the Kyrgyz Republic”; 7. The external pasture boundaries of all Pasture Committees were digitized and installed in Google Earth Pro; 8. Information from Hydrometeorological Center on climate change and weather forecast in Chui and Talas is procured and sent to pasture users; 9. A lawyer was recruited to strengthen the institutional capacity of the DPLF; 10. Computer and office equipment (KGS 1,520,000) were purchased to increase the RDAD of Chui and Talas regions and the central office of the MAFIM; 11. The Kyrgyz Livestock and Pasture Research Institute (KLPRI) was supported in developing “Caragana control plan”, which included: 1) pest control approaches; 2) use of herbicides and their management; 3) the Bank's policy regarding pesticides, regulatory legal framework and institutional capacity; 4) monitoring and evaluation, cost estimates. 12. 406 Community micro-projects were implemented to improve access to pastures: Micro-projects Chuy Oblast Talas Oblast Total # of PLIMP # of PLIMP # of PLIMP projects amount in projects amount in project amount in USD USD s USD Pasture infrastructure improvement 28 279 219 48 547 814 76 827 033 Repair of catchments and water points 3 34 852 20 198 972 23 233 825 Rehabilitation of bridges 9 81 926 16 161 614 25 243 539 Road rehabilitation 12 133 633 0 12 133 633 Road rehabilitation (km) 718.4 473.5 1191.9 Construction of dipper for small cattle 4 28 808 12 187 228 16 216 036 Procurement of machinery, equipment 260 3 298 046 70 1 784 595 330 5 082 642 Procurement of a/c machinery - tractors, bailers, 18 382 733 17 382 980 35 765 713 seeders etc. Procurement of road machinery - excavators, 81 2 233 353 23 1 059 172 104 3 292 525 bulldozers, loaders Procurement of trucks 18 317 451 7 276 846 25 594 297 Equipment for fodder production - crushers, 63 228 054 6 13 598 69 241 653 seeders, ploughs, sprayers Other - attachable equipment, shearing 80 136 455 17 51 999 97 188 454 machines, generators Page 37 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) 13. Extensive public awareness and communication campaign was conducted. Full list of communication campaign activities and links to outputs is given in Appendix C to Annex 1; Objective/Outcome 2 Improve Community Based Livestock Management in the Project Area 1. Number of AHHPs implemented satisfactorily based on PUU performance criteria – 120 (actual at completion) Outcome Indicators 2. Proportion of pasture users satisfied with the quality of services provided by Private Veterinarians – 95.5 percent (actual at completion) 1. Number of animal health and husbandry plans prepared by PUUs and approved by the project – 140 (actual at Intermediate Results completion) Indicators 2. Number of private veterinarians trained – 743 (actual at completion) 1. The draft Laws “On Animal Protection”, “On Control of Epizooty” and the proposed amendments to the Law “On Veterinary Medicine” were developed and cleared with relevant ministries; 2. 140 PUUs were supported in preparing, implementing and monitoring plans for animal health and animal husbandry, a small grant of about US$1000 was provided to each qualified PUU — with the established Animal Health Sub-Committee; 3. Training modules on animal health were developed on the following topics: “Foot and mouth disease”, “Common horse diseases”, “Methods of sampling and delivery of samples of pathological materials to the veterinary laboratory”, “Nodular dermatitis of cattle”; Key Outputs by 4. Training delivered on the following topics: Component # No of Of which % (linked to the Name of activities: training (workshop), advisory services participants of female achievement of the Planning and monitoring of animal health activities at the level of Objective/Outcome 2) 1 1048 8,87% aiyl aimak Animal Health Sub-Committee (AHSC): the need to create and the 2 principles of its formation. Information about the Veterinary 1031 9,60% Statutory Body 5 Animal pasteurellosis and arrangement control measures 262 3,44% Farm animal breeds bred in Kyrgyzstan and ways to improve their 6 230 4,35% breeding quality Page 38 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) 7 Artificial insemination of cows and its meaning 249 6,83% Organization and monitoring of activities to control fascioliasis and 8 164 7,32% dicroceliosis 9 Anthrax and rabies and control measures 510 4,12% 10. Cattle leukemia 223 2,69% 11 Technology of preparation of feed and farm animal feeding 238 4,20% 12 Foot and mouth disease of farm animals 183 7,65% 13 Nodular dermatitis of cattle 220 5,00% 14 Common Horse Diseases 220 5,00% 15 Training in skills on x-ray, ultrasound and other devices 59 10,17% 16 Epizootic lymphangitis of horses 243 2,47% 17 Foot-rot of sheep and goats 243 2,47% 18 Equine strangles 243 2,47% 19 Contagious ecthyma of sheep and goats 243 2,47% Total 5609 6,12% 5. 69 Community sub-projects were implemented to improve livestock management Micro-projects Chuy Oblast Talas Oblast Total # of PLIMP # of PLIMP # of PLIMP projects amount in projects amount in project amount in USD USD s USD Veterinary and livestock 45 410 945 24 44 262 69 455 207 Artificial insemination and shearing points 0 18 25 419 18 25 419 Pharmacies and vet points 8 201 989 0 8 201 989 Construction of Beccari pits 1 5 756 2 9 401 3 15 157 Other - sheds, races, folds 36 203 200 4 9 442 40 212 642 Page 39 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Appendix A to Annex 1 STANDARD STRUCTURE OF CPMUP 1. Introduction 2. General Information (in table / map format) - Geographical location, climate, settlement and population - Land resources - Brief information about Pasture Users Union (PUU) - Brief description of natural boundaries and pasture infrastructure 3. Livestock amount and required pasture and fodder resources - Actual livestock amount in conditional heads - Forecasted livestock amount in conditional heads for the next three years - Required pasture and fodder resources 4. Pasture availability and fodder balance (annual) - Dry matter availability on grazed pastures - Prepared fodder availability - Fodder and feed balance 5. Pasture monitoring and needs assessment - Analysis of pasture conditions under climate change scenario - Environmental safeguards plan - PUU needs assessment 6. Medium term PUU pasture management plan - 5-year plan for pasture usage - Activities and budget for 5 years - Medium term animal health and husbandry plan - 5-year plan for pasture monitoring and evaluation - 5-year plan for CPMUP monitoring and evaluation 7. CPMUP annual plan - 1-year plan for pasture usage (grazing plan) - Calendar plan for livestock grazing - Operational plan of Pasture Committee for the year - 1-yar animal health and husbandry plan - 1-year activities plan - 1-year budget - 1-year CPMUP monitoring and evaluation plan 8. PUU internal grazing rules 9. Sanctions for breach of PUU internal rules 10. Annexes (including list of PUU members, pasture monitoring data, agreements’ and pasture ticket templates, maps of PUU area Page 40 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Appendix B to Annex 1 PUU ASSESSMENT TOOL METHODOLOGY. − PUU assessment is based on the satisfactorily implementation of the CPMUP and AHHP Assessment of CPMUP CPMUP assessment is a sum of technical assessment and institutional assessment. The technical assessment score is averaged with two institutional scores (pasture usage fee collection and grazing pressure see below) to give an overall mark (expressed in percentage of maximum) for PUU performance. Satisfactory level is determined to be 75 percent or higher. Technical assessment The technical assessment is based on the evaluation of the pasture and feeding site management. It means rating on the four-point scale (poor, fair, good, and excellent) on the following characteristics: (i) pasture condition; (ii) grazing pastures and riparian areas; and (iii) feed, shelters and watering site management. Each pasture area is evaluated separately by a monitoring team that made an expert opinion on each criterion. Each point on the scale has well described reference criteria, which can be easily applied by a monitoring team. Overall grade then was averaged among those three criteria and was expressed as a percentage of maximum. For example, if average was 3.2 (out of 4), PUU got 80% for the pasture and fodder management. Table 5 Technical assessment scorecard SCORED CRITERIA SCORE/LEVEL Pasture and Feeding Site Management 1. Pasture Condition Four-point scale 2. Grazing Pastures and Riparian Areas Four-point scale 3. Feed, Shelters and Watering Site Management Four-point scale (1) TOTAL POINTS EARNED = Total Points Available - Minus Total Points Not Applicable (NA) (if any) (2) TOTAL APPLICABLE POINTS = (3) AVERAGE SCORE = Total Applicable Points/3 (4) PERCENT SCORE = Total Applicable Points/12X100 Institutional assessment Institutional assessment consists of valuation of the fee collection for the pasture usage and the level of grazing pressure experienced by pasture plots (pasture load). Fee collection was measured as a percentage of the planned amount (with 100 percent if equal or more than planned), and the grazing pressure as a percentage of the pasture carrying capacity (number of animals) to the actual number grazed at a certain period of time (with 100 percent if carrying capacity is equal or more than actual number of animals grazed). Page 41 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) − Pasture usage fee collection Rating: If actual equals or more than planned, PUU gains 100 points. If less, than the rating is the percentage of actual versus planned. − Pasture load Rating: If factually grazed animals equals or less than pasture load conditional heads, PUU gains rating 1. If less, than the rating is the percentage of actual versus planned. TOTAL RATING OF CPMUP = Average result of technical assessment “Pasture and feeding site management” + rating of “Pasture usage fee collection" + rating of “Pasture load"/3 Assessment of AHHP AHHP for each PUU is assessed against four important criteria: i) livestock living conditions; ii) animal nutrition; iii) feed storage; and iv) animal health. Each criterion is rated on a four-point scale. As in the pasture management assessment tool, each grade has a clear description to guide evaluators’ assessment and make final figures comparable. The resulted figures on each criterion are then averaged to give an overall mark (expressed in percentage of maximum) for PUU performance. Satisfactory level is again determined to be 75 percent or higher. Table 6 AHHP assessment scorecard SCORED CRITERIA SCORE/LEVEL Animal Health and Production 1. Living Conditions Four-point scale 2. Animal Nutrition Four-point scale 3. Feed Storage Four-point scale 4. Animal Health Four-point scale (1) TOTAL POINTS EARNED = Total Points Available - Minus Total Points Not Applicable (NA) (if any) (2) TOTAL APPLICABLE POINTS= (3) AVERAGE SCORE = Total Applicable Points/4 (4) PERCENT SCORE = Total Applicable Points/16X100 TOTAL RATING OF AHHP = Average score of “Animal health and productivity” Page 42 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Appendix C to Annex 1 COMMUNICATION MATERIALS Name of the work Target group Goal of the Video owner Permanent address of the work video on the APIU channel: Learning videos Training video on Pasture Training, Department https://www.youtube.com “Pasture assessment Committee, Capacity of Pastures /watch?v=4QVzAfDllPY&t using the crossed- lines ayil okmotu building =35s method” Training video on “Animal Pasture Training, Department https://www.youtube.com health plans Committee, Capacity of Pastures /watch?v=GP4DD2DhgSU development” ayil okmotu building Training video on Pasture Training, Department https://www.youtube.com “Pasture management Committee, Capacity of Pastures /watch?v=VxmKe0KvqDo and use plans ayil okmotu building development” Training video on Pasture Training, Department https://www.youtube.com “Varieties of pasture Committee, Capacity of Pastures /watch?v=6hSX4XoQtyo plants” ayil okmotu building Videos, films, special reports, news that were placed on TV Video about initiatives Pasture Experience Department https://www.youtube.com and experience exchange Committee, exchange, of Pastures /watch?v=PdvYbCGvESE by Pasture Committees ayil okmotu motivation Video about interaction Pasture Promoting Department https://www.youtube.com of Pasture Users Unions Committee, motivation of Pastures /watch?v=cJFGC0oVe9c ayil okmotu Video about current Pasture Motivation, Department https://www.youtube.com results of the Pasture Committee, image, results of Pastures /watch?v=MZBOLdF3YEc reform ayil okmotu Video about the Veterinarians, Increasing Veterinary https://www.youtube.com Veterinary Chamber farmers knowledge Chamber /watch?v=xuVDaNf4oM4 &t=3s Special video report Veterinarians, Increasing trust, Veterinary https://www.youtube.com about the Veterinary farmers image Chamber /watch?v=jxtq- Chamber bmh4PM&t=6s Special video report All population All population APIU https://www.youtube.com about the Pasture and /watch?v=Oi6hE- Livestock Management RSphw&feature=youtu.be Improvement Project Special video report All population All population APIU https://www.youtube.com about activities of the /watch?v=jxtq- Veterinary Chamber bmh4PM&t=6s Page 43 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) “Agro charba” program Farmers Increasing Department https://www.youtube.com knowledge, of Pastures /watch?v=z3HkiNR0xcA&t experience =666s exchange “Agro charba” program Farmers Increasing Department https://www.youtube.com knowledge of Pastures /watch?v=i5Ai9GGFX3M& t=381s “Agro charba” program Farmers Increasing Department https://www.youtube.com knowledge, of Pastures /watch?v=i5Ai9GGFX3M experience exchange “El Baylygy” film Entire Increasing Department https://www.youtube.com population knowledge, of Pastures /watch?v=ryxTIz57z_o results “Adamzat darygeri” film Entire Increasing Department https://www.youtube.com population knowledge, of Pastures /watch?v=MsswoQsJ224 results Video news about Entire Informing https://www.youtube.com transferring of population /watch?v=ci4j1vyBITM motorcycles to private vets Video news about the Entire Informing https://www.youtube.com work of the Pasture population /watch?v=ZxHFwNNQPW4 Committees of Chui oblast Video news about the Entire Informing https://www.youtube.com support of the veterinary population /watch?v=CJAarTCtcJY service of Talas oblast Video news about the Entire Informing https://www.youtube.com results of the competition population /watch?v=ZroqBHMcr4k among the media for the best publications on pastures Running line Pasture Informing On the need to pay for Committee pasture on time Radio “Kyrgyzstan Obondoru” Entire Increasing Live Radio recordings in the (news, games, population knowledge, APIU archive broadcasts) “Birinchi Radio” Entire Informing Live Radio recordings in the (news, broadcasts) population APIU archive Social networks APIU page Entire Informing APIU https://www.facebook.co population m/apiu2019/ Page 44 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) “Doctor Azamat” page Entire Informing APIU https://www.facebook.co population m/profile.php?id=1000179 29986522 YouTube channel Entire Informing APIU https://www.youtube.com population /channel/UCT0C- iTnihB5P2fgMDVFsfA Telegram Farmers Informing APIU https://t.me/apiu2019 Other works Website of the Entire Informing Department http://pasture.gov.kg/ Department of Pastures, population of Pastures Livestock and Fisheries Photobook about the Partners Results, image APIU https://cloud.mail.ru/publi project (link to the c/4c24/2wWTmpt1r English version) Newsletters Partners, target Informing APIU https://cloud.mail.ru/publi (2015, 2016, 2017, groups c/2hT1/W3kBd2qZx 2018гг.) Photos APIU, partners Informing APIU Photos in the APIU archive Press releases Mass media Informing APIU Press releases in the APIU archive Press tours Mass media Informing Articles and news stories Page 45 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Peter Goodman Task Team Leader(s) Alexander Balakov Procurement Specialist(s) Nodar Mosashvili Financial Management Specialist Gulana Enar Hajiyeva Social Specialist Talaibek Torokulovich Koshmatov Team Member Asli Gurkan Social Specialist Tamir Ibragimoff Social Specialist Cesar Niculescu Social Specialist Lidia Hvan Team Member Supervision/ICR David Tuchschneider, Artavazd Hakobyan Task Team Leader(s) Irina Goncharova Procurement Specialist(s) Garik Sergeyan Financial Management Specialist John Bryant Collier Environmental Specialist Mohammad Ilyas Butt Procurement Team Kristine Schwebach Social Specialist Alexander Balakov Procurement Team Talaibek Torokulovich Koshmatov Team Member Natalya V. Iosipenko Team Member Talimjan Urazov Team Member Rosalie Quong Trinidad Team Member Kunduz Ermekbaeva Procurement Team Page 46 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY13 4.197 20,668.12 FY14 30.824 250,243.52 FY15 17.026 55,869.24 FY16 13.734 62,926.25 FY17 0 0.00 Total 65.78 389,707.13 Supervision/ICR FY14 .200 306.80 FY15 7.325 44,430.93 FY16 8.066 28,587.20 FY17 19.666 93,325.02 FY18 24.670 148,092.16 FY19 24.803 138,301.89 FY20 12.663 57,309.21 Total 97.39 510,353.21 Page 47 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Community Based Pasture 11.44 8.7 76.0 % Management Community Based Animal Health and Husbandry 1.85 .8 43.2 % Services Project Management 1.71 1.4 81.9 % Total 15.00 10.9 72.67 % Page 48 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) ANNEX 4. EFFICIENCY ANALYSIS Underlying Assumptions To make it consistent with the PAD, the same methodological approach was followed in respect to the project benefits. Efficiency was evaluated based on the project benefits resulted from lower stocking rates in the with-project compared to without-project scenario because more pasture is made accessible due to infrastructure investment and better pasture management. The lower stocking rates result in more available dry matter, higher growth rates in fat-stock and higher milk yields. Higher fertility rates, lower mortality due to a lower parasitic burden and better body condition, and higher feed conversion efficiency due to better pasture quality resulted in higher livestock productivity that was observed in the project area. The critical assumptions which drive the analysis are therefore: a) Utilized pasture area; b) livestock number growth (stocking rate sheet); and b) pasture forage dry matter production response to stocking rate and efficiency of feed conversion. − Utilized Pasture Area: In the with-project scenario, the area of utilized pasture gradually increases over a 5-year period and then stabilises. This assumes that PCs develop and implement the main infrastructure projects which provide increased access to pasture within 5 years and assumes that thereafter the same level of access to pastures is maintained. At ICR actual increase in the pasture area was used to estimate benefit. − Livestock Numbers: In the with-project scenario, livestock numbers are assumed to increase more slowly than in the without-project scenario as PCs impose limitations on livestock numbers allowed onto pasture. However, livestock productivity increases more rapidly than in the without-project scenario, which lead to positive incremental benefits. Incremental benefit results not only from increased productivity but also from reduced cost of animal maintenance. − Stocking Rates: Consequently, the stocking rate declines as the utilised pasture area increases in the with-project scenario but decreases in the without-project scenario. Thereafter, stocking rates increase and stabilise in year 10 but growth is slower in the with-project scenario. The analysis estimates project benefits in year 10 - these are incremental gross margins (the difference between gross margins in the with-project and without-project scenarios) in year 10, the year in which the difference between with-project and without-project project stocking rates and therefore project benefits, are maximised and stabilise. The same benefits are achieved in years 11-20. Project benefits in years 1-9 are estimated as a percentage of year 10 benefits by calculating the difference in with-project and without-project stocking rates each year as a percentage of the maximum difference which is achieved in year 10. This avoids the need to calculate a full set of gross margins for each of the years 1- 10. Where the incremental benefits from improved pasture productivity and animal performance due to lower stocking rates are higher than the incremental costs of lower livestock numbers the project yields a positive IRR. Data Use Statistically observed livestock production data proved to be generally in line with assumptions. The volume of gross output of livestock production of farms in Chui and Talas regions has been increased from KGS 14.4 billion to KGS 15.1 billion or by 5 percent. Meat production in pilot regions for all types of farms Page 49 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) has been increased from 68.6 thousand tons in 2015 to 73 thousand tons in 2018 or by 6.4%. The production of raw milk has been increased from 448.7 thousand tons in 2015 to 477 thousand tons in 2018 or by 6.3%. This generally corresponds with 2 percent annual incremental increase in livestock production assumed by the model. In ICR actual increase in utilized pasture area and actual project costs were taken into account. Cost-Benefit Analysis Running the same model that was used at appraisal but with actual data yielded following results: Appraisal ICR Financial Results Incremental gross margin per farm (USD) 100 54 Incremental gross margin per ha total pasture area (USD) 3 2 FNPV of net incremental benefits (million USD) 29.22 10.54 FIRR (%) 49.2% 35.8% Economic Results ENPV of net incremental benefits (million USD) 32.13 16.3 EIRR (%) 52.4% 46.2% The full set of benefits planned by the project did not become visible due to the shortened period of the project implementation. This resulted in somewhat smaller figures than were expected; however, calculated benefits are still solid and are in line with figures expected in other projects in the sector. Decrease in IRR is less pronounced than in NPV because of the reduced project cost. In order to verify data that were obtained at the macro-level a cost-benefit analysis was made at a micro-level of a typical pasture committee. Aral PUU in Talas region implemented its pasture management plan and 5 micro-projects, which allowed this PC to improve its economic performance. Procured machinery was used to improve infrastructure/access to pastures, which maintained rotation and improved pasture productivity, which led to improved forage base and increased weight gain in livestock and milk yield. Procured machinery brought additional benefit to PUU, that is now able to lease this machinery to other users and earn additional cash. Calculation of cost-benefit at a micro-level is given below: Incremental cost Unit of Number Unit cost Item measure Total of units (KGS) # ment 1 Capital expenditures: 1.1 Front-end loader with bucket 1.8 m3 pc 1 2 490 000 2 490 000 1.2 Organization of veterinary service pc 1 309 500 309 500 1.3 Capital repair of water-catchment pc 1 680 496 680 496 Page 50 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) systems 1.4 Construction of dippers pc 1 941 544 941 544 Rehabilitation and reconstruction of 1 1 311 904 1 311 904 1.5 bridges pc Total: 5 733 444 Current expenditures: 1 Current expenditures: for earth works KGS 246 656 Current expenditures: KGS 706 415 2 Earth backing by bulldozer Total KGS 953 071 Incremental Benefit Unit of Number Item measur Unit cost Total of units # ement 1 Livestock Weight gain kg 10 716,75 250 2 679 187,50 2 Milk yield kg 40 523,25 15 607 848,75 Total: KGS 3 287 036,25 Additional Benefit vehicle 224 1531,43 343 039,31 1 Machinery leasing /hour Total: 343 039,31 Assuming annual incremental benefit of KGS 3 630 075 (3 287 036 + 343 039), one-time cost of KGS 5 733 444, and annual cost of KGS 953 071 the project yields IRR of 46 percent and NPV over 10 years of KGS 7.8 million with discount rate of 15 percent. IRR of 46 percent at micro-level generally confirm data that the model calculated at the macro-level. Page 51 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Page 52 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) Unofficial Translation “On the letterhead of the Ministry of Agriculture, Food Industry and Melioration of the Kyrgyz Republic” December 10, 2019 No. 01-4/6423 To: World Bank Kyrgyz Republic Office Re.: Pasture and Livestock Management Improvement Project The Ministry of Agriculture, Food Industry and Melioration of the Kyrgyz Republic expresses its gratitude to the World Bank team for making available to it the draft Implementation Completion and Results Report for the Pasture and Livestock Management Improvement Project (PLMIP), which was closed on June 28, 2019. Having reviewed the draft PLMIP Implementation Completion and Results Report (ICR) the Ministry in conjunction with the Agricultural Projects Implementation Unit, the Community Development and Investment Agency, the Department of Pastures, Livestock and Fisheries as well as the implementing organizations, would like to report that there are no comments on the draft. At the same time, I would like to point out that indeed, the delay at the implementation stage due to the expiration of the ECAPDEV grant, with the funds originally allocated for preparatory activities (development of an assessment tool for Pasture User Unions, community pasture management plans, engagement of international and local technical assistance); the later effective date of the project due to the lengthy ratification process in line with domestic procedures have strongly affected the PLMIP implementation period and evaluation, as it was indicated in the Borrower’s Project Implementation Completion Report and in draft Project ICR presented by the World Bank. The project became effective at the end of August 2015, more than a year following its approval. However, owing to the robust activities of the Project implementing organizations and the World Bank team, it is fair to say that all project activities have been completed, and thus the project goal has been achieved despite these circumstances. We are grateful to the World Bank team for their close cooperation and significant efforts to ensure the timely implementation of the scheduled project activities, and we look forward to further fruitful cooperation. The Ministry would be grateful to receive the final document – the Implementation Completion and Results Report for the Pasture and Livestock Management Improvement Project. Minister “signed” E. Choduev Executed by: Agricultural Projects Implementation Unit: 66 56 25, 66 27 65 Page 53 of 54 The World Bank Pasture and Livestock Management Improvement Project (P145162) ANNEX 6. SUPPORTING DOCUMENTS 1) Project Appraisal Document of June 5, 2014 (Report # PAD833) 2) Financing Agreement between Kyrgyz Republic and IDA, 5521- KG, H976-KG, December 10, 2014 3) Project Restructuring Paper (Report # RES35863) 4) Aide-mémoires for the missions of: - May 13-15, 2015 - July 27–August 3, 2016 - January 9 – 13, 2017 - August 23–September 1, 2017 (MTR) - April 16–20, 2018 - December 10-19, 2018 - June 6-19, 2019 5) Implementation Status and Results Reports (sequence #1–#12). 6) Country Partnership Strategy for the Kyrgyz Republic for the period FY14–FY17 (June 24, 2013; Report # 78500-KG) 7) Country Partnership Framework for the Kyrgyz Republic for the period FY19-FY22 (October 10, 2018; Report # 130399-KG) 8) PROMOTANK LLC PLMIP Completion Survey Final Report 9) ARIS Report on Pasture Users’ Satisfaction 10) Michael Hale, Assessment of Community Pasture Management and Use Plan 11) APIU of MAFIM and ARIS – Progress Reports Page 54 of 54