Document of The World Bank Report No: ICR00003055 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-44610) ON A CREDIT IN THE AMOUNT OF SDR 319.9 MILLION (US$ 521 MILLION EQUIVALENT) TO THE REPUBLIC OF INDIA FOR A NATIONAL VECTOR BORNE DISEASE CONTROL AND POLIO ERADICATION SUPPORT PROJECT June 27, 2014 Human Development Sector South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective March 31, 2014) Currency Unit = INR INR 60 = US$1 US$1.5456 = SDR 1 FISCAL YEAR April 1 – March 31 ABBREVIATIONS AND ACRONYMS ACT Artemisinin-based combination therapy (for treating malaria) AM Aide Memoire API Annual Parasite Incidence AS+SP Artesunate plus Sulfadoxine-Pyrimethimine (a co-blistered ACT) ASHA Accredited Social Health Activist BCC Behavior Change Communication BPO Block Program officer CBO Community Based Organization CDC US Centers for Disease Control and Prevention CFR Case Fatality Rate CHC Community Health Center CMO Community-based Organizations CMS Central Medical Stores CSKAEP Coalition for Strengthening Kala Azar Elimination Project DBS Domestic Budget Support DDT Dichloro diphenyl trichloroethane DGHS Directorate of General Health Services DHS Director of Health Services DIR Detailed Implementation Review DPO District Program Officer EAC External Aided Component ECoP Environmental Code of Practice EMP Environmental Management Plan EOP End of Project EQAS External Quality Assurance System FA Financial Agreement GFATM Global Fund for AIDS, TB and Malaria (the Global Fund) GIS Geographic Information System GoI Government of India HR Human resources ICMR Indian Council of Medical Research ICR Implementation Completion and Results Report IDA International Development Association IEAG India Expert Advisory Group (Polio) IEC Information, Education and Communication IMA Indian Medical Association IO Intermediate Objective IRS Indoor Residual Spraying ISR Implementation Status and Results Report ITN Insecticide Treated (bed) Net JMM Joint Monitoring Mission LLIN Long-Lasting Insecticidal Net LQAS Lot Quality Assurance Survey LMIS Logistic Management Information System MCH Maternity and Child Health MDA Mass Drug Administration MDG Millennium Development Goal MIS Management Information System MPW Multi-purpose health worker MoHFW Ministry of Health & Family Welfare MoU Memorandum of Understanding MTR Mid-term Review M&E Monitoring and Evaluation NGO Non-government Organizations NHP National Health Policy NHRM National Rural Health Mission NID National Immunization Days NIMR National Institute of Malaria Research NMCP National Malaria Control Program NVBDCP National Vector Borne Disease Control Program OPV Oral Polio Vaccine OR Operations Research PAD Project Appraisal Document PDO Project Development Objective Pf Plasmodium falciparum Pv Plasmodium vivax PHC Primary Health Care PIP Program Implementation Plan PPE Personal Protective Equipment PPP Public Private Partnership QA Quality Assurance QC Quality Control RDT / RDK Rapid Diagnostic Test / Rapid Diagnostic Kit SABA Social and Behavioral Assessment SOE Statement of Expenditure SOP Standard Operating Procedures SSH Sentinel Service Hospitals VBD Vector-borne disease VHAI Voluntary health Association of India VPP Voluntary Pool Procurement (GFATM) WCO WHO Country Office – India WHO World Health Organization WHP World health Partnership Vice President: Philippe H. Le Houerou Country Director: Onno Ruhl Sector Manager: Julie McLaughlin Project and ICR Team Leader: Ramesh Govindaraj ICR Author: John Paul Clark INDIA NATIONAL VECTOR BORNE DISEASE CONTROL AND POLIO ERADICATION SUPPORT PROJECT CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 1 3. Assessment of Outcomes ............................................................................................ 2 4. Assessment of Risk to Development Outcome........................................................... 3 5. Assessment of Bank and Borrower Performance ....................................................... 3 6. Lessons Learned ......................................................................................................... 3 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ............ 3 Annex 1. Project Costs and Financing ............................................................................ 4 Annex 2. Outputs by Component ................................................................................... 5 Annex 3. Economic and Financial Analysis ................................................................... 6 Annex 4. Bank Lending and Implementation Support/Supervision Processes .............. 7 Annex 5. Beneficiary Survey Results ............................................................................. 9 Annex 6. Stakeholder Workshop Report and Results................................................... 10 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 11 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 12 Annex 9. List of Supporting Documents ...................................................................... 13 MAP DATA SHEET A. Basic Information India: National Vector Borne Disease Control Country: India Project Name: & Polio Eradication Support Project Project ID: P094360 L/C/TF Number(s): IDA-44610 ICR Date: 06/10/2014 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: SIL Borrower: INDIA Original Total XDR 319.90M Disbursed Amount: XDR 150.59M Commitment: Revised Amount: XDR 151.46M Environmental Category: B Implementing Agencies: Ministry of Health and Family Welfare Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 07/07/2005 Effectiveness: 03/06/2009 03/06/2009 12/16/2010 Appraisal: 06/02/2008 Restructuring(s): 12/19/2012 12/30/2013 Approval: 07/31/2008 Mid-term Review: 04/18/2011 05/02/2011 Closing: 12/31/2013 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Low-negligible Bank Performance: Moderately Unsatisfactory Borrower Performance: Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Moderately Quality at Entry: Government: Unsatisfactory Unsatisfactory Moderately Implementing Quality of Supervision: Unsatisfactory Unsatisfactory Agency/Agencies: Overall Bank Moderately Overall Borrower Unsatisfactory Performance: Unsatisfactory Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 88 88 Public administration- Health 12 12 Theme Code (as % of total Bank financing) Child health 56 56 Malaria 33 33 Other communicable diseases 11 11 E. Bank Staff Positions At ICR At Approval Vice President: Philippe H. Le Houerou Praful C. Patel Country Director: Onno Ruhl Isabel M. Guerrero Sector Manager: Julie McLaughlin Benjamin P. Loevinsohn Project Team Leader: Ramesh Govindaraj Gandham N.V. Ramana ICR Team Leader: Ramesh Govindaraj ICR Primary Author: John Paul Clark F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project PDO is to enhance the effectiveness of Recipient's efforts to control malaria, eliminate kala azar and eradicate polio, by: (a) increasing the number of people benefiting from effective prevention, diagnosis and treatment services for malaria and kala azar; (b) strengthening central and state capacities for evidence-based policy development, strategic planning, and program management for effective control of vector-borne diseases; and (c) securing the timely supply of polio vaccines. Revised Project Development Objectives (as approved by original approving authority) NA (a) PDO Indicator(s)1 Actual Value Original Target Formally Achieved at Values (from Revised Indicator Baseline Value Completion or approval Target Target Years documents) Values Percentage of fever cases in project districts receiving a malaria test result no Indicator 1 : later than the day after the first contact Value 54.3% (range 38.4- quantitative or 10-58% 70% 91%) Qualitative) Date achieved 2009 2013 77% of the target was achieved in five phase one states. Baseline values are from population based household survey conducted by NIMR in 2009. End line Comments estimate is a population weighted summary program monitoring data (LQAS) (incl. % collected in 5 Phase 1 project states. 54.3% is a populated weighted average achievement) of point estimates from Andhra Pradesh 90.7%; Chhattisgarh 38.4%; Jharkhand 26.2%; Madhya Pradesh 54.8%%; Odisha 74.4% Percentage of individuals in project areas belonging to eligible Long Lasting Indicator 2 : Insecticidal Net (LLIN) target population who slept under an LLIN during the previous night Value 28.6% quantitative or 0% 50% (range 20-37%) Qualitative) Date achieved 2009 2013 Comments 57% of target achieved. This is a population weighted summary estimate from (incl. % program monitoring data (LQAS) collected in 5 Phase 1 project states (AP, MP, achievement) Chhattisgarh, Jharkhand and Odisha). Percentage of blocks that achieve the elimination goal of less than one kala azar Indicator 3 : case per 10,000 persons at the sub-district level Value 236 blocks in 3 states 50% 58.6% quantitative or (Bihar, Jharkhand, West 1 Population based surveys were planned for baseline, mid-term and end of project. However, only the baseline survey was conducted. The baseline survey is representative of the target population for 5 states from phase one only. There are no comparable data for phase 2 districts and states. In the absence of mid-term and end of project surveys aggregated population-weighted LQAS data from implementing districts in the 5 phase 1 states was reported. This data is not representative of the target population of 11 states and the validity of the estimates is questionable. Qualitative) Bengal) in 2010 reported to have annual incidence of greater than 1 per 10,000 population (being confirmed by NVBDCP) Date achieved 2009 2013 Comments (incl. % 117% of target achieved. achievement) At least 80% of the households with eligible children covered during the national Indicator 4 : and sub-national immunization days in high risk districts. Value quantitative or >80% >80% >98% Qualitative) Date achieved 2009 2010 Comments (incl. % 123% target achieved. achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Percentage of population in high risk project areas protected by ITNs or IRS Value (quantitative No baseline 80% range: 32.9 – 84.2% or Qualitative) Date achieved 2009 2012 35-1.05% of target achieved. Indicator not routinely monitored. Weighted Comments summary estimate from LQAS not available. 5 states reporting in 2012 as (incl. % follows Andhra Pradesh 65.3%; Chhattisgarh 84.2%; Jharkhand 27.9%; Madhya achievement) Pradesh 32.9%; Odisha 41.9% Percentage of Rapid Diagnostic Test (RDT) positive adults receiving ACT no Indicator 2 : later than the day after the fever started. Value 0% (District-wide ACT (quantitative introduction is being done 70% 33% (range 2-65%) or Qualitative) under the project) Date achieved 2009 2013 47% of target achieved. It is difficult to assess this indicator using program Comments monitoring data from LQAS because this indicator is based on front line workers' (incl. % reports of treatment of RDT positive cases and the number of RDT positive cases achievement) is very small per worker. Percentage of designated providers of malaria diagnosis and treatment who have Indicator 3 : not had an ACT or RDT stockout during last 3 months. Value 0%(ACT and RDK are 57.9% (RDT), 90% (quantitative being introduced under range 40-86%. or Qualitative) the project) Date achieved 2009 2013 Comments 64% of target achieved.(This is a population weighted summary estimate from (incl. % program monitoring data (LQAS) collected in 3 Phase 1 project states (AP, MP achievement) and Odisha). Percentage of villages with a trained designated provider of malaria diagnosis Indicator 4 : and treatment services Value Range: 19.1 to (quantitative 20% 70% 94.5% or Qualitative) Date achieved 2009 2012 27-135% of target achieved. Not routinely monitored in ISR. Weighted Comments summary estimate from LQAS not available. 5 states reporting in 2012 as (incl. % follows Andhra Pradesh 19.1%; Chhattisgarh 86%; Jharkhand 82.7%; Madhya achievement) Pradesh 94.5%; Odisha 66.6% Indicator 5 : Percentage of diagnosed kala azar cases completing the standard treatment Value 55% (based on (quantitative 80% 93.8% preliminary surveys) or Qualitative) Date achieved 2009 2013 Comments (incl. % 117% achieved. achievement) Percent of houses in identified [targeted] kal azar endemic areas covered with Indicator 6 : effective insecticide spray. Value (quantitative No baseline 80% 97.22% or Qualitative) Date achieved 2009 2012 Comments (incl. % 121% achieved. Data drawn from annual reports and not externally validated. achievement) Percentage of facilities reporting no stock-outs of rapid diagnostic test for kala Indicator 7 : azar and first line medicines. Value 73% (for first-line (quantitative 90% 91% for RDT medicines) or Qualitative) Date achieved 2009 2013 Comments (incl. % 100% achieved. achievement) Monitoring System established in 5+ sites to monitor the Quality of RDTs, drugs Indicator 8 : and insecticides delivered by the procurement system. Value (quantitative 1 5 1 or Qualitative) Date achieved 2009 2012 Comments 20% achieved. (incl. % achievement) Pharmaco-vigilance system established in at least 3 sites to monitor the first line Indicator 9 : medicines for kala azar elimination Value (quantitative 0 3sites 4 sites or Qualitative) Date achieved 2009 2010-2013 Comments (incl. % 133% achieved. Source RMRI report. achievement) Percentage of planned additional staff who are in position at central, state and Indicator 10 : district levels and received induction training. Value 68.1% (staff in (quantitative 0% 100% NA position) or Qualitative) 63.5% (trained) Date achieved 2009 2013 Comments 63.5% achieved. Difficulties with retention of staff resulted in high rates of staff (incl. % turnover and challenges in providing new staff with timely access to training. achievement) Proportion of eligible districts meeting the readiness criteria for each period of Indicator 11 : implementation. Value 0% (These criteria are (quantitative being applied under the 100% 100% or Qualitative) project) Date achieved 2009 2011-2013 Comments (incl. % 100% achieved. achievement) Percentage of endemic districts with quality controlled incidence data of vector- Indicator 12 : borne diseases stratified by age and gender Value (quantitative 0 50% 100% or Qualitative) Date achieved 2009 2012 Comments 200% achieved, but indicator not routinely monitored in ISR. NVBDCP reports (incl. % that value was 100% at baseline and remained so throughout the project. achievement) Indicator 13 : No stock outs of Oral Polio Vaccine for Supplemental Immunization Activities. Value (quantitative No stockout No stockout NA No stockout or Qualitative) Date achieved 2009 2009-2013 Comments (incl. % 100% achieved. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 12/08/2008 Satisfactory Satisfactory 0.00 2 06/23/2009 Satisfactory Satisfactory 5.00 Moderately 3 01/25/2010 Moderately Satisfactory 79.15 Unsatisfactory 4 06/25/2010 Moderately Satisfactory Moderately Satisfactory 81.65 5 01/02/2011 Moderately Satisfactory Moderately Satisfactory 81.65 6 06/28/2011 Moderately Satisfactory Satisfactory 90.78 7 04/05/2012 Moderately Satisfactory Moderately Satisfactory 167.94 8 11/13/2012 Moderately Satisfactory Moderately Satisfactory 211.58 9 04/10/2013 Moderately Satisfactory Moderately Satisfactory 227.22 Moderately Moderately 10 11/15/2013 235.83 Unsatisfactory Unsatisfactory Moderately Moderately 11 12/25/2013 238.09 Unsatisfactory Unsatisfactory H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions The GOI requested that the Bank allow M/s RITES (in lieu 12/16/2010 N MS MS 81.65 of UNICEF) to undertake OPV procurement to obtain better prices for OPV The GOI requested cancellation of US$184 million of IDA credit due to delay in central procurement of VBD commodities and a decision to 12/19/2012 N MS MS 211.58 use domestic resources for OPV (the amount cancelled was US$158 million, $70 million from VBD components and $88 million from polio component). The GOI requested a cancellation of the remaining 12/30/2013 N MU MU 233.09 credit amount of US$101.4 million due to low likelihood that Long Lasting Insecticidal ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Nets (LLINs) could be procured before project closure. I. Disbursement Profile 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Vector-borne diseases (VBD) pose immense public health concern and continue to be major causes of significant morbidity and mortality in India. At the time of project preparation, over 90% of the reported 2 million malaria burden was predominately from 11 states in India, while nearly all 40,000 new kala azar cases reported every year were from the states of Bihar, Jharkhand and West Bengal. The Government of India (GoI) made enormous progress in containing the VBD over more than three decades. Between 1997/8 and 2007/8, the total expenditure on VBD control increased by 170%. The country’s National Health Policy (2002) has pledged commitment to reduce mortality of malaria by 50% and eliminate kala azar by 2010. In 2005, GOI launched the National Rural Health Mission (NRHM), a flagship national program to improve rural health outcomes. NRHM serves as an umbrella program that links together disease control and other public health programs, including the National Vector Borne Disease Control Programme (NVBDCP). NRHM incorporates a number of innovative approaches in the sector, including a new national community health worker program, use of untied block grants, district-level planning, and new initiatives aimed at community mobilization and accountability. In 2010, this program became known as the National Health Mission, or NHM, as it incorporated urban health. Rational for Bank assistance Vector Borne Diseases: At the time of appraisal, the World Bank had been assisting GOI in developing effective infectious disease control programs for over a decade. A Malaria Control Project, partially funded by the International Development Association (IDA), was implemented in selected districts from 1997 to 2005. The project’s Implementation Completion Report highlighted the need for major reform in the technical content of the malaria program as well as of its implementation arrangements. Subsequently, the GOI developed new policies for vector borne disease control incorporating the latest evidence based prevention, diagnostic and treatment approaches for malaria and kala azar. The new Bank-financed project being assessed by this ICR, was therefore an opportunity to translate these policies into practice by introducing new malaria diagnosis and treatment strategies, long-lasting insecticidal nets (LLIN) as an alternative to indoor residual house spraying (IRS) with DDT, and new diagnostic technology and new drugs to treat kala azar. Polio: India joined the global polio eradication efforts in 1995 by starting the National Immunization Days (NIDs) Program. Polio eradication efforts progressed well until they suffered a set-back in 2002 with a major increase of polio cases in the state of Uttar Pradesh. This placed a substantial operational challenge as well as financial burden on the country due to steeply increasing operational and vaccine costs. Despite these set-backs, polio elimination remained technically and politically feasible in India. In May 2008, the India Expert Advisory Group (IEAG) for polio identified a substantial financing gap and expressed serious concerns about vaccine security. With limited world production of polio vaccine, the lack of multi-year secured financing made it difficult for the GOI to ensure adequate vaccine supplies. With India being one of only four polio endemic countries left in the world (at the time of project preparation), the 1 Bank agreed to support India’s effort to eradicate polio through provision of financial security for the multi-year purchase of polio vaccine, and expanded the project to include a polio component (original estimated cost US$271 million). 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The Project Development Objective (PDO) as stated in the PAD is “to enhance the effectiveness of government response to control malaria, eliminate kala azar and eradicate polio. This will be achieved by an increase in the number of people benefiting from effective prevention, diagnosis and treatment services for malaria and kala azar, and vaccination against polio”. The PDO in the Financing Agreement (FA) differs from the PAD: “to enhance the effectiveness of Recipient’s efforts to control malaria, eliminate kala azar and eradicate polio, by: (a) increasing the number of people benefiting from effective prevention, diagnosis and treatment services for malaria and kala azar; (b) strengthening central and state capacities for evidence-based policy development, strategic planning, and program management for effective control of vector-borne diseases; and (c) securing the timely supply of polio vaccines”. The disconnect between the two versions of the PDOs is that the FA includes an additional PDO (b) which more adequately reflects the project design since Project Component 3 (see below), and subcomponents 3a and 3b (but not 3c –M&E), deals with the strengthening of state capacities. The FA PDO also clarifies that the project is concerned exclusively with the supply of polio vaccine and not the process of immunization more broadly as may be implied in the PAD definition. The PDO key indicators are: For vector borne diseases: (i) Percentage of fever cases in project districts receiving a malaria test result no later than the day after the first contact; (ii) Percentage of individuals in project areas belonging to eligible LLIN target population who slept under an LLIN during the previous night. (iii) At least 50% of sampled blocks which at baseline have not achieved the elimination goal of less than one kala azar case per 10,000 persons, will achieve the elimination goal by endline. For polio: (iv) At least 80% of the households with eligible children covered during national and sub- national immunization days in high risk districts. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDO was not revised during the life of the project despite the differences between the PDO as articulated in the PAD and in the FA. 1.4 Main Beneficiaries The project was meant to benefit tribal people and the rural poor who are disproportionately affected by malaria and kala azar. By focusing on the geographic areas where malaria and kala azar morbidity and mortality were the highest, the project also focused on the poorest, most 2 remote and disenfranchised populations in India. Both diseases have strong but complex links with poverty, and the burden falls disproportionately on the poorest segments of the population. Within endemic areas, increased infection risk is linked to poor housing conditions and environmental sanitation, lack of personal protective measures and economically driven migration. Project beneficiaries also included all persons at risk of polio during the project (by definition those living in the most remote and inaccessible communities) and future generations who, once polio eradication was achieved, would never again be at risk from the disease. 1.5 Original Components (as approved) The project had four components with a total estimated cost of US$521 million. Component 1: Improving Access to and Use of Malaria Prevention and Control Services (total estimated costs US$119.5 million) This component was to finance centralized procurement of commodities and had three sub- components: (a) improve malaria case management by supporting rapid scale up of evidence based diagnosis and treatment polices; (b) strengthen malaria surveillance by supporting effective passive surveillance; and (c) rapid scaling-up of Long Lasting Insecticidal Nets (LLINs) and promoting integrated vector management. Component 2: Supporting Elimination of Kala Azar (US$41.9 million) This component was to finance the procurement of commodities and had three sub-components: (a) improve kala azar case management by supporting effective diagnostic and treatment services and ensuring treatment compliance; (b) strengthen kala azar surveillance by using sentinel sites complemented by active case search in locations where clustering of cases is noted; and (c) ensure effective kala azar vector control through piloting integrated vector management and supporting community mobilization. Given the concentration of kala azar geographically, forty- six (46) of the most endemic districts in 3 states were targeted for intervention. Component 3: Policy and Strategy Development, Capacity Building and Monitoring and Evaluation (US$52.1 million). This component of the project was designed to develop evidence based public health policies and strategies, and strengthening program management capacity and M&E. It had three sub- components, each with multiple associated activities as follows: (a) Policy and Strategy Development, through: (i) updating policies for prevention, diagnosis and treatment of vector-borne diseases based on evidence and the translation of these policies to operational guidelines, training materials and courses; (ii) improving quality assurance of, inter- alia, rapid diagnostic kits (RDKs) and microscopy; (iii) operations research including technology assessment of, inter-alia, different pumps for indoor residual spraying (IRS), RDKs for vivax malaria, and different delivery models in the private and public sectors; (iv) impact evaluation for malaria control; (v) periodic external reviews of the program; (vi) monitoring of drug resistance and insecticide effectiveness; (vii) assessment of quality of medicines; and (viii) the promotion and establishment of public-private partnerships (PPPs) for the distribution of state and district levels long-lasting insecticidal nets. 3 (b) Program Management and Capacity Building, through: (i) provision of additional state and district level staff for procurement, financial management, BCC, and M&E; (ii) deployment of additional entomologists at State levels; (iii) management training for state and district program teams; (iv) integrated vector management training for supervisors and State entomologists; and (v) training and exchange programs for national and state vector-borne disease control program managers. (c) Monitoring and Evaluation, through: (i) making the existing computerized management information system (MIS) functional, and monitoring and evaluation-related surveys and studies; and (ii) the strengthening of a geographic information system (GIS) for micro-stratification, planning, monitoring and advocacy. Contingency Fund for Vector Borne Disease Control (US$36.5 million) In addition to the funds allocated above for Components 1 and 2, additional funds were included in the project as unallocated to address unanticipated costs, such as more rapid scale up of project interventions such as LLINs or in the event that proposed finances from the GFATM for malaria control did not become available. Component 4: Supply of polio vaccine (US$271 million) The component was to finance the procurement of three types of oral polio vaccines required for coverage of eligible children under supplemental national and sub-national polio immunization rounds. 1.6 Revised Components Project Components were not revised during the course of the project although their scope was significantly reduced: 49.5% of the financing was cancelled by the end of project (see below). 1.7 Other significant changes Although the focus of the project did not change in terms of number of diseases targeted (malaria, kala azar and polio) and the concurrent focus on policy and strategy development, capacity building and monitoring and evaluation, the scale of the project changed significantly over its life due to delays linked to district readiness, HR recruitment and sub-contracting, financial management and procurement. A total of three restructuring exercises were carried out, two of which involved cancellation of a total US$258 million (49.5%) of the original US$521 million of IDA financing. See section 2.2 below for a detailed explanation of each restructuring. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry The project was the first centrally sponsored health operation following the Detailed Implementation Review (DIR) 2007 and was in conformity with the actions agreed under the Joint Action Plan between the Ministry of Health and Family Welfare (MoHFW) and IDA. Lessons incorporated into project design. Project preparation and design benefitted from the documented experience and lessons learned from the World Bank financed Malaria Control 4 Project (MCP) which was implemented between October 1997 and December 2005 and the findings and recommendations of the DIR. The following lessons were reflected in the project design: LESSON LEARNED PROJECT DESIGN Ownership by the states is essential for success States to review and approve district plans and provide implementation oversight to districts. Identifying and addressing procurement and Procurement to be centralized. UNOPS and supply chain problems upfront is critical UNICEF identified as procurement agents. Consultant agencies to be hired for supply chain management. Strengthening management at the state and Additional management support provided district levels is essential for successful (positions and training) and district readiness decentralization criteria established. External reviews and data triangulation are M&E framework included external WHO led necessary to maintain the program’s credibility reviews, household surveys, LQAS, sentinel surveillance and impact evaluations. Innovative approaches to prevention strategies Entomological surveillance to be strengthened. are needed IRS to be limited and LLINs introduced. Monitoring the therapeutic efficacy of anti- Support for drug resistance monitoring in 14 malaria medicines and insecticide resistance is regional sites and entomological units to essential monitor insecticide resistance. Both parasite and vector elimination is Support for integrated vector management and essential for kala azar elimination behavior change communications. Introduction of new treatment regimen. For polio [elimination] to succeed the timely Large scale procurement of three types of availability of quality vaccines is essential OPV. Project Phasing: The decision to implement the project in two phases was a design response to the identified critical risk that the new malaria case management policy did not have well established implementation models. The mitigation measure was to implement the project using a phased implementation approach with expansion subject to early implementation reviews. The districts included in phase 1 of the project were those with the greatest burden of malaria and kala azar. Technical Design. On a technical level, the project benefited from i) a close partnership with the WHO and other partners (including CDC, BMGF, GFATM, UNICEF, etc.) and active participation of Bank staff in the WHO-National Vector Borne Disease Control Program Joint Monitoring Mission in February 2007 (JMM-2007); ii) direct engagement of Bank staff in health sector dialogue broadly, and in discussions of policies and technical strategies for malaria control and kala azar in particular; and (iii) opportunities to engage with and build on the approaches (such as the national community health worker program, district level planning and new initiatives aimed at community mobilization and accountability) promoted under the National Rural Health Mission (NRMH) which was established in 2005. Ultimately, this led to the development of a project that aimed to bring malaria control and kala azar elimination into the 21st century by emphasizing the roll out of new policies and the introduction of new tools, technologies, service delivery and monitoring and evaluation strategies. Implementation of the new technical strategies was dependent on two important assumptions: 1) the general health services and a large cadre of village based volunteer providers would be able to effectively 5 deliver the services; and 2) there would be a consistent supply of quality assured pharmaceuticals, test kits and LLINs. Risk Assessment. As mentioned above, this project was the first health sector project in India to be approved after the 2007 DIR of five health projects in the India portfolio. The India DIR found significant indicators of fraudulent and corrupt practices in all five projects including, inter alia, a high risk of fraud and corruption in decentralized procurement (local contracting), a practice which was increasingly being used in Bank projects, but was subject to only minor procurement review. Therefore, during the preparation of this project, both the Bank and the Government of India were very concerned that steps be taken to ensure that risks of fraud and corruption were properly mitigated in project design. The DIR and the associated MoHFW-IDA Joint Action Plan had a significant impact on the design of the project, particularly with respect to procurement, financial management and implementation arrangements, including ensuring institutional strengthening arrangements at the central, state and district levels to address both operational and fiduciary concerns and the introduction of district readiness filters for decentralized funds. The DIR affected the way procurement was to be carried out under the project. Specifically, fiduciary safeguards recommended by the DIR were put into place including: (i) use the professional procurement agent for the procurement of goods at the central level, and (ii) exclude decentralized procurement from Bank's financing (until demonstration that the states had developed the necessary capacity). Lengthy project preparation period. The project was prepared over an extended period(2005- 2009) during which the Bank and the GOI were able to identify opportunities to position the project as a potential game changer with respect to the control of malaria and kala azar and an important contributor to the final push towards the elimination of polio. The project preparation period was prolonged in part by the need to conduct a Detailed Implementation Review (DIR) of the earlier Malaria Control Project (MCP) and other health sector projects and the subsequent development of agreements and measures to improve program governance and accountability in Bank financed projects. The PAD was significantly revised on two occasions. Following the Quality Enhancement Review (QER) on May 2, 2007, the scope of the project was narrowed from support for all VBDs nationally to malaria control and kala azar elimination in a select number of high burden states and district. The second major revision occurred during project appraisal and negotiation. The project was appraised on June 6, and negotiated June 10, 2008. Subsequent to negotiations, the Government of India requested that a component be added to the project to finance the purchase of polio vaccines from UNICEF. The Bank accommodated the request, added a polio component and US$271 million IDA financing, and renegotiated the project on June 20, 2008. The project was approved on July 31, 2008 and became effective on March 6, 2009 (see below the explanation for this effectiveness delay). Benefit of project reviews: To ensure good quality at entry, the project went through several reviews during the preparation phase including a QER in 2007 which helped to shape and focus the project on malaria and kala azar (rather than vector borne diseases more broadly) and limit the geographic scope to the districts with the highest endemicity for the two diseases. The project was also submitted for an Operational Committee (OC) Review cum Appraisal Review Meeting on May 28, 2008. In the OC review, the team specifically sought management guidance on issues related to risk and their mitigation and the balance between residual risk post mitigation and the potentially large benefits. The guidance from the OC review was incorporated into project design. 2.2 Implementation 6 The project became effective in March 6, 2009. The delay in effectiveness was due to the time required for obtaining GOI and parliamentary approvals, which took longer than expected due to national elections during the period. The technical and policy guidance by the World Bank team provided through the preparation and early implementation of the project, working closely with WHO, the National Institute of Malaria Research and its field officers, the Ministry of Health and Family Welfare (MoHFW) and the National Vector Borne Disease Control Program Directorate, led to key policy changes that were long overdue in malaria prevention and control in India. These changes include the adoption of Long Lasting Insecticidal Nets (LLINs), and case management at community level through the use of Rapid Diagnostic Tests (RDTs) and Artemisinin Combination Therapy (ACTs) by community based volunteers. Despite early progress with respect to policy and strategy development in the first two years, the following implementation challenges were reported in Aide Memoire from the beginning of project implementation throughout the life of the project: i) difficulty in recruitment and retention of field consultants due to issues of pay parity of project staff with NRHM staff at all levels, ii) delays in procurement of LLINs, pharmaceuticals and diagnostic kits, iii) delays in recruitment and payments to consultants who submitted their deliverables, iv) lack of decentralized funds for printing behavior change communication (BCC) materials, and v) limited use of monitoring data for targeting efforts. Mid Term Review (MTR) May 2-20, 2011 The MTR took place at a time when many of the issues confronted during the first two years of the project, including delays in district readiness, completion of the baseline household survey, recruitment of human resources, signature of consulting contracts and procurement of core commodities had been, at least temporarily, resolved and project implementation/performance was at its peak. . The MTR was conducted jointly with technical and development partners. The methodology included data analysis, the review of state reports, thematic reviews and in-depth field visits. The review concluded that there had been significant progress (with respect to prior performance) across the board in the implementation of the project, and particularly on program management, monitoring and evaluation, the social aspects and the implementation of the environmental management plan. It was also noted that disbursement increased to 20%, all delayed payments to suppliers under the project were being released, and all outstanding service contracts had been placed. The MTR noted that at the time of the review: 1) the malaria control program services were functioning in accordance with the national malaria control strategy. User friendly RDTs and pre-packaged ACTs were available, the health staff and the Accredited Social Health Activist (ASHA) were enabled to provide the diagnosis and treatment services, and there were no reported stock outs of RDT or malaria drugs. There was however variability in implementation progress across states with two of the phase one states, Jharkhand and Chhattisgarh, lagging behind the others; 2) the kala azar effort was showing progress, but a more vigorous effort was needed - especially in Bihar State, to overcome implementation constraints including the absence of operational guidelines, poor quality and sub-optimal indoor residual house spraying and stocks of obsolete drugs, and; 3) the procurement of oral polio vaccines under the project continued to be smooth after amending the Financial Agreement to allow M/s RITES to serve as procurement agent addition to UNICEF. The review also identified a large number of challenges and associated actions. Cross cutting issues included: the recruitment, retention, supervision and mobility of human resources and highlighted the problem of lack of pay parity; problems associated with procurement, commodity logistics and contract management, including technical specifications and prolonged delays in processing central procurement, and; financing and financial management including problems associated with the standard costs approach to decentralized financing, timely release of funds; delayed or non-payment of salaries to 7 consultants and lack of clarity on incentives for community level service providers (ASHA) and patients. Following the MTR, the ratings for both environmental and social safeguards were raised from “unsatisfactory” to “moderately satisfactory” and the overall IP rating was raised from “moderately satisfactory” to “satisfactory”. Project Restructuring The project was restructured three times. None of the restructuring exercises resulted in a change in the PDO, components, indicators or targets. Level Two: Amendment to the FA for procurement of Oral Polio Vaccines. Approved on 16-Dec- 2010: The Ministry of Health and Family Welfare (MoHFW) expressed concern over the perceived high vaccine prices, as well as the high management fees charged, when OPVs were procured through UNICEF. The GOI therefore requested that the Bank allow M/s RITES (the MoHFW procurement agent) to undertake OPV procurement. The Bank agreed to allow M/s RITES (or any other professional procurement agent selected per the Project procurement arrangements) to undertake procurements of OPV, with the proviso that any such arrangement provide supply security and flexibility to meet the program’s requirements and be consistent with the agreed quality assurance (QA) standards *i.e., that they be WHO pre-qualified). The FA was amended to allow payment to agencies other than UNICEF who could procure OPVs under the project. Level two: Cancellation of US$158 million of IDA Credit. Approved on 19-Dec-2012: Based on an assessment and recommendation by the Bank team, the GOI requested the cancellation of US$184 million of IDA Credit. The amount cancelled was US$158 million which was lower than the requested amount as the cancellation request did not take into account unpaid liabilities for goods and services under the polio component of the Project. This included cancellation of (a) US$70 million from the vector borne disease control components of the project due to chronic delays in central procurement of commodities; and (b) US$88 million from the polio component as the GOI elected to source polio vaccines from domestic suppliers with GMP certification but who were not WHO pre-qualified. Level Two: Cancellation of US$100 million. Approved on 31-Dec-2013 The GOI requested a cancellation of US$100 million IDA Credit (from the VBD components) of the remaining Credit amount of US$101.4 million. The project had been slow to disburse under the VBD components primarily due to delays in the central procurement of Long Lasting Insecticidal Nets (LLINs) and some other key diagnostic and treatment related commodities. Since the project was scheduled to close on December 31, 2013, it was unlikely that these procurements would be completed in time even if a rebidding process using the existing government system were to be initiated. Prior to the request for cancellation, the GOI had requested an extension of the project which was not accepted by the Bank. After cancellation, approximately, US$1.4 million remained with the Directorate to fulfill the remaining contractual commitments under the project. Summary: Key Factors Affecting Implementation (positive and negative) and response/action taken Procurement: Despite constant support from the Bank team, procurement delays for goods (medicines, diagnostic kits, insecticide products) and services (competitively bid contracts) and 8 the inability of the Borrower to conclude procurement of critical project inputs, particularly LLINs, prior to project closure severely limited the success of project implementation and the efficiency with which the project was executed. These delays resulted in the cancellation of project financing. The procurement aspects of the project are discussed in detail under section 2.4 Safeguards and Fiduciary Compliance. Financial Management: A standard cost approach to disbursement for decentralized expenditures was a key project innovation. The financial management arrangements put in place for decentralized expenditures however, posed several challenges during implementation, including a poor understanding of the approach by states and districts, delays in district readiness and delays in implementing integrated implementation and fiduciary reviews. In response to indications in the integrated review reports that the decentralized expenditures were significantly lower than the standard costs, the Bank requested the project team to submit state annual audit reports and reconcile standard costs with actual expenditures. The financial management aspects of the project are discussed in detail under section 2.4 Safeguards and Fiduciary Compliance. Human Resources: The successful implementation of the project was dependent on the recruitment of qualified human resources at all levels with the ability to implement and monitor the program. Project financing was used to staff key positions and cadres of personnel at central state and district levels as well as training at all levels of the health pyramid. Domestic financing supported the expansion of human resources, particularly multipurpose health workers and Accredited Social Health Activists (ASHA) at the block and village levels. The project and the program achieved a qualified success in building human capacity despite difficulties in recruiting and retaining consultants and staff and non-competitive compensation. Human resource issues and challenges are discussed in greater detail under section 3.2 Achievement of Project Development Objectives. Monitoring and Evaluation: M&E arrangements affected project implementation in both positive and negative ways. This is discussed in detail in section 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Indicators The Project Data Sheet as included in the ISRs (and the present ICR) provides information on all four Project Development Objective (PDO) level indicators and a sub-set (7 out of 13) of intermediate outcome (IO) indicators included in the PAD and the Financial Agreement. It also provides data for two World Bank core indicators that were not in place at the time of project appraisal. Some indicators are worded differently in the Results Framework, the Financial Agreement and the Project Data Sheet. The number of IO level indicators included in the Results Framework is high and the description of indicators is generally weak. These deficiencies were recognized by the Bank team and some adjustments and clarifications were made to obtain the best available data but there was no formal change in the indicators made. Despite these improvements, the following weaknesses remained:  Numerators and particularly denominators were not well defined. Numerators and denominators lack specificity and were not quantified, some of the denominators (from different indicators) appear to overlap and may or may not be interchangeable. These 9 include: population in high risk project areas; eligible districts; endemic districts; identified endemic areas; and targeted endemic areas.  There is a lack of definition and consistency with respect to some of the interventions being measured. For example, some indicators specify long lasting insecticidal nets (LLIN) and others use insecticide treated nets (ITN) which includes LLIN as well as conventional nets treated (or retreated) with an insecticide prior to or following distribution.  Some of the indicators measure the same or overlapping concepts. One PDO and two IO level indicators were designed to measure the coverage and/or use of insecticide based interventions for the control of malaria and/or kala azar. Only the PDO level indicator on LLIN use was monitored by the project, although the data reported did not come from household surveys as intended.  There are several compound indicators in the Results Framework that are either hard to measure or hard to interpret. “Percentage of planned additional staff that is in position at central, state and district levels and received induction training” is actually a composite of six separate indicators and of little value for project monitoring unless disaggregated. Only one of the compound indicators was routinely monitored..  Two of the PDO level indictors (LLIN use and treatment within 24 hours) correspond to the Roll Back Malaria (RBM) core indicators for malaria control; however, there are important deviations from the international standard. In both indicators, age group is not specified while the international standard is to report the indicators with reference to children less than five years of age for the purpose of international comparison. For the treatment indicator, the RBM core indicator attempts to measure treatment with an effective antimalarial (ACT) within 24 hours on onset of symptoms (fever) whereas the project reported on ACT treatment no later than on day after first contact, presumably meaning within 24 hours of providing a blood sample for diagnosis. M&E for Polio In order to attain elimination, the India Polio Eradication Program had put into place a very effective system for monitoring and surveillance over the previous decade (since joining the WHO launched Global Polio Eradication Initiative in 1995). The polio monitoring and surveillance system uses global standard indicators and engages independent monitors to constantly monitor the state of polio eradication. As the project was engaged only in financing the three types of oral polio vaccine required by the program, the project included one PDO level indicator on vaccine coverage which was validated by the independent monitors (including WHO and CDC) engaged by National Polio Surveillance Project (NPSP) and one results level indicator on stock outs of OPV for supplementary immunization activities which was available from routine program reports. M&E for Vector Borne Diseases For the vector borne disease control components of the project, the approaches to monitoring and evaluation were more ambitious and complex, involving: changes to existing data collection tools; the introduction of indicators and data collection strategies that were new to the NVBDCP; and engagement of multiple actors and institutions for proper execution. During project preparation it was recognized that there were considerable human resource capacity constraints for monitoring and evaluation at both central and state level which would need to be addressed through recruitment and training of M&E specialists, external technical assistance and outsourcing. 10 The total number of individual activities in the monitoring and evaluation plan is very large, but can be grouped into nine key activity clusters: (a) strengthening the health management information system, including the introduction of streamlined formats for data collection; (b) establishing sentinel surveillance for malaria deaths and kala azar hospitalizations and case fatality rates in district health facilities with high case loads of malaria or kala azar; (c) facility and population-based surveys, including cross-sectional household surveys (at baseline, mid- project and endline) and annual surveys using the lot quality assurance sampling (LQAS) approach; (d) creation of a logistics management information system (LMIS) to track commodities; (e) quality assurance monitoring for drugs and diagnostics; (f) monitoring drug and insecticide resistance; (g) pharmaco-vigilance using a protocol of prospective monitoring and drug susceptibility testing for ACTs and Miltefosine; (h) operational research and impact evaluations; and (i) joint technical program reviews led by the WHO. Each of the M&E activity clusters contained multiple specific actions to be undertaken by a variety of actors under the supervision and stewardship of the National Vector Borne Disease Control Program. The ambitious plans for monitoring and evaluation were not fully realized and the project was unable to generate the data required to effectively measure the malaria related indicators at PDO or IO level. Due to weak capacity in M&E and chronic delays in contracting, only the baseline household survey was completed and the approach for using LQAS data to generate point estimates for outcome indicators (Large Country LQAS or LC-LQAS) was not applied according to the proposed protocol. One exception may be in the state of Odisha where the process received external support from the Liverpool School of Tropical Medicine and the former Bank staff who developed the approach. Elsewhere, the primary function of LQAS which was to monitor coverage and use of project interventions at the PHC level, and the surveys were only carried out in districts and blocks that had the required inputs for program implementation. The sample of districts and blocks was the result of purposive rather than random selection and cannot be used to generate aggregate measures for the entire target population. Some elements of the monitoring and evaluation plan were successful and contributed to improve the quality and quantity of data collected as well as the technical and allocative efficiency of the Program, such as: • Technical assistance provided or financed by the Bank resulted in new and greatly simplified record keeping and reporting formats for case management and vector control activities, collection of data by community level providers and some progress toward integrating the VBD reporting system with the HMIS system under NRMH. The epidemiological data generated by this system provided credible evidence that there was a decrease in morbidity for malaria and kala azar. However, efforts to further automate the system were largely unsuccessful. • Two sentinel sites were identified in all of the project districts and received support to improve diagnostic services. A majority of sites (303 out of 348 identified under the Bank project) were at least partially functional at Closing and routinely reporting to state officials. • LQAS was (and continues to be) used for monitoring program implementation at sub district level and is valued by project staff responsible for tracking district and local implementation of malaria control and kaal-azar elimination. Aggregated LQAS data were generated routinely but were not adequate for tracking project indicators. • A general management consultancy firm, Strategic Alliance Management Services (SAMS) was engaged to strengthen supply chain management and logistics for malaria control and kala azar elimination and was successful in developing a functional, partially automated, logistics management information system (LMIS) and in improving the overall system in terms of quantification, routine reporting and tracking and monitoring 11 commodities with respect to their physical distribution, and expiration. The approach taken to the management of short-expiry commodities is an example of allocative efficiency in the project. • Operations research activities for monitoring the therapeutic efficacy of and adverse reactions to newly introduced drugs, insecticide resistance monitoring, and the quality assurance of rapid diagnostic kits for malaria and kala azar diagnosis were managed by the National Institute of Medical Research and clearly contributed to the quality and technical efficiency of the project. For example, decreasing therapeutic efficacy of the co-blistered ACT combining artesunate and sufadoxine-pyrimethamine (AS+AQ) was detected in malaria endemic states in the north-east and the treatment protocol was changed to co-formulated artemether-lumefantrine (AL) in response. Similarly, systematic quality assurance of RDTs, particularly important during the earlier years of the project (when there was significant quality variance between products as well as batches), was able to detect and eliminate from use, poorly performing batches of tests. • An impact evaluation was conducted to assess community perceptions on malaria care- seeking practices and service delivery through supportive supervision and community mobilization. • Two large scale reviews, referred to as WHO – NVBDCP Joint Monitoring Missions (which included the Bank and a large array of other development partners and stakeholders)framed the project implementation period and contributed significantly to project design and evaluation. JMM 2007 contributed to the technical design of the project and facilitated the introduction and roll out of new policies and technologies through the project. JMM 2014 was conducted concurrently with the World Bank ICR mission with full participation of the ICR team. It is also notable that WHO experts and other partners participated in the semi-annual project reviews conducted by the World Bank and borrower as part of routine project supervision.       2.4 Safeguard and Fiduciary Compliance The safeguard policies triggered by the project included: Environmental Assessment (OP/BP/GP 4.01); Pest Management (OP 4.09); and Indigenous peoples (OP) (BP 4.10). All three aspects of safeguards are rated as moderately unsatisfactory for compliance due to slow implementation progress despite strong policy foundation for the safeguards being in place. Environment: The project was classified as environmental safeguards category B, as it involved the use, storage, transportation and disposal of insecticides in various applications. The project also involved the introduction of rapid diagnostic tests which produce medical waste in the form of biologicals and sharps. During project preparation, field visits and assessments revealed that pesticide management was inadequate and that attention to worker safety was weak. With technical assistance from IDA, the National Vector Borne Disease Control Program (NVBDCP) prepared an Environmental Management Plan (EMP) which defined the critical steps required to strengthen pesticide management and the disposal of medical waste. The EMP was a framework with guidelines to be adapted and incorporated into practice by the states. The EMP and associated guidelines were communicated to the States by the NVBDCP. In order to facilitate implementation of the EMP the project provided for the recruitment of an environmental consultant at the central level, one entomologist for each state and malaria technical supervisors at district level. The consultant contract to support EMP implementation was signed with M/s Senes Consultants India Pvt. Ltd on March 26, 2011 – more than two years after project effectiveness. This firm was engaged to assist NVBDCP in 1) strengthen guidelines for the 12 handling and use of insecticides; 2) capacity building in environmental management at state, district and sub-district levels; and, 3) monitor and report on the implementation of EMP activities as per the required regulations and guidelines at all levels in project districts. The firm undertook an assessment of issues and practices and developed and disseminated an Environmental Code of Practice (ECoP) and associated pictorial training modules through workshops and various training activities. The firm also monitored and reported on indoor residual house spraying (IRS) rounds in 2012 and 2013. Although the EMP, the Environmental Codes or Practice (ECoP) and training materials form a solid basis for improving the management of insecticides and medical waste, in practice there are still major challenges. Observation made by Senes as part of its monitoring and reporting function, the Bank’s assessment of compliance with environmental safeguards during the final Project ISR and observations from the JMM 2014 and ICR review were consistent: (i) personal protection equipment (PPE) for spray workers (gloves, aprons and masks) are now being provided by the DDT manufacturer, (ii) DDT is now being supplied in 25kg rather than 50kg bags which facilitates transport and handling and reduces waste, and (iii) the ECoP and pictorial training materials may be considered international best practice. However, PPE is not consistently available and there is a need for BCC to consolidate worker willingness to use PPE. The availability of storage space, the conditions of storage and the disposal of insecticides is below prescribed norms. Training of spray teams and community mobilization in advance of IRS rounds has been inconsistent across districts and over time, compromising the quality, safety and coverage of control efforts for both malaria and kala azar. Efforts to reduce reliance on DDT for IRS through improved insecticide management practices, the introduction of synthetic pyrethroids for spraying and promotion of LLIN as an alternative to IRS has been patchy largely due to the delays in the procurement of commodities. The JMM 2014 also noted that the management of medical waste was sub-optimal and noted during field visits that universal precautions, particularly the use of latex gloves whilst taking and processing blood samples for diagnosis were not being implemented. Social Safeguards: Tribal people and other vulnerable communities in India bear a disproportionate burden of vector borne diseases and have low access to health care and other services. They are socially and economically disadvantaged suffering from poverty, low levels of education, social exclusion, isolation from the mainstream economy and often live in areas with little or no service delivery infrastructure. A social and beneficiary assessment (SABA) was carried out during project preparation to engage tribal communities in project design. The outcome of the assessment and consultation was a Vulnerable Communities Plan (VCP) which was meant to be integral to the project’s overall implementation and the GOIs project implementation plan (PIP) and Operational Plan. The NVBDCP engaged the Voluntary Health Association of India (VHAI) to enhance the effectiveness of malaria control and kala azar elimination through social mobilization and service delivery amongst the vulnerable communities in project states. The main objectives of their support was to (i) mobilize vulnerable communities for availing timely and appropriate preventive and curative services; (ii) to build capacity of the peripheral health workers involved in malaria control/ kala-azar elimination to sensitize on the needs of vulnerable communities; and (iii) strengthen service delivery in selected villages in high endemic districts. The contract was signed on February 2, 2011 –almost two years into the project implementation period. In parallel, on January 28, 2011, the NVBDCP awarded a contract to M/s New Concept to develop a 13 behavior change communications (BCC) strategy, implementation plan, tools, messages and materials. The Moderately Unsatisfactory rating for implementation of the indigenous peoples safeguard is due to slow progress in implementing the VCP. The issue was highlighted during the implementation support mission in September of 2010 and repeatedly flagged during subsequent missions throughout the project period. Although at times it was reported that some progress was being made, the assessments were qualified by statements indicating key issues remained and needed to be urgently addressed. The development of the BCC strategy, implementation plan and associated tools progressed fairly well, however actual implementation by the contracted consultants, states and districts was never fully realized. This is in part attributable to slow decision taking and poor contract management by the MoHFW and the NVBDCP which delayed payments for completed deliverables, fund release and approvals of the implementation plans of the contracted agencies. The implantation of the VCP was never evaluated and documented, despite repeated requests from the Bank. The final implementation support mission in November 2013 reported that the implementation of the VCP was less than satisfactory and that there was “no indication that information, education and communication (IEC) or behavior change communications (BCC) activities were being implemented on the ground.” During the last year of the project, the NRHM took the decision that the implementation of these activities should be the responsibility of the states and that the terms of reference (TORs) for the contracted agencies should be modified to provide capacity building and monitoring support at district and block levels. This decision had not been formally approved by the MoHFW at the time of the ICR review and no action had been taken with respect to the contractors’ TORs. Fiduciary Overall, fiduciary compliance is rated as moderately unsatisfactory. Both procurement and financial management were down-graded from moderately satisfactory to moderately unsatisfactory at the end of the project implementation period. Procurement was downgraded due to chronic delays in the procurement of critical project commodities which resulted in the cancellation of project financing and failure of the project and the NVBDCP to provide services, particularly malaria prevention based on use of LLIN, to the target population. Financial management was downgraded primarily due to repeated delays in obtaining the annual audited financial statements from the states and the FY12-13 audit report for the vector borne diseases components of the project. The ICR team concurs with the ratings, but notes that it finds procurement performance under the polio component of the project satisfactory while for the VBD components the rating would be unsatisfactory. As the project was very procurement intensive (almost 90% of the project cost was proposed to be used for procurement contracts), many risk mitigation measures were put in place. The project excluded decentralized procurement and required that all central procurement of goods and services be managed by a professional procurement agent acceptable to the Bank. Decentralized expenditures were limited to contractual staff, mobility and training costs and were provided on a predetermined standard cost basis with provisions for external integrated implementation and fiduciary monitoring as well as an enabling clause in the Project Financing Agreements allowing the Bank to request the state annual audit reports as part of due diligence. a. Procurement 14 During the first two years, procurement under the polio and VBDCP components was relatively effective. Polio vaccines were procured through UNICEF and then subsequently through m/S RITES. Between 2009 and 2012 procurement performance was good and US$167 million of the allocated US$271 was spent on the three types of oral polio vaccine (OPV) required by the Polio Eradication Program. Consistent with Bank requirements, OPV was only purchased from WHO pre-qualified manufacturers during this period. However in 2011, the MoHFW decided to drop WHO-prequalification in favor of good manufacturing practice (GMP) as a selection criterion in order to obtain lower pricing from vaccine manufacturers in India. As this approach was not consistent with Bank procurement requirements, the GOI elected to use domestic resources to finance vaccine purchase and requested cancellation of the remaining US$ 88 million of Bank financing for Component 4 of the project. The procurement performance of the vector borne disease component was more problematic. Initially, the United Nations Office for Project Services (UNOPS) was contracted to procure malaria and Kala Azar commodities, such as LLIN, synthetic pyrethroid insecticides, diagnostic test kits, pharmaceuticals and other commodities required by the program. However, procurement started deteriorating from 2011 when RITES replaced UNOPS as the procurement agent at the insistence of the client. At this time the tendency toward risk adverse behavior and inaction with respect to procurement was heightened. The Controller and Auditor General (CAG) heavily criticized the procurement process used in some non-Bank projects, resulting in investigations/sanctions against some bureaucrats and procurement officials). This in turn prompted project officials as well as the procurement agent to take a risk-averse approach, which resulted in substantial delays in procurement process. As a result many important commodities (notably LLINs) could not be procured before the closure of the project and funds allocated for this component were largely cancelled. Near the end of the project, to mitigate the delays in central level procurement, the Bank agreed to procurement of LLIN by some of the states but even this was not permitted by MoHFW. As a result only about US$43 million was used in vector borne disease control components of the project for procurement of goods and 68% of the allocated financing for these components was cancelled prior to project closure. The procurement challenges identified in the review of the project were also reflected in the preliminary findings of the 2014 Joint Monitoring Mission of the NVBDCP and the WHO, which examined both central level procurement and decentralized procurement (which was originally disallowed under the Bank’s Financing). The JMM noted that there has been continuous failure in the procurement of LLINs for three years (2011-2014) and delays in procurement of other items (pharmaceuticals, rapid diagnostic test kits, etc) at central level noting that some supply side factors and “apprehension of collusion” contributed to the delays. The report also noted that at state level, decentralized procurement of the core commodities –drugs, diagnostics and LLINs was also unsatisfactory resulting in delays, higher costs or concerns about quality. It was further noted that UNICEF provided assistance in some states for the procurement of LLIN using non- Bank financing. b. Financial Management Central level expenditures, including vaccines, were to be reimbursed on six monthly/annual basis and validated annually through audit reports. Although decentralized procurement was disallowed, project implementation did require the financing of decentralized expenditures. The essential categories of expenditure at State and District level were defined as contractual staff, 15 mobility (vehicles and transport) and training. In lieu of reimbursements based on detailed audits of state and district level accounts, an approach to financing decentralized expenditures based on predetermined standard costs was adopted. The standard cost approach provided the benefits of financing for critical activities at the State and District level without the need to maintain separate records for Bank funds or undergo separate audits, thereby reducing transaction costs for the borrower. The benefits to the Bank were an opportunity to test an innovative approach that responded to client needs and promoted greater collaboration between fiduciary and program teams to meet project objectives. For the decentralized expenditures financed through standard cost mechanism it was proposed that fiduciary assurance would be derived primarily through “integrated implementation and fiduciary review” by an external consultant and IDA supervision missions. The TORs for the implementation and fiduciary review of decentralized activities were discussed and agreed during project negotiations (9-13 June, 2008). The approach was intended to intensify field level input and output monitoring for a sample of project districts on a semi-annual basis, coupled with a review of project implementation. The objective of this monitoring was to document that under the standard cost mechanism (i) activities were being executed; (ii) states and districts were able to deliver the planned outputs; and (iii) the actual expenditures of the decentralized activities financed by the Bank were broadly comparable with the estimated standard costs. Implementation experience, as evidenced by the Aide Memoires and ISRs indicates that the financial management arrangements for the project remained valid and reflected `Moderately Satisfactory’ FM performance ratings through a large part of the project life. Delays in submission of audit reports for central level expend were noted but did not significantly impact the quality of fiduciary assurance arrangements. For the payments to suppliers under the Polio component, MoHFW opted for the direct payments method of disbursement. For each such direct payment made by the World Bank to the suppliers at the behest of MoHFW, complete documentation including copies of (a) Bank’s no objection fax message, (b) notification of award; (c) original invoices; (d) consignee acknowledgement receipt; (e) original packing note; (f) airway bills; (g) Manufacturer Warranty Certificate; (h) self-attested internal test analysis report of vaccines; (i) insurance and inland transit policy certificate; and (j) supplier’s certificate of origin were provided to support the payment requests. These due diligence measures were supplemented by third party verification of the utilization of the OPV in the field through WHO household surveys. Based on this, an exception was obtained from Financial Management Operations Review Committee on 08-Oct-2013 for submission of annual audit reports for the Polio component. The financial management arrangements put in place for decentralized expenditures however, posed several challenges during implementation. To start with, there was a delay in the recruitment of the consulting firm to carry out the integrated implementation and fiduciary reviews [first report received in early 2011]. Secondly, district readiness took much longer than expected and at some time during the project there was a redistricting which led to an increase in the number of districts in the project, and presumably costs. Thirdly, although the Bank team made repeated efforts to explain the concept and associated processes, the standard cost approach was poorly understood and implemented, particularly at State and District levels where the resources were perceived to be rigidly allocated for very narrowly defined uses within the three expenditure categories of staffing, mobility and training. Standard costs for salaries became ceilings and were not adjusted annually leading to problems with recruitment and retention as the salary levels were not competitive even with the NRHM. Inability to ensure commodity supplies at the field level exacerbated the staff motivation levels, resulting in significant levels of vacancies in staff positions. The need for greater flexibility for decentralized units to move funds 16 across categories and to broaden definitions was noted in several Aide Memoires. In response to indications in the integrated review reports that the decentralized expenditures were significantly lower than the standard costs, the Bank used the enabling clause in the Project Financing Agreements to request the project to submit state annual audit reports. This was part of the due diligence efforts initiated by the Bank to establish adequacy of the expenditures with respect to the standard costs. This exercise, however, had its own challenges; and included (a) delays in submission of state audit reports; and (b) difficulties in establishing the actual expenditures, as these were not easily identifiable in the financial statements. This exercise took considerable time and effort in terms of triangulation of information from several different sources (e.g., Utilization Certificates, statements of expenditures, and annual audit reports) and was concluded only in the last year of the project. 2.5 Post-completion Operation/Next Phase At the time of project closure there were no plans for a next phase or follow-on project, although the need for addition support, especially for malaria control, is high. Vector Borne Diseases The NVBDCP is a part of the National Health Mission (NHM) which is a high-priority GOI program that has been receiving an increasing budget allocation under successive five year plans. At the time of project preparation, 72% of the NVBDCP budget was expected from domestic financing with the remaining 28% from IDA and the Global Fund for AIDS, Tuberculosis and Malaria (GFATM) who were the only external donor partners of the program. Discussions were held with the Directorate of the NVBDCP and the MoHFW on the sustainability of the project interventions beyond project closure, with particular emphasis on the need to ensure that systems set up under the project and lessons learned are appropriately embedded in the NVBDCP and NMH as part of the implementation of the 12th five year plan (2012 to 2017). Given the increasing support from the GOI to the health sector and a focus on fully integrating administrative procedures under the NHM, the MoHFW has confirmed its intention to continue to fund positions funded under the Bank-financed project at central, state and district levels until the end of March 2014 and that a new proposal was made by the MoHFW for the continuation of this funding until the end of the 5-year plan (2012-2017). Although these positions are likely to be sanctioned by the MoHFW, many will remain vacant until policies promoting pay parity, at least within the NRMH, are implemented. As the project ended with a fairly substantial unfinished agenda with respect to the vector-born disease control components, it is important to note that the NVBDCP has indicated that several unfinished activities will be taken forward with domestic financing such as the end-line household survey. Furthermore, the states have started taking on responsibility for the implementation of the IEC and BCC activities and making use of the “innovations” developed under the project, although there is still a need for capacity building and monitoring support at the district and block levels. During project preparation and implementation, the Bank promoted donor coordination and the harmonization of support for malaria control and kala azar elimination. The Bill and Melinda Gates Foundation (BMGF), UK Department for International Development (DFID), the Global Fund, UNICEF and WHO participated in joint missions and provided technical guidance in a harmonized manner to ensure consistency across the programs run by the NVBDCP. Going forward, partners have agreed to map the implementation and technical gaps and attempt to mobilize appropriate technical and financial support for successful program implementation 17 throughout the country. There is preliminary evidence that the Global Fund may be able to address a portion of the gap in support for malaria control under their new implementation model and that the BMGF and other development partners will continue to support the elimination of kala azar. Polio With the elimination of polio from the borders of India, the focus of the National Polio Eradication program is now shifting to the prevention of re-introduction of the polio virus. “The risk of polio persists in India, with continuing poliovirus transmission in neighboring countries and outbreaks across the World. Globally since 2000, as many as 49 countries that had been polio free have suffered from one or more importations of wild poliovirus.” 2 To mitigate the risk of importation from across the border, vaccination posts have been set up at 102 locations along India’s international borders. The polio program network is also turning its attention to broader health issues in India and the state governments are capitalizing on the polio surveillance and social mobilization networks to strengthen routine immunization and convergent health initiatives. 3. Assessment of Outcomes Overall, the project is rated as Moderately Unsatisfactory because only two of the four PDO indicator targets were fully achieved (see discussion below). As mentioned earlier, project implementation was plagued by delays while the project design was ambitious, complex and was highly dependent on central procurement and the successful implementation of new approaches to monitoring and evaluation. Procurement delays for both goods and services and the failure of the project to conduct an adequate baseline survey and the complete absence of a mid-term or end- line surveys resulted in chronic implementation delays, low intervention coverage and poor documentation of progress –particularly for the malaria control component of the project. As a result of these delays, US$258 million (49.5%) of the US$521 million of IDA financing allocated to the project was cancelled. For VBD, there is epidemiological evidence suggesting that the project made a positive contribution to malaria control, kala azar elimination and capacity (primarily a temporal correlation between the availability of project inputs and concurrent downward trends in disease incidence and mortality), however the project did not fulfill its potential to contribute to malaria control and kala azar elimination. And, in the absence of end-line household surveys, the constructed values (derived from LQAS data not intended for this purpose) for the two malaria specific PDOs and several of the intermediary results cannot be accepted as reliable estimations. Still, the polio component of the project made a sizable and timely contribution to vaccine security during the “end-game” of polio eradication in India and clearly contributed to achievement of the eradication objective. 3.1 Relevance of Objectives, Design and Implementation Objectives: 2 “India’s Story of Triumph over Polio: reaching every child with two drops of life” The United Nations Chidren’s Fund (UNICEF), 2014. P.191. 18 The relevance of the overall project development objective - “to enhance the effectiveness of government capacity to control malaria, eliminate kala azar and eradicate polio” - was high and responded directly to the expressed needs of the borrower. India’s National Health Policy (2002) called for reducing malaria mortality by 50% and eliminating kala azar by the year 2010, and the GOI joined the global polio eradication efforts in in 1995. The World Bank had been assisting the GOI in its efforts to combat malaria since 1997 and to eradicate polio since 1999. The Bank’s Country Assistance Strategy (CAS) for India 2005-08 identified the control and, when feasible, the elimination of infectious diseases as a priority for IDA assistance, and noted its importance for the poor disproportionately affected by such diseases. Project relevance is further supported by the CAS 2009-2012 which emphasizes the importance of assisting India to achieve the Millenium Development Goals (MDGs) and to increase the effectiveness of service delivery in the health sector , particularly among the poorest states. At the time of project preparation the most deadly strain of malaria (p. falciparum) was on the rise due to high levels of resistance to chloroquine; kala azar cases were concentrated in a small number of states and districts and India reported more than half of all polio cases globally and was one of only four counties that had not successfully eliminated polio. The project objectives also aligned with India and the Bank’s commitment to meeting the Millennium Development Goals (MDGs) and the Bank-wide commitment to malaria control, as adopted in the Bank’s Global Strategy and Malaria Control Booster Program in 2005. Design: Project design was highly relevant but complex and based on several high risk assumptions. The project sought to capitalize on: the launch of the National Rural Health Mission (2005) which aims to provide affordable and effective health care to all citizens, in particular to the poorer and most vulnerable sections of the population; the GOIs commitment to strengthen national capacity for the control of vector borne diseases, including malaria and kala azar; significant shifts in policies and strategies for vector borne diseases which involved the transition to more effective tools for the prevention, diagnosis and treatment of malaria and kala azar; and an opportunity to provide financing security for OPV to assist the GOI achieve polio eradication. Three of the four project components were associated with vector born disease control: one focused on malaria control, the second on kala azar elimination and the third on capacity building at all levels of the health system for implementing the new policies and strategies. The fourth component was simply financing for the procurement of OPV. The malaria and kala azar components were to be implemented in two phases to accommodate a learning curve associated with the introduction of new policies and technologies. These components and the plan to implement in stages were logical given the assessed needs of the programs. The necessary ingredients for success were all identified and plans were made to put them in place. However, in retrospect, while the focus of the project design was highly relevant, the design of project implementing arrangements seemed overly dependent on centrally procured commodities; constrained by lack of flexibility particularly with respect to district readiness filters and decentralized funding; overly ambitious in including a second phase of implementation; and burdened with an overly complex approach to monitoring and evaluation. Given that the success of the project depended heavily on the design of these arrangements, the overall relevance of design is downgraded from High to Substantial. Therefore, on balance, the relevance of the objectives and design is rated as Substantial. 3.2 Achievement of Project Development Objectives 19 The project development objective is “to enhance the effectiveness of Recipient’s efforts to control malaria, eliminate kala azar and eradicate polio.” In accordance with the FA version of the PDO (which this ICR deems as the more fitting version given project design), this was to be achieved by: (a) increasing the number of people benefiting from effective prevention, diagnosis and treatment services for malaria and kala azar; (b) strengthening central and state capacities for evidence-based policy development, strategic planning, and program management for effective control of vector-borne diseases; and (c) securing the timely supply of polio vaccines. As mentioned in section 2.3 above, incomplete reporting and the absence of mid-term or end-line data from planned but unrealized population-based household surveys made it difficult to assess the adequacy of progress towards achievement of the PDO, particularly with respect to malaria control. PDO, Part (a): increase the number of people benefiting from effective prevention, diagnosis and treatment services for malaria and kala azar. MALARIA: Improve access to and use of malaria prevention and control services. PDO indicator 1: Percentage of fever cases in project districts receiving a malaria test result no later than the day after the first contact. TARGET 70%. Indicator was not reliably measured due to partial baseline survey and no follow-up surveys. Aggregated LQAS data (not intended to measure this indicator) suggest that the weighted average of all communities sampled in five Phase 1 project states was approximately 54.3% ( range 38-92%). In two states, the data from sampled communities showed that the target had been exceeded. PDO indicator 2: Percentage of individuals in project areas belonging to eligible Long Lasting Insecticidal Net (LLIN) target population who slept under an LLIN during the previous night. TARGET 50% Indicator was not reliably measured due to partial baseline survey and no follow- up surveys. Aggregated LQAS (not intended to measure this indicator) suggest that the weighted average of all communities sampled in five Phase 1 project states was approximately 28.6%% ( range 20-43.5%). None of the states achieved the target. Malaria remains a significant public health problem in India. While about 95% of the population resides in malaria endemic or malaria risk areas, where epidemics may occur, 80% of reported cases are confined to tribal, hilly, difficult and inaccessible areas with only 20% of the population. The approach to malaria control taken under the project was to promote implementation of new policies and technologies for the prevention (introduction of long-lasting insecticidal nets (LLIN) as an alternative to indoor residual spraying with DDT or synthetic pyrethroids), diagnosis and treatment of malaria (introduction of community-based diagnosis using quality assured rapid diagnostic tests (RDT) and treatment of P. falciparum positive cases with an artemesinin-based combination treatment (ACT) positive cases in 124 districts with the highest burden of the disease. The JMM 2014 concurred with the overall malaria strategy noting that it is based on a correct combination of effective case management and anti-vector measures, differentiated and targeted according to the local epidemiological, entomological and operational situations. The JMM also noted that malaria surveillance, program logistics and communication are also essential elements of malaria control, which have been rightly emphasized. 20 The approach remains relevant and there is a significant unfinished agenda at the end of project, particularly with respect to the very low coverage of LLIN in the target population due to chronic delays in central procurement and the apparent further deterioration of an already weak program of IRS. The introduction of ACTs for the treatment of p. falciparum and the associated monitoring of drug resistance is another element of the unfinished agenda. Already resistance to the co-blistered combination of artesunate + sulfadoxine-pyrimethamine has emerged in the North-East region of the country necessitating a change in treatment policy to co-formulated artemether-lumefantrine. Drug resistance monitoring was a critical (and successful) activity under the project and must be continued going forward, particularly given how little is known/documented about the use of antimalarial drugs, both combination and montherapy, in the private sector and the daunting emergence of resistance to artemesinin derivatives in neighboring South-East Asia. As previously noted, chronic delays associated with contractual actions, particularly LLIN procurement, severely limited access to and use of malaria prevention and control services. An analysis by Dr. Allan Schapira, co-Chair of JMM 2014 and technical advisor to the project, suggests that if the LLIN had been procured and distributed as planned, 394,460 reported cases of malaria and 5.160 malaria deaths could have been prevented. Nonetheless, the project did provide LLIN protection to approximately 16.6 million people (assuming 2.5 persons protected by each LLIN purchased); 24.1 million rapid diagnostic tests for malaria and 2.81 million courses of ACT for the treatment of P. falciparum malaria. Although there is no direct evidence that the project development objectives and intermediate results were achieved due to limited geographic coverage and late implementation of the baseline survey and the absence of any follow-on surveys, there is some indication from data reviewed by the JMM 2014 that progress has been made towards higher level objectives to which the project was intended to contribute, as follows: For malaria morbidity, the JMM 2014 found the reported trends reliable enough to concur that the objective of a 40% reduction in malaria morbidity between 2006 and 2012 has been achieved, although the actual number of malaria cases is estimated to be six times higher than the number reported as many people with fever do not seek care from public facilities. With respect to malaria mortality, the goal of reducing deaths by 50% by 2010 (compared to 2002) was achieved by 2013 according to NVBDC monitoring data. Unfortunately, monitoring of mortality is weak and the number of actual deaths due to malaria may be between 15 to 50 times the actual numbers reported. Nonetheless, the trends are consistent with sentinel surveillance data collected from two hospitals in each high burden district. The sharp decline in malaria mortality from 2010 to 2013 is plausible, as RDT-ACT services were implemented at a large scale in high-burden districts from that year, but the mortality surveillance is not strong enough to indicate by how much the real mortality has decreased. Trends of indicators in All World Bank Project Districts (124) of India Year Malaria Cases Deaths due to Malaria 2007 927054 524 21 2008 911421 499 2009 953289 523 2010 1013628 584 2011 842845 452 2012 655502 293 2013 540203 160 Rating: Modest. Although the reported data indicate encouraging trends, the PDO level indicator target were not adequately measured and are unlikely to have been met given indicative data. KALA AZAR: Improve access to and use of services for elimination of kala azar PDO indicator 3. At least 50% of the sampled blocks which at baseline have not achieved the elimination goal of less than one kala azar case per 10,000 persons, will achieve elimination goal by endline. TARGET 50% Data Source is routine data from program records. Target was exceeded: 58.6% Kala azar was and remains a significant public health problem primarily in three states: Bihar, Jharkand and West Bengal. The vast majority of those cases (80%) are from Bihar and most of those in only nine districts. The project was intended to contribute to the national objective of elimination of kala zar from India by 2015. The specific activities financed by the project reflect the findings and recommendations of the WHO – NVBDCP Joint Monitoring Mission 2007 (JMM 2007) which found deficiencies in the control of kala azar but which concluded that disease elimination was technically feasible. The target for elimination is to reduce the annual incidence of kala azar to less than one per 10,000 population at the block level. The key indicators for monitoring the progress includes incidence of kala azar cases per 10,000 populations at block level, the case fatality rate and the treatment compliance rate. The Bank financed project specifically sought to improve the diagnosis and treatment of kala azar by strengthening service delivery and introducing rapid diagnostic tests and new treatment options (oral Miltefosine and Paromycin injections) for the disease; strengthening kala azar surveillance (an essential strategy in disease elimination); and, promoting more effective vector control in 46 of the 54 endemic districts in the country. The project has facilitated major reforms in the diagnosis, prevention and treatment of kala azar. Orally administered Miltefosine has now been adopted nationally as the first line of treatment for the disease, and is being used in 16 of the 31 districts targeted under Phase I of the project. The expansion through the project of the use of RDTs to diagnose kala azar, and the subsequent national adoption of this method of diagnosis, has also accelerated progress towards the control of kala azar. As with malaria, the objectives and design of the project with respect to kala azar continue to be relevant. There is room for improvement with respect to several of the implementation strategies, including (i) revision of the current treatment guidelines to include new treatment regimens which are more effective, safer and ensure better compliance; (ii) updating selection criteria for kala azar diagnostic kits to address accuracy and usability in field conditions (including thermo stability and usability with whole blood); (iii) better and more consistent approaches to incentives for providers and patients, social mobilization and behavior change communication; and (iv) 22 updated approaches to integrated vector management to increase acceptability, quality and coverage. Although the limited geographic scope of kala azar may give the impression that it is not a significant public health problem for India as a whole, it is a very significant problem in effected communities. The NVBDCP has reported that many issues and challenges were encountered that had a negative impact on the implementation of the kala azar elimination program and the effective use of Bank financing including: (i) delays in the release of funds for implementation of program activities; (ii) a large number of vacancies of health functionaries resulting in inadequate supervision and monitoring; (iii) chronic delays or non-payment of patient and provider incentives; (iv) low acceptance of DDT house praying attributed to inadequate IEC/BCC; and (v) stock-outs of essential drugs and other commodities. Nonetheless, the Directorate has reported that the targets for the project development objective and one of two intermediate results associated with kala azar have been exceeded. The progress towards elimination, despite many challenges encountered in program implementation is highly encouraging. The NVBDCPE is likely to have made an important contribution improving access to and the use of community of health services for the elimination of kala azar, by financing and encouraging the provision of field level staff, drug treatments and tests, where only limited services existed prior to the project’s intervention. With a more aggressive and well managed approach “similar to how India addressed polio elimination” there is a viable opportunity to complete the unfinished business and eliminate kala azar from the borders of India. Malaria was allocated almost 75% of the budget associated with this part of the PDO as it represents a greater overall epidemiologic and economic burden in India than kala azar, effecting a larger number of people, districts and states. As such, on balance, the modest rating for malaria 23 and the successful rating for kala azar combine to provide an overall rating of modest efficacy for Part a of the PDO. PDO, Part (b): strengthen central and state capacities for evidence-based policy development, strategic planning, and program management for effective control of vector- borne diseases. There are no PDO level indicators associated with program management and capacity building. There are five relevant Intermediate Outcome indicators for this part of the PDO of which only two have been monitored: Percentage of planned additional staff who are in position at central, state and district levels who are in place and who received training; and Proportion of eligible districts meeting the readiness criteria for each period of implementation. In addition, there is a World Bank Core Indicator, Number of health personnel receiving training, which has been monitored. Nonetheless, to supplement the paucity of indicators, the ICR team was able to gather information about this part of the project which is presented below. The project was designed and implemented to scale up new policies and strategies and introduce new technologies for vector born disease control, whilst strengthening human resources and health systems to ensure ongoing effective implementation. Although the project did not succeed in meeting all of its targets, it did contribute to durable positive changes in policy and capacity through demonstration of concept and identification of challenges. New diagnostic tools and effective treatments for malaria and kala azar have been put in place and it has been adequately demonstrated that malaria can be effectively managed and documented by community workers. The value added of multi-purpose health workers (MPW) and accredited social health activists (ASHA) for vector borne disease control is now widely accepted. Quality assurance of diagnostics and routine monitoring of the therapeutic efficacy of drugs and pharmaco-vigilance is in place and has already had an impact on treatment policy. Program Management and Capacity Building (including policy and strategy development) One of the key issues that the project sought to address was the capacity of the borrower to carry out the normative functions associated with vector borne disease control and the supplemental activities proposed under the project. This included policy and strategy development; procurement and supply chain management; commodity quality assurance; drug and insecticide resistance monitoring; financial management; behavior change communications; environmental safeguards; training; and, program implementation, monitoring and evaluation. As such, much of the non-commodity financing was directed at 1) strengthening human resources at the central, state and district levels through recruitment and training, and 2) outsourcing and contracting specific tasks or areas of work requiring specialized skills. (1) Human resources and training:   The project proposed to finance 44 positions (14 professional and 30 support staff) at central level; 55 consultants at state level; and 124 vector-borne disease specialists (VBD), 744 malaria or kala azar technical supervisors (MTS or KTS) and 372 laboratory technicians (LT) at district level. In addition the project intended to support training and incentives to 6849 MPW and more the 455,000 ASHA. Central and state level positions were intended to address gaps in core skill sets associated with management and stewardship of the program while the positions at district, block and village levels were needed to ensure the day-to-day implementation, supervision and 24 monitoring of project activities. Training was required for health functionaries at all levels and a comprehensive training strategy was developed. The recruitment and retention of key staff as well as their weak capacity were key constraints to smooth implementation during the entire project period as has been the case in a number of Bank- financed operations in India. The national and state level positions and the district VBD consultants financed by the project were initially recruited with the support of the National Health System Resource Center (NHSRC) and later directly by the NVBDCP. District level MTS, KTS and LTs were recruited by the states. MPWs and ASHAs, who engage in community health activities other than vector-borne disease control were recruited under the NRHM programs but trained in malaria control and kala azar elimination under the project. Although the investment in human resources was critical to the limited success of the project, there were a high number of vacancies at all levels and across all states throughout the project implementation period; and, slow recruitment at the beginning of the project is often cited as the primary reason for delayed district readiness and consequently delayed disbursement of decentralized funds. The key factor that contributed to the recruitment delays and poor retention was non-competitive compensation. Salary and benefits were low in comparison to positions requiring similar or equivalent skills and qualifications in the private and public sector, including positions in the NRHM. This may have been, in part, due to the poor understanding and implementation of the standard cost approach to decentralized financing (see financial management above). Standard cost estimates for compensation made at the beginning of the project were not adjusted during the project and were seen as “ceilings” and no annual increment was applied resulting in the positions becoming less competitive over time. Some states have begun addressing this problem by adopting and implementing policies on “pay parity”. However, this did not have a major effect on the project during implementation. At the close of the project, 11 of the 14 key positions at central level were vacant. Vacancies at state and district level were 36% and 31% respectively. It might be asserted that non-competitive compensation would result in the recruitment of weak candidates. However, the core problem encountered was not the talent of individuals but the sheer number of vacancies and the continual need to recruit and train replacements. At no point in the project were central or state level teams complete and fully trained and often, and some of the most critical skill sets required for effective project implementation were completely absent. (2) Outsourcing and contracting:   The project financed the National Institute of Malaria Research (NIMR) and the Rajendra Memmorial Research Insitute, Patna (RMRI) to manage operational research, which included pharmacovigilance of newly introduced drugs, therapeutic efficacy of ACT and insecticide resistance monitoring, quality assurance testing of rapid diagnostic test kits for malaria and kala- azar and sentinel surveillance, and the baseline household. With the exception of the household survey (see monitoring and evaluation section below), these organizations had the specialized skills, disease focus and the infrastructure required and outsourcing this set of tasks and work areas was both sensible and effective. The VBDCP engaged consulting services to address the need for some of the products and services required for project implementation including training; fiduciary review; environmental management planning; supply chain logistics management information systems (LMIS); information, education and communications (IEC) and behavior change communications (BCC) support. On the whole, consultants performed well, although the award of consulting contracts 25 and the approval of proposals was subject to the same delays and weak management and stewardship that limited the procurement of goods. This was particularly injurious to the project with respect to multiple delays in recruitment of the fiduciary consultant as integrated implementation and fiduciary reviews of decentralized costs were not in place during 40% of the project period. Weak management and stewardship of contracts and contractors was associated with chronic delays in payments to contractors and in failure of the NVBDCP and MoHFW to take decisions in a timely manner with respect to contractor proposals. For example, one of the contractors successfully developed and pretested IEC and BCC materials and submitted a proposal to implement activities at a larger scale consistent with the contractor’s terms of reference. No action was taken with respect to the proposal for an extended period. Eventually, it was decided to hand over responsibility for IEC and BCC implementation to the states and modify the terms of reference of the contractor to focus on capacity building and monitoring at district and block levels. The project ended before these changes were approved. Rating: Modest. Sub-optimal performance was associated with persistent vacancies of key positions at the central and state level and delays in contracting. PDO, Part (c): secure the timely supply of polio vaccine PDO indicator 4: At least 80% of the households with eligible children covered during the national and sub-national immunization days in high risk districts. TARGET >80%. Data obtained through an independent monitoring system assocoiated with the Global Polio Eradication Initiative with support from WHO and CDC. Target was exceeded: >98% The World Bank contribution to the polio elimination program in India was in the form of financing for commodities. While the polio elimination program was well-manned, planned, monitored and supervised adequate buffer stocks of vaccines were not available to allow for the short term changes in vaccination approach needed to respond to the type of polio virus found in each vaccination round. The project financed the MoHFW to procure three types of oral polio vaccine (OPV) worth approximately US$167 million (out of US$271 allocated) in OPV. This was carried out at a critical point in elimination process and therefore the project can take credit for a substantial contribution to the elimination of polio in India. There have been no reported cases of polio since a two-year old girl was diagnosed with polio paralysis on January 13, 2011 in Howrah district of West Bengal. After three years with no new polio infections and the maintenance of an incredibly high rate of vaccination coverage even in the most remote areas (99%), India was declared and certified polio free on March 29, 2014.  This is an incredible success for a country, which reported more than half the global polio cases until the year 2009, and which was predicted by many experts to be the last to stop polio as its endemic pockets in parts of Uttar Pradesh and Bihar were among the most difficult places in the world for polio eradication. Rating: Successfully achieved. On balance, the rating for Efficacy is Modest given that objectives (a) and (b) were only modestly achieved while objective (c) was fully achieved. 3.3 Efficiency The Economic and Financial Analysis conducted during project preparation focused primarily on the evidence base concerning the cost effectiveness of the interventions to be financed by the project and an estimate of the costs and benefits of introducing the new diagnosis and treatment 26 protocols for malaria and kala azar. Public financing of polio eradication is considered well justified using standard public financing criteria. The elimination of polio from India is considered a global public good and the benefits of eradication are infinite. For the vector borne disease control objectives, the overall efficiency rating is low given the absence of key commodities to support planned interventions throughout the implementation period and given that most of the funds for this area were cancelled given procurement inefficiencies; however, there are some examples of good practices within the project that enhanced technical and or allocative efficiency. Overall, efficiency is rated as Modest. A detailed discussion of project efficiency is included in Annex 3. Economic and Financial Analysis. 3.4 Justification for overall outcome rating Rating: Moderately Unsatisfactory This rating is based on a combination of Substantial Relevance, Modest Efficacy and Modest Efficiency. Efficacy and efficiency were higher for the polio component of the project than for the VBD components. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Poverty Impacts: The project targeted services to the communities with a high burden of malaria and kala azar. Given the nature of these two diseases and their association with poverty, these communities are also amongst the poorest, most remote and underserved in India. Malaria is both a disease of poverty and a cause of poverty: poor children and women living in rural areas are at the greatest risk of death or severe debility from malaria and the disease drains the resources of families and keeps the poor in poverty. Poverty may prevent some households from spending on prevention or an effective cure, leaving malaria illness untreated, risking complications, and thus over time having a depletory impact on a household’s ability to cope with other contingencies. Studies on health care expenditures have consistently shown that most of the money spent on malaria prevention and treatment comes out of the pockets of individuals and households. Kala azar is also a disease of the poor and keeps families in poverty. The JMM 2007 found that 60-70% cases accessed the private sector for diagnosis and treatment. In addition to wage loss for a significant period for the kala azar patient, the high cost of under-regulated and likely poor or inadequate diagnosis and treatment impoverished families. Although the project did not succeed in providing access to free LLINs to all communities as planned due to delayed and incomplete procurement processes, community based diagnosis and treatment for malaria and kala azar was available to a large proportion of the target population. Free diagnosis and effective treatment provided by Accredited Social Health Activists (ASHA) and financial incentives to kaala azar patients to improve treatment compliance were very pro-poor strategies and the Mid-term Review (MTR) highlighted the observation that beneficiaries were increasingly seeking care in the public sector due to the availability of commodities and improved services. Polio eradication is a global public good and as such the benefits accrue to the rich and poor, men and women alike. However, in the final stages of polio eradication, pockets of polio transmission and cases are concentrated in the most geographically remote and hard to reach populations, who lack access to many basic services, and among people who do not perceive the benefits of vaccination. As such, the focus of efforts (supplemental immunization activities, mop-ups and 27 outbreak response) during the project period focused on poor, vulnerable and hard to reach populations. Gender: One of the initiatives carried out at district level with support of the NVBDCP was the “Mo- Moshari” (My Mosquito Net) program which provided conventional insecticide treated nets (ITNs that require periodic retreatment with insecticide) to pregnant women and to students at residential tribal schools. Recruitment and training of MPWs and ASHA took gender into account, two cadres of multi-purpose health workers, one male and the other female, were recruited under NHM and trained with project funding and all ASHA were women. Overall, the project contributed to the training and incentives for ASHA, mobilizing and equipping them to provide diagnosis and treatment for malaria and kala azar at the community level, which contributed to their status in the community. A review of ASHA performance during the JMM 2014 noted that one of the constraints to the recruitment of ASHA, particularly in remote tribal areas, was the low level of female literacy in these communities. Social Development: The project integrated the Vulnerable Communities Plan (VCP) into the project implementation plan and operational plan. The GOI has a legal and institutional framework for vulnerable communities who are described as “groups with social, cultural, economic and/or political traditions and institutions distinct from mainstream or dominant society that disadvantage them in the development process” (PAD p. 130). Indigenous peoples (tribal groups), scheduled castes (those who remain outside of the four Hindu castes, including socially or economically deprived Budhist and Sikh populations) and people living below the poverty line are considered to be vulnerable communities. The VCP contains many specific actions associated with the three vector borne disease control components of the project and the NVBDCP engaged the Voluntary Health Association of India to implement the plan, however implementation progress was slow and incomplete leading to a moderately unsatisfactory rating for social safeguards. Details are provided in section 2.4 Safeguard and Fiduciary Compliance. (b) Institutional Change/Strengthening Human resources for the implementation of vector borne disease control strategies have been strengthened at the district, block and community level (although most positions at central and state level are vacant) and domestic financing is being put in place to the end of the five year planning cycle ending in 2017 to ensure continuity of access to services. The project’s contribution appears to be more successful (in terms of vacancy rates) and more permanent at district and block levels than at state and national level. Establishment of the cadres of Vector Borne Disease Consultants, Malaria and Kala azar Technical Supervisors and the expansion of the network of Laboratory Technicians were important project contributions. If the GOI can continue to resolve the issue of pay parity, recruitment and retention will become less of a problem in the future. At central and state levels, where the project’s contribution to human capacity provided for the recruitment of more specialized skills (financial management, procurement, monitoring and evaluation, information, education and communication, etc) may be less durable. Although many of the positions have been sanctioned for continuation to the end of the planning cycle, many of those positions, particularly at central level, were not filled or are now vacant. If the NVBDCP and the states do not take a pro-active stance with respect to recruitment and retention the positions may remain vacant. 28 The simplification of reporting and reporting forms for malaria control and the establishment of sentinel sites for monitoring malaria morbidity and mortality have been important contributions of the project to monitoring and evaluation for vector borne disease control. Less progress has been made in automating the reporting system and integrating malaria and kala azar monitoring and evaluation with other health statistics. The project also initiated a culture of using LQAS for monitoring implementation at the district level. Although most stakeholders see value in in sentinel sites and local use of LQAS, it is unclear for how long and how broadly these practices will be maintained. Sentinel sites may be more durable as they are clearly part of a nation-wide system. Local use of LQAS is already quite variable across states and districts, and although popular where it has been used, without a champion and dedicated funding its usage may decline. Additional information is provided in section 2.3 Monitoring and Evaluation. (c) Other Unintended Outcomes and Impacts (positive or negative) N/A 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Although there were no beneficiary surveys of stakeholder workshops held as part of the ICR or project assessment, the ICR mission corresponded to the 2014 Joint Monitoring Mission by the National Vector Borne Disease Control Program and the World Health Organization (JMM 2014). The previous joint monitoring mission took place in 2007 with Bank participation and informed the design of the project. The JMM took place from 1 – 10 March 2014. The objectives were: to review technical policies and strategies in light of global policies and best practices; to assess the program implementation against targets defined in National Strategic Plan (NSP), Results Framework Document (RFP) and National Health Policy (NHP 2002); to assess the effectiveness of NVBDCP strategies on control, elimination and/or reduction in disease burden; and to formulate recommendations in order to move towards achieving the targets envisaged in National Health Policy 2002 and MDGs by 2015 and sustain these in the post 2015 agenda. The JMM 2014 involved meetings with stakeholders, program implementers and beneficiaries through consultations in Delhi and visits to 7 states and 14 districts across India. The JMM report is presently in draft stage; however, preliminary findings related to malaria control, kala azar elimination and health systems are referenced throughout the ICR. In general, the JMM 2014 concluded that for malaria and kala azar, progress has been made in decreasing the disease burden, particularly through improved diagnosis and treatment at community level. Some progress in vector control has been made, but the roll out of LLINs has been far from adequate and significant challenges still exist with respect to the quality, acceptability and coverage of IRS. 4. Assessment of Risk to Development Outcome Rating: Negligible-Low The GoI is highly committed to controlling malaria, eliminating kala azar and preventing reintroduction of polio virus to India. Malaria, kala azar and other vector borne diseases are within the purview of the National Health Mission (NHM), a flagship program of Ministry of Health mandated to provide accessible, affordable and quality health care to the rural and vulnerable populations. NVBDCP under the overall architecture of NHM is financed by 29 domestic budget support (DBS) and external aid component (EAC). Under NHM, there has been an increase in domestic financing for the disease control program and the budget allocation for NVBDCP has also increased proportionally. Malaria has been highlighted as one of the top development challenges under the 12th Five Year Plan (2012–2017) and the GoI has committed to supporting key positions at central and state level as well as the cadres of workers (VBD consultants, malaria and kala azar technical superviors) supported by the project and the NHM will continue to support the multi-purpose health workers and the accredited social health activists (ASHA) at block and community level. In many districts, District VBD Control Societies (under NRHM) have been established to assist with management of funds and planning and monitoring of program activities. States developed a stronger sense of ownership and capacities at central and districts levels have been enhanced to continue the program. EAC for malaria has been fairly limited with financial support primarily from the World Bank and the Global Fund for AIDS, TB and Malaria and there is some risk that progress in reducing the burden of malaria could stall in the absence of adequate external funding. Kala azar elimination appears to be less vulnerable due to a high level of interest in disease elimination by stakeholders and external financing partners such as the Bill and Melinda Gates Foundation. High quality technical support for malaria control and kala azar elimination continues to be provided by the WHO. The JMM 2014 will provide confirmation of the technical soundness of the disease control approaches rolled out under the project as well as guidance and recommendations on technical strategies going forward. Having eliminated polio from India the GoI is committed to preventing reintroduction of the virus from neighboring polio endemic countries and has established vaccination posts along India’s international borders. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory The Bank team worked closely with the GoI and development partners, particularly the World Health Organization (WHO), to develop a technically sound and ambitious project that incorporated lessons learned from previous investment lending for malaria control, the findings of an in depth review of vector borne disease control in India (JMM 2007), and international best practices for malaria control and kala azar elimination. The project was designed to fast track implementation of significant changes in disease control policy and build on increasing domestic investment in community level service delivery under the NRHM. During the preparation period the project benefitted from both a Quality Enhancement Review (QER) and an Operation Committee (OC) Review which helped the project team to narrow the geographic scope of the project and focus on malaria and kala azar as priority public health problems. The addition of a polio component late in the preparation of the project (during negotiations and without a full appraisal) added to the complexity and risks associated with the project. It also added significantly to the budget, more than doubling the cost of the project. The OC review of the Appraisal Decision Meeting Package noted that the project represented both a high opportunity and high risk for the Bank’s HNP program in India. As has been noted throughout the document, the project design was strongly influenced by the DIR and subsequent agreements between the Bank and the GoI. The conduct of the DIR and the subsequent development of a joint action plan between the MoHFW and the Bank contributed to the prolonged project preparation period and set the tone for project design and development. 30 The Bank and the Borrower took a very risk-averse approach to project design in an inherently high risk context. This resulted in very prescriptive implementation arrangements particularly concerning procurement and financial management, restrictions on decentralized costs and expenditures, district readiness filters and project phasing. The project lacked, or was perceived to lack, flexibility and in certain respects was “over-designed”. Monitoring and evaluation is one aspect of the project that has been highlighted as ambitious and overly complex, requiring multiple tools and approaches and the triangulation of data from different sources. Given the various innovations being promoted within the project (new diagnostic and treatment policies, new service delivery models, the introduction of standard costs for decentralized expenses, large country LQAS and population-based household surveys) the project needed to provide space for some risk taking by the implementing agency, but the implementation arrangements, coupled with lingering concerns about fraud and corruption, promoted risk averse behavior, caution and inaction. This included prolongation and cancellation of procurement processes; overly narrow definition of decentralized costs and significant delays in response to implementing partner proposal. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Unsatisfactory The task team carried out regular and in-depth bi-annual supervision missions, including site visits to states, districts and communities. The Bank team made a concerted effort to engage other development partners, including DFID, the WHO, the Global Fund and the Bill and Melinda Gates Foundation, in supervision missions in order to incorporate a broader set of observations and perspectives and expand the number of states and districts visited during each mission. In addition, implementation support missions to states were conducted between supervision missions when needed and there was frequent contact with the NVBDCP and key development partners and stakeholders such as CARE, MSF and DNDI to track implementation progress and challenges on a more day to day basis. Detailed Aides-Memoires with specific action points were generated during each mission and were presented to and discussed with the implementing agency and of state and national level officials at the highest level. Continuity in task team membership was fairly high. There was only one change in TTL that occurred just prior to the mid-term review of the project. The TTL transition, however, was fairly abrupt for such a large and complex project. No formal handover notes were prepared or transmitted and for a period of approximately six months an acting TTL bridged the gap between the outgoing and incoming TTLs. There were smooth transitions between two social safeguards specialists and two financial management specialists. The procurement, environmental and monitoring and evaluation specialists on the team did not change during the life of the project. Technical support for LQAS was available during project preparation, but not implementation. This resulted in a lack of continuity in support for Large Country LQAS (as part of the M&E plan) in particular. Given the size and complexity of this project and high periodic and day-to day demands on Bank staff the investment in supervision by the Bank, including number and location of staff may have been overly lean One observation on the quality of supervision is the apparent disconnect between the project implementation and project ratings. Even when the project processed the first cancellations of funds and could not document progress against two of the PDO level indicators due to the absence of survey results, the team still maintained “moderately satisfactory” DO, IP, Procurement, Financial Management and M&E ratings almost until project closure. While the 31 ratings remained above the line, the task team did not fail to observe or to document the challenges encountered during the project including chronic delays in in the procurement of goods, such as LLIN, or the contracting/outsourcing of critical services – particularly the external implementation and fiduciary reviews and the baseline, mid-term and end-line household surveys. However the mission Aides Memoires, upon which the ISR ratings were based, were consistently optimistic suggesting that the challenges documented were being resolved or on the cusp of being resolved. This optimism appears to have been a consequence of three factors: 1) strong assurances from the client at all levels that the challenges were being aggressively addressed. The team was continually expected procurement to turn around, even until the supervision mission in July 2013; 2) routine epidemiological program data that indicated downward trends in morbidity and mortality for malaria and kala azar; and 3) the very inclusive nature of supervision mentioned above (by aligning so many stakeholders with an interest in the ultimate success of the project to the supervision process some of the Bank’s objectivity may have been lost). Additionally, the satisfactory performance of the polio component, particularly with respect to procurement, seems to have been overly weighted compared to the less than satisfactory performance of the VBD components in the summary ratings. Ratings were still moderately satisfactory as reported in the July 2013 supervision mission Aide-Memoire and were only down-graded in the second to last project ISR (November 2013) when it was clear that a second cancellation of financing for malaria commodities would be required and there was very little likelihood of a follow-up household survey being conducted. This was not the case for either environmental or social safeguards where the ratings varied according to performance during the period under review and which are more consistent with the story of progress and challenges as documented.  The summary of ISR ratings is provided in the table below. Ratings of Project Performance in ISRs N Date ISR PRO M&E ENV SOC DO IP FM o. Archived C 1 12/08/2008 S S MS MUS S S 2 06/23/2009 S S MS MUS S S 3 01/25/2010 MS MU MU MUMU MU MU 4 06/25/2010 MS MS MS MSMU MU MU 5 01/02/2011 MS MS MS MSMS U U 6 06/28/2011 MS S MS MS MS MS MS 7 04/05/2012 MS MS MS MS MS HS MS 8 11/13/2012 MS MS MS MS MS HS MS 9 04/10/2013 MS MS MS MS MS MS MU 10 11/15/2013 MU MU MS MS MU MS MU 11 12/25/2013 MU MU MU MU MU MU MU The consistent use of the MS rating may have masked the seriousness and chronic nature of some of the difficulties encountered during the implementation of the project. As a result, the search for solutions, the purpose and scope of restructuring exercises and the attention of senior management may have been less than optimal. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory Overall Bank performance, which includes project preparation, supervision and evaluation, is rated as moderately unsatisfactory. Although the level of effort invested in project preparation 32 and supervision was extremely high the project was very complex and ambitious and suffered from chronic shortcomings that were not sufficiently appreciated through much of the implementation period. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory The strong commitment of the GoI to health in general, the establishment of the NHM with its mandate to focus on the most vulnerable populations and their priority health concerns, and the standing commitment to eradicate polio provided and enabling environment in which to launch the National Vector Borne Disease Control and Polio Eradication Project. The successes to which the project contributed, including polio eradication and the delivery of effective diagnostic and treatment services for malaria and kala azar at the community level, were largely dependent on the continuous commitment of the GoI and domestic investment in polio eradication and the strengthening of health systems and health manpower, particularly at the community level, under the NHM. However the GoI, at the level of the MoHFW, was unable to provide adequate stewardship to the project as evidenced by delays in approvals and decisions which affected project implementation. This includes a fairly broad array of issues including pesticide product registration processes, procurement clearances, decision-taking on implementation support proposals from contractors, communication and support to states, prioritization of monitoring and evaluation activities; and the resolution of pay parity issues within the NHM. Key informants have indicated that many of the procedural constraints and delays at the level of the government and the implementing agencies, particularly with respect to procurement and contracting actions were due to risk adverse behaviors and concerns about personal association with fraud and corruption allegations in the wake of the DIR and the context of concerns about the subsequent allegations of the misuse of government funds associated with preparation of the 2010 Commonwealth Games in India. (b) Implementing Agency or Agencies Performance Rating: Unsatisfactory. The performance of the National Polio Eradication Program was Satisfactory, as evidenced by the certification of India as polio-free in March 2014. Although 32% of the Bank financing for OPV was cancelled, the financing was replaced from domestic sources and there was no detrimental impact on OPV supply, program implementation, outcome or impact. The performance of the National Vector Borne Disease Control Program was Unsatisfactory, despite some notable progress in the diagnosis and treatment of malaria and kala azar. The NVBDCP was not adequately equipped or staffed throughout the project period to fulfill all of its normative functions as defined under the project or provide the necessary leadership to address the multiple implementation challenges that arose during project implementation in an effective or timely manner. The NVBDCP never achieved full staffing of the key central level positions financed under the project and many remained vacant for extended periods. With the exception of a brief period in 2010 the program directorate did not take a notably proactive approach to addressing delays and bottlenecks which ultimately resulted in the cancellation of 68% of bank financing for vector borne diseases control, poor coverage of LLINs and inadequate documentation of the malaria-related PDO and intermediate outcome indicators. Despite repeated assurances, the mid-term end of project population based household surveys were not contracted and clear evidence of project impact is not available. 33 (c) Justification of Rating for Overall Borrower Performance Rating: Unsatisfactory Moderately Unsatisfactory ratings for government performance and the Unsatisfactory rating for the implementing agency result in an overall borrower performance rating of Unsatisfactory. 6. Lessons Learned 1) Community-based service delivery models can work. This project has contributed further evidence that community-based health workers with limited training can effectively manage biological confirmation and treatment of malaria and kala-azar and contribute to epidemiologic and program monitoring. Investment by the GoI through the NRMH in building the cadres of MPW and ASHA to deliver essential health interventions to populations with poor social, geographic or economic access to publicly provided health services has provided a strong platform for the introduction of RDTs and new treatments for malaria and kala azar at the community level. As many other malaria endemic countries are beginning to introduce or scale up community case management (CCM) the example from India , which has several distinctly successful and replicable features, is timely. The project included impact evaluation research which assessed the introduction of CCM through deployment of community health workers. This paper shows that directed investments in community health workers can feasibly shift care- seeking behavior by improving the perceived quality of the lay provider. The results highlight how community mobilization and supportive supervision can improve utilization and delivery of malaria diagnosis and treatment. 2) The introduction of innovations requires a high level of capacity building for the implementing agency and financing for external technical support within the project budget. The project introduced several innovations including Large Country LQAS as a monitoring and evaluation tool and the Standard Costs Approach for disbursement of project funds for decentralized costs. These were important opportunities for project learning; however neither of these innovations was adequately implemented and evaluated due to the implementing agency’s weak capacity in M&E and financial management at central level and the absence of continuous external technical support or a planned formal evaluation of the tools themselves. 3) Operational research can have a very positive impact on policy and practice. By incorporating routine drug and insecticide resistance monitoring, batch testing of rapid diagnostic tests, pharmaco-vigilance and impact evaluation into the M&E framework for the project, new data were continually being generated for consideration by technical experts, state-level and national authorities. As has previously been noted, this provided confidence in the use of newly introduced diagnostic technologies and treatment regimens and the capacity of community health workers to conduct CCM. In the case of malaria drug efficacy monitoring resulted in a change in ACT treatment policy from co-packaged artesunate and sufadoxine pyrimethamine (AS-SP) to co-formulated artemether-lumefantrin (AL) in states with increasing levels of resistance to SP. 4) Client assurances with respect to the recruitment of human resources to address capacity constraints are not an adequate substitute for thorough assessment of past performance and the market for key skill sets. As has been noted recruitment and retention were major challenges 34 during implementation particularly at central and state level. This was identified as a substantial risk during project preparation and a more in depth assessment of past performance and the labor market may have helped identify problems such as non-competitive compensation packages and the absence of specific skill sets, potentially leading to a different approach to capacity building and/or risk mitigation measures. 5) Substantive alterations to projects late in the preparation process should be avoided. Although the polio component of the project contributed to the elimination of polio in India, there was not adequate opportunity for appraisal of the component which may have led to an overestimation of the required budget. 6) Monitoring and evaluation needs to be given highest priority. The delay in implementation of the baseline population based household survey and the complete absence of mid-term and end of project survey data for comparison is clearly not acceptable. Projects need controls in place to ensure that key monitoring and evaluation activities are conducted in a timely manner. This may include conditions of effectiveness or dated covenants. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Cofinanciers (c) Other partners and stakeholders 35 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Revised Actual/ Appraisal Estimate Latest Percentag Estimate Components (USD Estimate e of (USD millions) (USD Appraisal millions) millions) Component 1 Improving Access to and Use of 46% Services for Malaria Control 119.5 64.0 55.0 1.a.: Improving Malaria Case Management 45.0 36.0 30.1 1.b.: Strengthening Malaria Surveillance 5.9 0.0 0.0 1.c.: Effective Vector Control 68.6 28.0 24.9 Component 2: Improving Access to and Use of 6.0 14% Services for Kala Azar Elimination 41.9 5.8 2.a.: Improving Kala Azar Case Management 40.0 6.0 5.8 2.b.: Strengthening Kala Azar Surveillance 1.5 0.0 0.0 2.c.: Effective Vector Control 0.4 0.0 0.0 Component 3: Policy and Strategy Development, Capacity Building and Monitoring and 14% Evaluation 52.1 8.0 7.4 3.a.: Policy and Strategy Development 23.1 5.0 4.9 3.b.: Capacity Building and Program Management 24.1 3.0 2.5 3.c.: Monitoring and Evaluation 4.8 0.0 0.0 Component 4: Improving Polio Vaccine 61% Availability 271.0 185.0 166.2 Total Baseline Cost 484.5 263.0 234.4 48% Unallocated 36.5 0.0 0.0 0 Total Project Costs 521.0 263.0 234.4 45% (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Borrower 0.00 0.00 0 International Development 521.00 234.4 45% Association (IDA) 36 Annex 2. Outputs by Component   Component 1: Improving Access to and Use of Malaria Prevention and Control Services  1a: Improving Malaria Case Management Goods procured:  Rapid Diagnostic Test Kits for Malaria (P.f)  Rapid Diagnostic Test Kits for Malaria, Bivalent (P.f / P.v)  Supply of ACT Combination Blister Packs  Injectible Arteether Community level health workers providing diagnostic and treatment services for malaria.  1b: Strengthening Malaria Surveillance Routing reporting simplified and the total number of monitoring forms reduced. 2 Sentinel sites identified in all project districts: 331 of 348 sites are functional, but improvement required in analysis and reporting.  1c: Effective vector Control Goods procured:  LLIN 6.18 million LLINs distributed to 52 of the 124 project districts. High coverage levels achieved in the communities that received nets. Approximately 15.45 million people in phase 1 districts were protected from malaria (assumes 2.5 person protected per LLIN. New WHO guidance suggest this number is 1.8 persons protected per net). Component 2: Improving Access to and Use of Services for Elimination of Kala Azar  2a: Improving Kala Azar case Management Goods procured:  Rapid Diagnostic test Kits for Kala Azar  Miltefosine Capsules Community level health workers providing diagnostic and treatment services kala azar.  2b: Strengthening kala azar Surveillance SEE: Monitoring and Evaluation / Operational Research below.  2c: Effective Vector Control Goods procured:  Synthetic Pyrethroid (Powder)  Synthetic Pyrethroids (Liquid) 37 Two rounds of IRS carried out in both 2012 and 2013in the 3 kala azar endmic states although there are concerns about coverage and quality Component 3: Policy and Strategy Development, Capacity Building and Monitoring and Evaluation  3a: Policy and Strategy Development Development of Operational manual for program managers and updated training modules Environmental Management Plan (EMP) and Environmental Codes of Practice developed. Technical policies updated:  Shift from monovalent to bi-valent RDTs for malaria to diagnose both P. falciparum and P. Vivax malaria.  Transition from AS+SP to AL for malaria treatment in north east states based on routine drug efficacy testing.  Transition from 28 day treatment of kala azar with miltefosine to 10 day treatment with miltefosing + paromycin at block level and single dose ambisone injection at hospital level  3b: Program Management and Capacity Building Goods Procured:  Motor Cycles  Multi-Utility Vechicles [Four Wheelers] Human resources:  At central level 3 of the 14 sanctioned professional positions in place at EOP. MoHFW has approved continuation of 11 positions post project.  24 or 30 support staff are in position. MoHFW has approved continuation of 22 positions post project.  At state level 35 out of 55 positions are filled. 50% have been trained  At district level 95 out of 124 vector born disease consultants are in place, although not all have been trained.  535 out of 744 malaria technical supervisors are in place but there is a high rate of attrition.  223 of 372 laboratory technicians are in place of which 158 have been traine  At local level, the project trained 98% of 4741 MPW (out of 6849 sanction positions).  Traing provided to 52% of 407867 ASHA (out of 455595 sanctioned positions). Outsourcing and contract services:  For IEC/BCC activities- M/s New Concept Information Systems Pvt. Ltd, New Delhi was hired for conducting formative research on BCC and to develop BCC strategy and implementation plan for project states.  For Social Mobilization & Service Delivery activities- Voluntary Health Association of India (VHAI), New Delhi was hired for Capacity building and promotion of correct & continuous use of Insecticide treated bed nets (LLIN).  For Supply Chain & Management activities- M/s Strategic Alliance Management Services, New Delhi was hired for conducting trainings for Malaria control & Kala-azar elimination in project states.  For Implementation & Fiduciary Review of Decentralized Activities- J.P. Associates, New Delhi was hired for periodic implementation and fiduciary review of decentralized activities in project areas.  For Environmental Management Plan (EMP)- M/s Senses Consultants India Pvt. Ltd., 38 Kolkata was hired for conducting Pilot study and National workshop for environmental codes. Trainings were also organized at state level & M&E activities were organized.  For Trainings on VBDs Control- National Institute of Health & Family Welfare (NIHFW), New Delhi was hired for conducting trainings to Faculty of Medical Colleges, District level Officers. All trainings were completed by 31st March, 2013.  3c: Monitoring and Evaluation Baseline household survey completed in 5 phase one states by NIMR. All Phase 1 states have completed 2-8 rounds of LQAS surveys. Operations research conducted by NIMR and RMRI Patna  Therapeutic efficacy of anti-malarial drugs (to assess drug resistance for first line ACT and evaluate potential alternatives)  Pharmaco-vigilance for kala azar treatment regimens  Quality assurance of RDTs  Vector bionomics  Vector resistance to insecticides Component 4: Supply of Polio Vacines Goods procured:  Oral Polio Monovalent Vaccines  Oral Polio Bivalent Vaccines  Oral Polio Trivalent Vaccines   39 Annex 3. Economic and Financial Analysis The Economic and Financial Analysis conducted during project preparation focused primarily on the evidence base concerning the cost effectiveness of the interventions to be financed by the project and an estimate of the costs and benefits of introducing the new diagnosis and treatment protocols for malaria and kala azar. VECTOR BORNE DISEASE CONTROL: For the vector borne disease control objectives, the overall efficiency rating is low primarily due to implementation performance; however, there are some examples of good practices within the project that enhanced technical and or allocative efficiency. At the time of preparation, there were a number of influential reviews of the cost effectiveness of malaria control, many of which were cited in the project appraisal document. A fairly exhaustive review of the evidence associated with the cost effectiveness of malaria control interventions was published in 2000 by the Global Forum for Health Research. The central message from this analysis was that all of the malaria control interventions evaluated would be an attractive use of resources. In subsequent years, the interventions—already established as cost-effective—have been subject to continuous improvements to promote more effective use, reduce costs and improve the cost-effectiveness ratio. Kala azar, a disease that affects the poorest segments of rural populations wherever it is found, was considered in the economic analysis for its economic burden, particularly on poor rural households, and the cost effectiveness of kala azar treatment which compares favorably to other public health interventions in terms of the cost per death and disability adjusted life year averted. The analysis in the PAD estimated and compared the costs and benefits of the then current protocols for malaria control and kala azar elimination with the costs and benefits of a phased transition to the new protocols and concluded that the transition would produce significant net incremental economic benefits over the five year period of approximately 4,389 million rupees. Unfortunately, a similar analysis based on observed data and actual costs was not conducted at the end of the project. In the absence of these data, the ICR team looked for other evidence of efficacy in the context of project implementation and concludes that the overall efficacy of the VBD components of the project is low, despite the documented cost effectiveness of the project interventions. At very few points in time were goods, personnel, training and other services adequately aligned which resulted in fairly profound inefficiencies in project implementation and constrained the impact of the project. Some examples include the very limited coverage of LLIN in the target population because the borrower was unable to procure the commodity; the recruitment and training of staff to implement activities that were not adequately supported with required commodities (such as ACTs for the treatment of malaraia and Miltefosine for kala azar), while in other instances interventions could not be delivered or activities conducted because of the high level of staff vacancies. There were nonetheless some examples of good practice that enhanced the allocative and/or technical efficiency of project implementation.  A cost-effective response was taken to the severe and chronic shortage of LLIN. In other settings, when there have not been enough LLINs to cover the entire target population at 40 the designate level (often 80% of households or universal coverage), the coverage target has been lowered to ensure equal access to the entire at risk population. However, such an approach is not technically optimal. When LLIN coverage is high even people without nets in that community are protected from malaria. At lower levels of coverage there is no community protective effect and only those persons sleeping under the LLIN are protected from malaria. The program chose to maximize the potential benefits of LLIN and targeted communities with the highest malaria cases and only those for which they could provide high coverage.  Proper management of expiry of drug stocks at General Medical Store Depots (GMSD), state and district levels. Regular monitoring of short-expiry drugs and other supplies was implemented at all levels so that these commodities could be shifted to destinations with low stock and high demand to ensure utilization of close to expiry items within available shelf life. 138 episodes of shifting short-expiry drugs and other supplies were documented. Had this not occurred those drugs and commodities would have been destroyed or inappropriately used after expiry.  Routine monitoring of therapeutic efficacy (to detect drug resistance) was conducted of the new drugs (primarily ACTs) introduced by the Program. For example, decreasing therapeutic efficacy of the co-blistered ACT combining artesunate and sufadoxine- pyrimethamine (AS+AQ) was detected in malaria endemic states in the north-east and the treatment protocol was changed to co-formulated artemether-lumefantrine (AL) in response, improving the technical efficacy of malaria treatment in those states.  Systematic quality assurance of RDTs was undertaken, particularly important during the earlier years of the project when there was significant quality variance between products as well as batches, was able to detect and eliminate from use, poorly performing batches of tests. POLIO: Public financing of polio eradication is considered well justified using standard public financing criteria. The eradication of polio is considered a global public good and the benefits of [Global] eradication are infinite. The project contributed to the India program at a critical point in the disease elimination process and the contribution was simple and focused and responded to a specific need (availability of an adequate supply of three types of OPV). During the project, a change in procurement agent for vaccines did result in cost savings while maintaining adherence to WHO pre-qualification requirements. 41 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Dariush Akhavan Consultant SASHD Lawrence Barat Consultant SASHD Suprotik Basu Public Health Spec. AFTHE Mam Chand Consultant SARPS Mohan Gopalakrishnan Sr Financial Management Specia SARFM Hnin Hnin Pyne Senior Human Development Speci EASHH Shreelata Rao-Seshadri Consultant SASHN Suneeta Singh Consultant EASHD Ruma Tavorath Senior Environmental Specialis SASDI Supervision/ICR Rajeev Ahuja Economist (Health) SASHN Silvia M. Albert Temporary AES Neera Bhatia Temporary SASDO Meera Chatterjee Senior Social Development Spec SASDS Jed Friedman Senior Economist DECPI Mohan Gopalakrishnan Sr Financial Management Specia SARFM Shanker Lal Senior Procurement Specialist SARPS Arun Manuja Sr Financial Management Specia SARFM Shafali Rajora Program Assistant SASHD Allan Schapira Consultant SASHD Sridhar Srikantiah Consultant SASHN William Franklin Gayer Senior Operations Officer SASHN Starbuck Ruma Tavorath Senior Environmental Specialis SASDI (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY05 5.71 23.260 FY06 32.45 120.599 FY07 45.25 213.342 FY08 63.74 342.082 FY09 25.20 121.958 42 Total: 172.35 821.231 Supervision/ICR FY09 17.34 66.616 FY10 84.18 302.392 FY11 59.83 254.775 FY12 55.40 225.224 FY13 63.50 257.085 FY14 36.49 195.941 Total: 316.74 1292.034 43 Annex 5. Beneficiary Survey Results (if any) NA 44 Annex 6. Stakeholder Workshop Report and Results (if any) NA Annex 7. A) Summary of Borrower's ICR World Bank Assisted National Vector Borne Disease Control Support Project Executive Summary World Bank supported project “National Vector Borne Disease Control Support Project” for Malaria control and Kala-azar (KA) elimination was approved and signed by the Government of India and the World Bank for a period of five years from 2008-09 to 2012-2013. The Financing agreement was signed on 6th March, 2009. The project is to support NVBDCP to implement evidence based interventions for malaria control and elimination of kala-azar in districts with highest burden of these diseases in a phased manner. Some new initiatives have been taken up under the project for the control and management of two vector-borne diseases: malaria and kala- azar. The Polio component supported procurement of oral polio vaccines required for high coverage of eligible children under supplemental national and sub-national polio immunization rounds in the country. Project Objectives (Malaria) The Objective of the project is to:  Reduce Malaria morbidity by 25% by 2013 ( Base Year 2007)  Reduce Malaria mortality by 50% by 2013 ( Base Year 2007); &  Achieve Kala-azar elimination by 2015 Project Components (malaria): The project has four components: Component 1: Improving Access to and Use of Services for Control of Malaria: This component includes the following sub-components:  Improving Malaria Case Management;  Strengthening Malaria Surveillance; and  Effective Vector Control Component 2: Improving Access to and Use of Services for Elimination of Kala-azar: This component includes the following subcomponents:  Improving Kala Azar Case Management;  Strengthening Kala Azar Surveillance; and  Effective Vector Control; Component 3: Policy and Strategy Development, Capacity Building, Monitoring and Evaluation: This component includes the following sub-components:  Policy and Strategy Development; 45  Program Management and Capacity Building; and  Monitoring and Evaluation. The project has been implemented in two phases. In Phase I, the project started supporting since 2009, Malaria control & Kala-azar elimination activities in 50 malarious districts of five states namely Andhra Pradesh, Chhattisgarh, Jharkhand, Madhya Pradesh and Orissa; and 46 Kala-azar districts in three states namely Bihar, Jharkhand and West Bengal. From 3rd year, Phase-II had been implemented in remaining 43 malaria districts of Gujarat, Karnataka & Maharashtra after approval from the World Bank. It was requested to the World Bank to include 30 districts of Jharkhand, Odisha and West Bengal which were covered earlier under the Intensified Malaria Control Project-I supported by Rd.4 of Global Fund till June 2010 and one district (Adilabad) of Andhra Pradesh which is in tribal area and is having problem of malaria. The World Bank approved the proposal and accordingly, they were included in the Phase II of the Project after assessment for district readiness. Thus, the project had been implemented in a total of 124 districts of 9 states. District implementation plan was prepared by all project districts using a model action plan. The key components of which are case management, integrated vector management, logistics control and capacity building and sensitization of CBOs. The assessment for District readiness for implementation of World Bank supported implementation plan was carried out in a phased manner in Phase I districts in 2009 and for newly included Phase II districts in 2011. The project was implemented in all the certified districts. The World Bank missions have also reviewed periodically (every six monthly) the progress of the project activities. During the project period, seven missions have reviewed the progress of the project. Under World Bank project the following inputs are being provided:  Human Resources o State level Consultants:- M&E, Finance Training, IEC/PPP, Logistic & Procurement consultants. o District Level: - One VBD consultant for each of the project district. o Sub-district level: - Malaria Technical Supervisor o Sentinel site hospitals: - Laboratory technicians  Logistic support:-RDT, ACT, Artether injections and LLIN  Support for M&E which includes mobility support for field visit for monitoring and supervision. Support for training  Hiring of Central level Agencies for BCC and Vulnerable Community Plan (VCP) HR Inputs provided under the programme: HR status in the World Bank supported project is given as under: a. State Consultants- 6 Consultants per State, Sanctioned (79) & In-position (44) b. Consultant (Vector Control)- 1 Consultant at ROH&FW, Sanctioned (9) & In- position (6) c. GIS Data Entry- 1 per State, Sanctioned (9) & In-position (5) d. Accountant- 1 per State, Sanctioned (9) & In-position (4) e. Secretarial Assistant- 1 per State, Sanctioned (9) & In-position (4) f. Insect Collector- 2 per State, Sanctioned (18) & In-position (11) g. Dist. VBD Consultants- 1 Consultant per Dist., Sanctioned (170), In-position (130) 46 h. Financial & Logistic Assistant- 1 per District, Sanctioned (124) & In-position (84) i. Data Entry Operator- 1 per District, Sanctioned (124) & In-position (82) j. Lab. Technicians- 3 per District, Sanctioned (372) & In-position (230) k. Malaria Technical Supervisors- 6 per District, Sanctioned (744) & In-position (550) Capacity Building: Under the project, the capacity building has been taken up as a major component. A country wide training programme was undertaken in collaboration with NIHFW to train the faculty of medical colleges in VBD control programme activities.  40 zonal medical colleges were involved in the training which imparted training to regional medical colleges and district level officials with the result that a rich resource of Master trainers at district/ zonal and state level has been created which will be helpful in imparting future training to the personnel involved in VBD control programme.  District VBD Consultants of the project states were trained in batches of 25 at various central institutes like NIMR, PHFI, NCDC and RMRC  MTSs were trained at ROHFW, Bhubaneshwar, Patna and other training institutes  District VBD officers were trained at NCDC Delhi in 2 batches  Besides, MOs and other health field staff were trained at state/ District and sub- district levels by the trained staff of the programme and faculty from other research institutions District Readiness Assessment for Implementation of World Bank supported NVBDCP: A standard protocol for processes and certification to assure district level readiness for NVBDC project implementation was agreed between NVBDCP and the World Bank. All 124 districts have been certified as ready for implementation from the date of the visit of the team. Rapid Diagnostic Test (RDT): Rapid diagnostic kits have been introduced at community level (ASHAs & other health workers) for quick diagnosis of P.falciparum malaria. ASHAs in the project districts have been trained for using RDT for diagnosis and slide preparation. Guidelines for the use of RDT in the field were given to the states from the NVBDCP. Record is maintained by the ASHA in M-ASHA register and performance based incentive has been provisioned for the services provided by them. The programme has introduced bivalent RDT kits from 2012 as majority of the Phase II districts are Pv predominant. A total of 240.97 Lakhs of RDT Kits for Malaria and a total of 28.06 Lakhs of ACT Combi Blister Packs have been supplied in the World Bank States till the year 2010-11. Artimisinine Combination Therapy (ACT): All confirmed P.falciparum cases diagnosed by rapid diagnostic kits (RDK) or by malaria microscopy in districts under the project are being treated with effective Artemisinin-based combination Therapy (ACT). The programme is using at present Artesunate + Sulphadoxine + Pyrimethamine combination and as it is still very effective; the programme has not introduced other ACT combinations. However, four combinations are at present approved by the DGC (I) in India. From 2010ACT has been recommended as first line of treatment of all Pf cases in the country including project districts. Long Lasting Insecticide Nets (LLINs): The project has envisaged introduction of LLINs in most endemic inaccessible malaria areas where IRS operation is difficult. In 2009, 1 million LLINs have been supplied to the states. 5.18 million LLINs (Planned under 2009-10 and10-11) have also been supplied to Project states /districts (of Phase I) and distributed to the beneficiaries along with appropriate IEC/BCC activities. Its use will be monitored through local level field workers and through LQAS. The procurement of 11 million LLINs (Under 11-12 and 12-13) is in 47 process. It has been delayed due to various reasons. A total of 75.28 Lakhs have been procured and supplied in the World Bank States out of which 75.12 Lakhs have been distributed from states to districts by covering 52 districts and achieving 97% target of the LLIN distribution. Monitoring: With the technical support from the World Bank the existing recording and reporting (M&E) system was revised. New record keeping and reporting formats were developed for case management activity and the vector control activities. These formats were communicated to the states and are being implemented by the states. At the National level, the effort to make functional the web-based NAMMIS was made with the support of Tata Consultancy Service. However, the response at the implementation level by the states was not encouraging. The Government is planned to integrate the VBD reporting system with the HMIS - reporting system under NRHM. The format for reporting through HMIS by the District Programme Management Unit has been designed and is now uploaded on the HMIS of NRHM. After due training it will be made functional. However the data available will be of district level. So the states have been requested to use the NAMMIS for sub-center wise data entry. For monitoring of programme implementation at micro-level i.e at sub-district level, Lot Quality Assurance Sampling (LQAS) method is used by MTS for assessing the progress of impact of the project. The training of first batch of MTS for use of LQAS was completed in Orissa during 19th to 22nd October, 2009. A Pilot study of its use was carried out in Orissa with the technical support of DFID. Based on its observation necessary changes in the formats were made. Varying rounds of LQAS have been implemented in different states based on capacity and local problems. (MP have done data collection in eight rounds, Odisha in four rounds, Jharkhand Chhattsisgarh in three rounds and AP in six rounds). The results of the same are used at the local level for identification of weak areas. This method is also being used in GF supported IMCP project areas and it is one of the important indicators to be reported semi-annually. The state level trainers of IMCP states were also trained along with the trainers of the WB project states. It is a good example of ‘good practice’ of one project being extended in another project in the country. The district VBD officers and the Consultants are involved in monitoring and validation of data collection at the field level. The modules have been revised to include any missing indicator or to obtain data on new indicators. Monitoring and Evaluation has been strengthened by providing effective supervision and surveillance by involvement of ASHA and MPW at ground level. Besides the newly recruited KTS/ MTS, VBD Consultants and District/state officials regularly visit the field for better supervision of the disease control activities. They have a monthly schedule of mandatory visits which is submitted to the district/state officials in advance and visits are conducted as per the schedule. The district malaria officers and CMOs/CMHOs conduct monthly review meeting at district Headquarter and the state officials also participate in that meeting. The state also conducts monthly/ quarterly review meeting at state Headquarters wherein the representatives from Directorate of NVBDCP also participate and programme implementation activities are reviewed. For regular assessment of programme implementation activities and its impact, 6 monthly lot quality Assurance Sampling survey is conducted at block level, which is used as a tool for assessment of performance and for taking corrective actions at local level. At Directorate of NVBDCP, regular disease-wise review meetings are held for monitoring the programme at National level and taking corrective measure as and when needed. In all project districts, Sentinel Site Hospitals have been identified which provide data on the trend of diseases in the community and severity of illness which helps in taking decisions for specific causes. Every 6 months, project is reviewed by World Bank review mission team consisting of World Bank officials, International representative of various bilateral organizations. During this mission, extensive field 48 visit is also done through pre-mission visits by World Bank and NVBDCP teams for assessing ground level status of implementation activities. NAMMIS: data recording and reporting is strengthened by introduction of web-based MIS called NAMMIS. Efforts have been made to operationalize NAMMIS throughout the country. Mobility Support a. Motor bikes to the MTS b. Vehicles to DMOs (Tata Safari) to facilitate supervisory field visits and better monitoring of programme implementation activities. c. Budgetary provision for POL to the project activities for extra field activities. Sentinel sites: Two sentinel sites have been identified in all the project districts and Lab. Technicians (LTs) are provided for strengthening the diagnosis services at sentinel sites. Module including the guidelines for sentinel sites was developed with the technical support from the World Bank and WHO. This included a recording and reporting system for the sentinel sites. Based on this Module, Pilot training of SSMO and LTs was conducted in Orissa. This experience was expanded in the second year in other project districts. The states of Madhya Pradesh and Orissa have regularly started monitoring the reporting form these sentinel sites. Other states are in process of making those centres fully functional and now they have also started receiving regular reporting from these centres in 2012. The sentinel sites have also been made functional in districts involved in Phase II. So far out of 348 identified Sentinel sites under World Bank supported project states, 331 sites are functional and 303 sites are reporting to the concern state officials. Development of operational manual for programme manager & updating of existing training modules: An exhaustive Operation manual (2009) for the programme was developed with the technical support from the World Bank. The existing training module for Medical Officers and MPWs were also revised to include the new initiatives in the module. As MTS and ASHAs have been included in the project / programme, a training module for both of these cadres was developed. Recently, the Caritas India which is a project partner in GF supported IMCP-II project has helped in developing the revised ASHA module and the guide for facilitator. For newly recruited VBD consultants, a VBD training module was developed with the help of PHFI, NCDC, NIMR for their 3 months training by PHFI at Delhi. The 3 months training for the first batch of 26 Distt. VBD Consultants was completed during September to December 2009 at PHFI. The 2nd batch training of 23 VBD Consultants was completed during November, 2010 to January, 2011 by PHFI. The 10-days MTS training was completed at RMRC (ICMR) Bhubaneshwar for the first Phase I districts. The training for the VBD consultants for the newly included districts has been done with the help of NIMR and NCDC from June 2012 in two batches. The trainings for MTSs of the Phase II districts are being organized at regional levels in different states with the help of NIMR and ROHFW. Status of manpower and their training is given in the chapter of Human Resources. Operational research activities of rapid assessment of Artemisinin monotherapy, baseline Household survey, monitoring of therapeutic efficacy, insecticide resistance, pharmaco-vigilance exclusively focusing on side effects of drugs, quality assurance of rapid kits and household survey were entrusted to NIMR for malaria in the year 2008-09. NIMR has completed the studies of rapid assessment of Artemisinin monotherapy and the baseline household survey and submitted the reports. Remaining studies are undergoing at NIMR through its 15 field stations scattered throughout the country and their progress reports are submitted regularly at the 49 NVBDCP and are periodically reviewed. Operation research studies on alternative ACT drugs are also being conducted by the NIMR. The status of OR studies is given in other document. Hiring of Agency: For specialized activities in the programme for which technical expertise is not fully available in the Directorate; specific agencies have been hired to facilitate the programme related activities these are as under (details are given at Annexure): a. For IEC/BCC activities- M/s New Concept Information Systems Pvt. Ltd, New Delhi was hired for conducting formative research on BCC and to develop BCC strategy and implementation plan for project states. b. For Social Mobilization & Service Delivery activities- Voluntary Health Association of India (VHAI), New Delhi was hired for Capacity building and promotion of correct & continuous use of Insecticide treated bed nets (LLIN). c. For Supply Chain & Management activities- M/s Strategic Alliance Management Services, New Delhi was hired for conducting trainings for Malaria control & Kala-azar elimination in project states. d. For Implementation & Fiduciary Review of Decentralized Activities- J.P. Associates, New Delhi was hired for periodic implementation and fiduciary review of decentralized activities in project areas. e. For Environmental Management Plan (EMP)- M/s Senses Consultants India Pvt. Ltd., Kolkata was hired for conducting Pilot study and National workshop for environmental codes. Trainings were also organized at state level & M&E activities were organized. f. For Trainings on VBDs Control- National Institute of Health & Family Welfare (NIHFW), New Delhi was hired for conducting trainings to Faculty of Medical Colleges, District level Officers. All trainings were completed by 31st March, 2013. Environment Management Protection is a novel innovation in the programme for better environment management and for better handling, storage, use and disposal of hazardous insecticides used under the public health programme for community and environment protection. Under this project capacity has been built up. Epidemiological Impact Indicators: The epidemiological impact has been positive of the interventions done like LLINs, provision of ACT and RDT for EDCT. The district wise and state wise details are given at Annexure. However, a comprehensive and concise assessment is represented below in the tables and graphs: Achievement of the Objective of the project No. Objective Base Year Target by 2013 Achievement 1. Reduce Malaria morbidity (Cases) 2007 by 25% 41.87% 2. Reduce Malaria mortality (Deaths) 2007 by 50% 69.47% Trends of indicators in All World Bank Project Districts (124) of India 50 Year Malaria Cases Deaths due to Malaria 2007 927054 524 2008 911421 499 2009 953289 523 2010 1013628 584 2011 842845 452 2012 655502 293 2013 540203 160 Year wise Status of Cases & Deaths of 124 Districts of World Bank supported Project 1200000 700 584 1000000 600 524 525 Pv, Pf & Total Malaria Cases 503 452 500 800000 Deaths 400 600000 293 300 400000 200 160 200000 100 0 0 2007 2008 2009 2010 2011 2012 2013 PV PF Malaria  Cases Deaths % Achievement in 2013 as compared to 2007: Pv Cases: 52.45 % ; Pf Cases: 35.09 %;  Malaria Cases: 41.87 ; Deaths: 69.47 % * Data of 2013 is provisional The status of malaria in 50 districts which were covered throughout the project period is shown in the following graph: 51 Malaria Cases and Deaths in World Bank supported Project ‐50 Dist.  Of Phase I   (2007‐2013*)   700000 300 600000 250 Pv, Pf & Total Malaria Cases 500000 200 400000 Deaths 150 300000 100 200000 100000 50 0 0 2007 2008 2009 2010 2011 2012 2013* PV PF Malaria Cases Deaths • 31.56 % reduction in cases • 70.74 % reduction in deaths in 2013 compared to 2007 Status of malaria cases and deaths during 2007-13 in the Phase II(74) districts is as follows: Kala-azar 1.1. Objective The Kala zaar set for elimination by 2015 i.e reducing the number of KA cases less than one per 10 thousand population. 1. To reduce Kala azar cases 2. To reduce the case fatality 3. Sustain the achievement 1.2. Epidemiological trends The State of Bihar contributes about 80% of the total disease burden of the country. In Bihar, 9 districts out of 31 districts contribute between 65-70% of Kala-azar case. 52 2. Capacity Building:  Capacity building is an integral and important component of any disease control programme. For strengthening the programme components in the field mainly the surveillance and complete treatment, Kala-azar Technical Supervisors (KTS) were provided to three endemic states for above purposes. This provision necessitated increasing the technical capacities of these supervisors and therefore the training programmes on all aspects of Kala-azar elimination programme were organised at State/district/block levels.  A total of 183 KTS from the three states trained by RMRI, Patna (Bihar - 149, WB - 16 & Jharkhand -18) for 10 days during 2011. 23 KTS from West Bengal were trained in May 2013.  26 VBD Consultants have been imparted 3 months training by RMRI Patna and PHFI from 17th Jan to 16th April 2011. Another batch of untrained VBD (21), Consultants training is scheduled was schedule in October/ November 2013 but could not be done due to spray activities during that period. Now proposed in middle of 2014 after completion of 2nd round IRS activities in the states. 3. Diagnostics, Drugs & Insecticide: All the KA endemic districts are being provided free rapid diagnostic kits and first line drug Miltefosine and second line drug Amphoterecin B injection. Both drug and diagnostics are available upto block level. DDT 50% insecticide is being provided by Govt of India free of cost to all endemic states during indoor residual spray. Seeing the long duration of treatment of 28 days with Miltefosine and treatment compliance, the expert committee suggested short duration of treatment in the programme in August 2013 with the flowing combination: i) Miltefosine + Paromomycin for 10 days at block level ii) Ambisome injection single dose at district /referral /medical college. 53 At present introduced in seven identified districts and shall be expanded in all the districts in phased manner. States are in process for purchasing of drugs. RMRI,DNDi, MSF is helping the resource mobilisation, training and updating heath system. 4. Indoor Residual Spray:  During 2012, two round Indoor Residual spray with DDT 50% was carried out in all the three Kala-azar endemic states covering a population of 38.37 million for 1st round and in the 2nd Round spray with DDT covering 27.61 million populations.  In 2013 all the states have completed first and second round of IRS Central teams from NVBDCP, NCDC, RMRI, Regional office visited the Kala-azar endemic area for supervision and Monitoring.  28 districts were covered for IRS quality by central teams consisting officers from NVBDCP, NCDC, RMRI, Patna and RD Office Patna. The overall observations of the visiting team have indicated moderate coverage of IRS during both the rounds; the PPE has been used by the spray teams for IRS. However the quality of spray was not very good though districts have made efforts to cover as many rooms for larger impact of IRS. The problem of leakage of pumps, lack of supervision and monitoring on the part of district authorities have been noted, the efforts on the part of IEC has been done by many districts.  As done in 2012, training of trainers on IRS by M/s. Hindustan Insecticides Ltd the was also done in 2013 (before 1st round) in Bihar involving district VBD officers, VBD Consultants. 5. Incentives:  Rs. 200/- as incentives to ASHA/Health Volunteer for referring a suspected case and ensuing complete treatment.  Loss of wages @ Rs. 50/- per day to the confirmed case of kala-azar during the entire period of treatment.  Free diet to the patients and one attendant during treatment period. 6. Endemicity of Kala-azar (Per 10,000 Population at block level) The table below shows that nos. of blocks showing <1 per 10,000 population has been increased in 2012 in comparison with 2011. The reason attributed is better case detection and complete treatment. In 2011 against 584 blocks 309 blocks (53%) have reported cases <1 per 10,000 population. In 2012 out of 579 blocks 342 block (59%) have reported cases <1 per 10,000 population. 7. Initiatives  The reporting formats for recording Kala-azar cases vary from district to district and state to state. A necessity was felt for developing a uniform Health Management Information System (HMIS). One such meeting held at Siliguri on 16th -17th October 2012 involving SPOs of three KA states, DMOs, Consultants for discussion on five proformas/formats developed by NVBDCP with the help of World Bank Consultant. The piloting in two districts (Saharsa & Madhepura-Bihar), (Sahibganj & Pakur – Jharkhand) and (Malda and Darjeelling – West Bengal) has completed. Dte. has developed online software for capturing the data with the help of CARE which is now completed and shortly the HMIS shall be rolled out in the country. 54  Programme has also carried out situation analysis on KA prevalence on the facilities available and the man power situation in the state of Bihar in 2012-13. The preliminary analysis indicates that there is no great difference between the epidemiology on the ground and that reported in the programme. About 30% of the suspected cases contact the private sector and most returned to the public health facilities and ultimately only 4% are seeking treatment exclusively by the private sectors i.e. 96% cases are going to public health facilities.  Mobile phone with IRS apps provided to District VBD Officer on day to day reporting activities on IRS.  State Health Mission, Bihar has taken a decision to provide mobile phone to ASHAs.  Hindustan Insecticides Ltd. services are being utilised for quality training of Indoor Residual Spray to the trainers. In 2013 HIL has imparted training to trainers DMOs, BMO, VBD Consultant and supervisors at Patna. The training held in 2012 has given dividends and the quality of spray has improved.  Voluntary Health Organisation representatives for strengthening social mobilization and service delivery for improving the implementation activities and involvement of new concept- an agency hired by NVBDCP for developing IEC /BCC materials on Kala-azar including training. 8. Involvement of Stakeholders There are multiple challenges affecting the KA elimination program particularly in the state of Bihar including underreporting of cases and deaths, lack of awareness in the community, no quality monitoring of Indoor Residual Spraying (IRS), prolonged duration of treatment, PKDL (post Kala Azar Dermal Leishmaniasis) cases, TB co-infected cases, weak referral system and no proper active case search. Seeing the gravity of the problem some of the potential partners have come forward to work with the Govt. health system in the field to improve access to and use of services for elimination of Kala-azar, expanding the effectiveness, reach and visibility of Kala Azar elimination efforts through IEC/ BCC, Community mobilization and Civil Society support and improving Indoor Residual Spraying (IRS) in endemic blocks of the project. The Partners are: MSF; DNDi & RMRI; BHU 1. BMGF (CARE, PATH, DFID, World Health Partners (WHP) Coalition for Strengthening Kala Azar Elimination Project (CSKAEP): This is the new initiative taken by CARE with support from the Bill Mellinda Gates Foundation (BMGF) along with stakeholders like B-TAST (DFID), PATH, RMRI, State Malaria Office, CBOs and private sector like World Health Forum to bring various stakeholders (state/govt. and NGOs) on one platform to improve access to and use of services for elimination of KA, reach & visibility efforts through IEC/BCC, community mobilization and Civil Society support, improving IRS coverage in project (8 districts in Bihar). This coalition has held two meetings in December 2012 and April 2013. The purpose of this coalition is to increase awareness of and promote Kala Azar elimination strategies for informed choices by all stakeholders. Objectives of the Coalition:  To advance Kala Azar Elimination policy initiatives to government and other key decision makers in the State. 55  To advocate and raise awareness around common challenges to elimination  To initiate activities to achieve common goals  Synergising efforts between stakeholders and cross learning between partners Activities:  Coalition activities include but are not limited to:  Research and production of position papers and briefing notes  Presentations at forums, conferences, meetings  Engagement with partners and other organizations  Advocacy with stakeholders including the Government for addressing operational problems  Development and implementation of an annual advocacy strategy CARE Patna branch has hired 68 link volunteers at district level (@ 6 each district) for 8 districts and were trained for 2 days at RMRI, Patna to help the programme on monitoring and supervision. These link volunteer have been oriented for monitoring and supervising of IRS Spray. The HR by CARE shall be expanded in all the 33 districts. 9. Meeting /Workshops (Jan – 15th July 2013)  WHO team Dr. Jean Jennin and Dr. Daniel, WHO HQ Geneva visited Bihar from 28th Feb to 4th March 2013 to oversee Kala-azar elimination programme at Village level.  World Bank team with NVBDCP officer visited district Muzaffarpur from 20th – 23rd March 2013 for 1st round IRS supervision and programme activities.  Meeting of expert committee on Kala-azar held on 18th April 2013 to review the phase III clinical trial of Amphomul injection (funded by the Dept. of Science and Technology).  NVBDCP officer visited Patna to attend Coalition for Strengthening Kala Azar Elimination Project (CSKAEP) meeting on 28th – 29th April 2013.  World Bank team and NVBDCP Officer visited Saran district Bihar to review Kala- azar elimination programme from 10th – 12th May 2013.  NVBDCP officer participated a meeting with Bill Gates Melinda Foundation on KA elimination on 29th May 2013.  A Regional Review meeting of State & District Programme Officers was organized at Patna on 11th – 12th July 2013 to review Kala-azar elimination. 10. Issues  Delay in release of funds to districts for implementation of programme activities and delay in submission of utilization certificate to Govt. of India.  Inadequate supervision & monitoring of the programme due to large number of vacancies of health functionaries like DMOs, Malaria Inspectors at various levels. The monitoring and supervision is also suffering due to non positioning of contractual staff. ASHAs honorariums are not being paid timely resulting their reluctance to work in KA programme.  Dedicated and more sincere efforts from District Malaria Officers of high endemic districts of Bihar is required.  Timely payment of loss of wages to Kala azar patient is not being adhered. 56  Due to weak IEC/BCC and inadequate social mobilization the acceptance of DDT spray is poor among the community.  Training to all categories needs to be strengthened.  District Collectors need to be involved in regular review of the programme at district level. 57 Year wise Status of Key Project Outcome Indicators in Project NVBDCSP (a) Increase the number of people Achievements (in percentage) Final benefiting from effective Targe prevention, diagnosis and Name of the Yr.1 Yr.2 Yr.3 Yr.4 Yr.5 t treatment services for malaria and State 08/09 09/10 10/11 11/12 12/13 kala-azar, (Baseline) *% % % % # (i) Percentage of fever cases in Andhra 10-58% # NA* 60.7 75.6 90.7 70 % project districts receiving a Pradesh malaria test result no later than Chhattisgarh 27.4 NA 38.4 the day after first contact. Jharkhand 36.4 35.3 40.3 Madhya 28 41.1 54.8 Pradesh Odisha 39.3 44.5 74.4 (ii) Percentage of individuals in Andhra 77% # NA* 4.5 19.1 22.5 50 % project areas belonging to eligible Pradesh (Slept (Slept LLIN target population who slept Chhattisgarh under 24.6 NA 37 under under an LLIN during the Jharkhand ITN) 11.2 6.3 43.5 LLIN previous night. = Madhya 5.5 5.3 20 ) Pradesh Odisha 49.7 31.0 22.17 (iii) At least 50%of the sample 0.0 % # NA* 20.9 52.9 58.6 50% blocks which at baseline have not achieve the elimination goal of less than one kala-azar cases per 10,000 persons (584), will achieve elimination by end line. (iv) At least 80% of the (from MCH >80%# NA* >80 households with eligible children division) % covered during the National and Sub-National immunization days in high risk districts. # This baseline data is the data from the Household surveys conducted by NIMR in 2008-2009 = In the baseline survey this indicator gave the information on percentage of individuals who slept under ITN (Insecticide Treated Net). LLIN( Long lasting insecticidal Net ) distribution was started under the project, in year 2010 and so the final target is for use of LLINs where it has been distributed. *NA = Not available. The data was to be collected by LQAS surveys to be done by MTSs/KTSs. As the recruitment and trainings of MTSs/KTSs were done in Year 2, the LQAS surveys were started in year 3 onwards after their training for LQAS and so the data for year 2 are not available. (iii) The blocks that not achieved the goal of elimination were targeted for intervention under the project and so, the baseline status was 0 %. It was estimated that it will be brought up to 50 % Data for polio coverage are available from MCH division 58 Intermediate Result: Component One: Improving Access to and Use of Malaria Prevention and control Services Achievements (in percentage) Final Name of the Yr.1 # Yr.2 Yr.3 Yr.4 Yr.5 Target State 08/09 09/10* 10/11 11/12 12/13 (Baselin % % % % e) 1. Percentage of Rapid Andhra 20%# NA* 7.4 NA 65 70% Diagnostic Test (RDT) Pradesh positive in Project District Chhattisgarh 2 NA 1.9 receiving Artemisinin based Jharkhand 32.5 32.5 33 combination therapy ( ACT) Madhya 11.2 8.9 15.3 no less than the day after the Pradesh first contact. Odisha 33.3 32.2 31.7 2.Percentage of designated Andhra 0%# NA* ACT- ACT- ACT-67.5 90% providers of malaria Pradesh 61.1 56.9 RDT-52.8 diagnosis and treatment who RDT- RDT- have not had an ACT or RDT 69.1 77.8 stock out during the last 3 Chhattisgarh ACT- 59 NA ACT-41 months RDT- 56.7 Jharkhand NA RDT- NA 41.5 Madhya ACT- ACT- ACT-80.7 Pradesh 25.7 85.1 RDT-85.9 RDT- RDT- 23.7 26.5 Odisha ACT-7.5 ACT- ACT-80.7 RDT- 61.9 RDT-40.3 32.7 RDT- 70.3 3.Number of long lasting Nil 1,000,000 0 5,179,11 9,262,625 8,350, insecticide treated malaria 4 (purchase 000 nets purchased and /or in process) distributed Component Two: Improving Access to and Use of Services for Elimination of Kala azar 1. Percentage of diagnosed 55% NA* 90.09 91.03 93.75 >80 % kala-azar cases completing the standard treatment as per the National guidelines. 2 Percentage of facilities 73% NA* RDT- RDT- RDT-9% 90% reporting no stock-outs of 59% 76% ‘Rapid Diagnostic Test for 1st line 1st line 1st line Kala azar’ and first line drug - drug - drug - medicines. 33% 35% 66% Component Three: Policy and Strategy Development, Capacity Building and Monitoring and Evaluation Percentage of planned 48-100 48-100 45 to 40-100% 40-100% 100 % additional staff who are in 100% 59 position at central, state and district levels and received induction training Proportion of eligible districts 0 68 100 100 100 100% meeting the readiness criteria for each period of implementation. Number of Health personnel 0 445 600 780 950 receiving training. Indicators Statu Name of the 2010 2011 2012 Data s in State (%) (%) (%) collectio 2009 n (%) method Intermediate Result: Component One: Improving Access to and Use of Malaria Prevention and control Services Percentage of population in high-risk NA Andhra 32.6 53.5 65.3 LQAS project areas protected by ITNs or IRS Pradesh Survey Chhattisgarh 62.5 NA 84.2 report Jharkhand 29.2 41.5 41.5 Madhya 19.1 9.4 32.9 Pradesh Odisha 68.6 48.2 41.9 Odisha Percentage of villages with a trained NA Andhra 97.7 NA 19.1 LQAS designated provider of malaria Pradesh diagnoses and treatment services. Chhattisgarh 80.9 NA 12.4 Jharkhand 59 74.9 NA Madhya NA 86.4 94.5 Pradesh Odisha 88.7 82.7 66.6 Component Two: Improving Access to and Use of Services for Elimination of Kala azar Percentage of houses in identified in NA 93.1 95.74 97.22 Annual kala azar endemic areas covered with report effective insecticide spray. Component Three: Policy and Strategy Development, Capacity Building and Monitoring and Evaluation Monitoring system established in 5+ 1 1 1 1 Record sites to monitor the quality of RDTs, drugs and insecticides delivered by the procurement system Pharmaco-vigilance system established 0 3 4 sites 4 sites RMRI in at least 3 sites to monitor the first line sites report medicines for kala azar elimination 60 Indicators Statu Name of the 2010 2011 2012 Data s in State (%) (%) (%) collectio 2009 n (%) method Percentage of endemic districts with 100 100 100 100 Record quality controlled incidence data of vector-borne diseases stratified by age and gender NA: Not Available Notes:  Status of year 2009 is derived from the data of baseline survey conducted in 2009.  Status in 2010, 2011 and 2012 are derived from the LQAS surveys done in project States.  Some of the states could not implement the LQAS in all the years.  The shortage of 1st line drug for Kala-azar was met through DNDi. RDT for Kala-azar expired in Nov. 2012 61 Annex 7. B) Borrower's Comments on Draft ICR Directorate of National Vector Borne Disease Control Program Comments on Implementation Completion and Results Report, World Bank The report is well written and comprehensive. The point wise specific comments on the report of NVBDCP are as under:- (A)Overall general comments on the report:-  Considering the impact of the project on the malaria and kala-azar morbidity and mortality the project outcome may be rated as ‘moderately satisfactory’ instead of ‘moderately unsatisfactory’.  C.1 Ratings summary/Performance rating by ICR - Borrower performance- instead of unsatisfactory it may also be rated as moderately satisfactory. Further the World Bank implementation Support Missions reviewed the progress of the project from time to time and has rated the project moderately satisfactory and Implementation Support Mission 13th September to 15th October 2010 has rated project as Satisfactory. Therefore, overall rating should be moderately Satisfactory.  Considering the steps taken by the Banks in the prevailing circumstances their performance may also be rated as ‘moderately satisfactory’ instead of ‘moderately unsatisfactory’.  Similarly, in the report at various sections the rating as ‘moderately unsatisfactory’ may be replaced with ‘moderately satisfactory’.  It is understood that by making such modification the report can be made positive without changing overall implication. Further it is informed that the National Health Mission (NHM) came into existence in 2013 merging National Rural Health Mission (NRHM) and National Urban Health Mission (NHUM). (B) Specific comments on the report:-  Under PDO indicators (a) and Intermediate Outcome Indicators (b) some changes are suggested and may be seen in the track mode. May be reviewed and changed accordingly.  Few minor changes/ additions i.e. at p.no.11, 20 & 48-49 may be seen in track mode for review/consideration and changes. (C) Some factual explanation on the project implementation:-  ASHA incentive- it should be considered as one of the important innovations of the programme for ensuring diagnosis and treatment at community level in time. It was met from Domestic Budget Support and should be considered as counterpart financing.  The observation made on delay decision making/engaging consultant agencies / recruitment / procurement etc , it is submitted that the all laid down codal formalities/ checks & balances are to be observed to follow procedures for meeting the administrative and financial requirement . Accordingly, the final view may be taken in the report.  It is also observed that the World Bank norms for procurement also impeded the progress of project work as decentralised procurement was not allowed by the bank even for the 62 items like motor cycle, vehicles, computer etc. For these items the states and PSUs have very clear and transparent procedure at any part of the country, hence decentralized procurement of these items would not have made any difference in financial implications but certainly the expeditious procurement could have facilitated effective supervision and to monitoring of implementation of program. 63 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders 64 Annex 9. List of Supporting Documents  Project Appraisal Document (PAD), National Vector Borne Disease Control and Polio Eradication Support Project. July 30, 2008  Financing Agreement (National Vector Borne Disease Control and Polio Eradication Support Project) between INDIA and International Development Association, February 13, 2009  Aide Memoire of all missions conducted under the project.  Implementation Status and Results (ISR) documents from the project.  India NVBDCP Quality Enhancement Review (QER) May 2, 2007  India Detailed Implementation Review (DIR) Report 2006-2007  National Vector Borne Disease Control Program: Joint Monitoring Mission Report, February 2007  National Vector Borne Disease Control Program: Joint Monitoring Mission Draft Report March 2014  WHO India Country Cooperation Strategy 2012 – 2017 65