44821 HEALTH, NUTRITION, and POPULATION SERIES Performance-Based Contracting for Health Services in Developing Countries A Toolkit Performance-Based Contracting for Health Services in Developing Countries A Toolkit Health, Nutrition, and Population Series Performance-Based Contracting for Health Services in Developing Countries A Toolkit Benjamin Loevinsohn © 2008 The International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org E-mail: feedback@worldbank.org All rights reserved 1 2 3 4 11 10 09 08 This volume is a product of the staff of the International Bank for Reconstruction and Development / The World Bank. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgement on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development / The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; fax: 978-750-4470; Internet: www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@worldbank.org. ISBN: 978-0-8213-7536-5 eISBN: 978-0-8213-7537-2 DOI: 10.1596/978-0-8213-7536-5 Library of Congress Cataloging-in-Publication Data Loevinsohn, Benjamin. Performance-based contracting for health services in developing countries : a toolkit / Benjamin Loevinsohn. p. ; cm. -- (Health, nutrition, and population series) ISBN-13: 978-0-8213-7536-5 (alk. paper) ISBN-10: 0-8213-7536-9 (alk. paper) 1. Medical care--Contracting out--Developing countries. 2. Performance contracts-- Developing countries. 3. Public-private sector cooperation--Developing countries. I. World Bank. II. Title. III. Series. [DNLM: 1. Contract Services--organization & administration. 2. Health Services-- economics. 3. Developing Countries. 4. Private Sector--organization & administration. 5. Public Sector--organization & administration. W 74.1 L826p 2008 RA410.55.D48L64 2008 362.1068'4--dc22 2008009421 Cover photo: Tran Thi Hoa/World Bank. Cover design: Naylor Design Inc. Contents Boxes, Figures, and Tables vii Why and How to Use This Toolkit xi Outline of the Toolkit xiii Acknowledgments xv Abbreviations xvii 1. Summary of the Toolkit 1 Background: What Is Performance-Based Contracting? 1 How to Contract 2 Whether to Contract 6 2. What Is Performance-Based Contracting? 9 Introduction 9 What Is Contracting? 10 What Is Performance-Based Contracting? 11 What Kinds of Health Services Can Be Contracted? 11 How Is Contracting Different from Other Approaches to Organizing Health Services? 12 What Approaches to Contracting Work Best in Common Situations? 15 How Is Contracting Related to Pay for Performance? 16 3. How to Contract 19 Step 1: Conduct Dialogue with Stakeholders 20 Step 2: Define the Services 26 Step 3: Design the Monitoring and Evaluation 37 Step 4: Decide How to Select Contractors and Establish the Price 44 Step 5: Arrange for Contract Management and Develop a Contract Plan 52 Step 6: Draft the Contract and Bidding Documents 55 Step 7: Carry Out the Bidding Process and Manage the Contract 64 v vi · Contents 4. Checklist for Contracting 67 5. Whether to Contract 69 The Evidence on Contracting 69 Methodology 70 Results 71 Discussion 72 Why Does Contracting Work? 82 What Are the Concerns about Contracting? 83 In What Context Will Contracting Work (or Not Work)? 93 Appendix A: Example of a Contracting Plan (or Manual) Strengthening Primary Health Care Services in Country X 97 Appendix B: World Bank Procurement Approach to Contracting for Health Services 119 Appendix C: Description of Evaluated Contracting Experiences 125 Appendix D: Terms of Reference for a Third-Party Evaluation of the Contracting Effort in Country X 161 Appendix E: Terms of Reference Examples 167 References 189 Index 195 Boxes, Figures, and Tables Boxes 3.1 Establishing Worker-Controlled Health Cooperatives in Costa Rica 24 3.2 Paying for Performance in Haiti 32 3.3 Accuracy of the Routine Administrative Records Part of the HMIS 40 3.4 Lot Quality Assurance Sampling 41 3.5 Collaborating with Private Practitioners to Treat Tuberculosis 46 3.6 Results of a Government Maintaining Control of Staff Assignments 57 3.7 Attracting Health Workers to Underserved Areas 57 3.8 Decentralizing Drug Procurement in Afghanistan 59 5.1 Contractor versus Government Performance in Industrial and Middle-Income Countries 73 Figures 1.1 The Seven Steps of Contracting 3 3.1 The Seven Steps of Contracting 20 5.1 Median Double Differences in Coverage Rates from Studies with Controlled, Before-and-After Methodology 72 C.1 Coverage of Prenatal Care by Approach Used 127 C.2 Change between Baseline and Endline Surveys 128 C.3 Scores on a Broad Index of Quality of Care 139 vii viii · Boxes, Figures, and Tables C.4 Use of BHU by Respondents Reporting Illness in the Last Month 156 C.5 Change in the Average Number of Outpatient Visits to BHUs per Month, Compared with Same Month of Year before in NGO (PRSP)-Managed Districts and Government-Managed Districts 157 Tables 1 Toolkit Appendixes xiv 2.1 Typology of Service Delivery Arrangements 14 2.2 Types of Contracting to Consider in Some Common Situations 15 2.3 Types of Pay for Performance and Their Relationship to Contracting 17 3.1 Reasonable Performance Indicators in a Contract for Primary Health Care 27 3.2 Some Less-than-Ideal Indicators in a Contract for Primary Health Care 28 3.3 Examples of Input, Process, Output, and Outcome Indicators 29 3.4 Examples of Indicators for a Performance-Based Contract 38 3.5 Means of Data Collection 39 5.1 Summary of Evaluated Contracting Experiences 74 5.2 Some Common Concerns about Contracting and the Global Experience 86 5.3 Financing and Provision of Services 91 5.4 Common Concerns about the Environment for Contracting 94 A.1 Key Performance Indicators and Targets 99 A.2 Next Steps for Agreement Implementation 106 AA.1 Key Performance Indicators and Targets 109 B.1 Procurement Approaches to Use in Different Situations 120 C.1 Changes in the Concentration Index of Health Services, Follow-Up Minus Baseline 129 C.2 Annual Total Health Expenditures per Capita, 2003 129 C.3 Changes in Key Indicators from Baseline to Endline in Project and Control Upazilas 132 C.4 Changes in Coverage, 2000­03 134 C.5 Changes in Coverage among the Poorest 50 Percent, 2000­03 135 C.6 Changes over Time in NGO and Publicly Managed Districts in El Alto 137 C.7 Reported Data on Service Delivery in Butare Intervention Area Compared with the Rest of the Country (Rural Areas) 142 Boxes, Figures, and Tables· ix C.8 Before-and-After Results of Household Surveys in Cyangugu 142 C.9 Comparison of Cooperatives and CCSS-Managed Clinics, 1990­99 144 C.10 Characteristics of the Areas Implementing the Different Approaches Used in Guatemala 148 C.11 Results of the Different Approaches to Service Delivery in Guatemala 149 C.12 Performance of NGOs in Haiti as Use of Bonuses Increases 151 C.13 Results from 14 Evaluated Initiatives 153 C.14 Change in Private Provider Behaviors 160 D.1 Key Performance Indicators and Targets 163 E.1 Targets for the PBA 171 E.2 Management Indicators for the PBA 172 E.3 Impact, Output, and Process Indicators for FSW HIV Prevention Services Contract 178 Why and How to Use This Toolkit Why Use This Toolkit? This toolkit provides practical advice to anyone involved in, or who is interested in becoming involved in, performance-based contracting of health services with nonstate pro- viders in the context of developing countries. It addresses many of the issues that may be encountered. Input from experienced contract- ing professionals will give newcomers increased confidence as they go forward. Experts directly involved in contracting on a large scale have contributed to the development of this toolkit. What Is Contracting? Contracting is a mechanism whereby a financ- ing entity procures a defined set of services from a nonstate provider. What Is This Toolkit About? Performance-based contracting is a type of contracting with a clear set of objectives and indicators, sys- tematic efforts to collect data on the selected indicators to judge con- tractor performance, and consequences for the contractor, either re- wards or sanctions, based on performance. Intended Audience. This toolkit has been developed for individuals working for government agencies, other purchasing entities (such as insurance companies and social insurance funds), nongovernmental organizations, faith-based organizations, private (for-profit) health care providers, World Bank staff, and development partners. Where to Start. The short summary gives an overview of the tool- kit. Readers can then delve into increasing levels of detail by working through the various sections and appendixes. xi xii · Why and How to Use This Toolkit Limitations of the Toolkit. The actual task of contracting is more of an art than a science. Although a reasonable amount of evidence suggests that contracting works (see section 5), many of the "how-to" issues are a matter of experience rather than systematic evidence. As we learn more about contracting, some of what we believe now may change. Keep in mind that contracting occurs in different contexts. Some issues raised in the toolkit are not relevant in certain situations but may be crucial in others. Don't Panic! A number of issues have to be considered in contracting, but everything need not be exactly right on the first try. Beethoven scratched out more than a few musical ideas as he was writing his Ninth Symphony, and scientific scans of Leonardo da Vinci's Mona Lisa show that he changed his mind about her more than once. Just like creating great art, every detail doesn't have to be perfect every time, and certainly not right off the bat. Other Useful Materials. Additional materials including key docu- ments, such as a contracting plan, draft contract, terms of reference, and checklist, are included in the toolkit and are available in MS Word format at http://www.worldbank.org/hnp/contracting. Outline of the Toolkit Section 1: Summary of the Toolkit. This overview is useful for ref- erence and for quick refreshers later. It will be helpful to read the sum- mary before moving on to the main part of the toolkit. Section 2: What Is Performance-Based Contracting? This sec- tion provides background on contracting, including definitions of key terms, the types of services that can be contracted, how contracting relates to other ways of organizing health services, and which con- tracting approaches work in different settings. Section 3: How to Contract. This section provides a systematic way of thinking about contracting and how to do it in practice. It looks at seven aspects of the contracting process from initial dialogue with stakeholders through carrying out the bidding process and managing contracts. This framework will help ensure a systematic consideration of the choices and challenges (see figure 1.1). Section 4: Checklist for Contracting. This checklist contains tasks and issues to address while designing and implementing a contract. The checklist can also be used to review an existing contract to see what is missing or could be improved. Section 5: Whether to Contract. This toolkit assumes that the reader has an interest in contracting, but it is useful to keep asking questions. This section reviews the evidence for contracting in devel- oping countries, explores why contracting appears to work, and ad- dresses concerns that have been expressed about contracting. xiii xiv · Outline of the Toolkit Table 1 outlines the five appendixes to this toolkit, the purpose of each, and when they should be used. Table 1 Toolkit Appendixes Appendix Purpose Point of use A Contracting Plan: Example of a contract- During the design phase Strengthening Primary ing plan or contracting of a contracting effort, Health Care Services in manual, which should step 5; includes a draft Country X be developed for all agreement between contracting efforts the purchaser and the contractor B World Bank Procurement Guidance for World Bank During the design phase Approach clients and staff on the of a contracting effort, procurement methods to steps 4 and 6 use in different contexts C Description of Evaluated Real-world experience During the design pro- Contracting Experiences that can be used to see cess; it may be helpful what others have done to read this appendix before dialogue with stakeholders, step 1 D Terms of Reference for Example of TORs for an In cases in which the Third-Party Evaluation independent evaluation purchaser would like to of a contracting effort evaluate how well the contracting effort has performed using a third party E Terms of Reference Examples of TORs for pri- During the design phase Examples mary health care service of the contracting effort; that lays out objectives, it would also be included scope of work, roles, and in the contract responsibilities Note: TOR = terms of reference. Your Suggestions and Comments Are Welcome. Please send us your suggestions about how this toolkit can be made more useful: http:// www.worldbank.org/hnp/contracting or healthpop@worldbank.org. Acknowledgments This toolkit was written by Benjamin Loevinsohn with the sup- port and insights of many people, including Anabela Abreu, Silvia Albert, Asha Ayoung, Patricia Baquero, Bernard Becq, Indu Bhushan, Sekhar Bonu, Emanuele Capobianco, Sadia Chowdhury, Mariam Claeson, Laura Coronel, Agnes Couffinhal, Tania Dmytraczenko, Rena Eichler, Phoebe Folger, Nancy Fronczak, Pablo Gottret, April Harding, Peter Harrold, Zafrul Islam, Barbara Kafka, Vijay Kala- vakonda, Shahnaz Kazi, Zahed Khan, Kees Kostermans, Gerard La Forgia, Mattias Lundberg, Jean-Pierre Manshande, Tonia Marek, Alison Micheli, Patrick Mullen, Homira Nassery, Barbara O'Hanlon, Patrick Osewe, Toomas Palu, Aakansha Pande, Miyuki Paris, Harry Anthony Patrinos, Plan International (Shehlina Ahmend, Josef De- cosas, Sandy Fortuna, Tom Miller, Sovannarith Sok), John Roome, Robert Saum, Julian Schweitzer, Yolanda Tayler, Inaam ul Haq, Joseph Valadez, Jacques Wangata, and Abdo Yazbeck. Special thanks to the World Bank Institute. xv Abbreviations ARI acute respiratory tract infection ARV antiretroviral BF breast fed BHC basic health center BHU basic health unit BINP Bangladesh Integrated Nutrition Project BPHS basic package of health services BPS basic package of services BWP Bahawalpur CBO community-based organization CCC Chittagong City Corporation CCSS Centro Costaricense de Seguridad Social CHC comprehensive health center CHW community health worker CI contracting in CIDA Canadian International Development Agency CMU contract management unit CO contracting out DP development partner DPT3 diphtheria, pertussis, and tetanus (vaccine, 3 doses) EPI expanded program on immunization FBO faith-based organization FSW female sex worker xvii xviii · Abbreviations G government without support GFATM Global Fund to Fight AIDS, TB, and Malaria GOX Government of Country X GS government with support HFA health facility assessment HHS household survey HIV human immunodeficiency virus HMIS health management information system IBBS integrated behavioral and biological surveillance LFA local fund agent LQAS lot quality assurance sampling M&E monitoring and evaluation MC management contract (contracting in) MCH maternal and child health MOH Ministry of Health MOHFW Ministry of Health and Family Welfare MOPH Ministry of Public Health MSW male sex worker NGO nongovernmental organization NSP nonstate provider NSSP new sputum smear positive NTP national tuberculosis program P4P pay for performance PAA partnership agreement area PBA performance-based agreement PHC primary health care PHCC primary health care center PHD provincial health director PLWHA people living with HIV/AIDS PPA performance-based partnership agreement PPHCC Provincial Public Health Coordination Committee PPP public-private partnership PubMed U.S. National Library of Medicine Abbreviations· xix RFP request for proposal RYK Rahim Yar Kahn SDC service delivery contract (contracting out) SSS single-source selection STI sexually transmitted infection TB tuberculosis TOR terms of reference UNICEF United Nations Children's Fund USAID U.S. Agency for International Development VCCT voluntary confidential counseling and testing WHO World Health Organization All dollar amounts are U.S. dollars unless otherwise indicated. SECTION 1 1 IONT Summary of the Toolkit SEC Background: What Is Performance-Based Contracting? Contracting is when a financing agency (government, insurance en- tity, or development partner), also known as a "purchaser," provides resources to a nonstate provider (NSP, such as a nongovernmental or- ganization [NGO] or private sector firm), also known as a "contrac- tor," to provide a specified set of services, in a specified location, with specified objectives and a set of measurable indicators of success, over a defined period. Performance-based contracting is a form of contracting that explic- itly includes a clear definition of a series of objectives and indicators by which to measure contractor performance, collection of data on the performance indicators, and consequences for the contractor based on performance such as provision of rewards (such as performance bonuses or public recognition) or imposition of sanctions (such as ter- mination of the contract or public criticism). Many different types of health services have been successfully con- tracted, including offering primary health care (PHC) services in rural or urban areas, providing HIV prevention services among high-risk groups, establishing health insurance systems, acting as an interme- diary with many for-profit providers to strengthen the management of tuberculosis patients, managing hospitals, and operating diagnostic services within health facilities. 1 2 · Performance-Based Contracting for Health Services in Developing Countries Contracting is different from providing grants because it is the pur- chaser, not the NSP, who determines which services will be delivered, where, and how performance will be measured. Contracting is also different from internal contracts between one level of government and 1 another, in part because the sanctions that can be used against local IONT governments are usually limited (for instance, they cannot generally be "fired" for poor performance). It is difficult to be prescriptive, but SEC there are some situations that are more conducive to specific forms of contracting than others. For example, in areas in which existing gov- ernment services are not achieving adequate results, there may be an opportunity to contract in management services. Contracting is often a form of "pay for performance" (P4P) and has been used sometimes to introduce P4P for individual health workers. How to Contract If you believe that contracting may be an approach worthwhile to try in your situation, then you need a systematic approach. Contracting is robust and has worked well even in difficult circumstances, so don't worry excessively about making everything perfect. Contracting is more of an art than a science, but a systematic approach can avoid some of the most common mistakes. The "contracting cycle" shown in figure 1.1 can help in this regard, as can the checklist in section 4, which summarizes in two pages most of the issues that need to be addressed. Step 1: Conduct Dialogue with Stakeholders Designing and implementing an effective contracting system requires close consultation with stakeholders, such as government health work- ers, government health officials, local politicians, NSPs, development partners, and the community. After initial discussions about contract- ing, it is worthwhile to draft a contracting plan (see the example in appendix A) and then go back to stakeholders with concrete proposals to discuss. There is usually a need to identify "champions" among the stakeholders to support contracting efforts and overcome resistance. A few common stakeholder concerns are frequently encountered that can be addressed in a number of creative ways. For example, resistance Summary of the Toolkit · 3 Figure 1.1 The Seven Steps of Contracting 1. Conduct dialogue with stakeholders 7. Carry out the bidding 2. Define the services process and manage 1 the contract 3. Design the monitoring IONT and evaluation 6. Draft the contract and bidding documents SEC 5. Arrange for contract 4. Decide how to management and develop select contractors a contracting plan Source: Author. from government health workers may be overcome by paying higher salaries or providing performance bonuses. Step 2: Define the Services The process of designing a contract may involve thinking about many issues, but five are usually of central importance: 1. Defining the objectives of the contract and selecting the indicators of success 2. Ensuring that equity and quality of care are addressed 3. Ensuring that contractors (and purchasers) focus on achieving the stated objectives, possibly through the use of paying for performance 4. Defining the size and location of service of each contract "lot" 5. Defining the scope of services to be delivered. The process of contracting must start with a clear definition of objectives and indicators. Indicators should focus mostly on outcome and outputs, be measurable, be defined as precisely as possible, and be few in number (generally fewer than 10). It is worthwhile to include indicators that ensure that the poorest people and other marginalized groups receive high-quality services. Ensuring that purchasers and contractors pay attention to the key indicators can be accomplished by P4P (for example, by using bonuses or linking payment to the number 4 · Performance-Based Contracting for Health Services in Developing Countries of services provided). Paying for performance is attractive, but indica- tors must be independently verified and perverse incentives must be avoided. For example, paying for each additional cesarean section may lead to an excessive number being performed. 1 The purchaser needs to define each contract "lot," that is, the size IONT and location of service of each individual contract. The lots should be relatively large to achieve economies of scale, facilitate management SEC and monitoring, and increase competition among bidders. The scope of services needs to be defined in sufficient detail so that contractors know what is expected, although the terms of reference (TORs) should generally avoid telling contractors "how" they should deliver services except to ensure compliance with national technical standards. Step 3: Design the Monitoring and Evaluation Considerable argument is often given to which indicators should be used, but little time or effort may be given to deciding how data will be collected, for instance, through the use of a routine health manage- ment information system, household surveys, health facility assess- ments, or supervisory checklists. A schedule for data collection needs to be established with particular attention to collection of baseline data. It needs to be clear who is responsible for implementing the monitoring and evaluation (M&E) plan, and a sufficient budget should be allocated to ensure that the plan can actually be implemented. Step 4: Decide How to Select Contractors Competitive Selection Process. The best way of selecting a contrac- tor is through an open competition, according to clear selection criteria developed in advance, that uses a transparent and independent evalu- ation process. An independent and diverse bid evaluation committee comprising experts from various institutions is an important part of the process. The number of interested bidders can be maximized in a number of ways, including carrying out consultations with potential contractors before the selection process and advertising widely. Selection of Contractors under World Bank Guidelines. There are two distinct methodologies for selecting contractors. When the output is an easily measured physical result (for example, maintenance of equipment or cleaning), the contractor should be selected using a Summary of the Toolkit· 5 wide, competitive process in which the contract will be awarded to the bidder with the lowest price who meets the technical criteria estab- lished by the purchaser. When the output is of an intellectual nature (for example, managing a hospital or providing treatment to people 1 living with HIV/AIDS), the contractor should be selected in confor- mity with procedures normally used for the selection of consultants. IONT There are three methods that can be used in this case: selection based on quality and cost among short-listed firms, fixed budget, and single- SEC source selection. Hybrids of these approaches exist that can also be useful. Although open competition is the preferred approach, circum- stances that will be described later in the toolkit may justify the fixed budget and the single-source selections. Step 5: Arrange for Contract Management and Develop a Contracting Plan Managing contracts requires full-time attention by a clearly defined, reasonably sized team with explicit responsibilities and authority. To function effectively, such a team requires people with a variety of skills and a sufficient budget to cover salaries, equipment, and transporta- tion. When the number of contracts is large, consideration should be given to using contract management software. Effectively implement- ing a contracting effort requires a written contracting plan (most is- sues can be dealt with in six or seven pages, as can be seen from the example in appendix A). Step 6: Draft the Contract and Bidding Documents Maximizing Managerial Autonomy and Accountability. There is increasing evidence that autonomy improves contractor performance. Maximizing managerial autonomy also allows purchasers to hold con- tractors accountable. Safeguarding autonomy can be accomplished by clearly defining the respective authority of the contractor and govern- ment officials, using lump-sum budgets rather than line-item budgets, giving contractors control of personnel functions (such as hiring, fir- ing, posting, and handling pay and benefits), and leaving procurement of various supplies up to the contractors. Protecting the Interests of Both Parties. Contract duration should be at least four to five years to allow both parties to get used to 6 · Performance-Based Contracting for Health Services in Developing Countries the contractual relationship. To facilitate implementation of the con- tracts and to reduce opportunities for corruption, clear procedures must be in place for processing both mobilization payments and ongo- ing payments. Enforcement of a contract requires a clear process for 1 termination, use of other sanctions, and a practical mechanism for IONT resolving disputes. Reporting and Other Obligations. Contracts should specify the SEC content of the contractors' regular progress reports and may mandate independent financial audits. The TORs should also address whether and how contractors can levy user charges, what their responsibilities for building health worker capacity are, and who is responsible for maintenance, repair, and rehabilitation of physical infrastructure. Formulating a Request for Proposals. On the basis of the con- siderations above, a request for proposal (RFP) or similar document should be drafted that includes instructions to the bidders on how to prepare their bids and the criteria by which contractors will be selected, the TORs, and the draft contract. Step 7: Carry Out the Bidding Process and Manage the Contract In almost all situations, it should be possible to complete a competi- tive bidding process in six months. Long delays should be treated with suspicion. Because contracting involves complex relationships, the contract management unit should meet regularly with contractors and regularly report to major stakeholders. During the implementa- tion of the contracts, the contracting plan and the contract should be reviewed regularly. Whether to Contract A review of global experience with contracting for health service deliv- ery is described in section 5. Fourteen evaluated examples of contract- ing were found; it appears that in developing countries contracting with NSPs to deliver primary health or nutrition services can be very effective and that improvements can be achieved rapidly. These results apply for a variety of services and settings. All of the studies found that contracting was successful. Ten of the 14 studies compared contractor Summary of the Toolkit· 7 performance with government provision of the same services, and the contractors were found to be consistently more effective. The current weight of evidence indicates that contracting with NSPs will provide better results than government provision of the same services. Future 1 contracting efforts still deserve to be evaluated rigorously. The advantages of contracting include the following: IONT · Ensuring a greater focus on the achievement of measurable results SEC · Tapping the private sector's greater flexibility and avoiding bureaucratic "red tape" and unhelpful political interference · Reducing important aspects of corruption, such as absenteeism, selling of positions, and theft of drugs · Using constructive competition to increase effectiveness and efficiency · Overcoming "absorptive capacity" constraints that often plague government health care systems · Improving the availability and distribution of health workers · Broadening the autonomy of managers on the ground · Allowing governments to focus on other roles that they are uniquely placed to carry out, such as planning, standard setting, financing, and regulation. The review provides some information on the concerns that have been expressed about contracting, including: · There is a concern that contracting is unlikely to enable service delivery on a large scale. However, four of the examples studied involved populations of tens of millions of beneficiaries, and one now covers 30 million people. It does appear that contracting can be conducted successfully on a large scale. · There has also been concern that contracting might be more expensive than government provision of the same services. Six studies provided an opportunity to test this hypothesis, and each found that NSPs performed better even when public institutions were provided similar amounts of resources. 8 · Performance-Based Contracting for Health Services in Developing Countries · Sustainability also does not appear to be an issue. Twelve of the 14 examples of contracting have been continued and expanded. Six cases have been sustained for seven years or more. Provision of a basic package of primary health care by contractors costs between 1 $3 and $6 per capita per year, an amount that should be affordable IONT even in low-income settings. Thus, financial sustainability does not appear to be a serious threat. SEC · There is a concern that contracting will decrease equity. Three of the cases studies explicitly examined this issue; two found that NSPs were able to significantly improve health services for the poor and did a better job than government provision. The other case found no difference. · A number of observers have been concerned that ministries of health have limited capacity to manage contracts. This review found that contract management was a significant issue in at least three of the examples studied; however, it did not prevent contractors in those instances from being successful. In addition, there were at least six examples in which contract management was done reasonably well, which suggests that the problem can be overcome. According to the global experience so far, it appears that contract- ing is a practical means for improving health service delivery. Its use should be expanded, but continuing evaluation is warranted. Many concerns have been raised about contracting, but the experience so far suggests that few are so serious as to render it ineffective. The con- cerns that seem most serious are the following: first, contract man- agement capacity is often weak and needs attention; second, tenders and contract management may create opportunities for fraud and cor- ruption (even if they reduce other forms of corruption that afflict the public sector); and, third, bureaucratic opposition to contracting is sometimes deep-seated. SECTION 2 What Is Performance-Based Contracting? 2 IONT SEC Introduction This section provides some background about performance-based contracting and sets contracting within the context of other ways of organizing health services. It answers the following questions: What is contracting? Contracting is a mechanism for a financing entity to procure a defined set of health services from a nonstate provider (NSP). The definition of services includes what services, where, to which group of beneficiaries, for how long, and so on. What is performance-based contracting? It is a type of contracting with (1) a clear set of objectives and indicators, (2) systematic efforts to collect data on the progress of the selected indicators, and (3) consequences, either rewards or sanctions for the contractor, that are based on performance. What kinds of services can be contracted? Many different types of health services have been successfully contracted, including primary health care, HIV prevention services, and hospital management. How is contracting different from other approaches to organizing health services? Contracting implies that it is the 9 10 · Performance-Based Contracting for Health Services in Developing Countries purchaser (rather than the NSP) that defines what services will be provided, where, and how they will be evaluated while leaving to NSPs how resources will be managed. What approaches to contracting work best in common situations? It is difficult to be prescriptive, but some situations are more conducive to specific forms of contracting than others. How is contracting related to paying for performance (P4P)? Performance-based contracting is a form of paying for performance and can be used as a way of implementing P4P. 2 IONT What Is Contracting? SEC Contracting is a mechanism for a financing entity (such as a govern- ment ministry, insurance entity, or development partner) to acquire a specified set of services, with specified objectives, of a defined quantity, quality, and equity, in a particular location, at an agreed-on price, for a specified period, from a particular NSP (such as an NGO, private sec- tor firm, or private practitioner). Like all contracts, contracts for health services are voluntary, meaning both parties enter them freely. A few other terms related to contracting should be defined: Nonstate providers. Nonstate providers of services include any nongovernmental entity such as NGOs, faith-based organizations (FBOs), community-based organizations (CBOs), or private for- profit entities or individuals. Contractor. The contractor is the NSP implementing and managing the services defined in the contract. Another useful term for contractor is "partner," although the term "contractor" is used throughout this toolkit for the sake of simplicity. Purchaser. The purchaser is the entity that awards the contract, provides the financial and other resources for the services, and has the fiduciary responsibility for ensuring that the terms of the contract are met. Purchasers of health services are typically government agencies, parastatal organizations, insurance entities, or development partners. What Is Performance-Based Contracting?· 11 Public-private partnerships. Contracting is one form of public- private partnership. A partnership sometimes implies that both parties bring financial or other resources into the relationship, but this is not always the case. What Is Performance-Based Contracting? Performance-based contracting is a form of contracting that explicitly includes three characteristics: 2 · Clear definition of a series of objectives and indicators by which to measure contractor performance IONT · Collection of data on the performance indicators to assess the SEC extent to which the contractors are successfully implementing the defined services · Performance leading to consequences for the contractor, such as provision of rewards or imposition of sanctions. Rewards can include continuation of the contract in situations in which there is a credible threat of termination, provision of performance bonuses, or public recognition. Sanctions can include termination of the contract, financial penalties, public criticism, and debarment from receiving future contracts. What Kinds of Health Services Can Be Contracted? A large variety of health services can be, and have been, contracted. These include the following: · Providing primary health care services in rural or urban areas · Offering HIV prevention services among high-risk groups · Providing HIV/AIDS treatment services to people living with HIV · Establishing a health insurance system · Setting up and operating a voucher project 12 · Performance-Based Contracting for Health Services in Developing Countries · Acting as an intermediary in providing P4P to public health care providers · Offering behavior change communication activities and information, education, and communication · Providing maintenance and cleaning services in a hospital · Providing social marketing of health products, such as contraceptives · Working as an "umbrella" agency that oversees the work of many 2 smaller NGOs and CBOs involved in delivering primary health IONT care, nutrition services, or HIV services · Operating an ambulance system SEC · Acting as an intermediary with many for-profit providers, for example, in strengthening the management of tuberculosis patients · Managing a hospital · Operating diagnostic facilities within public health care facilities · Providing ancillary services such as equipment maintenance, cleaning, waste management, food preparation, and security. Sample terms of reference for some of these services are provided in appendix E; others are available at http://www.worldbank.org/hnp/ contracting. How Is Contracting Different from Other Approaches to Organizing Health Services? Typology of Service Delivery. A number of different approaches exist for organizing health service delivery, so clarifying definitions will facilitate meaningful dialogue. Although this is a bit of a simpli- fication, there are five important functions related to service deliv- ery: (1) designing the services, that is, what services will be delivered, where, and with which indicators of success; (2) selecting the service provider; (3) actually managing the services; (4) establishing and con- trolling the "production infrastructure," which includes personnel, What Is Performance-Based Contracting?· 13 equipment, drugs, clinics, and other facilities; and (5) financing the system (see table 2.1). As an example, under a management contract (arrangement 3 in table 2.1), a government will contract with an NSP or an individual to manage existing government services in a speci- fied area. Under a service delivery contract (arrangement 4 in table 2.1), the government decides which services the contractor will pro- vide and where, while the contractor will both manage and supply the production infrastructure. The arrangements described in table 2.1 are not exhaustive, and hybrids clearly exist. For example, the line between a management contract and a service delivery contract blurs when the contractor uses government health workers but pays them 2 significantly more than their civil service salaries. Intergovernment "Contracts." There has been some experience IONT with national governments signing agreements with local governments SEC (arrangement 2 in table 2.1) that pertain to achieving certain goals. Although potentially interesting, this arrangement rarely involves a true contract that the parties enter into voluntarily and in which the contractor can be "fired" for nonperformance (although other rewards and sanctions may be available). Another issue with such contracts is that denying resources to poorly performing areas can be politically or ethically challenging. The Difference between Grants and Contracts. Grants by gov- ernment or donors to NSPs, often NGOs (see arrangement 5 in table 2.1), are quite common, particularly in HIV/AIDS prevention and treatment. These grants are usually given to organizations that sub- mit a proposal to a funding agency. The most important difference between grants and contracts is who defines the services to be deliv- ered. For grants, it is generally the NSP that decides what kinds of services will be delivered, where they will be delivered, and how they will be evaluated. As the funding agency defines more and more of the details of the services to be provided, the distinction between grants and contracts blurs. Grants can be very useful and have worked well in many situations. They are particularly helpful in beginning new types of services or providing an opportunity for creative innovations to address health problems. The downside to grants is that they can lead to an irrational distribution of services with gaps in some areas and duplication in others. For example, in Ghana epidemiologists believed that about 70 percent of HIV transmission involved female sex workers (FSWs). However, when a grant mechanism was introduced to control SECTION 2 14 Table 2.1 Typology of Service Delivery Arrangements Services Arrangement Service design Provider selection management Infrastructure setup Financing Example 1. Government Government Government Government Government Government Government sets up public services primary health care centers 2. Inter- Government-1 Government-1 Government-2 Government-2 Government-1 Government transfers funds governmental from federal to provincial agreements governments 3. Management Government Government Private sector Government Governmenta Government hires a private contracts sector manager to manage existing government health services 4. Service deliv- Government Government Private sector Private sector Governmenta Government hires NGO to ery contracts provide services where none exist 5. Government Private sector Government or Private sector Private sector Government NGOs submit proposals to grants to NSPs donor (w/ or w/o NGO government for needs identi- or community fied by community or NGO contribution) 6. Private sector Private sector Consumer Private sector Private sector Consumer or · NGO establishes health services NGO/donor services in slum areas using its own funds · For-profit providers establish private clinic Source: Author. Note: Government-1 and Government-2 refer to different levels of government. NGO = nongovernmental organization; NSP = nonstate provider. a. Financing may be supplemented by formal or informal user charges. What Is Performance-Based Contracting? · 15 the epidemic, fewer than 1 percent of the grants went to NGOs work- ing with FSWs, reflecting a shortage of grant proposals for this impor- tant type of service. What Approaches to Contracting Work Best in Common Situations? Because situations and contexts vary considerably, it is difficult to be prescriptive about which types of contracts will work best in a given set of circumstances. However, some situations are more conducive to some forms of contracting than others. Table 2.2 explores some of the 2 options. This table is not meant to limit creativity; on the contrary, it should be seen as an encouragement to innovate and explore new ways IONT of contracting. SEC Table 2.2 Types of Contracting to Consider in Some Common Situations Context/situation Options to consider Limited rural health services but with · SDC for specified geographical "mission" clinics or other faith-based "catchment" area with a matching organizations grant or subsidy · SDC plus MC for the management of existing government services Poorly performing districts, provinces, or · MC for management services states with existing government health · MC for management services plus P4P services for health workers · SDC where government health workers join NSP Uncoordinated NGO-delivered services · SDC for specified geographical areas with multiple donors (for example, post- conflict situation) Few services of any kind, or new kinds · SDC for specified geographical areas of services required (for example, HIV with emphasis on innovation and prevention, nutrition services, early careful evaluation childhood development services) Existing government services where · MC for management services improved management is needed or · MC for management services plus P4P innovations are required for health workers Urban primary health services with many · SDC for specified geographical areas different providers but limited coverage with focus on reaching the poor (slum of preventive services for the poor and nonslum residents) with preven- tive and promotive services Source: Author. Note: MC = management contract (contracting in); NGO = nongovernmental organization; P4P = pay for performance; SDC = service delivery contract (contracting out). 16 · Performance-Based Contracting for Health Services in Developing Countries How Is Contracting Related to Pay for Performance? Pay for performance is a broad term that covers a number of approaches to rewarding the provision of more and better services. The basic idea of P4P is attractive because it compels providers to focus on important objectives and uses financial rewards to reinforce good performance. In some ways any contract that specifies explicit, measurable outcomes and allows for termination of the contract for nonperformance is a type of P4P. Contractors are rewarded for good performance by con- tinuation of the contract and ongoing payment while poor performers 2 have their contracts terminated. However, P4P is often used to refer IONT to a more explicit link between performance and payment. Table 2.3 describes some of the forms of P4P and their relationship to contract- SEC ing. Details of applying P4P in a contracting situation are given in section 3 (task 6). For some types of P4P the evidence is compelling. Fee-for-service payments to individual providers (approach 3 in table 2.3) consistently lead to increased service provision (sometimes even too much). For other types of P4P the evidence, so far, is less strong. The use of per- formance bonuses in contracts makes sense and has worked well in some contexts, such as Afghanistan and Haiti, but less well in oth- ers, such as Uganda. For rewards to local governments (approach 1 in table 2.3) the evidence is still modest. This approach also suffers from an ethical issue because poorly performing areas that need the most help may receive fewer resources, which may only reinforce preexist- ing inequalities. Table 2.3 Types of Pay for Performance and Their Relationship to Contracting Who receives What the funds can Who provides Type of P4P the funds be used for the funds Example Relationship to contracting 1. Rewards for local Local Programs of local National Burkina Faso "perfor- "Performance agreements" governments governments governments government mance" agreements rarely true contracts 2. Rewards National Programs of national Development GAVI Alliance payments for Not related to national governments governments partners increased DPT3 coverage governments 3. Payment per Individual Personal uses Government, Rwanda, where NGOs paid May be easier to introduce service (fee for health workers individuals, or health workers based on in the context of contracting service) NSPs number of services provided with NSPs 4. Performance NSP Other programs or at Purchaser Haiti, where NGOs received Sometimes used in health bonuses the discretion of the bonuses for achieving care contracting, very NSP specified targets often used in other forms of contracting 5. Performance- NSP At discretion of the Purchaser Amount paid to an NSP is Can be incorporated fairly based payment NSP a function of the number of easily into contracts patients seen Source: Author. Note: DPT3 = third dose of diphtheria/pertussis/tetanus vaccine; GAVI Alliance = formerly the Global Alliance for Vaccines and Immunization; NGO = nongovernmental organization; NSP = nonstate provider. 17 SECTION 2 SECTION 3 How to Contract If you believe that contracting may be an approach worth trying in your situation, then a systematic approach to doing it is necessary. This chapter provides a framework for contracting, but keep a few things in mind: · Don't be overwhelmed! Contracting has proven fairly robust and 3 has worked well even in difficult circumstances, so don't worry ex- cessively about getting everything perfect. Usually a limited num- IONT ber of key issues are present to worry about, and typically these SEC center on · Defining the objectives of the contract, selecting the indicators of success, and taking equity and quality into account · Ensuring that contractors (and purchasers) focus on achieving the stated objectives · Defining the size and location of each contract "lot" · Defining the scope of services to be delivered · Maximizing the managerial autonomy of the contractors · Developing a contracting plan that addresses, among other things, how the contracts will be managed and how the monitor- ing and evaluation will be done. · Contracting is more of an art than a science, but a systematic ap- proach can avoid simple mistakes; the point should be to make new and interesting mistakes, not to repeat old ones. · Contracting also allows for, and requires, creativity and adaptation to local circumstances. 19 20 · Performance-Based Contracting for Health Services in Developing Countries Figure 3.1 The Seven Steps of Contracting 1. Conduct dialogue with stakeholders 7. Carry out the bidding 2. Define the services process and manage the contract 3. Design the monitoring and evaluation 6. Draft the contract and bidding documents 5. Arrange for contract 4. Decide how to management and develop select contractors a contracting plan Contracting is not a simple linear process, but some logical steps can be represented in the "contracting cycle": conducting dialogue with stakeholders, defining the services, designing monitoring and evaluation, deciding how to select a contractor and establishing the price, arranging for contract management, drafting the contract and 3 bidding documents, and carrying out the bidding process and manag- IONT ing the contracts (see figure 3.1). SEC Step 1: Conduct Dialogue with Stakeholders Task 1: Establish a Consultative Process with Stakeholders Designing and implementing an effective contracting system is an iterative process that requires close consultation with stakeholders. This process is inevitable because contracting is about balancing the interests of various stakeholders, who often hold competing interests. Stakeholders usually include government health officials, government health workers, local politicians and government officials, NGOs, community-based organizations (CBOs), the for-profit private sector, the community, and other development partners or donors. Having a good consultation process during the design phase of contracting can prevent problems later during implementation. There are a couple of important aspects of the process: · Hold a few discussions with each set of stakeholders. Contract- ing is often a new approach for many stakeholders, and so they need to hear about it a few times before they can fully appreciate its im- plications. Having a few discussions is generally worth the effort. How to Contract · 21 · Get back to stakeholders with draft proposals and contracts. After initial discussions it is worthwhile to draft the documentation (a contracting plan and the actual contract; see task 25) and then go back to stakeholders with something concrete to discuss. Other- wise, discussions can go on for a long time at a theoretical level with little progress. Task 2: Identify Champions There is almost always a need for "champions"--people who understand the evidence for contracting and are willing to see it at least pilot-tested on a reasonable scale. These people can use their influence to overcome resistance from a number of quarters. Champions can often be found in unusual places, and some may not necessarily be in the government or even in the health sector. They may be advisers to political figures, well- respected businesspeople, or leading community figures. 3 Task 3: Address the Legitimate Concerns of Stakeholders Section 5 provides the evidence for and some of the commonly ex- IONT pressed concerns regarding contracting. The discussion there, as well SEC as in appendix C (case studies), should be read before beginning dis- cussions with stakeholders. On the basis of experience in a number of settings, a few common concerns often voiced by stakeholders have been identified that can be addressed in a number of creative ways: · Concerns of government health officials. Government health of- ficials are often deeply suspicious of contracting and may resist it vig- orously. They are often the key decision makers, particularly when it comes to externally funded contracting efforts. Some ways of ad- dressing their concerns include reviewing the evidence for contract- ing and suggesting that improved health services will reflect well on them. However, there are often other concerns, for example: · Implicit criticism. Health officials may perceive contracting as an implicit criticism of their own competence or performance. This impression can be dealt with by (1) indicating that much of the advantage held by private sector contractors is that they are less burdened by bureaucratic rules and regulations (which can then shift the discussion productively to how to reduce "red tape" and increase managerial autonomy in the public sector), (2) allowing 22 · Performance-Based Contracting for Health Services in Developing Countries officials to take leaves of absence from the government to work with contractors, (3) allowing some "lots" (see step 2, task 8) to be managed by government officials with more autonomy than they would otherwise have, and (4) allowing new parastatal entities to be established to bid on contracts (so long as they have an arm's- length relationship with the purchaser). Parastatal entities are linked to the government but are not constrained by civil service rules and regulations; they are run on a quasi-commercial basis. · Competition. Related to the above concern, health officials may also be nervous about possible competition from nonstate pro- viders (NSPs). Such competition, they sometimes fear, may put publicly delivered services in a poor light and lead to unflattering comparisons. A legitimate way of addressing this concern is to ensure that NSPs are provided the same level of resources as the public sector and held to the same performance standards. What is not helpful in addressing this issue is to put additional con- 3 straints on NSPs so that they face the same issues as the public IONT sector. For example, forcing NSPs to obtain multiple clearances when recruiting staff will just reduce their flexibility and mana- SEC gerial autonomy. · Less control. Government officials often fear giving up con- trol. They may worry about having less power and prestige be- cause they will exert less control over recruitment, postings, and transfers of health workers and procurement. In many countries, hiring and managing personnel take up much of health officials' time and can be exasperating work. Being able to focus on in- teresting, strategic issues may be attractive to them. One way of aligning the incentives for government officials and the contrac- tors is to provide performance bonuses to government officials if the contractors achieve certain measurable outputs. This incen- tive has been implemented successfully in Afghanistan. · Lost opportunities for corruption. Dishonest officials will be concerned about losing opportunities for corruption or influence peddling. There are a number of ways of avoiding corruption in the contracting process that are discussed below (see tasks 8, 16 to 19, and 31), including not making the size of the contracts too small and using transparent competition for contractor selection. How to Contract· 23 · Opposition to change. Many officials oppose contracting be- cause it represents a change in the way they do their business, and they resist any change because they are comfortable with the status quo. This concern can be difficult to address. However, experience suggests that contracting is a pretty low-risk activity. It can increase the officials' prestige if services improve, and they can carry out tasks that are of greater prestige and greater inter- est, such as strategic planning, health care financing, and track- ing performance. Discussions with officials from countries that have implemented contracting can help allay fears. · Concerns of local politicians and local government officials. Politicians, including local government officials, have sometimes been helpful in the introduction of contracting. Conversely, they have often been the most vociferous opponents of contracting for many of the same reasons held by national government officials. Local officials and politicians often have the most to gain from 3 improved services, and many are the first to hear the complaints of the community about existing services. The concerns of these IONT stakeholders generally relate to ensuring that they retain some in- volvement and influence over the contractor's performance. There SEC are a few ways to satisfy their need to have a part in the contracting process, including making them signatories or witnesses to the con- tract, having them review the quarterly reports of the contractors before payment, and giving them an explicit role in monitoring the contractor's performance. · Concerns of government health workers. The difference between a management contract and a service delivery contract is the degree of managerial authority the contractor has over the staff needed to deliver the services. In situations in which the staff belong solely to the contractor, this authority is generally not an issue. However, in many circumstances, including all management contracts and some service delivery contracts, government staff are still much needed, but they often fear that they will lose their job, see their wages low- ered, or lose some benefits. Some options to address this issue have been tried and found reasonably successful: · Health workers can form cooperatives to take over existing pub- licly managed clinics (see box 3.1) 24 · Performance-Based Contracting for Health Services in Developing Countries Box 3.1 Establishing Worker-Controlled Health Cooperatives in Costa Rica Costa Rica has a long-established social health insurance system that provides nearly universal coverage to the population. In response to rapidly increasing costs, long waiting times, and a need to strengthen the primary health care sys- tem, the government allowed the formation of worker-controlled cooperatives that took over the management of clinics previously managed by the social secu- rity institute. The cooperatives were autonomous legal entities provided with full autonomy on the use of funds, but they had to deliver standard services and follow government guidelines and objectives. According to more than a decade of data from the social security institute, it appears that the cooperatives provided more patient services than the publicly managed clinics but at a significantly lower cost per patient. (See case 7 in appendix C and Gauri, Cercone, and Briceno 2004 for more details.) 3 · Government workers can take a leave of absence to work with the contractor IONT · Salaries of workers selected by the contractors can be raised SEC above the civil service norms · Performance-based bonuses can be paid to health workers se- lected by the contractors · Government workers can be selected by the contractors, and the remaining ones can be transferred to another location within the government health services. · Concerns of the private sector, including NGOs. Potential con- tractors (NGOs, CBOs, and other private sector entities) need to be consulted to ensure their participation and genuine commitment to the process. Potential contractors often have the following fears: · Nontransparency in the ways contractors will be selected. This fear can be dealt with by having a clear selection process with explicit criteria included in the bidding documents and the over- all contracting plan. The selection process is discussed in more detail in step 4. · Delayed payments that interfere with their ability to implement the services. Given that most NGOs and CBOs do not have the capital to pay the staff until they receive payments, delays in dis- How to Contract· 25 bursements by the purchaser can lead to interruption of services. Mechanisms for decreasing the risk of delayed payment are de- scribed in step 6 (task 31). These include making explicit the doc- umentation needed for payment and using professional contract managers. · Kickbacks that must be handed over to get paid. This issue can often be dealt with by having a clear payment process described in the contract (task 31), a dispute resolution mechanism (task 33), and a third party that can certify the performance of the NGO independently. · Unrealistic expectations coupled with inadequate resources. Po- tential contractors often fear being set up for failure by govern- ment officials with a vested interest in maintaining the status quo. To some extent this can be addressed by making objectives explicit, having independent assessment of performance, ensur- ing sufficient managerial autonomy (task 26), and having a com- 3 petitively established price for the contract. · Dislike of the idea of being "contractors." Some nonprofit or- IONT ganizations (NGOs and CBOs) dislike the commercial conno- SEC tations of contracting; they feel it impugns their humanitarian motives. They prefer to be seen as, and actually be, partners of the purchaser. This concern can be addressed in part by using terms such as "partnership agreements" and "public-private part- nerships." More important, NGOs should be allowed to carry out other development activities in the same area using other sources of funds (or funds that are left unspent at the end of the contract). · Concerns of the community. Generally, communities care less about who is delivering services than that the services are actually being delivered and that they are of high quality. Contracting af- fords an opportunity to ensure greater community participation in the design, implementation, management, and monitoring of health services. Nongovernmental organizations are usually quite interested in increasing community participation, and this should be included in the contract's terms of reference (TOR). · Concerns of development partners/donors. Many development partners (DPs) support contracting, and many have much experi- 26 · Performance-Based Contracting for Health Services in Developing Countries ence working closely with NSPs. Most of DPs' concerns center on ensuring that there is a continued focus on improving results, that the interests of governments and NSPs are respected, and that the contracting efforts are coordinated with existing services. Step 2: Define the Services 1. Conduct dialogue with stakeholders 7. Carry out the bidding 2. Define the services process and manage the contract 3. Design the monitoring and evaluation 6. Draft the contract and bidding documents 5. Arrange for contract 4. Decide how to management and develop select contractors a contracting plan 3 IONT A critical step in contracting is defining the services in sufficient de- SEC tail. Many issues must be considered in drafting a contract. However, in most cases the following issues are the most important: · Defining the objectives of the contract and selecting the indicators of success · Considering pay for performance and other means of ensuring that contractors (and purchasers) focus on achieving the stated objectives · Defining the size and location of each contract "lot" · Defining the scope of services to be delivered. These issues become most of the TORs for the contract. Task 4: Define the Objectives of the Contract Possibly the single biggest advantage of contracting is that it allows purchasers and contractors to focus on results. This focus means that objectives need to be explicit and measurable, which is why drafting a contract should start with a clear definition of the objectives and the How to Contract · 27 Table 3.1 Reasonable Performance Indicators in a Contract for Primary Health Care Indicator Number of consultations per person per year provided by the contractor Percentage of couples of reproductive age currently using a modern family planning method Number of sputum-positive cases of tuberculosis detected as a percentage of target based on estimated prevalence Proportion of children age 6 to 59 months who received vitamin A supplements within the past 6 months Percentage of children age 12­23 months who received measles immunization cover- age before age 12 months Percentage of all women pregnant during the past year receiving at least one ante- natal care visit from a skilled health care provider (doctor, nurse, trained midwife) Proportion of births in the past year attended by skilled attendants, including institu- tional delivery but excluding trained traditional birth attendants Score out of 100 on an index of quality of care as judged by a third party, including the availability of drugs, quality of patient-provider interaction, patient satisfaction, and so on 3 Larger improvements for the bottom two income quintiles (based on asset index) and women or girls on immunization coverage and consultations per capita per year IONT Source: Author. SEC indicators by which they will be assessed. (See table 3.1 for an example of reasonable indicators included in a contract for delivery of primary health care. Table 3.2 includes examples of indicators that should not be used in contracts.) A few basic principles should be applied to the selection of indicators, including the following: · Limit the number of indicators. The process of selecting indica- tors and keeping many stakeholders happy often results in a large number being identified, an outcome that should be actively re- sisted. Experience has shown that, not surprisingly, having many indicators leads to less data actually being collected. It also leads to "indicator inflation" in which both purchasers and contractors pay little attention to any of the indicators, even the most critical ones, because there are so many. Selecting fewer than 10 indicators is good practice. 28 · Performance-Based Contracting for Health Services in Developing Countries Table 3.2 Some Less-than-Ideal Indicators in a Contract for Primary Health Care Indicator Issues Percentage of district hos- Mostly an input or upstream process indicator. It pitals with functioning X-ray doesn't show whether the machines are being used machines or how many X-rays have been taken. It also doesn't indicate anything about the quality of the films. Number of health workers Mostly an input or upstream process indicator. receiving appropriate training It doesn't indicate if the health workers actually learned anything or if they apply their knowledge to treating patients. The definition of "appropri- ate" training is unclear. There is no definition of a numerator or denominator. Percentage of community Unclear definition of "happy." It's also likely that members using the clinic who people who are not using the clinic are more are happy with the services unhappy with the services ("voting with their feet") than those who do use the clinic. People who live near the clinic may use it more, so this indicator may hide inequity in access to services. Low rate of health workers Hard-to-collect data for this indicator. Numera- 3 leaving their positions tor and denominator are unclear, as is the point at which the rate is "low." Those health workers IONT who leave may be difficult to trace. This is also a process measure that may or may not affect the quality of services. SEC Percentage of inpatients willing Your assessment:a to pay extra for private bed Proportion of health centers Your assessment:b having enough drugs Source: Author. a. It is not clear what this indicator is trying to measure. Is it quality of care or use of services by better off people? The definition of "willing" is not clear, as the number could be constrained by the number of private beds available or the cost. b. The issue is what "enough" means and which drugs are the most important. · Design bias toward outcome and output indicators. As one se- lects indicators, it is worthwhile to be biased toward outcomes and outputs rather than inputs or processes. Outcomes and outputs are those achievements that have direct meaning for patients or com- munities and, according to available scientific evidence, will likely lead to improved health status. (To be clear about terms, this toolkit uses the terminology described in table 3.3.) There is some value in using a few input and process indicators because they are often easier to measure, can be measured more frequently, and can pro- How to Contract · 29 Table 3.3 Examples of Input, Process, Output, and Outcome Indicators Nature of indicator Input Process Output Outcome Vaccination Vaccines Vaccinators Levels of DPT3 Decreased coverage and syringes and vaccines or measles measles ordered available in coverage incidence or health center under-five mortality rate Antimalaria Budget avail- Antimalaria Children with Decreased drug able for anti- drugs avail- malaria obtain under-five availability malaria drug able in health appropriate mortality rate procurement centers drugs Training of Number of Health work- Health workers Decreased health workers health workers ers' level of apply their under-five trained knowledge six knowledge to mortality rate months after actual patients training Source: Author. Note: DPT3 = third dose of diphtheria/pertussis/tetanus vaccine. 3 IONT vide useful milestones along the road to outcomes or outputs. How- ever, an excessive focus on input and process indicators can negate SEC one of the main advantages of contracting, a focus on results that matter to real patients. Even within input and process indicators, it is worthwhile to concentrate on those that are as far "downstream" as possible. For example, rather than focusing on "the budget avail- able for malaria drugs" or the "timely procurement of drugs," it is more useful to concentrate on the availability of drugs in peripheral health facilities. Recommendations. Most indicators chosen for a contract should be outputs or outcomes. A few input and process indicators may be useful to include in a contract as a means of seeing whether perfor- mance is "on track." · Ensure that indicators are independently measurable. Step 3 gives some suggestions about how to measure indicators and en- sure that they are independently verifiable. Generally, much more time is taken arguing about indicators than about how and whether they can actually be measured. A good rule of thumb is to spend as much time on discussing how to measure indicators as on which 30 · Performance-Based Contracting for Health Services in Developing Countries indicators to examine. One issue to watch out for is indicators that can be measured only by the contractors themselves. This mea- surement can create an obvious conflict of interest. · Define indicators as precisely as possible. Both the numerator and the denominator of indicators must be carefully defined. Also, it is important that terms are clearly defined, that is, being care- ful with words like "appropriate" or "sufficient." For example, the indicator "health workers have appropriate knowledge of diarrhea management" is problematic because the numerator, the denomina- tor, and the word "appropriate" are not defined. · Set targets broadly. Much time and energy can go into specifying targets in exact detail, which is probably not worth the effort. Usu- ally what is needed is a statistically significant and programmati- cally important improvement, which implies that targets should be reasonable, not obsessive. For example, if immunization coverage 3 at baseline is 50 percent and the target is 80 percent, a contractor that achieves 78 percent coverage should be congratulated for mak- IONT ing significant progress. That they didn't achieve 80 percent should SEC not much matter given measurement error (confidence intervals for immunization coverage measured through standard cluster surveys are typically plus or minus 10 percentage points) and the fact that a large improvement has been achieved. Conversely, a large sample survey could indicate that immunization coverage improved by 3 percentage points, which could be statistically significant but of not much programmatic importance. Task 5: Include Objectives Related to Equity and Quality It is important to make concerns about equity and quality as explicit as possible in a contract. Equity can be defined by use of services by the poor (for example, the poorest two income quintiles or people liv- ing below the national poverty line), coverage of services among the population living in underserved geographical areas, or a concentra- tion index. Measuring equity can be challenging and often entails special efforts, including additional questions on household surveys (more details on how to do this are available at http://www.worldbank. org/analyzinghealthequity). Quality of care can also be challenging How to Contract· 31 to measure and often entails the conduct of specialized health facility surveys (see task 10). Task 6: Consider Pay for Performance Once the effort has gone into defining the objectives of the contract, it is important to ensure that purchasers and contractors actually pay at- tention to achieving those results. One way of doing this is by paying for performance, sometimes referred to as performance-based financ- ing (PBF) or output-based aid (OBA). There is increasing experience with bonuses linked to accomplishments, as in Haiti (see box 3.2) and Afghanistan, or with relating payment directly to the number of ser- vices provided, as in Rwanda. Paying for performance (P4P) is attrac- tive, and experience in health and other sectors shows that it works well. However, a few issues must be addressed in designing P4P: · Independent verification. Pay for performance often requires data that may be most easily available from the service providers 3 themselves, which may put them in a conflict-of-interest situation. Because the data they collect are used to determine how much they IONT get paid, they have an incentive to overstate the amount of services SEC they have provided. For example, service providers who are paid for the number of infants immunized on the basis of reports they generate may deliberately include in their reports children who are older than age one. · Perverse incentives. There is also the danger of perverse incen- tives in which providers do too much of a good thing or cut corners on quality so they obtain the performance payment. Examples of potentially perverse incentives are cases in which service providers are given an incentive to increase the number of cesarean sections or to put people living with HIV on antiretroviral (ARV) therapy. Contractors may place people on drug therapy who don't really need it. To some extent, this possibility can be avoided by strict application of treatment protocols, but even they require important clinical judgments. Giving ARVs to people who don't need them exposes them to the risk of serious side effects and heightens the risk of ARV resistance in the broader population. Hence, in design- ing P4P, care is needed to avoid perverse incentives. 32 · Performance-Based Contracting for Health Services in Developing Countries Box 3.2 Paying for Performance in Haiti To improve the performance of NGOs delivering primary health care in Haiti, USAID through Management Sciences for Health (MSH) began paying them a bonus based on their performance on key indicators, such as immunization cover- age, skilled birth attendance, and prenatal care. These indicators were measured using the health management information system but were independently verified through a data audit. As the percentage of NGOs being paid on a performance basis (that is, offered performance bonuses if they achieved specified results) increased, the coverage of these services also increased (see table). Performance of NGOs in Haiti as Use of Bonuses Increased (Percent) Indicator 2000 2002 2003 2004 2005 Children fully vaccinated 34 65 91 92 100 Women received at least three prenatal 29 50 41 48 60 visits 3 Deliveries assisted by skilled attendant 58 64 57 63 77 NGOs paid on performance basis 0 35 37 44 93 IONT Source: Based on Eichler and others (2006). Note: Data for 2000 from Demographic and Health Survey; data for 2002 to 2005 from SEC audited management information system. · Quality and equity. Pay for performance can sometimes lead to an excessive focus on the quantity of services provided without equal attention given to the quality of service or to who benefits from the services. For example, if a P4P approach pays per consultation (outpatient visit), then there is no guarantee that the quality is ac- ceptable or that many people living around the facility are using it but people living farther away are not. · Easy-to-understand terms. It is important that NSPs and individ- ual providers understand when a bonus will be paid. Without this understanding, the P4P approach may not be particularly effective in influencing performance. · Performance incentive amount. Setting the amount of the per- formance incentive is challenging. If it is too small, it may not be useful, and if it is too high, it becomes too expensive for the pur- How to Contract· 33 chaser and may result in perverse incentives. In those contracts with NSPs that have included performance bonuses, the maximum bonuses that the NSPs could earn have been about 10 percent of the contract value. · Clearly stated financial risk. NSPs, individual providers, and pur- chasers need to understand the financial risk they are taking in a P4P approach. Depending on how the performance bonus is de- signed, if NSPs and individuals fail to meet performance criteria, they may receive less money than they were counting on. If the performance bonuses exceed the established contract price, then the financial risk lies with the purchaser. · Reasonable payment criteria. The decision to pay a performance bonus can be based on different factors, including (1) improvements from the provider's own baseline (for example, a 10 percent im- provement in prenatal care coverage), (2) achievement of a specified benchmark (for example, more than 80 percent measles coverage), 3 (3) per unit of service provided (for example, $0.50 for each con- sultation), and (4) "tournament" style where the top 10 percent of IONT providers receive bonuses. SEC Recommendations. When feasible, it is often best to use some combination of improvement from baseline and achievement of a specified benchmark. However, the resulting combination must be easy to understand. Task 7: Ensure That Purchasers and Contractors Focus on Objectives Besides P4P, there are other means to get contractors and purchasers to focus on key objectives: · Ensure that contractors realize they face a credible threat of being fired for nonperformance · Have purchasers and contractors regularly discuss the status of indicators · Ensure that field monitors discuss results on key indicators during field visits · Provide baseline and follow-up survey data to the contractors and purchasers 34 · Performance-Based Contracting for Health Services in Developing Countries · Ensure that both purchasers and contractors understand the con- tents of the contract (it is remarkably common for key people in the process not to fully understand or even to have read the entire contract). Task 8: Define the Size and Location of Each Contract "Lot" A contract "lot" is the size and location of an individual contract, which is usually defined by the purchaser. For example, a primary health care contract could cover a district with a population of 300,000, or the lot could cover a province with a population of 1.2 million. Lots are also sometimes referred to as "packages," although this is different from a "package of services," which describes the scope of services to be delivered (see task 9). Relatively fewer, large lots are recommended rather than many small ones. Some arguments have been made for using smaller contracts; those are explored next. 3 The rationale for having a few large lots includes the following: IONT · Financial economies of scale. Large fixed costs are associated with implementing health services, including management and ad- SEC ministration. Hence, larger contracts will spread these fixed costs over a larger base, thereby reducing the cost per beneficiary served. This is not just a theoretical concern. In Afghanistan, using a sys- tem of competitive bidding that is based partly on price, contracts for delivering primary health care in entire provinces were 52 per- cent of the cost per beneficiary of contracts that covered only parts of a province ($4.10 per person per year for the large contracts compared with $7.80 per person per year for the smaller contracts). This lower cost per beneficiary means that with larger contracts more people can obtain services for the same amount of money. · Economies of scale in contract management. Large lot size means far fewer contracts to manage. A number of examples exist in which purchasers who originally designed contracts for small lots realized the problems of managing multiple contracts and changed their strategy during their second round of funding. · Economies of scale in monitoring and evaluation. Having fewer lots makes it easier and less expensive to monitor and evaluate each contractor's performance. When multiple contracts must be moni- How to Contract· 35 tored, more field visits are required. Similarly, if a household survey is used to collect information, the cost can be exorbitant if the per- formance of many small contractors must be assessed. · Increased competition. Large lots provide sufficient financial in- centives to encourage more organizations to compete for the con- tract. This competition allows the purchaser to choose from among more organizations, increasing the chances that a high-performing organization will be selected. Small lots limit competition and force the purchaser to choose among fewer organizations. · Economies of scale in capacity building. In situations in which there are few potential contractors with adequate experience, it is easier to build the capacity of a few larger NSPs rather than a large number of small NGOs or CBOs. For example, teaching 70 NGOs how to work with female sex workers (FSWs) is much more difficult than building the capacity of 7 NGOs. · Decreased opportunities for corruption. Larger contract lots 3 appear to reduce the opportunities for corruption. Unscrupulous officials have an easier time intimidating small NGOs or CBOs IONT working on small lots into paying kickbacks or bribes. There is SEC also sometimes political pressure to have many small contracts so they can be spread around among many political supporters. As mentioned earlier, larger contracts will attract greater competi- tion, which generally reduces corruption. Reasons for having smaller lots include the following: · Increased diversity. The purchaser should not become dependent on only a few large NSPs because in the long run this dependency can limit competition, which results in higher costs and poorer per- formance. In addition, there are situations in which there is a need to study different approaches to learn which is most effective. Some people fear that having only a few large contracts will limit innova- tion and diversity in approaches. · No disruption to existing providers. There are concerns that large contracts will overlap and possibly interfere with existing small-scale programs. In these cases, it may be better to contract with an umbrella organization for building capacity, coordinating existing efforts, and filling gaps in service delivery. 36 · Performance-Based Contracting for Health Services in Developing Countries · Uncertain capacity of NSPs. Where the capacity of NSPs to carry out services on a large scale is an issue, one finds an argument for smaller lots. However, it is often difficult to assess an NSP's ca- pacity in advance, and purchasers often underestimate the capac- ity of NSPs and overestimate the alternative (the ability of state providers). Recommendations. Many advantages to having large contract lots can be named, and it generally makes sense to have between 7 and 20 lots. This number is manageable and allows sufficient diversity to ensure innovation and to reduce dependence on a few contractors. (In primary health care, lots should generally cover groups of at least 400,000­500,000 people.) Building the capacity of newer and smaller organizations can be accomplished by encouraging a larger organi- zation to enter into joint ventures with them, recruiting umbrella organizations that can subcontract with local organizations, or setting aside a few smaller lots. 3 IONT Task 9: Define the Scope of Services--Focus on "What" Not "How" SEC The scope of services to be provided needs to be defined in the con- tract in sufficient detail so that contractors know what is expected of them. As one defines the scope of services, the focus should be on in- dicating "what" services need to be delivered, but one should generally avoid telling contractors "how" they should deliver the services. For example, specifying that immunization coverage should increase is im- portant and sensible, but indicating how immunization services should be delivered is not. Whether the contractors want to go house to house, stand on street corners, or set up mobile clinics in schools should be left up to them. What matters is that coverage increases, not how it is achieved. Two exceptions to this principle are when there is strong sci- entific evidence for a particular approach to how services are delivered or when it comes to ensuring that contractors comply with national technical standards regarding quality of care. For example, contractors must follow the national schedule for child immunization and should not be allowed to alter it (for example, they should immunize children with measles vaccine at 9 months and not change it to 6 months or 15 months). Many types of health services have been contracted; TORs for some of these services are included in appendix E and on the con- tracting Web site at http://www.worldbank.org/hnp/contracting. How to Contract · 37 Step 3: Design the Monitoring and Evaluation 1. Conduct dialogue with stakeholders 7. Carry out the bidding 2. Define the services process and manage the contract 3. Design the monitoring and evaluation 6. Draft the contract and bidding documents 5. Arrange for contract 4. Decide how to management and develop select contractors a contracting plan Agreeing on indicators is the first part of monitoring and evaluation (M&E), but this approach is meaningless if data are not actually col- lected, analyzed, and used for making decisions. Discussions often abound about the difference between monitoring and evaluation, but definitions are often inconsistent, and the distinction often blurs. 3 Task 10: Decide How Data Will Be Collected IONT In the overall contracting plan (see task 25) and the contract, it is useful SEC to have a specific list of indicators similar to table 3.4 (but with a few more indicators). However, it is also important to describe the details of each means of data collection and construct a table similar to table 3.5. A number of different methodologies can be used to collect infor- mation on the selected indicators, including (1) routinely collected data from the health management information system (referred to below as the HMIS for simplicity, even though there are other aspects to a typical health management information system), (2) household surveys, (3) health facility assessments, and (4) supervisory checklists, which are a way of systematically collecting and recording informa- tion during supervisory visits. Each approach to data collection has advantages and disadvantages. · Routine administrative records part of the health manage- ment information system (HMIS). The routine administrative records part of the HMIS is the way in which contractors regu- larly record and report their activities and should be the same as the purchaser's system. Using this part of the HMIS has a num- ber of advantages, for instance, it can, if working properly, provide 38 · Performance-Based Contracting for Health Services in Developing Countries Table 3.4 Examples of Indicators for a Performance-Based Contract Baseline Approximate Indicator Means of data collection value target Percentage of all women · HMIS 53% 75% pregnant during the last year · HHS (HHS) receiving at least one antenatal care visit from a skilled health care provider Score out of 100 on an index of · HFA 16 65 quality of care as judged by third · Supervisory (HFA) party, including adequacy of checklist waste management, drug avail- ability, provider knowledge, and patient-provider interaction Source: Author. Note: HFA = health facility assessment; HHS = household survey; HMIS = health management information system. 3 near real-time information, and it allows managers to easily track IONT their own performance. The routine recording part of the HMIS has several disadvantages: (1) it is often inaccurate (see box 3.3), SEC (2) contractors may overstate their own performance, (3) it is very expensive if you factor in staff time (see Stansfield and others 2006) although government officials see little incremental expenditure because these costs are already being met, and (4) it usually does not provide representative data on a number of important aspects of services, such as equity, community satisfaction, expenditure, and use of other services, such as from the private sector. · Household surveys (HHSs). Household surveys, including lot quality assurance sampling (LQAS; see box 3.4), look at a statisti- cally valid sample of households in the community and have the advantage of generally being more accurate, not being dependent on information collected by the contractors, and providing com- munity data on coverage, equity, health care expenditures, use of the private sector, and satisfaction. Disadvantages of HHSs include that they are relatively expensive to do, although the cost depends a great deal on the local avail- ability of firms or organizations that can carry out surveys and the sampling strategy, and they cannot be done too often and so will not usually provide real-time information. Table 3.5 Means of Data Collection Means of data Responsibility for data Schedule and arrangements Schedule for follow-up Counterfactual or collection collection and analysis for baseline data collection data collection Budget requirement comparison group Household Third party with July 200_, firm will be Every two years $250,000 per round Some noncontracted survey assistance from MOH recruited by March 200_ after the baseline times 3 rounds = areas (5 districts) will primary health care $750,000 also be surveyed. department Health facility Third party with July 200_, firm will be Every year Total = $400,000 Health facilities nation- assessment assistance from MOH recruited by March 200_ wide will be included contract management in sample. Sample size unit will allow comparison of contracted and non- contracted facilities. Health Contractor, MOH Already available and Quarterly assess- Total = $300,000 for Contracted and non- management HMIS department, archived at ment of data HMIS unit (does not contracted areas can information and MOH contract include health work- be compared (if data system management unit ers' time) quality independently assessed). Source: Author. Note: HMIS = health management information system; MOH = Ministry of Health. 39 SECTION 3 40 · Performance-Based Contracting for Health Services in Developing Countries Box 3.3 Accuracy of the Routine Administrative Records Part of the HMIS One of the reasons that a purchaser should not rely too heavily on data from the HMIS is that there are real issues with its reliability. For example, a number of stud- ies have indicated that reported vaccination coverage--that is, coverage estimates that come from these systems--typically overstates success (Murray and others 2003). The figure here compares results of the HMIS from districts in Pakistan with household survey results for three doses of the vaccine for diptheria, pertussis, and tetanus (DPT3) immunization coverage. It indicates that the HMIS-reported coverage is usually higher than the household survey data (on average 14 percent- age points higher). In addition, districts that reported more than 100 percent cover- age (obviously impossible) often had very poor performance. Forty percent of such districts had less than 50 percent coverage via the household survey, which means that if purchasers relied on HMIS data alone, they would have incorrectly identi- fied these districts as good performers when, in fact, they needed to improve their performance considerably. DPT3 Coverage Rates from Routine Reports versus Household 3 Surveys for Districts in Pakistan, 1999­2002 IONT 120 SEC 100 (%) 80 60 coverage 40 reported 20 0 0 20 40 60 80 100 household survey (%) Source: Loevinsohn, Hong, and Gauri 2006. · Health facility assessments (HFAs). This is a sample survey of health facilities using a standardized methodology and instrument that looks at various aspects of quality of care, such as knowledge and skills of providers, patient satisfaction, and availability of inputs including drugs, supplies, equipment, and staff. The advantages of HFAs include the following: (1) they can assess quality of care from a technical perspective, (2) they are an independent and impartial How to Contract · 41 Box 3.4 Lot Quality Assurance Sampling Lot quality assurance sampling (LQAS) is a statistical technique that can be used to quickly and inexpensively judge performance in a defined geographical area. As such, it can be used to determine whether a contractor has achieved the specified target in a defined contract area ("lot"). LQAS is now widely used, with more than 800 such surveys carried out worldwide in the past 20 years in applica- tions ranging from HIV/AIDS to immunization, women's health, nutrition, diarrheal disease control, quality management, and neonatal tetanus mortality (Robertson and Valadez 2006). To use LQAS, health system managers identify a coverage target (for example, 80 percent coverage of DPT3 immunization) and a lower threshold that represents an unacceptable level of coverage (for example, 50 percent coverage). LQAS then uses random sampling to classify lots as (1) successful, that is, at or above the target; (2) areas with unacceptably low levels of coverage; and (3) areas in between the target and the lower threshold, which are treated as similar to the nearest of the two (for example, 70 percent is treated as similar to the target of 80 percent coverage, and 60 percent is treated as similar to the lower threshold of 50 percent). 3 The advantages of LQAS include that only a small sample is needed to judge whether a lot has reached the predetermined coverage target, data from indi- IONT vidual lots can be combined into an estimate of coverage for the entire program areas that consists of multiple lots, and because LQAS is essentially a hypothesis SEC test, it can provide supervisors with a decisive judgment about whether a target has been accomplished, or not. However, LQAS also has its disadvantages: it can be used only to determine if a predefined target has been met; it does not provide point estimates of coverage; without proper training and manuals, it can be difficult to implement and under- stand; and it relies on random sampling rather than cluster sampling. With these limitations in mind, several easy-to-use manuals have been developed: · Lwanga, S., and S. K. Lemeshow. 1991. Sample Size Determination in Health Studies: A Practical Manual. Geneva: World Health Organization. · Valadez, J. J. 1998. A Training Manual for Lot Quality Assurance Sampling to Manage Decentralized Health Programs: A User's Handbook. Washington, DC: OMNI. · Valadez, J. J., W. Weiss, C. Leburg, and R. Davis. 2003. Assessing Community Health Programmes Using LQAS: A Participant's Manual and a Trainer's Guide. 2 vols. St. Albans, England: Teaching-Aids at Low Cost. (Available in English, French, Russian, and Spanish.) 42 · Performance-Based Contracting for Health Services in Developing Countries assessment of what is happening in the facilities operated by the contractor, and (3) they can be done more frequently than house- hold surveys and provide near-real-time information. Disadvantages of HFAs include that (1) they are complex and challenging to design and implement (examples of HFA question- naires are available at http://www.worldbank.org/hnp/contracting), (2) they can be relatively expensive to carry out, (3) they cannot provide information on coverage of services, and (4) the informa- tion they provide on patient satisfaction may be too optimistic (dis- satisfied people stay away from the facilities and so are not included in the sample of exit interviews). · Supervisory checklists (SCs). Supervisory checklists are like a short HFA, but they also yield a quantitative score that summarizes health facility performance. They can be used by the purchaser, local governments, contractors, or a third party to assess performance quickly. Although they may contain some of the same information 3 as an HFA, they focus on fewer items and provide a way for allowing IONT multiple visits to be included in the same form so as to judge progress. Their advantages include that (1) they can assess parts of quality of SEC care, (2) they can be done frequently, (3) they are relatively cheap to implement, and (4) there is evidence that they improve health worker performance (Loevinsohn, Guerrero, and Gregorio 1995). Disadvantages of SCs include that it is challenging to design a good one (there are many poorly designed SCs, and they are not often used) and that it is a challenge to ensure that they are used regularly. Recommendations. It is usually necessary to use a few different approaches to collecting information, even for the same indicator. No method is perfect, but all have their uses. One thing to keep in mind in designing data collection mechanisms is ensuring consistency over time. For example, changing the sampling methodology, ques- tions, or response group for household surveys may make the results noncomparable. Task 11: Collect Baseline Data One of the biggest issues in M&E is the lack of baseline data, which makes progress difficult to measure. Collecting baseline information is often tricky because it usually needs to be done at the same time as How to Contract· 43 services are being designed and service delivery is starting. Hence, one of the first tasks during the contracting process should be recruitment or mobilization of the organization that will actually do the baseline data collection (advanced procurement action is often required before funding is fully secured). In some situations, it may even make sense for contractors to collect the baseline data using a standard methodol- ogy, although this introduces the possibility of bias. Even if HMIS data will be used, these data should be collected and archived so that the data are readily available and consistent. Task 12: Devise a Clear Schedule for Data Collection It is important to have a clear schedule for data collection. Household surveys should be conducted every year or two, health facility assess- ments should be carried out every year, and supervisory checklists should be used every two months. HMIS data should be reviewed at least quarterly. 3 Task 13: Look for Comparison/Control Groups IONT Often judging the success of contracting and contractors requires SEC "benchmarking," that is, comparing the performance of contractors to each other or to other health service providers. Particularly in set- tings where contracting is controversial, it is worthwhile having a controlled, before-and-after comparison. This is a powerful way of learning lessons and evaluating the benefits and costs of contracting. A comparison or "control" group can be areas in which services are provided by the government sector or for-profit private sector provid- ers. (Where possible, randomly assigning lots to contracting and con- trol groups allows for the most rigorous type of evaluation.) Deciding on a comparison/control group has implications for data collection and the budget required. Task 14: Assign Responsibility for Collection, Analysis, and Dissemination of Data In situations that involve many contracts, it is sensible to make M&E someone's full-time job. Purchasers should consider recruiting a third-party firm to help with M&E design and data collection. The advantages of this approach include factors such as (1) the purchaser 44 · Performance-Based Contracting for Health Services in Developing Countries can obtain expertise it may not necessarily have; (2) the approach can allow the purchaser to focus on other aspects of contracting, includ- ing field visits and training; (3) the purchaser can collect data that does not rely on information generated by the contractor; and (4) this approach generally provides for a more impartial assessment of con- tractor performance. The major challenge with using a third party is maintaining purchaser involvement and commitment to M&E. Draft TORs for a third-party firm to help with M&E design, data collec- tion, and analysis are found in appendix D. Task 15: Budget Sufficient Funds for Monitoring and Evaluation The conduct of M&E activities will not happen without a budget that includes funds for (1) recruitment of a third party where appropriate, (2) conduct of a sufficient number of household surveys and health fa- cility assessments, and (3) staff within the contract management unit 3 to work full time on M&E. Depending on the size of the contracting effort, it is not unreasonable to budget 5 percent of the value of all the IONT contracts for M&E. SEC Step 4: Decide How to Select Contractors and Establish the Price 1. Conduct dialogue with stakeholders 7. Carry out the bidding 2. Define the services process and manage the contract 3. Design the monitoring and evaluation 6. Draft the contract and bidding documents 5. Arrange for contract 4. Decide how to management and develop select contractors a contracting plan Task 16: Use a Competitive Selection Process The fairest and best way of selecting a contractor is through open competition based on clearly defined criteria. This approach has many advantages, such as (1) potential bidders want to know that there is a "level playing field" so that it is worth their while to bid and partici- pate in the contracting process; (2) competition will generally lead to How to Contract · 45 the selection of the best organizations, the best managers, and the best ideas; and (3) competitive selection reduces the chances for cor- ruption and is usually required under government regulations and the rules of external financiers. By contrast, single-source selection, that is, a noncompetitive pro- cess, can be quick but it (1) is not fair to all potential contractors; (2) is not transparent and can easily become corrupt; (3) often creates resent- ment in the nonstate sector because contractors will be assumed to have received their contracts because of political connections or cor- rupt practices; (4) leads to "fat and happy" contractors who, shielded from the rigors of competition, usually become ineffective and inef- ficient; and (5) limits creativity and innovation. In a small number of situations, competitive selection may not be easy or the most practical solution: · An NGO or FBO may already be providing a significant number of services in a particular area (for example, a mission clinic that has 3 been providing health services for 60 years) · There is very limited competition (for example, providing services IONT in a conflict-affected area where few NSPs or government staff want SEC to work, and everyone is happy when even one NGO volunteers to work there) · The NSP brings significant funds with it into a partnership · Contracts will be made with existing private providers as is being commonly done for control of tuberculosis (see box 3.5). In most of these situations, the purchaser can avoid many problems by using P4P approaches that put at least some of the financial risk on the contractor. Recommendations. In most situations, it is best to use a competi- tive process to select contractors. Noncompetitive processes should be used only when there are compelling reasons. For World Bank bor- rowers and staff, the matrix in appendix 2 provides guidance on what kind of competitive selection process to use. Task 17: Develop Clear Selection Criteria During the design phase, it is important to develop explicit criteria for selecting the contractors. These criteria should be clearly defined 46 · Performance-Based Contracting for Health Services in Developing Countries Box 3.5 Collaborating with Private Practitioners to Treat Tuberculosis Traditionally, TB control efforts have focused on the public sector even though many patients with TB symptoms, including the poor, receive their care in the private sector. There has been broad concern about the quality of care provided by private practitioners because they have not been using state-of-the-art tech- niques for diagnosis, treatment, and case management. To address this issue, national TB programs (NTPs) in more than 15 countries have begun collaborat- ing with private providers through the use of "soft contracts." In return for free drugs and recognition, private practitioners are expected to follow NTP technical guidelines, including recording and reporting requirements. Fourteen of these kinds of public-private collaborations, in eight countries, have been evaluated, and they appear to be quite successful. Treatment success rates have averaged almost 90 percent, and case detection rates have increased by a median of 16.5 percentage points. In 7 of the 14 evaluated cases, NGOs have been used as intermediaries, and it appears that these efforts were even more successful. Where NGO intermediaries were used, the treatment success rate was 8.1 percentage points higher (90.1 percent vs. 82.0 percent) than those cases 3 in which the NTP itself managed the collaboration with private providers. (See appendix C, case 11, or Lonroth, Uplekar, and Blanc 2006 for more details.) IONT SEC before the beginning of the selection process and be known to all participants, should not be excessively detailed so as to prevent the evaluators from using their judgment, and should not be unrealis- tic. Purchasers sometimes set the criteria too high and then cannot find contractors who meet the exacting standards. (For an example of suggested selection criteria, see the contracting plan in appendix A.) Often it is useful to first short-list bidders on the basis of some minimum criteria before going to the trouble of evaluating their de- tailed technical proposals. (However, this approach does not apply in situations in which World Bank borrowers are using the Bank's pro- curement guidelines [Red Book], in which case the criteria below are judged on a pass-fail basis for all organizations submitting bids.) · Short-list criteria. These criteria are meant to ensure that the or- ganization has the minimum size, skills, and reputation to deliver the health services described in the contract. They can include (1) copies of three years of audited accounts for the organization, (2) proof (in the audited accounts) that the organization had a turnover How to Contract· 47 of some reasonable amount of funds in the last three years, (3) a copy of the organization's registration as a company or an NGO under the relevant acts of the particular country, and (4) some indi- cation that the organization has experience of the type that would allow it to deliver the services. · Technical proposal evaluation criteria. For organizations that meet the short-listing criteria, it makes sense to evaluate their tech- nical proposals using the following criteria: (1) the experience of the organization (its track record) in delivering similar services, because this is a critical means for judging how they are likely to perform; (2) the qualifications, experience, and reputation of key managers (usually the most important three or four in the team that will actu- ally implement the contracted services), because their abilities and expertise are much of what the purchaser is actually buying; and (3) the proposed work plan or strategy to see whether the bidder's ap- proach is practical and creative. Experience in the field shows that it 3 is reasonably easy to put together a plan that sounds good, but that this provides little insight into what the organization will be able to IONT accomplish. For this reason, it is recommended that the evaluation of the work plan be given the least weight compared with the other SEC two criteria. Task 18: Establish a Transparent and Independent Evaluation Process As important as the selection criteria is the evaluation process that must both be transparent and appear to be transparent. The best way to ensure this is by having an independent and competent evaluation committee whose members individually rate proposals. It is recom- mended that members of the evaluation committee come from outside the purchaser's organization and include representatives of interna- tional agencies (such as UNICEF or WHO [but not the World Bank, whose procurement guidelines prohibit Bank staff from being involved in the evaluation of bids that will be financed with Bank funds]) and the NGO community (for example, someone from the umbrella or- ganization of NGOs), although the latter must not work for an NGO that has submitted any proposals to avoid conflict of interest. Experi- ence indicates that having members of the committee from outside the 48 · Performance-Based Contracting for Health Services in Developing Countries purchaser ensures a more transparent evaluation process that follows established procedures, and it actually speeds up the bidding process (for example, there are fewer complaints). Task 19: Maximize Interest of Possible Contractors Given the value of transparent and competitive selection process, it is worthwhile to try to maximize the number of organizations that apply for each lot. Some actions that can increase interest include the following: · Carrying out consultations with potential contractors before the selection process · Advertising widely, such as through Development Gateway, United Nations Development Business, and other Web sites · Sending information directly to organizations that might be 3 interested · Holding information sessions or "pre-bid conferences" IONT · Keeping the request for proposal (RFP) documents as simple as SEC possible and the selection process transparent and understandable · Allowing smaller NGOs to form consortia · Avoiding bid and performance bonds. Task 20: Select a Contractor (For World Bank borrowers and staff, appendix B and task 20A pro- vide guidance on how to select a contractor.) For purchasers not bound by World Bank procurement guidelines, there are basically three ways of selecting a contractor and setting the price of the contract: (1) competition, wholly or partly based on bid price; (2) negotiation; and (3) establishing a fixed budget before the competition. · Competition has the following advantages: (1) it provides the low- est prices in the long run, thereby allowing limited resources to cover the largest number of beneficiaries; (2) it is a very transpar- ent means for establishing the price because each bidder's price is called out publicly at the opening of the financial proposals; (3) it How to Contract· 49 facilitates innovation by encouraging bidders to develop the most efficient means for delivering services; and (4) it reflects the local realities of the particular lot. The disadvantages of competition include the following: (1) it takes a few weeks longer than other approaches; (2) it can result in inconsistent prices so that similar-looking lots have very different prices, which can result in difficulties wherein journalists or others raise concerns about corruption; and (3) especially early in the con- tracting experience, competition can result in prices that are either too high or too low because bidders have limited experience and knowledge of the true cost of delivering the services. · Negotiation between the purchaser and the bidder achieving the highest score on its technical proposal has the following advantages: (1) it is a fairly rapid way of establishing the price, (2) it can reflect local realities, and (3) if the purchaser is well motivated and savvy, it can result in a reasonably good price. 3 The disadvantages of negotiation include the following: (1) it is not transparent and can easily be, or perceived to be, corrupt; (2) it IONT can also result in inconsistent prices for similar-looking lots; and (3) it is difficult to know whether the negotiated prices are "reasonable" SEC from the purchaser's perspective. · Fixed budget is where the purchaser fixes the budget available for each "lot" in advance of publishing the request for proposal. It has the following advantages: it is transparent because every bidder knows what the fixed budget is, and it is fairly quick. The disadvantages of the fixed budget approach are the following: (1) it is rigid as if "one size fits all," (2) it discourages innovations and efforts to reduce costs through more efficient means of service deliv- ery, and (3) it is often hard to estimate in advance how much services will cost resulting in prices that are too low or too high. Costing stud- ies to determine what the fixed budget should be are complicated to carry out, can take considerable time, and often come up with results that subsequently appear to be wrong. For example, in Bangladesh, the bid prices (that is, the market price that organizations thought the services would cost) were 35 percent lower than the estimates from a detailed costing study. In Cambodia the bid prices were nearly three times higher than the costing study indicated they should be. 50 · Performance-Based Contracting for Health Services in Developing Countries Recommendations. Transparent approaches are important in avoiding corruption and argue against negotiations behind closed doors. Approaches can also be modified to make them more appro- priate. For example, the fixed-budget approach can specify a cost per beneficiary and increase the price for mountainous or remote areas. Task 20A: Select a Contractor For World Bank borrowers and staff, two distinct ways may be used for selecting contractors. When the contract results in an easily mea- sured physical output, such as cleaning services or equipment main- tenance in a hospital, the contractor should be selected using a wide competitive process in which the award will go to the bidder with the lowest price who meets the technical criteria established by the pur- chaser. When the output is of an intellectual nature, such as manage- ment of a hospital or designing and implementing a behavior change campaign, the contractor should be selected in conformity with pro- 3 cedures normally used for the selection of consultants. There are IONT three methods that can be used in the latter case: (1) selection based on the quality of the technical proposal and costs, (2) fixed budget, SEC and (3) single-source selection. Hybrids of these approaches exist that can also be useful. Open competition is the preferred approach. However, circum- stances may justify other selection methods, such as the following: · Selection based on quality and cost. This method uses a com- petitive process among short-listed organizations that takes into account the quality of the proposal and the cost of the services in the selection of the successful bidder. Cost as a factor of selection is used judiciously. The relative weight to be given to the quality and the cost should be determined for each case, depending on the nature of the assignment. · Fixed budget. This method is appropriate only when the assign- ment is simple and can be precisely defined and when the budget is fixed. The RFP should indicate the available budget and request that the bidders provide their best technical and financial propos- als in separate envelopes, within the budget. The TORs should be particularly well prepared to make sure that the budget is sufficient How to Contract· 51 for the consultants to perform the expected tasks. Evaluation of all technical proposals should be carried out first. Then the price pro- posals should be opened in public and the prices read aloud. Propos- als that exceed the indicated budget should be rejected. The bidder who has submitted the highest-ranked technical proposal among the rest should be selected and invited to negotiate a contract. · Single-source selection (SSS). This method of selecting bidders does not provide the benefits of competition in regard to quality and cost, lacks transparency in selection, and could encourage unaccept- able practices. Therefore, SSS should be used only in exceptional cases. It may be appropriate only if it presents a clear advantage over competition: (1) for tasks that represent a natural continuation of previous work carried out by the firm; (2) in emergency cases, such as in response to disasters and for consulting services required dur- ing the period of time immediately following the emergency; (3) for very small assignments; or (4) when only one firm is qualified or has 3 experience of exceptional worth for the assignment. The disadvantages of SSS or negotiations include that it is not IONT transparent and can easily be, or perceived to be, corrupt (it can SEC also result in inconsistent prices for similar looking lots), and it is difficult to know whether the negotiated prices are "reasonable" from the purchaser's perspective. · Hybrids. Hybrids of the above approaches are also possible, and there is much scope for creativity. For example, a fairly common and sensible compromise between selection on the basis of quality and cost and fixed budget is where the amount of money for service delivery itself is fixed but the bidders compete on the basis of their overhead costs. (The RFP documents for such an approach are available at http://www.worldbank.org/hnp/contracting.) Another option, although it is a little complicated, is to have contractors compete on what proportion of the fixed budget would be provided to them on the basis of performance. (For example, one bidder would have 50 percent of the funds provided as a fixed budget and the remainder would be provided on the basis of performance as prescribed by the purchaser. Another bidder could bid 70 percent as fixed and 30 percent on the basis of performance.) 52 · Performance-Based Contracting for Health Services in Developing Countries Step 5: Arrange for Contract Management and Develop a Contracting Plan 1. Conduct dialogue with stakeholders 7. Carry out the bidding 2. Define the services process and manage the contract 3. Design the monitoring and evaluation 6. Draft the contract and bidding documents 5. Arrange for contract 4. Decide how to management and develop select contractors a contracting plan Contract management is the aspect of the contracting process that usually gets the least attention, often because it is not explicitly ad- dressed in any document (such as the contract or the RFP). This is one reason that drafting a written contracting plan (or contracting 3 manual) is so important. The major functions involved in contract management include the following: IONT · Ensuring both parties adhere to the terms of the contract SEC · Paying contractors on time so as to avoid disruption of services · Avoiding corruption or the appearance of corruption · Ensuring that proper monitoring and evaluation of contractor per- formance is carried out so that the purchaser can be confident that resources are being effectively used · Solving the problems that can arise in any complex relationship · Maintaining purchaser stewardship and ownership while avoiding micromanagement of the contractors. Addressing these issues requires considering who is responsible, the staff needed, and the available budget. Task 21: Define Responsibility and Clarify the Contract Management Structure Managing contracts requires full-time attention by a clearly defined, reasonably sized team with explicit responsibilities and authority. Dif- How to Contract · 53 ferent approaches have been taken to the location of the contract man- agement unit, including within the Ministry of Health (or other state organizations such as national AIDS committees), local governments, or special government procurement units. Although different approaches have been used in many countries, issues have arisen with all of them. · Central Ministry of Health. Many ministries of health have lim- ited experience and knowledge of contract management, and they are sometimes constrained from hiring staff or consultants who do have the knowledge. In addition, there is often real opposi- tion within the MOH to contracting, and contractors fear that the MOH will deliberately sabotage their efforts. Advantages of having contract management in the MOH (or related organization) include greater ownership and better technical knowledge of the sector. This approach has been used in many countries, including Bangla- desh and Cambodia. · Local government. Economies of scale exist in contract manage- 3 ment, and there is a fear that building the capacity for effective management at local levels is inefficient. However, the possible IONT advantages of this approach include greater involvement of local SEC officials and better monitoring of contractor performance. This approach has been used in a few countries, including Guatemala and Pakistan. · Specialized procurement unit. This approach limits the steward- ship function of the MOH, and specialized procurement units may be less motivated than organizations whose mandate is to improve health. Conversely, such units usually have extensive procurement experience, have more experience with contract management, and generally do a good job in regard to timely payment of contractors. This approach has been used in Africa, including in the Democratic Republic of Congo and Senegal. Recommendations. Where possible, it appears that having con- tract management based in the MOH or other line agency is the best long-term approach. It ensures ownership and long-term stewardship of the sector. It also ensures that people who are technically expert on the health services to be delivered will be involved in the process. This approach can be strengthened by (1) recruiting consultants or other 54 · Performance-Based Contracting for Health Services in Developing Countries skilled staff to support the MOH, (2) involving local governments in contract management (for example, review of quarterly reports) and field monitoring, and (3) ensuring that members of such a unit have an incentive to help contractors perform well. Task 22: Ensure Proper Staffing of the Contract Management Unit Without making contract management units too large, they will need (1) a senior person who can effectively manage relationships and com- municate with stakeholders, (2) skilled people who can spend consid- erable time in the field troubleshooting and monitoring performance, and (3) someone with a background in financial management. If the unit will also be responsible for recruitment of contractors, then some- one with procurement expertise may be required. Because many min- istries of health have limited experience with contracts, it often makes sense and leads to better results to recruit local consultants (paid mar- 3 ket wages) into the unit. IONT Task 23: Allow Sufficient Budget for Contract Management SEC Purchasers tend to underestimate the resources that are required for a contract management unit to be effective. The budget for managing the contracts needs to reflect (1) the cost of staff or local consultants; (2) equipment, computers, and software; (3) the cost of transportation and per diems that facilitate field visits for monitoring; and (4) pos- sible incentives for the contract management staff linked to the results achieved by contractors or timely payment of contractors. Task 24: Consider Computerization of Contract Tracking When more than six or seven different contracts must be managed, it becomes difficult to do it by hand. In cases in which larger numbers of contracts are involved, it makes sense to obtain contract management software. Task 25: Develop a Written Contracting Plan The purchaser needs to develop an overall contracting plan (some- times referred to as a manual of procedures or contracting manual) to avoid serious gaps or mistakes. Such a contracting plan should sum- marize the entire contracting process, including how contracts will be How to Contract · 55 managed and how they will be monitored and evaluated. (An example of a contracting plan is found in appendix A.) Developing such a plan should not be seen as a burden, because most important aspects can be dealt with in six or seven pages. Step 6: Draft the Contract and Bidding Documents 1. Conduct dialogue with stakeholders 7. Carry out the bidding 2. Define the services process and manage the contract 3. Design the monitoring and evaluation 6. Draft the contract and bidding documents 5. Arrange for contract 4. Decide how to management and develop select contractors a contracting plan 3 Drawing on the considerations in steps 1 to 5 and the content of the IONT contracting plan, the next step in the process is to draft a contract in- SEC corporating the scope of work developed in step 2 and the issues listed below. Appendix A provides an example of a contract (or an agree- ment). Finally, it will be necessary to formulate bidding documents, such as the request for proposal. An example of a World Bank RFP document, including the draft contract, is available at http://www. worldbank.org/hnp/contracting. Task 26: Maximize Managerial Autonomy One of the principal advantages of contracting is that NSPs are less constrained by the "red tape" and political interference that often plague governmental efforts at service delivery. Giving contractors significant managerial autonomy improves results for several reasons: · Management decisions are made by the people who are closest to the reality on the ground and can make the most informed deci- sions about how to tackle problems that arise. · It is easier for purchasers to hold contractors accountable for results because the latter cannot claim that the action, or inaction, of others has interfered with their performance. For example, if a contractor 56 · Performance-Based Contracting for Health Services in Developing Countries is responsible for hiring staff and setting wages, it cannot blame any- one else if it is unable to deliver services because of a staff shortage. · Managerial autonomy allows and actually encourages the inno- vations that can lead to improved performance. In Cambodia, an NGO introduced performance-based bonuses for the staff, which increased performance dramatically (Soeters and Griffiths 2003). · It allows the contractor to take advantage of the nonstate sector's inherent flexibility. For example, in urban Bangladesh an NGO manager arranged a raffle of a TV set as a way of informing the community about the opening of a new clinic, which was soon packed with patients. It would have been very difficult for a public sector manager to arrange for such a raffle. There are a number of ways of increasing the managerial autonomy of contractors: 3 · Clarify the authority of both parties. When the authority of the purchaser and the contractor are not clear, the purchaser usually IONT tries to assert control. Making the roles and responsibilities of both parties explicit in the contract can avoid important problems that SEC may later interfere with effective delivery of services. · Focus on "what" not "how." As mentioned, it is important to de- fine the scope of services by what services are delivered rather than how they are delivered. · Discover other ways of increasing autonomy. There are a few other important ways of providing contractors with appropriate au- tonomy that are discussed in more detail below, such as (1) allowing contractors normal management prerogatives over health workers; (2) using lump-sum budgets rather than line-item budgets; (3) leav- ing procurement of goods, supplies, and medicines to the contrac- tor; and (4) not applying inappropriate public sector accounting rules and procedures to the nonstate sector. Task 27: Ensure That Contractors Can Manage Personnel Effectively A very important part of successfully implementing health services is to be able to manage the staff effectively. Giving contractors control of the personnel function is an important aspect of managerial auton- omy without which performance can be compromised (see box 3.6). How to Contract · 57 In practice, this approach means that contractors should be given the normal management prerogatives of hiring, firing, posting, handling pay and benefits, setting terms of employment (such as performance bonuses), and establishing staffing levels. In management contracts in which the contractors use existing MOH health workers, the manage- ment prerogatives of the contractor may be limited by civil service rules. However, it still should be maximized to the extent possible. In a number of countries, contractors have succeeded in getting health workers into isolated locations that had previously been underserved (see box 3.7). Purchasers can reasonably expect that contractors re- cruit qualified staff with recognized credentials, ensure that health Box 3.6 Results of a Government Maintaining Control of Staff Assignments In the Democratic Republic of Congo contractors were allowed to manage gov- 3 ernment health workers. However, decisions about the assignment of staff were made by the Ministry of Health, and during the first round of contracts the MOH used its authority regarding staff posting. The result was that an excessive num- IONT bers of health workers were transferred into the contracted areas because the SEC pay was higher and the working conditions were better. For the second round of contracting, the contracts specified that the government would follow predeter- mined staffing patterns that ensured reasonable numbers of health workers. Box 3.7 Attracting Health Workers to Underserved Areas In Cambodia, a contractor in one district was able to increase the number of doc- tors working there dramatically by paying wages that were much higher than the government rate (but still only $250 per month). Combining a higher salary with certain benefits, such as a motorcycle and a performance-based bonus, allowed the NGO to attract five doctors to work in the district that previously had no doctors. In Afghanistan, contractors offered inducements to female health workers to relocate to remote rural areas. Besides better pay, this offer included employment for a male relative (very important in the cultural context), housing near the health center, and a small generator that provided light and entertainment (such as a DVD player). The results were impressive. Before the contracts, only 24 percent of health centers had trained female health workers. After two years of the con- tracts, 82 percent of the health centers had trained female staff. 58 · Performance-Based Contracting for Health Services in Developing Countries workers receive training to further enhance their skills, and respect national labor laws. Task 28: Use Lump-Sum Contracts Lump-sum contracts are those in which contractors receive an agreed- on amount of funds on a regular basis that is not reimbursement for specific expenditures they have incurred. (Even payment according to the number of services provided is a type of lump-sum payment because it is not tied to reimbursement of expenditures.) Lump-sum budgets provide an amount up to which managers can spend without worrying about the amount of a specific line item (line items limit the amount that can be spent on a particular type of expenditure, such as $85,000 for drugs and $120,000 for equipment). Lump-sum budgets have a number of important advantages: · They give managers the flexibility to move money to where it is 3 most needed IONT · They prevent "micromanagement" by the purchaser's contract man- agers, which can stifle creativity (micromanagement can result where SEC line-item budgets are used) · They facilitate implementation because it is not necessary to seek permission for changes in line items or to engage in the endless arguments between accountants over what items should be reim- bursed, which often result in delays · They ensure that purchasers and contractors focus on the outcomes and outputs of the contract, not merely the inputs, of which money is one. Lump-sum budgets do not preclude careful financial manage- ment. They are consistent with proper accounting and allow line-item descriptions of expenditures in reports provided by contractors (these reports facilitate subsequent costing studies). Task 29: Leave Procurement of Supplies, Equipment, and Services to Contractors In most situations it is better to leave the procurement of medicines, supplies, and equipment to the contractor. Experience has generally shown that decentralizing procurement to individual contractors en- How to Contract · 59 Box 3.8 Decentralizing Drug Procurement in Afghanistan In Afghanistan, the government signed contracts with NGOs that gave them responsibility for procurement of medicines and other supplies. At the same time, other contracts stipulated that medicines would be procured centrally by an orga- nization with long-term experience in drug procurement and distribution. In addi- tion, in a few provinces drug procurement and distribution were handled by the central Ministry of Health. An independent assessment showed that the availabil- ity of drugs improved much more in the places where individual contractors were responsible for procurement and distribution (see table). Effects of Decentralized Procurement on Drug Availability Index (Maximum Score = 100) in Afghanistan Approach to drug procurement and distribution Baseline Endline Change Decentralized to individual NGOs 62.4 92.1 29.7 Centralized, nonstate organization 70.3 83.0 12.7 responsible for procurement and 3 distribution Centralized, public sector procurement 57.1 71.2 14.1 IONT and distribution SEC Source: Bloomberg School of Public Health, Johns Hopkins University. sures better availability of supplies at the point of service delivery (see box 3.8). Although there may be economies of scale gained from central procurement, they must be weighed against not having supplies where needed. As for the quality of drugs, there appears to be no consistent difference between medicines procured by governments and those pur- chased by the nonstate sector (if anything, the latter may have an advan- tage). The purchaser should indicate in the contract the list of items to be procured and include specifications and standards (such as for drugs). The contract should also stipulate that the purchaser can carry out spot checks to evaluate drug quality. Some supplies and drugs, particularly vaccines, should be procured centrally. Task 30: Ensure That Duration of the Contract Is Sufficiently Long Experience in other sectors and increasingly in health sectors indi- cates that the minimum duration of a contract should be four to five 60 · Performance-Based Contracting for Health Services in Developing Countries years. There are a number of good reasons: (1) it takes time for both parties to understand and get used to the contractual arrangements and to develop a solid working relationship, (2) it makes sense to give contractors a reasonable amount of time to implement their plan, and (3) continuity--especially when dealing with local communities--is a clear advantage. No advantage can be seen to short-duration con- tracts that need to be renewed frequently. They just take up time, lead to delays, increase opportunities for conflict and corruption, increase costs, and result in gaps in services. Purchasers, legitimately, want to have the opportunity to terminate the contracts of organizations that are performing poorly. However, better than having frequent renew- als of short-duration contracts is to have clear termination clauses and regular reviews of contractor performance. Task 31: Have Clear Procedures for Making Payments There are two types of payment in a typical contract: an initial mobi- 3 lization payment and regular payments during the life of the contract IONT (usually every three or six months). SEC · Mobilization payments. Because contractors for health services are often NGOs or CBOs, they have no "capital," and it is impor- tant for them to receive a mobilization payment on signing the contract. This can usually be 10 percent of the contract amount. It is not recommended that a performance bond or bank guarantee be used with nonprofit contractors because it can lead to serious delays, it is often unfair since NGOs are not set up on such com- mercial terms, and there are other ways of ensuring that money will be properly used by relying on the social collateral of such NGOs. (Suggested wording for a letter from an NGO in lieu of a performance guarantee is available at http://www.worldbank. org/hnp/contracting.) Where NGOs have had previous contracts and have a "track record," the risk of providing a mobilization payment is low (partly because poorly performing NGOs can be "blacklisted"). · Regular payments. Subsequent, regular payments should be made on a lump-sum basis on submission by the contractor to the pur- chaser of acceptable reports (see task 34) that are explicitly described in the contract. Payments should be made on a quarterly or six- How to Contract · 61 month basis because this reduces the transaction burden on both parties and increases the likelihood that contractors are paid on time. As argued above, lump-sum payments reduce transaction time and effort, increase managerial autonomy and flexibility, and reduce the number of disputes. The prerequisites for claiming payment and processing payments should include submission of a quarterly activ- ity report, an invoice, and a financial report. The actual calendar dates for payment should also be agreed to by the parties. Task 32: Establish a Clear Process for Termination and Imposing Other Sanctions The contract needs to spell out clearly the procedures and rules gov- erning the termination of the contract. This clarity is important for the purchaser's peace of mind and helps it meet its responsibilities for carefully managing public funds. It makes sense to use a dispute reso- lution mechanism (task 33) before terminating the contract. Besides 3 termination, the contract should spell out other sanctions that the purchaser can employ in the case of poor contractor performance. A IONT few things that appear to work include (1) having face-to-face meet- SEC ings with key contractor officials, (2) writing letters first to the project manager and then to the board of directors of the organization (em- barrassment seems to work quite well, particularly when there is ob- jective evidence of poor performance), (3) demanding the replacement of key staff if other efforts have not produced the desired results, and (4) limiting the opportunity of the contractor to avail itself of other purchaser-sponsored contracting opportunities (for example, it can- not bid on new contracts, so-called blacklisting or debarment). Task 33: Establish Dispute Resolution Mechanisms Most disputes can be settled through discussion and, although rarely required, it is worthwhile to describe in the contract a fair method for resolving disputes. These descriptions should be kept fairly simple, and the first step can simply be appointment of a mediator acceptable to both parties. If this step fails to resolve the issue, it is worthwhile using an arbitration panel composed of people of known integrity. (The annex to appendix A has an example of a contract clause on dis- pute resolution, part K of the draft agreement.) 62 · Performance-Based Contracting for Health Services in Developing Countries Task 34: Define Reporting Requirements of the Contractor The reporting requirements of the contractor should not be exces- sive. We recommend that contractors submit a quarterly report that includes the following: · A description of progress made against the work plan · Problems encountered and solutions undertaken · A summary of health management information system data (HMIS forms should also be sent to the purchaser regularly, usu- ally monthly) · A financial statement · A bank account statement; contractors should also be expected to provide an annual external financial audit report. 3 Task 35: Have an Explicit Policy on User Charges IONT The imposition of user charges or fees for health services is contro- versial. However, many NGOs or private providers that end up being SEC contractors may already be implementing user charges, and so the con- tract should be explicit about whether and how contractors can levy such charges. If user charges are permitted, they should comply with guidelines established by the purchaser. These should specify that (1) user charges will not interfere with the accomplishment of greater eq- uity as stipulated in the objectives of the contract, (2) the level of the user charges will be reasonable and publicly displayed, (3) there will be explicit exemption procedures for the poor, and (4) the funds collected can be retained in the location where they are collected. Task 36: Ensure That Contractors Use Independent, Private Sector Auditors In other types of contracts, such as for construction of a road, the purchaser would not usually ask for an audited financial statement from the contractor. This makes sense because the purchaser is really interested in the timely construction of a road of acceptable quality. A similar approach should be used in contracting for health services. However, some purchasers, to allay their own fears, expect contractors How to Contract· 63 to provide regular financial reports and audited financial statements at the end of the year. If such audited statements are required, experience in the field strongly suggests that contractors should be allowed to use independent auditors, which ensures proper financial management while avoiding overly constraining bureaucratic rules. Task 37: Ensure That Contractors Build the Capacity of Health Workers The contractor should be responsible for ensuring that the health work- ers delivering services have the capacity to do a good job and to meet the technical standards stipulated in the contract. This task is particularly important in cases in which quality of care needs to be improved (which is almost everywhere). The contract should stipulate the following: · The qualifications of the health workers to be employed (purchasers need to be careful that they are realistic in what they expect and con- tractors should not be employing cardiac surgeons in rural clinics) 3 · The obligations of the contractor for training and capacity building IONT for their staff SEC · How contractors will obtain access for their staff to government- run or -financed training courses. Task 38: Address the Capacity Needs of Contractors Purchasers are sometimes reluctant to address the capacity needs of contractors; they argue that they want to hire NSPs who already have the needed capacity. However, building the capacity of contractors is in the purchaser's best interests. Techniques and approaches are evolving in many aspects of service delivery, and it is helpful if every- one is aware of the state of the art. For example, in HIV prevention services, there has been a steady movement toward more participa- tory approaches in which members of the high-risk groups actually manage and implement services themselves. Showing NGOs how this works will make them more effective, which will be a benefit to the purchaser as they have an interest in seeing that public funds are used efficiently. Purchasers may also want to help contractors with their M&E systems, financial management and reporting, and the facilita- tion of the sharing of experiences among NSPs. 64 · Performance-Based Contracting for Health Services in Developing Countries Task 39: Clarify Responsibilities for Physical Infrastructure The contract needs to stipulate what happens to the physical infra- structure during and after the contract period. It is recommended that equipment become the property of the purchaser after the con- tract, but that during the life of contract it should be the contractor's responsibility to maintain. The ownership of buildings should also be clearly stipulated in the contract. During the life of the contract, maintenance, repair, and rehabilitation of buildings should be the re- sponsibility of the contractor. Task 40: Formulate the Bidding Documents The request for proposal (RFP, or its equivalent) is the document that will be given to interested bidders (NSPs) and will guide the bidding process. Typically an RFP contains the following components: · A letter of invitation to bid 3 · Instructions to the bidders on how to prepare and submit their bids, IONT which will also describe the process and criteria by which contrac- tors will be selected SEC · The form of the technical proposal · The form of the financial proposal · The terms of reference · The draft contract. Examples of World Bank RFPs are available online at http://www. worldbank.org/hnp/contracting. Step 7: Carry Out the Bidding Process and Manage the Contract 1. Conduct dialogue with stakeholders 7. Carry out the bidding 2. Define the services process and manage the contract 3. Design the monitoring and evaluation 6. Draft the contract and bidding documents 5. Arrange for contract 4. Decide how to management and develop select contractors a contracting plan How to Contract · 65 Once the previous steps have been completed, the bidding process needs to be carried out in compliance with the procedures laid out in the RFP document (or its equivalent). Once the contract with the winning bidder is signed, M&E activities and contract management need to be implemented in keeping with the contracting plan and the signed contract. Task 41: Track the Schedule of the Bidding Process It is worthwhile to keep careful track of the bidding process described in the bidding documents and the contracting plan. In almost all situ- ations, it should be possible to complete a competitive bidding process in six or seven months. This schedule means that recruitment of con- tractors should begin as soon as possible, even before financing is fully secured. One way of diagnosing corruption is to keep careful track of how long the evaluation process takes and how long it takes to finalize the contract. In negotiations in which processes are corrupt, it takes 3 time for the parties to go back and forth arranging their deals. A use- ful rule of thumb is that the likelihood of corruption increases if it IONT takes more than three weeks from the time of the final bid evaluation to the time the contract is ready for signing. SEC Task 42: Conduct Regular Monitoring Visits Successful contract management entails regular monitoring visits to the sites where the health services are being provided. As with other types of supervision, monitoring visits should be systematic and use a check- list that examines key aspects of performance, such as the following: · Assessment of results from the routine recording system · Availability of key inputs (such as medicines, equipment, and vehicles) · Availability and morale of health workers · Satisfaction of key stakeholders · Quality of care or related processes. In addition to assessing performance, monitoring visits are also good times to cement relationships among all stakeholders, identify issues, and solve problems early on. If monitoring visits are carried out systematically, doing them frequently will have a real effect. 66 · Performance-Based Contracting for Health Services in Developing Countries Task 43: Meet with Stakeholders Frequently Contracting involves complex relationships that need to be estab- lished and then nurtured. To do this takes frequent discussions that allow issues to be identified and solved or prevented. Experience in the field shows that three actions can be particularly worthwhile: (1) hold regular and frequent (every one or two months) meetings between the contract management unit and contractors, (2) establish a mechanism for contractors to share experiences and ideas among themselves with- out the purchaser necessarily being present, and (3) regularly report to major stakeholders, particularly the purchaser and local governments, on the progress of the contracts. Task 44: Review the Contracting Plan and the Contract The unit responsible for contract management for the purchaser and the contractors should use the contract as the basis for ensuring 3 smooth implementation of the services. The purchaser should review the contract and the contracting plan to ensure that the contractor is IONT performing well on its contractual obligations, as well as systemati- SEC cally implementing the contract management and M&E functions. SECTION 4 Checklist for Contracting Task numbers in this checklist correspond to paragraph numbers in Section 3, "How to Contract." Highlighted tasks should receive par- ticular attention. Task Comments Step 1: Conduct Dialogue with Stakeholders 1. Establish a consultative process with stakeholders: · Hold a few discussions with each set of stakeholders · Get back to stakeholders with draft proposals and contracts 2. Identify champions 3. Address the legitimate concerns of stakeholders: · Government health officials · Local politicians and government officials · Government health workers · Private sector, including NGOs · The community · Development partners and donors Step 2: Define the Services (Develop TORs) 4. Define the objectives of the contract: · Limit the number of indicators · Design bias toward outcome/output indicators · Ensure that indicators are independently measurable · Define indicators as precisely as possible · Set targets broadly 4 5. Include objectives related to equity and quality 6. Consider pay for performance IONT 7. Ensure that purchasers and contractors focus on objectives SEC 8. Define the size and location of each contract "lot" 9. Define the scope of services--focus on "what" not "how" Step 3: Design the Monitoring and Evaluation 10. Decide how data will be collected: · Health management information system · Household surveys · Health facility assessments · Supervisory checklists 11. Collect baseline data 12. Devise a clear schedule for data collection 13. Look for comparison/control groups 14. Assign responsibility for collection, analysis, and dissemination of data 15. Budget sufficient funds for monitoring and evaluation 67 68 · Performance-Based Contracting for Health Services in Developing Countries Task Comments Step 4: Decide How to Select Contractors and Establish the Price 16. Use a competitive selection process 17. Develop clear selection criteria 18. Establish a transparent and independent evaluation process 19. Maximize interest of possible contractors 20. Select a contractor Step 5: Arrange for Contract Management 21. Define responsibility and clarify the contract management structure 22. Ensure that proper staffing of the contract management unit 23. Allow sufficient budget for contract management 24. Consider computerization of contract tracking 25. Develop a written contracting plan (or contracting manual) Step 6: Draft the Contract and Bidding Documents 26. Maximize managerial autonomy: · Clarify authority of both parties · Focus on "what" not "how" · Discover other ways of increasing autonomy 27. Ensure that contractors can manage personnel effectively 28. Use lump-sum contracts 29. Leave procurement of supplies, equipment, and services to contractors 30. Ensure that duration of the contract is sufficiently long 31. Have clear procedures for making payments: 4 · Mobilization payments · Regular payments IONT 32. Establish a clear process for termination and imposing other sanctions SEC 33. Establish dispute resolution mechanisms 34. Define reporting requirements of the contractor 35. Have an explicit policy on user charges 36. Ensure that contractors use independent, private sector auditors 37. Ensure that contractors build the capacity of health workers 38. Address the capacity needs of contractors 39. Clarify responsibilities for physical infrastructure 40. Formulate the bidding documents Step 7: Carry out the Bidding Process and Manage the Contract 41. Track the schedule of the bidding process 42. Conduct regular monitoring visits 43. Meet with stakeholders frequently 44. Review the contracting plan and the contract SECTION 5 Whether to Contract This section reviews the global experience with contracting with non- state providers, including nongovernmental organizations (NGOs), for improving health care delivery. Short descriptions (one to three pages) of specific examples are also provided in appendix C. The 14 evaluated examples indicate that contracting for the delivery of primary care appears to be effective and improvements can be rapid. These results were achieved in a variety of settings and services. There have been one randomized trial and five other controlled, before-and-after stud- ies of contracting in developing countries, and many of the anticipated difficulties with contracting were either not observed in practice or did not compromise its effectiveness. Twelve of the 14 cases have been sustained and expanded, often to a very large scale. Many contracts have been in place for more than seven years. Provision of a pack- age of basic services by contractors costs between approximately $3 and $6 per capita per year in low-income countries. The current evi- dence suggests that contracting for health service delivery should be expanded, and future efforts should include rigorous evaluations. 5 The Evidence on Contracting IONT SEC Substantial improvement in the delivery of health services will be nec- essary to achieve the health-related Millennium Development Goals (MDGs). For example, it has been estimated that 63 percent of child- hood deaths in developing countries could be prevented through the full implementation of a few effective and low-cost interventions (Bryce and 69 70 · Performance-Based Contracting for Health Services in Developing Countries others 2003). Hence, discovering better ways of delivering these services is critically important. Although many countries undoubtedly need to allocate more resources to health services, previous experience suggests that more money alone may not solve the problem of service delivery. There appears to be no consistent relationship between the amount spent on health and health outcomes (World Bank 2004). Another re- sponse to the challenge of improving service delivery has been to use public funds to contract with nonstate providers (NSPs). This section reviews the global experience with contracting for primary health care and related services, taking into account the methodological rigor of the evaluations, and summarizing the evidence from 14 evaluated examples. Appendix C provides more detailed descriptions of each example. Methodology This review focuses on evaluated instances in developing countries of contracts between governments, international organizations, or their agents with identifiable NSPs for delivery of primary health care ser- vices, including nutrition (but excluding referral hospital care or an- cillary services, such as food provision in hospitals). Inclusion Criteria. To be included in the review, the evaluation had to measure quality of care, outputs such as increase in volume of services provided, or effect on health status. The evaluations also had, at a minimum, to involve before-and-after or controlled designs. Hence, evaluations were excluded that described the contracting pro- cess but that did not measure some tangible outputs. Also excluded were studies that just provided "after" evaluations with no "before" 5 data or without contemporaneous controls. Although instances of contracts with for-profit entities were not excluded, only two of the IONT contracting initiatives reviewed (TB control with private providers, case 11, and improving management of childhood diseases among for- SEC profit providers, case 14) involved for-profit entities. Search and Review Methodology. To find as many examples of contracting as possible, experts from a variety of institutions were asked for examples of contracting of which they were aware. Previ- ous reviews of contracting in developing countries were also examined (Mills 1998; Abramson 1999; Palmer 2000; Slack and Savedoff 2001; Loevinsohn and Harding 2005; Liu, Hotchkiss, and Bose 2008). A Whether to Contract· 71 computerized search of the published literature was also carried out using ECO (Electronic Collections Online, a broad database cover- ing scholarly journals in a wide variety of fields); Periodical Abstracts (covering general and academic journals in business and economics, including transcripts of television and radio news programs); Econ- Lit (covering journals, books, working papers, and dissertations in economics); WorldCat (covering books and other resources in a large number of libraries); PAIS (Public Affairs Information Service, pro- viding selective subjects and bibliographic access to periodicals, books, hearings, reports, gray literature, government publications, Internet resources, and other publications from 120 countries); and PubMed (the U.S. National Library of Medicine's search service for access to Medline and other related databases). The electronic search was supplemented by a manual review of jour- nals that often publish articles related to health systems in develop- ing countries (Health Policy and Planning, Bulletin of the World Health Organization, and Social Science and Medicine). Personal contact was also made with people known to be involved in contracting efforts. Writ- ten reports or presentations of the evaluated examples were reviewed and summarized, and considerable attention was given to the evalua- tion methodology used. Then an attempt was made to conduct struc- tured interviews with people who had intimate knowledge of particu- lar examples. The summaries were modified accordingly, and the same people were asked to review the summaries before they were finalized. Analysis. In the instances where before-and-after data were available from experimental and control groups, the "double differences" were calculated. The double difference is the difference between follow-up and baseline results in the experimental group minus the difference between follow-up and baseline results in the control group. Wherever possible, 5 differences are expressed as percentage points (see note on page 95). IONT SEC Results Contracting Can Quickly Improve Service Delivery. According the 14 studies summarized in table 5.1 (pages 74­80), contracting with NSPs appears to deliver effective primary health or nutrition services, and im- pressive improvements can be achieved rapidly. Good results have been achieved in a variety of settings and for a variety of different services. 72 · Performance-Based Contracting for Health Services in Developing Countries All the studies found that contracting yielded positive results, and the most rigorously evaluated cases demonstrated the largest effect. In the six studies in which it was possible to calculate the double difference, the median double differences ranged from 8.3 to 26 percentage points (fig- ure 5.1).1 All the median double differences were positive. Larger dou- ble differences were observed for parameters that are easier to change, such as immunization, vitamin A, and antenatal care coverage. Smaller changes were observed in parameters that require important behavioral changes, such as family planning. Some of the other studies, while not using controlled, before-and-after designs, provide some compelling additional evidence of the effectiveness of contracting. Contractor versus Government Performance. Of the 14 studies, 10 compared contractor performance with government provision of the same services. All 10 found that the contractors were more effec- tive than government, on the basis of a variety of parameters related to both quality of care and coverage of services. In the studies reviewed here, the differences between contractor and government performance also tended to be large (see box 5.1). Discussion Methodological Limitations of This Review. This review was based partly on papers in the gray literature, some of which had not Figure 5.1 Median Double Differences in Coverage Rates from Studies with Controlled, Before-and-After Methodology 30 25 5 20 difference points) IONT 15 double 10 SEC (percentage median 5 0 Cambodia Cambodia Bangladesh Bolivia Afghanistan Bangladesh Rwanda SDC MC Nutrition Urban PHC Services Source: Author. Note: MC = management contract; PHC = primary health care; SDC = service delivery contract. Whether to Contract · 73 undergone peer review (8 of the 14 examples are described in articles from scholarly journals). The review included 14 contracting exam- ples, which is a reasonable, but not huge, number. It is likely that other unidentified examples of contracting could be found. Other experi- ences may not have been written up because the results were poor, which would lead this review to more positive conclusions than are warranted. This type of positive results bias is usually more profound when only published articles are used. Applicability of Findings. All the cases summarized in table 5.1 focused on primary care and nutrition services, in which outputs are relatively easy to measure. Other health services, such as spe- cialist inpatient care, present much larger measurement challenges related to quality of care. Also, the providers in these cases were usually nonprofit organizations. Contracting with for-profit enti- ties, especially self-employed physicians, is common in industrial and middle-income health systems, but there is limited experience in low-income countries (there are only two examples among the cases reviewed). Box 5.1 Contractor versus Government Performance in Industrial and Middle-Income Countries There are few examples of contracts initiated for primary health care services in industrial countries. Most countries that contract have always done so. In countries where both contracted and public (salaried) physicians deliver primary health care services, the distinct reimbursement approaches make it virtually impossible to assess the effect of contracting alone (as distinct from that of the payment basis). Hence, opportunities to compare performance are rare. Never- theless, a few scholars have attempted to assess the difference. 5 Several European countries have initiated contracting for packages of pri- mary health care services. Where contracted services have been compared with IONT those still provided by salaried physicians, results have generally been favorable. SEC In Croatia, evidence of higher productivity was found in contracted practices, including indicators of patient accessibility (Hebrang and others 2003). In Estonia, where salaried physicians converted to contracted status, a "before-and-after" analysis found improvements in allocative efficiency indica- tors, technical efficiency indicators (for example, annual number of visits per doc- tor, number of visits per inhabitant), and immunization rates (from 74 percent to 88 percent) (Koppel and others 2000). SECTION 5 74 Table 5.1 Summary of Evaluated Contracting Experiences Location and Contract and Contracting Evaluation Subsequent service type intervention type Scale of effort arrangement methodology Main results history Comments 1. Cambodia SDC and MC 1.5 million Competitively Randomized SDC and MC much Expanded Impressive results in compared with population bid, formal controlled better than CC. to twice contracted districts Rural PHC and government contract, man- study with 12 Median double as many achieved quickly. district hospital provision of aged by special districts as differencea on districts SDC performed bet- services services (CC). unit of MOH. experimental seven indicators ter than MC initially, Some problems units. House- for SDC vs. CC was but both outper- ensuring good hold and health 10.4%;bfor MC vs. formed government relations with facility surveys CC double differ- provision. Rigor- provincial conducted ence was 8.7%. ous methodology. officials. NGOs before and Poor better served SDC and MC more were paid on after 4 years of in contracted dis- expensive than CC, time. implementation. tricts, and quality but reduced out-of- of care was 19.3% pocket expenditure better. by community. 2. Bangladesh SDC with NGOs 15 million Fixed-price Controlled Malnutrition rates Expanded NGOs were able to compared with population MOU. Initially before-and- declined 18% in to more implement nutri- Rural commu- control areas sole-source after study SDC upazilas com- than 30 tion interventions nity nutrition with no orga- selection of with 6 experi- pared with 13% in million successfully on a services nized nutrition NGOs, then mental and 2 controls (double population very large scale. services (that competi- control upazilas difference = 5%). Modest difference in is, normal gov- tive. Serious (subdistricts). Double difference change in nutrition ernment health problems with Household sur- for vitamin A was status, but larger in services with payment and veys conducted 27%. coverage of related no nutritional other aspects by third party. services. No indica- component). of contract tion of how govern- management. ment would have done implementing nutrition services. Table 5.1 Summary of Evaluated Contracting Experiences (continued) Location and Contract and Contracting Evaluation Subsequent service type intervention type Scale of effort arrangement methodology Main results history Comments 3. Bangladesh 15 SDCs with 4 million Competitively Controlled Median double Contract- NGOs performed NGOs. NGO population bid formal before-and- difference favoring ing with better in cover- Urban PHC working in contracts, after study the NGO of 8.3% NGOs age of services Chittagong managed by with an NGO on six coverage expanded and quality of care compared with special unit of contract in indicators. 10.1% to many in spite of having government local govern- Chittagong double differ- more the same amount provision of ment division. compared ence among the cities, of resources. CCC services by Difficulties with two areas poor. NGO did 11% more than unable to provide CCC. encountered implemented by better on a broad doubling broad package of paying contrac- CCC. Household quality-of-care coverage services in newly tors on time and health index. built facilities. NGOs and adequate facility survey better able to reach monitoring. by third party. the poor. 4. Bolivia Limited MC in 250,000 Single-source Controlled, Double difference Unknown Relatively small- phase 2. MC population contract with before-and- for deliveries scale study showed Urban PHC with expanded NGO. Contract after design, between MC and large changes in MC authority management but data from control was 21%, district. Method- in phase 3. by commit- routine report- and 1% for bed ological issues. Control area tee, including ing system. occupancy. Contract giving had continued community Only a few greater autonomy public sector representatives. indicators to NGO resulted in management. examined. larger changes. (continued on next page) 75 SECTION 5 SECTION 5 76 Table 5.1 Summary of Evaluated Contracting Experiences (continued) Location and Contract and Contracting Evaluation Subsequent service type intervention type Scale of effort arrangement methodology Main results history Comments 5. Afghanistan SDC and MC 6 million Competitively Controlled, Double difference Contracts Large-scale perfor- compared with population bid contracts before-and- for SDC and MC expanded mance-based con- Rural PHC grants and gov- for the SDC with NGOs for after study. compared with and now tracts (using perfor- ernment provi- and MC SDCs. Competi- Third-party government itself cover mance bonuses) led sion without tively recruited evaluation of was 24.3%. Com- more than to a rapid increase management individuals for quality of care pared with grants 20 million in quality of care. support. MC. Contracts using broad to NGOs, double people Awaiting results managed suc- index based on difference was from coverage cessfully by satisfaction, 5.3%. surveys. government. knowledge, NGOs paid on and quality time. of patient- provider interaction. 6. Rwanda Provision of 388,000 and NGO signed In one province Median double dif- Expanded NGO as an interme- performance- 620,000 (two contract with a controlled, ference of 12.9%, to most of diary in implement- Performance- based pay to provinces) managers before-and- including 17% the rest ing pay-for-per- based pay for centers (and of individual after study double difference of the formance project. PHC in rural indirectly to health facili- with a com- in TT2+ and 9% country Program appears to areas health workers) ties. Monitoring parison group double difference have been success- using NGO as done by district and based on for DPT3. Large ful but little data on intermediary. health team. reported data. increase in institu- quality of care. Con- Other province tional delivery but trolled part of the before-and- no effect on family case uses reported after study planning. data of unknown based on quality. household surveys. Table 5.1 Summary of Evaluated Contracting Experiences (continued) Location and Contract and Contracting Evaluation Subsequent service type intervention type Scale of effort arrangement methodology Main results history Comments 7. Costa Rica Worker-owned 150,000­ Cooperatives Time series 22% more general Expanded Worker coop- cooperatives 300,000 given con- over 10 years visits and 42% to include eratives taking over PHC provided given SDC to beneficiaries tracts granting comparing more dental visits other publicly managed by worker manage clinics considerable cooperatives in cooperatives but nonstate clinics results in cooperatives previously autonomy on to publicly at lower cost. providers, increase in services managed by use of capita- managed facili- continuing at lower cost. No government. tion payment. ties based on for more data on patient reported data than 15 satisfaction or years technical quality of care. 8. Global Fund Global Fund 448 grants Grants (rather Grading of Taking into GFATM Very large-scale (GFATM) grants to programs than contracts) project after account other continues evaluation of NSP Support for to NSPs and in 136 coun- in which NSPs 2 years of factors, NSPs to fund vs. government AIDS, TB, and government tries worth and govern- implementa- 16.7% more likely NSPs on a performance in malaria entities to $6.8 billion; ments designed tion based to receive high- very large implementing GFATM fight the three 134 grants the interven- on progress, est rating and scale grants. Endpoints diseases. included tions. CCM and disbursement, 16.8% less likely were more outputs in the LFA monitored and country to receive lowest and process rather evaluation. performance. context. ratings. than outcomes. (continued on next page) 77 SECTION 5 SECTION 5 78 Table 5.1 Summary of Evaluated Contracting Experiences (continued) Location and Contract and Contracting Evaluation Subsequent service type intervention type Scale of effort arrangement methodology Main results history Comments 9. Guatemala MC in selected 3.4 million Competitively Controlled Median difference Started MC appeared to municipalities population bid contracts design based between MC and as small make modest dif- Rural PHC in and SDC in with NGOs. on household control on five pilot but ference in service mountainous more remote Difficulties survey con- indicators was 11% expanded delivery. Difficult to areas areas, com- with financial ducted by third (range 5­16%). rapidly, assess SDC because pared with management party 3 years now cov- of remoteness. government and supervision after contract- ers 27% of No baseline data provision of contracts. ing began. country available. (control). 10. Haiti NGOs with 1.3 million Input type Before-and- Average of fol- Initially Large change in SDCs offered population of contract after (over 6 low-up minus covered important indicators Bonuses for performance changed to years) design baseline ranged 500,000 with the introduc- NGOs deliver- bonuses based one focused based on base- from +19% (skilled people, tion of performance ing PHC in rural on agreed-on on outputs line household birth attendance) now bonuses and areas targets. with greater survey carried to +64% (vaccina- expanded increased autonomy. managerial out by third tion coverage). to cover "Dose-response" autonomy. party. Follow- Results improved 1.3 million curve makes for up data from as percentage people, compelling evi- HMIS validated of NGOs using 19% of dence, in spite of no by third party. performance- Haitian control group. based contracts population increased. Table 5.1 Summary of Evaluated Contracting Experiences (continued) Location and Contract and Contracting Evaluation Subsequent service type intervention type Scale of effort arrangement methodology Main results history Comments 11. Various 14 examples 11­17 million In 7 of 14 Combination Private providers Initiatives Working with countries of "soft total popula- cases, there of before-and- achieved 89.6% 3­12 years private practitioners contracts" tion (17,000 were NGO after and con- treatment suc- old, and was successful in TB control with private sputum-pos- intermediaries trolled designs cess rates and now 40 improving treatment services providers who itive cases) with formal with data from increased case such initia- success and case through private received drugs contracts with recording detection rates tives in 15 detection. Having providers in return for fol- TB programs. system verified by 16.5%. Having countries NGO intermedi- lowing national Some NGOs by TB experts, NGO intermediary ary achieved even guidelines. provided their including some led to 8% better better results, often Sometimes own resources. from third treatment success at lower cost than compared with party. rate. Cost per suc- public sector. publicly man- cessful treatment aged services. lower than public services. 12. Pakistan MC for the 3.3 million Sole-source Controlled com- 50% higher use of Expanded Large increase in 104 BHUs in population MOU with parison with BHU services, less to 12 outpatient visits Rural PHC one district NGO by local another similar use of unquali- districts in achieved with same compared government. district using fied providers, the Punjab budget. Vague with neighbor- Vague objec- household and and increased and now contracts interfered ing district as tives. Received health facility satisfaction in MC being with improving control. monthly surveys and district. No change replicated coverage of preven- tranche of before-and- in coverage of pre- in other tive and promotive funds regularly. after data from ventive services. provinces services. routine record- ing system. 79 (continued on next page) SECTION 5 SECTION 5 80 Table 5.1 Summary of Evaluated Contracting Experiences (continued) Location and Contract and Contracting Evaluation Subsequent service type intervention type Scale of effort arrangement methodology Main results history Comments 13. Madagascar Madagascar: 460,000 in In Madagas- Before-and- Severe and moder- Continued Modest effect on and Senegal SDCs with 50 Madagascar; car, contract after (17 ate malnutrition with NGOs malnutrition, but NGOs. 490,000 in management months) house- declined 6% and in both effect size uncertain Community Senegal: SDCs Senegal was done by hold survey of 4%, respectively. countries, because of absence nutrition with NGOs that unit in office of nutrition status Participation was but in of control group. services worked through the president; in Senegal. 72% in project and different NGO successfully small groups in Senegal, Third-party 35% in control format used unemployed of unemployed by parastatal. survey of areas. youths to deliver youth. No serious participation services. problems in project and encountered. control areas. 14. India SDC for NGO 54,000 NGOs applied Before-and- Rapid improvement Unknown Small-scale, working with population for grant from after (6 months in provider skills uncontrolled study, Improved private provid- U.S. Agency for later) design ranging from 25% but large and quality of care ers to improve International based on to 57% compared rapid improvement by private MCH services. Development household sur- with baseline. achieved in quality practitioners and then veys by com- of care. informally munity health contracted workers. with private providers. Source: Author, based on interviews and cited references. Note: BHU = basic health unit; CC = control/comparison; CCC = Chittagong City Corporation; CCM =country coordination mechanism; DPT3 = third dose of diphtheria/ pertussis/tetanus vaccine; GFATM = Global Fund to Fight AIDS, Tuberculosis, and Malaria; HMIS = health management information system; LFA = local fund agent; MC = management contract; MCH = maternal and child health; MOH = Ministry of Health; MOU = memorandum of understanding; NGO = nongovernmental organization; NSP = nonstate provider; PHC = primary health care; SDC = service delivery contract; TB = tuberculosis; TT2+ = two or more dosages of tetanus toxoid. a. Double difference = difference between follow-up and baseline results in the experimental group minus the difference between follow-up and baseline results in the control group. Whether to Contract· 81 Possible Study Biases. Of the 14 examples of contracting that met the inclusion criteria (that is, that had at least before-and-after data or data from contemporaneous controls), 6 had before-and-after con- trolled designs, 2 involved long time series (that amounted to more than just before-and-after designs), 2 had controlled designs with a single measure in time, 2 were before-and-after evaluations, and 1 was a mixed design because it reviewed a large number of different studies. There was one randomized trial. A pilot-test bias may have been present in the examples considered because contracting is novel. These efforts may have received extra attention from managers, donors, and the NSPs, thereby limiting the external validity of the studies. It is difficult to know how serious this problem is. There is some comfort in the fact that many of the contracting examples were done on a very large scale and provided services to many millions of beneficiaries. In addition, the history of contracting for social service delivery in the United States and Australia suggests that the initial experiences were problematic and that results improved as govern- ments and contractors ironed out the difficulties they encountered (Domberger 1999; Savas 2000). In light of the methodological concerns about the cases studied, there is still a need for future contracting efforts to include rigorous evaluations. However, the current weight of evidence indicates that contracting with NSPs will provide better results than government delivery of the same services. Alternatives to Contracting: Some Success Stories of Govern- ment Provision. The apparent success of contracting and its superior performance when compared with government delivery of services raises an obvious question: Why are there successful examples of pub- lic provision of health services such as in Costa Rica, Cuba, Kerala 5 State in India, Sri Lanka, and Vietnam? These examples indicate that it is certainly possible for governments to do a reasonable job of deliv- IONT ering health services. The cases included in the review suggest that SEC NSPs will perform better, not that governments will always perform badly. In these five jurisdictions, there have been few counterfactuals. Except for the example of Costa Rica, contracting with NSPs in these countries has rarely been compared with the existing public system. In addition, these particular cases have been singled out (since at least the early 1980s), and their successes have not been widely replicated. 82 · Performance-Based Contracting for Health Services in Developing Countries Why Does Contracting Work? Contracting for health service delivery has a number of attractive fea- tures that may help explain its success (Loevinsohn 2000). Focus on Results. The very act of drafting a contract can help the purchaser define exactly what services are needed and help make objectives explicit. Particularly when the contract defines objectively verifiable outputs and outcomes, the contracting process can ensure that both the purchaser and contractor focus on the achievement of measurable results. The presence of performance bonuses and the credible threat of sanctions or termination of the contract probably help motivate contractors to achieve better results. Flexibility. NSPs have the important advantage of being less con- strained by "red tape" (excessive regulation), bureaucratic inertia, and unhelpful political interference. In many circumstances, this is the largest advantage of NSPs over government delivery of the same ser- vices. Because of its greater flexibility, contracting allows for more innovation and creative responses to problems. It may also help avoid patronage and unhelpful political interference, such as hiring not solely based on merit. Reduction of Important Aspects of Corruption. Contracting appears to reduce some aspects of corruption that plague public health care systems, such as absenteeism, theft of drugs, selling of positions, leakage of funds on their way to peripheral health facilities, and infor- mal payments to providers. The most serious corruption issues sur- rounding contracting involve the selection process and the payment of invoices. The issue of corruption is dealt with in more detail below. Constructive Competition. Contracting uses constructive compe- 5 tition to increase effectiveness and efficiency. Nonstate providers are impelled through competition to develop the most effective and efficient IONT ways of delivering services, both during the bidding process and during implementation. Friendly competition (when NSPs or the government SEC work in different areas) over which organization can best deliver ser- vices is a big gain for poor people living in isolated areas, or people with AIDS who are not yet receiving services. In a number of settings, com- petition from NSPs under contract has pushed public sector providers to improve their own performance and deal with issues within govern- ment, such as limited managerial autonomy or "red tape." Whether to Contract· 83 Improved Absorptive Capacity. Nonstate providers are usually better at overcoming "absorptive capacity" constraints that often plague government health care systems and prevent them from effec- tively using the resources made available. This results in part from the greater flexibility of NSPs and the fact that they face fewer bureau- cratic regulations that constrain the efficient use of funds. Better Distribution of Health Workers. As a result of greater flexibility and innovative approaches, NSPs can often improve the dis- tribution of health workers and help ensure that skilled health workers are available and working in underserved areas. This has been par- ticularly noticeable in Afghanistan and Cambodia, where the distribu- tion of health workers, including doctors, improved very quickly as a result of contracting. Managerial Autonomy. Contracts, if drafted properly, provide managerial autonomy and decentralize decision making to manag- ers closest to the ground. Instead of people in distant capitals making decisions on issues they can't know much about, the people closest to the real-life situation are given authority to solve problems. Government Focus on Stewardship Role. Contracting pro- vides a greater opportunity for government to focus on roles that it is uniquely placed to carry out, such as planning, evaluation, stan- dard setting, financing, and regulation. When government is directly involved in service delivery, it often has little time or inclination to play a stewardship role. This is not simply a theoretical concern. In one South Asian Ministry of Health, a study found that 75 percent of all letters written by the ministry and 78 percent of all files had to do with postings and transfers of personnel. Contracting can free up time normally spent on personnel issues so that government officials can spend more time on stewardship and public health functions. 5 IONT What Are the Concerns about Contracting? SEC A number of potential issues have been raised in connection with con- tracting (England 1997; Mills 1998; Abramson 1999; Palmer 2000; Slack and Savedoff 2001; Frick-Cardelle 2003; Soderlund, Mendoza- Arana, and Goudge 2003; Liu, Hotchkiss, and Bose 2007). Some of the common concerns expressed by decision makers are summarized 84 · Performance-Based Contracting for Health Services in Developing Countries in table 5.2 (see pages 86­87) along with the global experience thus far. Most of the concerns are addressed in more detail in the follow- ing discussion. Scale of Contracted Services. The concern that contracting is unable to provide services on the large scale needed to make a differ- ence at the country level is no longer warranted. It used to be true that contracts (or, usually more often, grants to NGOs) were implemented on a small scale. That is no longer the case. Four ongoing examples of contracting (Afghanistan, Bangladesh, the Democratic Republic of Congo, and Pakistan) involve populations of tens of millions of ben- eficiaries. Contracts for nutrition services now cover a third of rural Bangladesh, more than 30 million people, and close to 20 million peo- ple in Afghanistan currently receive primary health care services from contractors. There are a number of other countries (many described in the review) where contracts covering millions of beneficiaries have been successfully implemented. Cost of Contracting. Contracting is not necessarily more expen- sive than government provision of the same services, as the examples from Afghanistan, urban Bangladesh, Costa Rica, Pakistan, Rwanda, Uganda, and the work on TB by private providers suggest. Nonstate providers performed better, even when public institutions were pro- vided with similar amounts of resources. Two examples of contract- ing exist in which it was more expensive than government delivery of the same services (Cambodia and the Democratic Republic of Congo), although baseline expenditures were very low in these cases. In every example where it was assessed, the contracting efforts saved people in the community money by reducing out-of-pocket health expendi- tures. Although the services provided under the different contracts 5 are not strictly comparable, a basic package of primary health services ranged from $0.65 per capita per year in urban Bangladesh to $6.25 IONT per capita per year in rural Guatemala. The low price in Bangladesh likely reflects the fact that the better-off urban residents received their SEC care from for-profit providers. Nonetheless, large examples from low- income countries demonstrate that basic primary health care services, including first-level hospital care, can be delivered for $3 to $6 per person per year. Equity. The concern that contracting will make health services less equitable appears to be unwarranted, although the data on this issue are sparse. The effect of contracting on the poor was measured Whether to Contract· 85 in three cases. In two of the cases (urban Bangladesh and Cambodia) contractors did better than the government in improving services for the poor. In the third case (Pakistan), there was no difference between the NGO and the government. In this last case, there was no mention in the contract about a focus on reaching the poor, whereas this was explicit in the contracts used in urban Bangladesh and in Cambodia. When contracts include explicit targets for reaching the poor, contrac- tors appear able to improve health services for them significantly. The idea that NSPs will not want to work in remote or difficult areas or do not want to focus on the poor is not consistent with global experience. Sustainability. Given its apparent success, the sustainability of contracting is a genuine concern and a difficult one to address simply. There are a few important findings that have a bearing on the sustain- ability of contracting: · In 12 of the cases that were reviewed above, the contracting efforts have been continued and usually expanded, often dramatically. Some of the contracts have been in place for 12 years, and 6 of the cases have been sustained for more than 7 years. In at least 10 cases the scope of contracting has more than doubled from its initial scope. · As described above, real-world experience, based on actual contract prices, in more than seven low-income countries indicates that con- tractors can successfully deliver a basic package of primary health care services for between $3 and $6 per capita per year. This sug- gests that contracting is not particularly expensive and can achieve good results at a level of investment most countries should be able to afford. · In most settings, NSPs on contract have achieved better results than the public sector even when provided the same or fewer re- 5 sources. This implies that contracting is more efficient and that in many cases sustainability is only a matter of maintaining health ex- IONT penditures at their current levels. SEC · In those cases in which contracting was more expensive than public provision of services, such as the service delivery contracts in Cam- bodia, out-of-pocket expenditures by the community, particularly the poor, were reduced by a larger amount than the contract cost. In these cases, sustainability is an issue of whether there is sufficient political will to improve services and efficiently subsidize the poor. SECTION 5 86 Table 5.2 Some Common Concerns about Contracting and the Global Experience Concern Experience so far "Contracting can be done only on Contracting was done on a small scale until the late 1990s; however, it is now being implemented on a very a small scale." large scale. In Afghanistan, Bangladesh, the Democratic Republic of Congo, and Pakistan, contractors provide services to tens of millions of beneficiaries. In other low-income countries, many millions of people receive their health services from contracted providers. "Contracting is more expensive This statement has been true in some settings such as Cambodia and the Democratic Republic of Congo, than government delivery of where previously expenditures on health were very low. In other places such as Bangladesh, Guatemala, services." and Pakistan, contractors achieved good results even if provided the same or fewer resources than those made available to the public sector. "Contracting will worsen inequity, The effect of contracting on equity has been measured in a few cases. Where it has been measured, equity partly because NGOs and other has either been better or no worse than in noncontracted areas. If given clear responsibilities and direc- NSPs will not want to work in tion, NGOs and other NSPs have successfully worked in remote and underserved areas. remote areas." "Even if successful, contracting Some contracting efforts have been going on for more than 10 years, and 85 percent of the examples evalu- will not be sustainable." ated so far have been sustained and even expanded. "Contracting will lead to increased Preventing corruption is a difficult issue and requires constant vigilance. Although empirical data are lim- opportunities for corruption." ited, contracting may avoid some aspects of corruption that plague publicly provided health services, such as absenteeism. Nonetheless, corruption has been an issue in some settings where contracting has been implemented, particularly during contractor selection and payment. There appear to be actions that can prevent or limit these types of corrupt practices. "Governments have limited Contract management has been an issue in some settings and deserves careful attention. In a number of capacity to manage contracts countries, including some with limited experience in this area, contract management has been done well, effectively." indicating that the problem may be solvable. Even where contract management has been done poorly, NSPs have still been able to deliver services effectively. "Contracting reflects a political Contracting is fully consistent with public financing of health services. By bringing patients back from the agenda aimed at weakening the private sector (particularly from nonqualified providers) into publicly financed facilities, it actually strength- role of the state and privatizing ens the role of the state. In most developed countries, publicly financed services are provided through some health care." form of contracting, and that has not weakened the involvement of the state in financing health services. "Contracting will increase the In countries with many existing NGOs, most contracts have been won by local NSPs. However, in countries presence of foreign NGOs." with few local NGOs, the first batch of contracts went to international NGOs, but subsequent contracts have often been awarded to local NGOs. In the long run, it appears that local NGOs will predominate, at least in part, because they have lower costs. "Contracting in post-conflict Where contracting has been implemented in postconflict settings, it does not appear to have weakened the situations will weaken the state state. This seemingly paradoxical finding arises from the government being better able to coordinate pri- at a time when it should be vate efforts and the fact that patients and communities just want better services with little concern about strengthened." who is actually delivering them. "Contracting will lead to duplica- Contracting has been successfully used to strengthen existing public services in a number of countries tion of effort where there are including Afghanistan, Cambodia, the Democratic Republic of Congo, Guatemala, Pakistan, and Rwanda. By existing public services." using management contracts, existing health services have been significantly improved without duplica- tion of services. "Contracting is a donor-driven It is true that many contracting initiatives in low-income countries have been financed by development initiative." partners. However, this has not always been the case. In Pakistan, a very large contracting effort was developed, implemented, and financed entirely through a local initiative. In southern Sudan, the govern- ment is using its own resources to finance contracted services. "Contracting will `co-opt' NGOs There are some situations in which NGOs may not wish to enter into a contractual relationship with a pur- and limit their advocacy role." chaser because of concerns about the legitimacy of the latter. Clearly, such judgments should be respected because a contract, by its nature, is voluntary. Discussions with NGOs actually involved in contracting indicate that they do not feel constrained in their advocacy role. "NSPs and governments are wary In many settings, NSPs, particularly NGOs, and the government do not have an easy relationship. Contract- of each other and so will not be ing has in some, but not all, situations improved the relationship by making explicit the roles and responsi- able to work together effectively." bilities of each party. "Contracting will lead to `fragmen- In almost all developing countries, the health care system is already fragmented with a large part of expen- tation' of the health care system." ditures occurring in the private sector. By increasing the use of publicly funded services, contracting may actually reduce fragmentation and strengthen the stewardship role of government. "Contracting does not improve the As with equity, quality of care has been measured in only a few cases, a situation that should be improved. quality of health services." In those cases in which it has been measured, NSPs have done better, often considerably better, than governments in improving quality of care. 87 Source: Author. SECTION 5 88 · Performance-Based Contracting for Health Services in Developing Countries · Many of the NSPs that have been given contracts are local or have a long-term presence in the country where they operate. For example, some of the faith-based organizations (FBOs) in Africa and South Asia have been present in the countries for many decades (some- times longer than the countries themselves have been in existence). · There are examples of low-income countries, including Pakistan and southern Sudan, that have used their own resources to contract for health services. · Contracting, especially for primary health care, is an approach that is the norm in high-income countries and is fully consistent with public financing or social health insurance. The use of contracting for health services in developed countries has been going on for decades and has been sustained. · In other sectors, governments in developing countries have success- fully changed over to a contracting model with improved efficiency. For example, many developing countries had their civil works built by "public works departments" up until the 1960s and 1970s. These public works departments now do very little construction themselves and instead have become purchasers of construction services. Long-Term Prospects. Some argue that contracting does not need to be sustainable because it is just a temporary approach until the government can take over the health services. Others argue that, so long as the costs are comparable, there is no reason why contracting should not become the norm, just as it is in developed countries. Peo- ple should take an empirical approach based on whether contracting does a better job in their setting of effectively, efficiently, and equi- 5 tably delivering health services. If it does, then there is no obvious reason why it should be only temporary. IONT Corruption. Currently only limited empirical data are available on the extent to which corruption affects contracting, although it has SEC clearly been an issue in some settings. As more contracting is carried out, the issue of corruption is likely to become more prominent. In publicly provided services, there is increasing evidence on the extent of corruption, in part reflecting the fact that public provision is the dominant mode in developed countries and has been for a long time. Corruption turns out to be a very serious and widespread problem in Whether to Contract· 89 public sector provision of health services (Lewis 2006; Transparency International 2006) and includes issues such as the following: · Absenteeism, which has been found to be a widespread problem, particularly in rural areas (among doctors working in rural areas in Bangladesh and Uganda, 74 percent and 68 percent were absent, respectively) · Purchase of public positions (25 percent of public hospital positions were bought in Ghana) · Drug leakage and poor procurement (32 percent of respondents in Costa Rica knew of drug theft from public facilities, and 30 percent of drugs in China were counterfeit or expired) · Corruption in the supply chain (11 percent of costs could have been saved in Colombia if public tendering rules had been followed, and in four other Latin American countries there is very wide variation in prices paid for standard commodities, suggesting irregularities in procurement) · Leakage of public funds, where public expenditure tracking surveys have found between 40 percent to 81 percent leakage in four coun- tries (Ghana, Peru, Tanzania, and Uganda) · Informal payments to providers, which are very common (55 per- cent of patients say they had to make informal payments in Cam- bodia; the figure is 82 percent in Sri Lanka) and often quite high (informal payments for hospital visits are greater than 25 percent of monthly per capita income in 8 of the 12 countries where the issue has been studied). 5 The experience with contracting thus far suggests that it can pre- vent or limit some of these forms of corruption, including absentee- IONT ism. Health facility assessments in Afghanistan, Bangladesh, and SEC Cambodia, among other countries, found almost no absenteeism in contractor facilities. Purchase of positions is simply not an issue in contracting. Drug supply management is almost always better under contracts as judged by the availability of medicines in clinics, and it is likely that more resources actually make it to the health facility, which explains in part the better absorptive capacity of contractors. 90 · Performance-Based Contracting for Health Services in Developing Countries Exit surveys carried out as part of health facility assessments indicate much lower rates of informal payments in clinics managed by con- tractors compared with publicly managed facilities in the same coun- try. In one example, it was only through the presence of an NGO contractor that the central government and the donor discovered the extent of local government corruption. In contracting for health services, the major issues of corruption arise during the selection of contractors and the processing of pay- ments. The best way of preventing corruption is constant vigilance, but there also appear to be some specific, practical ways of dealing with these issues, including (1) using competitive selection procedures, (2) having large contract "lots" that attract more competition and larger NSPs, (3) making the bid evaluation process more transparent by having clear selection criteria and involving multiple stakeholders in the evaluation process, and (4) having clear and explicit conditions for payment of invoices. These and other means for reducing corrup- tion are described in more detail in section 3. Contract Management. Contract management is often diffi- cult for governments, partly because they (particularly Ministries of Health) often have limited experience with service delivery contracts. Experience thus far shows that contract management is an impor- tant issue that requires careful attention. Contract management has been done well in some settings, such as Afghanistan and Senegal. (For example, the Ministry of Public Health in Afghanistan had no experience with contract management but has still been able to man- age 14 large contracts successfully as judged by careful monitoring of performance and timely payment of contractors.) Generally, countries with successful contract management appear to have benefited from 5 recruitment of local consultants, provision of sufficient resources, and having a manageable number of larger contracts rather than hundreds IONT of smaller ones. While ensuring competent contract management is important, contracting has been successful even in those situations SEC where it has not been done well. In rural and urban Bangladesh and Guatemala, where observers believed contract management was prob- lematic, contractors still successfully implemented large programs and achieved good results. Role of Government. Some people argue that contracting with NSPs arises out of an ideological desire to "privatize" publicly financed Whether to Contract · 91 health services and ultimately to limit or end government involvement in health care (Turshen 1999; Pfeiffer 2003). Discussion with people actually involved in contracting in developing countries suggests that they are motivated by more practical concerns. Most contracting ini- tiatives arise either from an absence of government services or from frustration with the poor quality and coverage of health services, espe- cially for poor people. Most advocates of contracting express a desire for increased government financing so that services in the community can be expanded and improved. They also want to see governments engage with private providers, which already deliver the bulk of cura- tive services in developing countries (in South Asia, for example, more than 70 percent of outpatient care is provided by the private sector), to improve quality of care, access, and coordination. Much of the disagreement over the role of contracting comes from a failure to recognize the difference between provision of services and financing of services. Table 5.3 outlines the different scenarios of financing and provision and emphasizes that contracting does not entail privatizing the financing of services. In fact, contracting is fully consistent with public financing of health care from tax revenues or social insurance premiums. Most developed countries in the Organisa- tion for Economic Co-operation and Development (OECD) use some form of contracting and still have public financing of health services. (Only 4 of the 28 members of the OECD provide primary health care through salaried civil servants.) For example, in Norway in 2001, 66 percent of primary care services were delivered by private, contracted physicians and only 19 percent by salaried physicians (Sorensen and Grytten 2002). Seventeen of the top 20 performing health systems in Table 5.3 Financing and Provision of Services 5 Financing IONT Provision Public Private SEC Government Typical public services Typical public services without user fees or with significant informal informal payments payments or user fees Nonstate providers Contracting Typical private provision, usually fee for service Source: Author. 92 · Performance-Based Contracting for Health Services in Developing Countries the world (as judged by the World Health Organization in its 2000 World Health Report) use some form of contracting, either implicit or explicit, to deliver health services. No part of this discussion is meant to diminish the challenges asso- ciated with contracting in low-income countries. The intent is merely to underscore that contracting for primary health care services is a widely practiced and accepted means of ensuring delivery of publicly financed primary care. Under contracting, the government's role becomes more strategic and less directly involved in service delivery. Governments will still need to finance health services and carry out essential public health functions, such as setting policy and technical standards, collecting information, monitoring and evaluating performance, responding in emergencies, and coordinating all the partners. Because actual service delivery takes up so much time and attention, contracting may well make governments better able to carry out their other, unique roles. If governments want to be stewards of the health sector, they will have to find creative ways of working with the private sector. Influence of Foreign NSPs. In countries with many indigenous NGOs, such as Bangladesh, competitive bids have been almost always won by local NGOs. The extent to which contracting efforts encour- age the growth of local NGOs is not yet clear, although access to predictable funding flows is a key determinant of NGO sustainabil- ity in virtually all settings (Green 1987). In Afghanistan, large-scale contracting has increased the number of local NGOs working in the health sector, such that now they are winning most of the new con- tracts. In Cambodia, contracting seems to have had only a modest effect on the number of local NGOs working in the health sector. 5 Bidding procedures that are at least partly based on price will, in the long run, select for local NGOs because expatriates are much more IONT expensive than local professionals. Strengthening of the State in Postconflict Settings. The expe- SEC rience in a few postconflict settings, particularly Afghanistan and the Democratic Republic of Congo, is that, far from weakening the state and how it is perceived, contracting can actually strengthen the state. This, somewhat counterintuitive, finding results from the following factors: · The NGO Sector Is Better Coordinated. In most postconflict settings, there are usually many NGOs that are providing health Whether to Contract· 93 services but with limited coordination. In addition, because the NGOs receive funds from many different sources, they often fol- low the direction of their financiers, rather than the priorities of the government. Contracting allows the Ministry of Health to ensure that NGO activities are better aligned with government priorities. · Communities Care Mostly about Better Services. Because it rapidly improves service delivery, contracting enhances the pop- ulation's perceptions of publicly funded services. For example, in Afghanistan, extensive focus group discussions found that people in the community did not differentiate between publicly and NGO- provided services. What most interested them was improved ser- vices, rather than who delivered the services. · Government Can Focus on Key Issues. Most postconflict states have difficulties directly managing health services themselves, and when they do, the services take up so much of their time that they cannot undertake all their other key roles. Contracting services allows them to focus on strategic issues, such as establishing tech- nical guidelines, setting priorities, and coordinating their develop- ment partners. Quality of Care. While measuring the quality of health services can be technically challenging, it is something that should be done more often. Quality of care has been measured in four of the evaluated cases, Afghanistan, urban Bangladesh, Cambodia, and Pakistan. In three of the four settings, NSPs performed significantly better on quality of care, and in the fourth there was little difference. Using scores derived from health facility assessments, the double difference was 24 percent- age points in Afghanistan and 19 percentage points in Cambodia. The single difference in Bangladesh was 11 percentage points. 5 IONT In What Context Will Contracting Work (or Not Work)? SEC Contracting has been successfully implemented in many situations and for a wide variety of health services. It is not clear from the experi- ence thus far if there are environments where contracting is not pos- sible. Certainly, there are concerns that arise in some contexts that may require creative solutions. However, for the most common concerns 94 · Performance-Based Contracting for Health Services in Developing Countries Table 5.4 Common Concerns about the Environment for Contracting Concern Comment Example Mitigation measures "The legal envi- In many set- The Democratic Ensure the contract ronment needed tings, particularly Republic of Congo has dispute resolu- to enforce a con- postconflict, the has success- tion mechanisms tract is missing." legal environ- fully implemented that are not entirely ment is difficult, contracts in spite dependent on the but it still allows of having limited legal system. Parallel contracts to be contract law and efforts to establish successful. an underdeveloped or strengthen con- judicial system. tract law should be initiated. "There are This lack can Cambodia success- Advertise widely and few potential be a problem, fully implemented limit obstacles to contractors." although new contracting in "market entry." Hire NSPs may enter spite of limited individuals (rather the "market" or competition. than organizations) shift from related on contract for the activities. management of services. "Potential Rarely is the In Pakistan, most Invest in develop- contractors have capacity of exist- NGOs working on ing the capacity of limited capacity." ing public provid- HIV/AIDS had very NSPs, particularly ers questioned. limited experience. in new areas like It is difficult to The majority, but not HIV/AIDS. judge in advance all, worked out well which NSPs will when provided a work well. contract. "The political This attitude In Afghanistan there Engage in extensive and bureaucratic can be a serious was initial opposition dialogue and discuss opposition to problem and may to contracting from global experience. contracting is require consider- many civil servants Have officials from strong." able dialogue and a few politi- other countries who (see details cians. In spite of this, have implemented below). contracting has been contracting discuss successful. issues with their 5 counterparts. IONT Source: Author. Note: NGO = nongovernmental organization; NSP = nonstate provider. SEC there are examples in which contracting has been successful in spite of what, at first, looked like an unpromising environment (see table 5.4). Political and Bureaucratic Opposition. Often the contextual issue that makes contracting the most difficult is opposition from govern- ment officials and, sometimes, politicians. Their concerns are often Whether to Contract· 95 legitimate and meant to ensure the best use of public resources. Under- standing their concerns is a prerequisite for having a useful dialogue. In addition to the concerns mentioned above ("What Are the Concerns about Contracting?"), there are a number of issues that officials will be reluctant to talk about openly but have been stumbling blocks in some settings, including the following (means for addressing these concerns are described in section 3, task 3): · Implicit Criticism of Their Performance. Many government officials see contracting as an implicit criticism of their ability to manage health services effectively. They sometimes argue that things are going well, so there is no need for contracting. If they admit that there are problems in service delivery, they will argue that the solution is more resources. · Competition Not Wanted. Contracting can provide competi- tion to the public sector, and some officials fear that this will put public performance in an unfavorable light. Having NSP-provided services with which public sector services can be compared may increase criticism of public sector performance. It is very likely to increase accountability. · Reduced Control. Government officials fear that contracting will reduce their control, power, and prestige. For example, they some- times fear that giving up control over the recruitment, posting, and transfer of health workers will reduce their bureaucratic and po- litical power that comes from "clientelism" and they will lose the ability to provide favors to influential people. Similarly, they fear giving up direct control over some of the budget. · Fear of Change. As with any new approach, some officials will re- 5 sist change simply because it is different from the system in which they have worked (often for many years) and feel comfortable. IONT SEC Note 1. As an example of the way the double differences were calculated, in Cambodia, baseline prenatal care coverage was 13.5 percent in ser- vice delivery contract (SDC) districts and 10.4 percent in the control 96 · Performance-Based Contracting for Health Services in Developing Countries districts. The prenatal care coverage rates found at the end-line survey were 66.8 percent and 34.6 percent, respectively. This yields a double difference (follow-up minus baseline in the experimental group minus follow-up minus baseline in the control) for SDC versus control of 29.1 percentage points ([66.8 ­ 13.5] ­ [34.6 ­ 10.4]). The range and median double differences for the seven indicators included in the contracts are described in the Cambodia example in appendix C. 5 IONT SEC APPENDIX A A IXD Example of a Contracting Plan NEP AP (or Manual) Strengthening Primary Health Care Services in Country X A. Background Challenges. The Government of Country X (GOX) has committed itself to dramatically improving the health and well-being of Country X. Achieving this improvement will require, among other things, a strengthened primary health care system that can deliver high-impact preventive, curative, and promotive services. These services, if of suf- ficient quality and reach, will help reduce child and maternal mortality, contain the spread of infectious diseases, and reduce other threats to good health. Currently, the primary health care system faces a num- ber of important challenges including (1) low use of primary health care centers (PHCCs) for curative and promotive services, (2) low cov- erage of preventive services, (3) inconsistent and often poor quality of care, (4) frequent absenteeism of staff and a shortage of staff in remote areas, and (5) inequitable coverage of services so that the poor and those living in remote rural areas have limited access. Basic Approach. In response to these challenges, the GOX is inter- ested in undertaking public-private partnerships (PPPs). The current proposal for a PPP envisages an area/community-based approach for delivering primary health care (PHC) services in PHCCs and their surrounding catchment areas. The basic approach for the PPP is (1) the 97 98 · Performance-Based Contracting for Health Services in Developing Countries private sector partner of the GOX (hereafter referred to as the "Part- ner") is accountable for achieving tangible results (described below) in delivering a package of PHC services; (2) in order for the Partner to achieve the expected results, the GOX will provide it adequate and A IXD timely financial resources; (3) careful monitoring and evaluation will NEP be carried out so that the GOX can be confident it is getting value for its money; (4) both parties will focus on outputs and outcomes more AP than inputs and processes, which means that the GOX will specify what results should be achieved and services delivered but that the Partner will be given latitude on how to implement those services; (5) the Partner will be provided with sufficient managerial autonomy so it can flexibly respond to local conditions and introduce innovations aimed at improving service delivery; and (6) an explicit agreement will facilitate the relationship between the GOX and the Partner by mak- ing roles and responsibilities explicit. Consultations with Stakeholders. Initial consultations have already been carried out with the existing government health workers, local gov- ernment officials, members of the communities, and 15 different non- governmental organizations (NGOs). This plan tries to address their expressed concerns, and a series of meetings will be held with stake- holders in which it will be discussed before being finalized. B. Objectives and Indicators of Success Objectives. This PPP project aims at achieving the following objectives: 1. Significantly strengthen PHC in PHCCs and their associated catchment areas to ensure the widespread delivery of a standard package of preventive, curative, and promotive services that will help improve the health and well-being of Country X. 2. Dramatically improve the coverage and use of services, quality of care, and equity of access to services by geographical areas, income levels, and women and children. 3. Ensure that patients and communities are increasingly involved and satisfied with the publicly financed health services and facilitate the community's participation in the design, delivery, and evaluation of health services. Example of a Contracting Plan · 99 Indicators of Success. Achievement of those objectives will be assessed by the indicators and targets described in table A.1. By the end of the four-year period covered by this PPP, it is expected that significant progress will have been made toward the targets in table A A.1. These targets may be revised as data become available and are meant to be indicative rather than exact. What will matter is signifi- IXD cant progress along these parameters. NEP AP Table A.1 Key Performance Indicators and Targets Means of Approximate measuring Indicator Baselinea target indicator Number of consultations per person per year 0.3 1.0 HHS and provided by the PHCC and its outreach activities HMIS Contraceptive prevalence rate: percentage of 9% 15% HHS couples of reproductive age currently using a modern family planning method TB case detection rate (number of sputum- 26% 45% HFA and positive cases detected as percentage of target HMIS based on estimated prevalence, that is, case finding) Proportion of children 6 to 59 months who have 36% 55% HHS and received vitamin A supplement within last 6 HMIS months Vaccination: measles immunization coverage 28% 50% HHS and before 12 months of age HMIS Coverage of antenatal care: percentage of all 22% 50% HHS and pregnant women receiving at least one antena- HMIS tal care visit from a skilled provider Proportion of births attended by skilled 14% 25% HHS and attendants (includes institutional delivery but HMIS excludes trained traditional birth attendants) Score out of 100 on an index of quality of care 46 70 HFA as judged by third party, which includes pro- vider knowledge and patient satisfaction Improved equity: ratio of poorest to richest 0.42 0.60 HHS income quintiles (based on asset index) on number of consultations Source: Author. Note: HFA = health facility assessment; HHS = household survey; HMIS = health management information system; PHCC = primary health care center; TB = tuberculosis. a. Baseline values may be modified based on available data. 100 · Performance-Based Contracting for Health Services in Developing Countries Pay for Performance. Failure of the Partner to achieve adequate progress on the above indicators will result in the termination of its contract or the imposition of other sanctions listed in the draft agree- ment. To ensure a focus on achieving tangible results, the Partner will A IXD also be able to earn performance bonuses based on achievement of NEP certain measurable improvements. Based on the annual health facility assessment, a Partner can earn a performance bonus of 2 percent of the AP contract amount if its score on the quality-of-care index improves by 10 points from its previous highest score or it achieves a score of 85 per- cent. When the midterm household survey results are available, a Part- ner will receive a bonus worth 5 percent of the contract amount if it has achieved the targets on four of the nine indicators listed in table A1.1. When the end-line household survey results are available, a Partner will receive a bonus worth 3 percent of the contract amount if it has achieved the targets on seven of the nine indicators listed in table A1.1. C. Description of Services Package of Services. The PPP aims to strengthen the delivery of a package of standard PHC services, including (1) maternal and child immunization; (2) antenatal, obstetric, and postpartum care; (3) fam- ily planning services, including all modern methods (injectables, in- trauterine contraceptive devices, condoms, and pills) and including family planning operations (no-scalpel vasectomies and tubectomies); (4) diagnosis and treatment of major infectious diseases, including tu- berculosis, malaria, and kala-azar; (5) basic curative services normally available at PHCCs; (6) nutritional support, including improving mi- cronutrient deficiencies (vitamin A, iron, iodine), therapeutic feeding, and breast-feeding promotion; (7) participating in special health ac- tivities, such as national immunization days and other types of cam- paigns; and (8) carrying out public health functions, such as disease surveillance and recognition/response to epidemics. The partners will comply with the current national technical guidelines (such as for TB and the expanded program on immunization [EPI]) and those that are developed during the life of the agreement. Size of Each PPP Lot. On the three districts of Province Y, four districts of Province Z, and three districts of Province W, all the 105 Example of a Contracting Plan· 101 PHCCs and their catchment areas will be involved in the PPP proj- ect. Each district will constitute one "lot" and has an average popu- lation of 450,000 (the smallest has a population of 380,000 and the largest 520,000). A IXD D. Monitoring and Evaluation NEP AP Evaluation Methodology. The achievement of the indicators in table A1.1 and certain other aspects of health care (such as community sat- isfaction, health expenditures, and use of unqualified providers) will be assessed using the following methodologies: 1. Household surveys. In each of the 10 PPP lots and 10 compari- son areas, 10 clusters will be randomly selected and 20 households will be interviewed per cluster (that is, 200 households per PPP lot, 2,000 for the contracted area, and another 2,000 households for the comparison area, for a total of 4,000 households). The baseline sur- vey will be done in June 2009 with a midterm survey in July 2011, and the endline survey will be carried out in June 2014. 2. Health facility assessments. Five PHCCs in each of the 20 areas (10 PPP lots and 10 comparison areas, for a total of 100 PHCCs) will be randomly selected and visited annually to assess the quality of care and other related indicators, such as staff morale. 3. Health management information system. Health management information system data will be used by the provincial health de- partments to monitor, on a monthly basis, the performance of the PPP and the comparison PHCCs. Arrangements for Surveys. The household and health facility surveys will be carried out by a third-party firm, which will also be responsible for analysis of the results. The contract management unit of the Ministry of Health (MOH) will be responsible for recruiting the third-party firm, overseeing the surveys, and disseminating the results to stakeholders, including the Partners. The estimated cost of the surveys is US$900,000. For true baseline data to be available, the third-party firm will be recruited as soon as possible. 102 · Performance-Based Contracting for Health Services in Developing Countries E. Agreement between the GOX and Partners Content of the Agreement. The relationship between the GOX and A its private sector Partners will be governed by an agreement, a draft IXD of which is in the following annex. The agreement sets out the basic NEP approach of PPP and includes (1) the objectives of PPP and the indica- tors selected to judge its success, (2) the scope of services, (3) the loca- AP tion and duration of services, (4) reporting requirements of the private sector Partners, (5) the authority and responsibilities of the GOX and the private sector Partners, (6) methods of payment and financial re- porting, and (7) dispute resolution mechanisms and sanctions. Linkages. The draft agreement specifically addresses the issue of how to ensure strong linkages of the private sector Partners with the subcenters and health workers below the level of the PHCC. It also addresses how to ensure appropriate referrals to the PHCCs and first- referral hospitals above the PHCC. F. Price of the Services Fixed-Cost Approach. The private sector Partners will receive a fixed amount per year based on the population of the district. According to budget allocations in the last few years and actual expenditures, it is estimated that the annual cost per capita will be US$4.10. A similar amount will be allocated to the comparison districts. G. Selection of Partners Competitive Selection of Partners. The private sector Partners of the GOX will be selected on the basis of an open competition that is carried out transparently. The announcement of the competition will be published in the newspapers and on the GOX Web site and given to potentially interested Partners through associations of health care providers, such as the Country X Voluntary Health Association. Suffi- cient time (about 45 days) will be provided to interested parties to sub- Example of a Contracting Plan· 103 mit their proposals to the contract management unit (CMU) of the MOH. The details of what should be included in the proposals, the se- lection criteria, and the content of the agreement will be available on the government Web site, which will be listed in the advertisement. A Selection Criteria. A two-stage selection process will be used. During the first stage, those proposals that have arrived on time will IXD be assessed quickly to see whether they meet the initial screening cri- NEP teria: (1) three years of audited accounts for the organization, (2) proof AP (in the audited accounts) that the organization had a turnover of at least US$100,000 per year in the last three years, (3) a copy of its registration under the Societies Act or other relevant acts, and (4) a certified check made payable to the GOX for US$100 as proof of the bidder's earnest- ness. Only those organizations that meet these criteria will have their technical proposals reviewed by the evaluation committee during the second stage. The suggested criteria and weights for this second stage are the following: 1. Experience of the organization (its track record) in deliver- ing health services in Country X or similar settings--45 percent, comprising: · Documented success of the organization in delivering health services in Country X or similar countries (25 percent) · Proven ability to manage a large budget, including financial, ad- ministrative, and logistic aspects of health services (15 percent) · Knowledge and understanding of the social, physical, political, and economic characteristics in the geographical area of the par- ticular lot (5 percent). 2. The work plan/strategy devised by the organization for delivering the services stipulated in the agreement--20 percent, comprising: · Practicality and innovativeness of the approach to service deliv- ery (10 percent) · Practical means for working with the existing public health sys- tem, including PHCCs, subcenters, and communities (6 percent) · Practical means for working with existing private sector organi- zations and health care providers (4 percent). 104 · Performance-Based Contracting for Health Services in Developing Countries 3. Qualifications and experience of key managers who will be re- sponsible for implementing the PPP--35 percent, comprising: · Project manager with more than six years of management expe- A rience, particularly in the successful delivery of health services IXD (15 percent) NEP · Deputy project manager (technical) with at least four years of experience in field implementation of health services delivery (10 AP percent) · Deputy project manager (administrative) with at least four years of managing the administrative aspects of successful field imple- mentation of social services (10 percent). The individual candidates will be judged according to the following criteria: · General education, length of experience, and positions held (35 percent) · Adequacy for the project: track record and experience in health or social sector project management and implementation, knowledge of health programs, and knowledge and experience with primary health care (65 percent). Structure of Proposals. To keep the review process feasible, tech- nical proposals should be kept to a maximum of 30 pages comprising three sections: (1) experience of NGO in delivering health services, (2) work plan and strategy for implementing the PPP, and (3) the signed curriculum vitae of the three managers. Selection Process. The contract management unit (CMU), in keeping with its existing practice, will establish an evaluation com- mittee having diverse membership. The committee will include a total of seven members, two from the provincial health departments, two from the CMU, and one each from UNICEF, WHO, and the Coun- try X Voluntary Health Association (any representative from the last named must not have formal linkages with any of the organizations that have submitted proposals). The members of the evaluation com- mittee will individually rate proposals based on the above criteria. For each lot, the organization whose proposal receives the highest average score will be invited to finalize the draft agreement. Example of a Contracting Plan· 105 Consortium of Organizations. An organization may associate with a maximum of three other organizations to submit a proposal. However, the lead organization ("member in charge") must be clearly identified. All the other organizations must sign letters indicating A their agreement to work under the lead organization's direction. Fail- ure to submit such letters may result in rejection of the proposal. The IXD lead organization must provide copies of its last three years' audited NEP financial statements. AP Number of Bids per Organization. An organization or consor- tium of organizations may submit proposals for a maximum of 3 of the 10 lots. H. Contract Management Arrangements The existing CMU of the MOH will be responsible for managing the contracts, including support, monitoring, and payment. The CMU will assign one public health professional from one of the concerned provinces, an epidemiologist and monitoring specialist from the uni- versity, a financial management specialist hired from the private sec- tor, and two assistants to work on management of the 10 contracts. The special subunit will be provided a budget of US$30,000 per year and an initial capital budget of US$40,000 for appropriate transporta- tion and office equipment. I. Schedule Next Steps. The next steps for the implementation of the PPP de- scribed in this proposal are described in table A.2. 106 · Performance-Based Contracting for Health Services in Developing Countries Table A.2 Next Steps for Agreement Implementation Start/completion Action Party responsible dates Comments A Evaluation firm CMU May 9­June 15, Terms of reference will be IXD recruited 2009 shared with the GOX for comment NEP Contract signed CMU June 15, 2009 Contract will not be signed with evaluation until approval of proposal by AP firm the GOX and availability of other funds Baseline survey Evaluation firm June 15­ Survey instruments already conducted September 30, exist but will need to be 2009 customized Broad advertis- GOX (CMU) June 15, 2009 Will need to finalize texts of ing of PPP advertisements and informa- tion for Partner organizations Closing date for Private sector July 31, 2009 45 days for organizations to proposals organizations submit bids Evaluation of Evaluation August 15, 2009 Evaluation committee will proposals committee have to be formally notified Agreements CMU and September 1, October 2, 200x, at the latest signed Partner 2009 Source: Author. Note: CMU = contract management unit; GOX = Government of Country X; PPP = public-private partnership. ANNEX XE N Agreement between Ministry AN of Health and (Nonstate Partner): Strengthening Primary Health Care Services in Country X Reader: This draft agreement/contract is meant for discussion as part of the development of the contracting plan. It is not a contract that can be used by World Bank borrowers, but it contains most of the elements needed in a contract. A. General Approach To strengthen primary health care, the Government of Country X (GOX) would like to use public-private partnerships (PPPs) for the delivery of services in primary health care centers (PHCCs) and their surrounding catchment areas. The basic approach for this PPP is the following: (1) the private sector partner of the GOX (hereafter re- ferred to as the "Partner") is accountable for achieving tangible re- sults (described below) in delivering a package of primary health care services; (2) for the Partner to achieve the expected results, the GOX will provide it with adequate and timely financial resources; (3) careful monitoring and evaluation will be carried out so that the GOX can be 107 108 · Performance-Based Contracting for Health Services in Developing Countries confident it is getting value for money; (4) both parties will focus on outputs and outcomes more than inputs and processes, which means that the GOX will specify what results should be achieved and services delivered but that the Partner will be given latitude on how to imple- XE ment those services; (5) the Partner will be provided with sufficient N managerial autonomy so it can flexibly respond to local conditions and AN introduce innovations aimed at improving service delivery; and (6) an explicit agreement such as this agreement facilitates the relationship between the GOX and the Partner by making roles and responsibili- ties explicit. B. Objectives The Partner of the GOX will be responsible for achieving the follow- ing objectives: 1. Significantly strengthen primary health care in its assigned PHCCs and their associated catchment areas so as to ensure the widespread delivery of a standard package of preventive, curative, and promo- tive services that will improve the health of Country X. 2. Dramatically improve the coverage and utilization of services, qual- ity of care, and equity of access to services by geographical areas, income levels, and women and children. 3. Ensure that patients and communities are increasingly involved and satisfied with the publicly financed health services and facilitate the community's participation in the design, delivery, and evaluation of health services. C. Key Performance Indicators and Targets Achievement of the above objectives will be assessed by the indicators and targets described in table AA.1. By the end of the two-year period covered by this agreement, it is expected that the Partner will have achieved the targets in that table, although these targets may be revised as data become available and are meant to be indicative rather than exact. What matters is significant progress along these parameters. Example of a Contracting Plan · 109 Table AA.1 Key Performance Indicators and Targets Means of Approximate measuring Indicator Baselinea target indicator Number of consultations per 0.3 1.0 HHS and HMIS person per year provided by the XE PHCC and its outreach activities N AN Contraceptive prevalence rate: 9% 15% HHS percentage of couples of repro- ductive age currently using a modern family planning method TB case detection rate (number of 26% 45% HFA and HMIS sputum-positive cases detected as percentage of target based on estimated prevalence, that is, case finding) Proportion of children 6 to 59 36% 55% HHS and HMIS months who have received vitamin A supplement within last 6 months Vaccination: measles immuniza- 28% 50% HHS and HMIS tion coverage before 12 months of age Coverage of antenatal care: 22% 50% HHS and HMIS percentage of all pregnant women receiving at least one antenatal care visit from a skilled provider Proportion of births attended 14% 25% HHS and HMIS by skilled attendants (includes institutional delivery but excludes trained traditional birth attendants) Score out of 100 on an index 46 70 HFA of quality of care as judged by third party, which includes provider knowledge and patient satisfaction Improved equity: ratio of poorest 0.42 0.60 HHS to richest income quintiles (based on asset index) on number of consultations Source: Author. Note: HFA = health facility assessment; HHS = household survey; HMIS = health management information system; PHCC = primary health care center; TB = tuberculosis. a. Baseline values may be modified based on available data. 110 · Performance-Based Contracting for Health Services in Developing Countries Monitoring and Evaluation. The achievement of the above indi- cators and the objectives listed in part B will be assessed through base- line and follow-up assessments using the following methodologies: 1. Household surveys. The government will use the results of a spe- XE cial household survey to assess, for example, coverage of services, N use of family planning, and use of services by the poor. AN 2. Health facility assessments. These assessments will be conducted by third parties and will examine, for example, the quality of care and TB treatment success rates. The Partner will provide unim- peded access to any of its PHCCs for such purposes. 3. Health management information system. Examination of the HMIS data will be done jointly by the GOX and the Partner. 4. Monitoring visits. Unannounced visits by the district and/or pro- vincial health officials will also be conducted to guide, support, and supervise implementation. D. Scope of Services Rapidly Ensuring Delivery of a Package of Primary Health Care Services. The Partner will be responsible for ensuring the delivery of a package of standard primary health care (PHC) services, includ- ing (1) maternal and child immunization; (2) prenatal, obstetric, and postpartum care; (3) family planning services, including all modern methods (injectables, intrauterine contraceptive devices, condoms, pills, and referral for voluntary surgical contraception) and including family planning operations (no-scalpel vasectomies and tubectomies); (4) diagnosis and treatment of major infectious diseases, including tu- berculosis, malaria, and kala-azar; (5) basic curative services normally available at PHCCs; (6) nutritional support, including improving mi- cronutrient deficiencies (vitamin A, iron, iodine), therapeutic feeding, and breast-feeding promotion; (7) participating in special health ac- tivities such as national immunization days and other types of cam- paigns; and (8) carrying out public health functions, such as disease surveillance and recognition/response to epidemics. The Partner will be responsible for managing all aspects of PHC services. Example of a Contracting Plan· 111 Following National Guidelines and Ensuring Quality of Care. In carrying out the services described above, the Partner will com- ply with the current national technical guidelines (such as for TB and expanded program on immunization [EPI]) and those that are developed during the life of the agreement. The Partner will also be responsible for ensuring high quality of care more generally through XE N quality assurance mechanisms appropriate for rural areas. AN Staffing PHCCs. The Partner will mobilize an adequate number of skilled health workers to deliver the package of primary health care services in their PHCCs and the associated catchment areas. The Partner may recruit additional health workers if necessary. The Part- ner will ensure that there is a fully trained and certified doctor, prefer- ably a woman doctor, working in the PHCC and its catchment area. Capacity Building. The Partner will be responsible for ensuring that its health workers have the necessary skills to deliver services and will be accountable for training and supervising its health workers. The capacity of its health workers will be judged by their knowledge and the quality of the patient-provider interaction as determined by the third-party health facility assessments. Procurement of Drugs, Supplies, and Basic Equipment. The Partner will be responsible for the procurement of essential drugs (those on the GOX's essential drug list, although the Partner may procure additional drugs if it feels this is appropriate) and supplies of acceptable quality from reputable suppliers. The Partner will be able to access the rate lists for drugs established by the GOX. It is the responsibility of the Partner to ensure the quality of the drugs in PHCCs, which may be tested on a random basis during the health facility assessments. The Partner will also be responsible for ensuring the presence of basic equipment (for example, examining tables and stethoscopes) and furniture needed for its PHCCs. The Partner will provide the provincial and district governments with lists of equip- ment and furniture procured under the agreement along with location and unique identifying number of each. Linkages with Health Workers. Existing health workers will con- tinue to be paid by the GOX, and the Partner will work closely with them to improve health services. The Partner, at its sole discretion, may provide performance bonuses to health workers that it believes are doing a particularly good job in strengthening services. 112 · Performance-Based Contracting for Health Services in Developing Countries Communications. The Partner will ensure that each PHCC has the ability to communicate, by radio or telephone, with the nearest first-referral hospital. Recording and Reporting. The Partner will have to implement XE the standard recording and reporting system, including the monthly N HMIS reports. Other reporting requirements are detailed in sec- AN tion G. Medicolegal Cases. The Partner will refer all medicolegal cases to the closest appropriate GOX facility. However, it should provide all necessary first aid to the patient prior to transfer. Health Care Waste Management. The Partner will be respon- sible for properly implementing health care waste management proce- dures in keeping with the rules and regulations of the GOX. Refurbishment of Facilities and Outreach. The Partner may use part of the funds it receives to refurbish the PHCCs in the districts in which it works. It may also purchase or rent a vehicle for provision of mobile services. E. Location and Duration of Services The above mentioned services will be delivered to the entire popula- tion in District . The services will be provided over four years beginning . F. Data, Services, and Facilities Provided by the CMU The contract management unit (CMU), on behalf of the GOX, will provide the Partner with the following inputs: (1) relevant available information about facilities, health care status of population, results of surveys and special studies, and other factors; (2) use of the PHCCs in the listed districts at nominal rent; (3) copies of standard report- ing and recording forms; (4) access to government training courses; (5) existing technical guidelines and new ones developed during the agreement period; (6) an inventory of the buildings provided to the Partner; and (7) technical assistance when needed for special services or campaigns. Example of a Contracting Plan· 113 G. Reporting Requirements The Partner will provide the CMU and the concerned provincial health departments quarterly reports related to activities undertaken in fulfillment of this agreement. The report will include the follow- ing sections: (1) progress made against achieving the objectives of XE N the agreement; (2) problems encountered and solutions undertaken; AN (3) relations with stakeholders, such as community institutions, the community, and other NGOs operating in the district; (4) a financial statement limited to simple line-item expenditures (remuneration, capital costs, nonremuneration recurrent costs); and (5) a summary of HMIS results with analysis. Such reports will be furnished within one month of the end of the calendar quarter (that is, by January 31, April 30, July 31, and October 31 of each year). H. Authority and Responsibilities of the Government The CMU and the provincial health departments have the authority to perform the following: 1. Visit any Partner-managed facility at their sole discretion at any time and to obtain such relevant information as to allow proper support, guidance, and monitoring of the Partner and its staff 2. Convene a meeting with the management of the Partner to discuss and resolve issues related to the agreement and its implementation 3. Ensure that the conditions of the agreement are met by the Partner 4. Ensure that the GOX's facilities are properly maintained through out the period covered by this agreement 5. Terminate the agreement in keeping with the specific provisions of part L if the Partner fails to meet the conditions of the agreement 6. Take other actions short of termination (also described in part L) to ensure that the services covered by this agreement are being prop- erly implemented 7. Consult with the local village councils and directly with communities regarding the implementation of services covered by this agreement 114 · Performance-Based Contracting for Health Services in Developing Countries 8. Review the quarterly report of the Partner and provide feedback to the Partner and the GOX. The government has the following responsibility: XE 1. Paying the Partner in a timely manner N 2. Not subjecting the Partner to excessive and unnecessary bureau- AN cratic procedures 3. Resolving quickly such reasonable complaints brought by the Part- ner that the latter feels interferes with its performance under this agreement 4. Paying the remuneration and emoluments of existing government staff, in a timely fashion. I. Authority and Responsibilities of the Partner The Partner will have the following authority: 1. It will enjoy sole discretion in the procurement of drugs, supplies, equipment, furniture, and other resources needed to meet obliga- tions under this agreement. 2. It will also enjoy sole discretion in the use of resources purchased or provided under the agreement. (However, the Partner will not re- fuse reasonable requests for the use of such resources by the GOX officials for implementing important health-related services, such as the use of the facility during emergencies or immunization days.) 3. It may raise voluntary contributions in kind or cash from the com- munity (not patients; that is, no user charges are permitted) or from outside the community for delivering health services. 4. It will enjoy sole discretion over the hiring, firing, posting, remu- neration, and customary managerial prerogatives over staff re- cruited by the Partner. The staff recruited by the Partner shall have no claims against the CMU, the GOX, or local governments. 5. For existing GOX health workers, the Partner may provide perfor- mance bonuses in a manner it considers appropriate and at its own Example of a Contracting Plan· 115 discretion so long as the criteria and process are clear and written. It may ask district health officials to transfer health workers that it feels are not working properly or who are actively interfering with the Partner's work under the agreement. The Partner will be responsible for the following: XE N 1. Ensuring the proper maintenance and repair of government physi- AN cal assets provided to it, as well as physical assets procured with funds provided under the agreement 2. Cooperating with any monitoring and evaluation process autho- rized by the CMU 3. Resolving quickly such deficiencies that are reasonably pointed out by the CMU 4. Ensuring government resources are used efficiently for delivery of health care services. J. Financial Management and Payment Financial Management. The Partner will maintain a separate set of accounts for the funds received under this agreement and will maintain those funds in a separate bank account. The Partner will also provide an annual, independently audited financial report to the CMU and/or its representatives. Such an audit report will be furnished within three months of the end of the fiscal year. The cost of the audit will be borne by the Partner out of the funds it receives under this agreement. The Partner will also provide the CMU with a list of equipment and furni- ture procured with funds provided under the agreement. Payment. It is understood that this is a lump-sum contract based on performance and not on the reimbursement of specific expendi- tures. The Partner will be paid the amount of [US$ ] upon signing this agreement as a mobilization advance. The total annual payment to the Partner will be [US$ ]. Payments to the Partner will be made within two weeks of receipt of an acceptable quarterly report and will be equal to one-quarter of the annual payment men- tioned above. There is no other requirement on the Partner before payment is affected. 116 · Performance-Based Contracting for Health Services in Developing Countries Performance Bonuses. According to the annual health facility assessment, the Partner will be paid a performance bonus of 2 per- cent of the contract amount if its score on the quality-of-care index improves by 10 points from its previous highest score or has achieved XE a score of 85 percent. When the midterm household survey results N are available, the Partner will receive a bonus worth 5 percent of the AN contract amount if it has achieved the targets on four of the nine indi- cators listed in table AA.1. When the endline household survey results are available, the Partner will receive a bonus worth 3 percent of the contract amount if it has achieved the targets on seven of the nine indicators listed in the table. K. Settlement of Disputes The parties to this agreement will use their best efforts to settle ami- cably all disputes arising out of or in connection with this agreement or its interpretation. The Partner may bring to the attention of the CMU any serious complaint that it feels interferes with accomplish- ment of the objectives and services described in this agreement. Any dispute between the parties as to matters arising pursuant to this agreement that cannot be settled amicably within 30 days after receipt by one party of the other party's request for such amicable settlement may be submitted by either party for settlement in accordance with the following provisions: 1. First, referral to an arbitrator appointed by the Secretary of Health of the GOX. To encourage reasonableness in disputes that are pri- marily monetary, the arbitrator will use "swing arbitration," that is, both parties will state their "most reasonable offer," and the com- mittee can accept only one or the other. To discourage frivolous referrals, the arbitrator can assess costs against the losing party. 2. If the dispute has not been resolved to the parties' satisfaction, either party may refer it to a committee comprising two members of the provincial government, two members from the NGO community (not involved in the dispute), and one member from the federal gov- ernment. The findings of the committee shall be binding. Example of a Contracting Plan· 117 L. Sanctions and Termination Sanctions. The government may, at its sole discretion, (1) insist on a meeting with the senior management of the Partner regarding any failures to meet the conditions of the agreement; (2) write directly to the board of directors of the Partner and expect a specific written reply XE N to its concerns; (3) ask for the replacement of field-level managers of AN the Partner, although it is understood that this will not be requested in a frivolous manner; (4) bar the Partner from receiving further con- tracts; and (5) make public the results of independent assessments of the Partner's performance. Termination of the Agreement by the Government. The GOX may terminate this agreement, by not less than 30 days' written notice of termination to the Partner to be given after the occurrence of any of the events specified below, but in keeping with part K, regarding the settlement of disputes: 1. If the Partner does not remedy a failure in the performance of its obligations under the agreement, within 30 days after being noti- fied or within any further period as the government may have sub- sequently approved in writing 2. If the Partner becomes insolvent or bankrupt 3. If, as the result of force majeure, the Partner is unable to perform a material portion of the services for a period of not less than 60 days 4. If the Partner, in the judgment of the government, has engaged in corrupt or fraudulent practices in competing for or in executing the contract. "Corrupt practice" means the offering, giving, receiving, or soliciting of anything of value to influence the action of a public official in the selection process or in contract execution. "Fraudu- lent practice" means a misrepresentation of facts in order to influ- ence a selection process or the execution of the agreement. Termination of the Agreement by the Partner. The Partner may terminate this agreement, by not less than 30 days' written notice to the government, such notice to be given after the occurrence of any of 118 · Performance-Based Contracting for Health Services in Developing Countries the events specified below, but in keeping with part K, regarding the settlement of disputes: 1. If the government fails to pay any monies due to the Partner pur- suant to this agreement and not subject to dispute within 45 days XE after receiving written notice from the Partner that such payment N is overdue AN 2. If, as the result of force majeure, the Partner is unable to perform a material portion of the services for a period of not less than 60 days. Signed and dated by: on behalf of the Contract Management Unit Signed and dated by: on behalf of the Partner Signed and dated by: on behalf of the Provincial Health Department APPENDIX B World Bank Procurement Approach to Contracting for Health Services B IXD NEP To help guide clients and task teams on how to procure health ser- AP vices, services should be judged according to (1) how easy they are to define in terms of outcomes, technical content of the services, and the organization of services (if that is likely to matter); (2) the "intellectual content" of the services; and (3) the ease, precision, and frequency of independent measurement of results at a reasonable cost. Generally, the more difficult services are to define, the greater their intellectual content, and the more complicated their measurement, the more the Consulting Services (Green Book) guidelines should be used. Some detailed examples are provided on subsequent pages. Examples of the Application of the Typology Below are two examples of how the typology in table A2.1 can be ap- plied to real-world situations. Example A: Managing a Clinic That Provides Antiretroviral Medicines to People Living with HIV/AIDS Definition of Services. Providing antiretroviral (ARV) drugs to people living with HIV/AIDS (PLWHAs) is hard to define in all its details. Although national guidelines often exist about how such ser- vices should be provided, their application usually relies on clinical judgment that takes into account the entire social and clinical con- 119 APPENDIX B 120 Table B.1 Procurement Approaches to Use in Different Situations Ease of definition Measurement of results Intellectual Recommended Type Examples Outcomes Organization Services content Ease Precision Frequencya guidelines Ancillary Equipment Easy Easy or Easy Low Easy High Frequent Procurement services maintenance, does not or very (Red Book) cleaning, security, matter frequent waste manage- ment, ambulance services Private- Build-operate- Moderate Does not Moderate Moderate Easy High Frequent Procurement public transfer approach matter (Red Book) financing for hospitals, partner- private provision ships for and operation infrastruc- of equipment in ture public hospitals Private- Public subsidies Moderate Moderate Variable Moderate Hard Moderate Infrequent According to public to existing mission to hard to hard (moderate to high to low or rare the Project financing clinics, contracts to hard) Imple- partner- with existing mentation ships for private provid- (Operational) services ers (for example, Manual "soft" contracts related to TB control), provi- sion of financing to public facilities by private sector entities Table B.1 Procurement Approaches to Use in Different Situations (continued) Ease of definition Measurement of results Intellectual Recommended Type Examples Outcomes Organization Services content Ease Precision Frequencya guidelines Hospital Individual or firm Hard Hard Hard High Hard Low Infrequent Consulting manage- contracted to and does Services ment manage existing matter (Green Book) hospital, treat- ment provided to PLWHA Inter- · Franchise with Moderate Hard Variable High Hard Low Infrequent According to mediary private provid- to hard and does (moderate or rare the Project services ers, voucher matter to hard) Imple- program mentation (Operational) Manual · Establish health Moderate Hard Variable High Hard Low Infrequent Consulting insurance, to hard and does (moderate or rare Services performance- matter to hard) (Green Book) based pay to public provid- ers, umbrella NGO overseeing many smaller NGOs/CBOs (continued on next page) 121 APPENDIX B APPENDIX B 122 Table B.1 Procurement Approaches to Use in Different Situations (continued) Ease of definition Measurement of results Intellectual Recommended Type Examples Outcomes Organization Services content Ease Precision Frequencya guidelines Population- Behavior change Moderate Hard Moderate High Hard Low Infrequent Consulting based communications, and does or rare Services services condom social matter (Green Book) marketing Delivery of Service delivery Moderate Hard Variable High Hard Moderate Infrequent Either preventive, contract for PHC, to hard and does (moderate to low or rare Consulting promotive, management matter to hard) Services and simple contract for PHC, (Green Book) curative provision of HIV or Procure- services prevention ser- ment (Red vices to high-risk Book) (will groups, provision be reviewed of antiretroviral based on medicines to experience) persons with HIV/AIDS Source: Author and agreed to by the World Bank's Operations Policy and Country Services unit. Note: CBO = community-based organization; NGO = nongovernmental organization; PHC = primary health care; PLWHA = people living with HIV/AIDS; TB = tuberculosis. a. Very frequent = more than once a month; frequent = between once a month and once a quarter; infrequent = once a quarter to once a year; rare = less than once a year. World Bank Procurement Approach to Contracting for Health Services · 123 text. Aspects of the organization of the clinic, such as making clients feel welcome, ensuring that they receive other support services, and ensuring confidentiality, are hard to write into a contract and mat- ter very much to the broad outcomes of the services. The outcomes for such services are also hard to define. PLWHAs who meet cer- tain clinical criteria need to be on ARVs; however, it may be perverse to use the percentage of patients on ARVs as a measure of success. (These are toxic drugs to which the virus becomes resistant if they are not properly used. Hence, putting people on ARVs who don't re- ally need them should be actively discouraged.) Similarly, using the B death rate of people who are put on ARVs may also have unintended consequences. Clinics may not take on patients who are very ill and IXD have a poor prognosis or other "difficult" patients, such as injecting NEP drug users who don't adhere easily to therapy. AP Intellectual Content. Perhaps the criterion for judging the intel- lectual content is whether a reasonably intelligent person, with no spe- cial training in the particular field, could do a good job in providing the services. By this standard, the intellectual content of such services is extremely high. It requires a specialized physician with extensive training and very good clinical judgment. There are aspects of each case that are different and important to take into account while devel- oping the optimal management strategy for the patient. Also, the knowledge in this field is rapidly changing, which means that even specialists need to keep up to date with recent developments. Measurement of Results. Not only are the outcomes of such ser- vices difficult to define, but they are also difficult to measure. They would likely involve chart reviews by experts who would still need to be specially trained and go through a reasonable number of records. (The desired content of the patient records would also have to be defined in advance.) The judgments of these experts would also be subjective (low precision), and this could be done only once or twice a year. Example B: Cleaning Services in a Hospital Definition of Services. The definition of such services is obvious and easy. The location and type of cleaning to be done could be specified. The organization of the services generally doesn't matter as long as they get done. The outcomes are evident to the nose and can also be easily tested by carrying out bacterial swabs of specified surfaces. 124 · Performance-Based Contracting for Health Services in Developing Countries Intellectual Content. Using the criterion of whether a reason- ably intelligent person, with no special training in the particular field, could do a good job in providing the services, cleaning is pretty low in content. Almost anyone could understand what needs to be done and could actually do it. Measurement of Results. Not only are the outcomes of such ser- vices obvious, but they are easy to measure, the precision of measure- ment is high, and the outcomes of interest can be measured frequently (daily if desired) at low cost. B IXD NEP AP APPENDIX C Description of Evaluated Contracting Experiences Case 1. Cambodia: Rural Primary Health Care Services C Background. Many years of war and political upheaval left Cambodia IXD with a limited health infrastructure, particularly in rural areas. Health NEP worker morale was poor, management capacity at the district level was AP very weak, and access to service was inadequate. A 1998 demographic and health survey found that, nationwide, only 39 percent of children were fully immunized and 21 percent of mothers had received any prenatal care during their last pregnancy. To address these serious is- sues, the Ministry of Health (MOH) devised a "coverage plan" that focused on delivery of a minimum package of activities comprising basic preventive and curative services, such as immunization, family planning, antenatal care, and provision of micronutrients. Description of Intervention. With financing provided by the Asian Development Bank, the government contracted with nongov- ernmental organizations (NGOs) in two different ways: (1) a service delivery contract (SDC, called "contracting out" in the original study) in which the contractors had complete line responsibility for service delivery, including hiring, firing, and setting wages; procuring and distributing essential drugs and supplies; and organizing and staff- ing health facilities; and (2) a management contract (MC, referred to as "contracting in" in the original) in which the contractors worked 125 126 · Performance-Based Contracting for Health Services in Developing Countries within the MOH system and had to strengthen the existing district structure. The MC contractors could not hire or fire staff, although they could request their transfer. Drugs and supplies were provided to the district through the normal MOH channels. The MC contractor received a budget supplement of $0.25 per capita per year to spend on incentives for staff, operating expenses, and other costs. In a con- trol area, the management of services remained in the hands of the district health management team (DHMT); and drugs and supplies continued to be provided through normal MOH channels. As with the MC, the DHMT received a budget supplement of $0.25 per capita per year to spend on incentives for staff, operating expenses, and so on. Technical assistance and training on management were provided to the DHMT. This approach is referred to as "government with sup- port" (GS). Three districts that were assigned to contracting were not successfully contracted (in two cases because there were no techni- cally responsive bidders, and in the other district because government C considered the bid price to be too high). These three districts had IXD health services implemented by the MOH with no budget supplement, NEP technical assistance, or training and are referred to as "government without support" (G). AP Contracting Arrangements. The MOH used a formal competitive process for selecting the NGOs based on both the quality of the techni- cal proposals and cost. Contracts were for four years. All the winning bidders were international NGOs with previous experience working in Cambodia. At the time the contracts were bid, there were almost no Cambodian NGOs active in the health sector. Contract management was carried out by a special unit of the MOH using local consultants. The unit appears to have functioned well in ensuring that NGOs were paid on time; however, much time and effort were needed to solve prob- lems between provincial and district authorities and the NGOs. Evaluation Methodology. Twelve districts with a combined popu- lation of 1.5 million were randomly assigned to the three different approaches and baseline household, and health facility surveys were carried out in late 1997. Contracts were signed with the NGOs in December 1998, and follow-up surveys were carried out in August 2001 and June 2003, about four years after implementation began. The surveys were carried out by a third party, and data were collected on the parameters stipulated in the contract. The wealth of each house- hold was assessed using an asset index. Description of Evaluated Contracting Experiences · 127 Results. There were much larger improvements in prenatal care coverage in the SDC and MC (contracting out [CO] and contracting in [CI], respectively) districts than in the control districts (see figure C.1) although they were quite similar at baseline. The government with support did no better than the government without support, although both groups made progress during the four years of the experiment. The story is similar for other important indicators, as illustrated in figure C.2. For example, use of a health facility if the respondent was sick in the last month increased 33 percentage points in the SDC (CO) group, compared with 25 percentage points in the MC (CI) districts, 12 percentage points in the government with support control group, and 10 percentage points in the government without support group. At baseline, use was similar in all the groups (0.7 percent, 0.9 percent, 1.0 percent, and 1.5 percent, respectively). The median double difference between SDC and the GS districts for the seven indicators explicitly mentioned in the contract was 10.4 percentage points (range ­1.3 to 29.1), and for the MC districts the C comparable figure was 8.7 percentage points (range 2.9 to 32.5). These IXD changes represent more than half a baseline standard deviation, a very NEP large improvement. The changes occurred even in parameters that AP are generally believed to change slowly, such as use of family planning and institutional deliveries. The SDC and MC groups also did signifi- cantly better on improving the quality of care compared with the GS Figure C.1 Coverage of Prenatal Care by Approach Used 80 70 (%) 60 50 coverage 40 care 30 20 prenatal 10 0 baseline midterm endline contracting out (SDC) contracting in (MC) government with support government without support Source: Author. 128 · Performance-Based Contracting for Health Services in Developing Countries Figure C.2 Change between Baseline and Endline Surveys 60 50 (%) 40 baseline suveys 30 between endline 20 and change 10 0 FIC Vit. A ANC TDEL FDEL MBS Use contracting out (SDC) contracting in (MC) government with support government without support Source: Based on Schwartz and Bhushan (2003). C Note: ANC = antenatal care coverage, one or more visits; FDEL = proportion of infants delivered in a health facility; FIC = percentage of children 12­23 months of age fully IXD immunized; MBS = proportion of couples with children 12­23 months of age and who are NEP using a modern form of spacing births/contraception; TDEL = proportion of deliveries attended by a trained health worker; Use = proportion of people sick in the last month who used a government health facility; Vit. A = proportion of children 6­59 months of age who AP received vitamin A supplements in the last 6 months. and G groups, even though this was not specified in the contract. The double difference between the contracted and government districts was 193 points on a scale of 1,000 (that is, 19.3 percent). The poor appear to have benefited disproportionately from con- tracting, as can be seen in table C.1, which describes the change in the concentration index from baseline to follow-up evaluation, that is, the extent to which health services became more or less pro-poor. (Nega- tive values mean that services became more pro-poor, and positive values indicate that services became less pro-poor.) With the excep- tion of the use of modern methods of spacing births and vitamin A, services in SDC districts became more pro-poor followed by the MC districts. The control districts became less pro-poor. The cost of SDC to the government was higher than the MC or the control groups but led to a considerable savings in out-of-pocket expenditures by people in the community (table C.2). Thus, contract- ing in this context was an effective and equitable way of subsidizing Description of Evaluated Contracting Experiences · 129 Table C.1 Changes in the Concentration Index of Health Services, Follow-Up Minus Baseline Type FIC Vit. A ANC TDEL FDEL MBS Use Average SDC ­0.213 0.013 ­0.151 ­0.081 ­0.222 ­0.116 ­0.175 ­0.135 MC ­0.115 ­0.036 ­0.007 ­0.014 0.033 ­0.006 ­0.036 ­0.026 GS ­0.018 ­0.030 0.088 0.061 0.210 ­0.196 0.104 0.031 G 0.010 ­0.007 0.083 0.112 0.027 ­0.129 0.041 0.020 Source: Based on Schwartz and Bhushan (2003). Note: Negative values indicate a more pro-poor distribution of services. ANC = antenatal care coverage, one or more visits; FDEL = proportion of infants delivered in a health facility; FIC = percentage of children 12­23 months of age fully immunized; G = government without support; GS = government with support; MC = management contract (contracting in); MBS = proportion of couples with children 12­23 months of age and who are using a modern form of spacing births/ contraception; SDC = service delivery contract (contracting out); TDEL = proportion of deliveries attended by a trained health worker; Use = proportion of people sick in the last month who used a government health facility; Vit. A = proportion of children 6­59 months of age who received vitamin A supplements in the last 6 months. C Table C.2 Annual Total Health Expenditures per Capita, 2003 (US$) IXD Government Contract out Contract in Government with without support NEP Source of funds (SDC) (MC) support (GS) (G) AP Private 14.29 16.72 19.86 19.99 out-of-pocket Public, 4.83 3.47 1.74 1.70 recurrent Total 19.12 20.19 21.60 21.69 Source: Based on Schwartz and Bhushan (2003). Note: MC = management contract; SDC = service delivery contract. the cost of health services. From society's point of view, contracting was also the most efficient way of delivering health services. Comments.Theresultsofthisrandomizedcontrolledstudyshowed that contracting with NGOs can significantly improve service delivery within a short time. It appears that the greater autonomy provided to NGOs under SDC enabled a faster improvement in performance, but the MC NGOs later caught up. Overall, contracting with NGOs was considerably more successful than government delivery of the same services, benefited the poor disproportionately, was the most efficient 130 · Performance-Based Contracting for Health Services in Developing Countries approach, and was achieved at a reasonable cost. One possible argu- ment is that if the control districts had been given the same amount of resources, they could have done just as well. The success achieved with contracting has been expanded during a subsequent project, financed by the U.K. Department of International Development (DFID), the Asian Development Bank, and the World Bank, that now covers about 20 percent of the population of Cambodia. (More information is avail- able in Bhushan, Keller, and Schwartz 2002; Bloom and others 2006; Loevinsohn 2000; Schwartz and Bhushan 2003, 2004; Soeters and Griffiths 2003.) Case 2. Bangladesh: Rural Nutrition Services Background. Bangladesh suffers from one of the highest malnutrition levels in the world, and until the 1990s the government of Bangladesh C (GOB) had not mounted a coordinated effort to address the problem. IXD Based on experience gained from the Tamil Nadu Integrated Nutri- NEP tion Project in India, the GOB was interested in introducing com- munity nutrition services on a large scale. It was initially thought that AP these services would be delivered by the Ministry of Health and Fam- ily Welfare (MOHFW). However, two major concerns with this ap- proach were expressed. The finance ministry was reluctant to expand the size of the MOHFW staff greatly, and there was general skepti- cism that the MOHFW could rapidly and effectively deliver commu- nity-based services, something with which they had little experience. A consensus emerged that it would be sensible to work systematically with NGOs to deliver community nutrition services. Out of this con- sensus grew the Bangladesh Integrated Nutrition Project (BINP), un- dertaken by the MOHFW with funding from the GOB, the World Bank, and the United Nations Children's Fund (UNICEF). Description of Intervention. The Bangladesh Integrated Nutri- tion Project employed a large number of female community nutrition promoters (CNPs), women from the community who have at least an eighth-grade education, to undertake growth monitoring and promo- tion among young children, nutritional support for pregnant and lac- tating women, behavior change communication about nutrition and related issues for the whole community, and supplementary feeding for Description of Evaluated Contracting Experiences· 131 severely malnourished or growth-faltering children under two years of age, as well as pregnant and lactating women. Training, supervision, payment, and support of the CNPs were carried out by NGOs, who signed a memorandum of understanding with the MOHFW. Non- governmental organizations were provided a fixed budget, the amount of which had been decided in advance by the MOHFW, to run the program in selected upazilas (administrative entities comprising more than 300,000 people). By the end of the BINP, 59 upazilas, comprising more than 15 million people, were covered by seven NGOs. The cost of the community nutrition intervention was $0.96 per capita per year, or $20.00 per child under age two per year. Contracting Arrangements. In the initial phase of the BINP, NGOs were selected on a sole-source basis based on their track record. During subsequent phases, NGOs were selected competitively based on the quality of their technical proposals. Winning bidders were given a fixed-price contract and assigned to specified upazilas by the MOHFW. Contract management was done by a special project unit of C the MOHFW but turned out to be problematic. Significant delays in IXD payments to NGOs and limited field supervision occurred. Obstacles NEP to managerial autonomy under the BINP also arose, including (1) line- AP item budgets wherein contractors could not use savings in one line item to offset expenditures in another, (2) the need for prior approval for innovations suggested by the contractors even when these involved no additional cost, and (3) the MOHFW ordering contractors to under- take new activities without any discussion or consideration of cost. Evaluation Methodology. A controlled, before-and-after design was used to evaluate the BINP. Baseline, midterm, and endline house- hold surveys in 1995, 1998, and 2003, respectively, were conducted in six upazilas that were included in BINP and two control upazilas that were not subject to the BINP nutrition activities. The surveys were conducted by a third party under contract to the MOHFW. The stated objective of the evaluation was to discern a change in nutri- tional status, particularly in the proportion of young children who were moderately and severely underweight. Results. Reductions in rates of moderate and severe malnutrition were slightly greater in project upazilas compared with control upazi- las, although both sets of upazilas made considerable progress (see table C.3). Although they were not specifically mentioned as indicators of 132 · Performance-Based Contracting for Health Services in Developing Countries Table C.3 Changes in Key Indicators from Baseline to Endline in Project and Control Upazilas Baseline Endline Change (Oct. 1995) (Feb. 2003) (% points) Double Project Control Project Control difference Parameter (%) (%) (%) (%) Project Control (% points) Child was moder- 52.6 50.7 34.9 37.3 ­17.7 ­13.4 4.3 ately and severely underweight (weight for age Z score < ­2 SD) Women attended 11.8 13.7 81.0 55.1 +69.2 +41.4 27.8 antenatal checkup Children received 17.1 16.8 83.2 56.1 +66.1 +39.3 26.8 vitamin A capsule Children initiated 14.5 11.9 84.0 56.2 +69.5 +44.3 25.2 breastfeeding immediately after C birth IXD NEP Source: Based on Karim and others (2003). AP success during the design of the project, levels of vitamin A coverage and prenatal care coverage improved more in the project upazilas than in the control upazilas. The cost-effectiveness analysis indicated a cost per malnutrition-related death averted of $1,745, and a cost per case of severe malnutrition averted (that is, changing a severely malnourished child to at least a moderately malnourished child) of $366. Comments. In a controlled, before-and-after evaluation, NGO delivery of community nutrition services appeared to have had a mod- est effect on nutritional status, although significant improvements were seen in other health services. The intervention was likely better than the control; however, the benefits were achieved at a relatively high cost. Although the effectiveness and cost-effectiveness of the BINP nutri- tion interventions remain controversial, the issue with regard to con- tracting is whether the NGOs successfully implemented the program as designed. They appeared to do this despite difficulties with MOHFW management of the contracts and the very large scale on which the NGOs had to work. Based on the perceived success of the BINP, the National Nutrition Program (NNP) was undertaken to expand commu- Description of Evaluated Contracting Experiences· 133 nity nutrition services further. The NNP currently covers 105 upazilas, or about a third of rural Bangladesh, encompassing more than 30 mil- lion people, with continued reliance on NGOs to deliver services. (More information is available in Karim and others 2003.) Case 3. Bangladesh: Urban Services Background. As in other developing countries, the provision of urban primary health care (PHC) services in Bangladesh is chaotic and often ineffective at reaching the poor. In the large cities of Bangladesh, pri- mary care is provided by a bewildering number of providers, includ- ing (1) a large for-profit private sector with many individual providers of varying level of skill, (2) a large number of for-profit hospitals and clinics, (3) a few federal government clinics, (4) a few city corporation clinics, and (5) a relatively large number of NGO facilities. The result of these uncoordinated efforts is that the coverage of PHC services C among the poor is worse than in remote rural areas. IXD Description of Intervention. The government, with funding NEP from the Asian Development Bank, divided four large cities into part- AP nership agreement areas (PAAs) and contracted with competitively selected NGOs to deliver a package of PHC services. Each PAA cov- ered a population of about 250,000­400,000 and included five to eight project-financed health centers that were built on government land. An NGO was expected to provide outreach services as well as to oper- ate health centers, and its contracts specified coverage targets as well as provided considerable autonomy. The cost of the contracts averaged $0.65 per capita per year, although the low cost partly reflected the fact that many urban residents obtain their PHC services from other sources. In the center of Chittagong, the country's second largest city, the Chittagong City Corporation (CCC) was given the opportunity to operate two PAAs with the same level of funding on a per capita basis as was provided to the NGOs. Another PAA in Chittagong, similar to the one operated by the CCC, was successfully tendered to an NGO. Contracting Arrangement. Competitive bidding was carried out for each of the PAAs with NGOs competing both on the technical quality of their proposals and price. All the winning bidders were Ban- gladeshi NGOs, which were awarded four-year contracts. Contract 134 · Performance-Based Contracting for Health Services in Developing Countries management was done by a project unit working in the Dhaka City Corporation. Project managers admitted problems with paying NGOs on time and carrying out adequate monitoring of performance. Evaluation Methodology. This project used a prospective, con- trolled before-and-after design to evaluate the difference between NGO and CCC provision of services. Baseline household surveys were carried out by an independent third party in all the PAAs, including the two operated by CCC. Three years later, a health facility survey was carried out and was used to create a broad index of quality of care (scaled from 0 to 100). A household survey was also undertaken among the general population and then among the poorest half of the popula- tion (mostly people living in slums or dilapidated housing). Results. As can be seen in table C.4, the NGO did considerably better in improving the coverage of maternal and child health services (except for immunization) with a median double difference of 8.3 per- centage points. The broad index of quality of care was also significantly C higher in the NGO area compared with the government-run facilities. IXD The NGO also did better than the government in improving the cov- NEP erage of services among the poorest 50 percent of the population with a median double difference of 10.1 percentage points (see table C.5). AP Comments. Using a before-and-after controlled evaluation, it appears that the NGO was able to provide more and better-quality health services even though it had the same amount of resources as the Chittagong municipal government. Based on the success of this experiment, the government, working with the Asian Development Table C.4 Changes in Coverage (Percentage Points), 2000­03 Area ANC CPR SBA FIC Vit. A BF QOC CCC (government) 18.8 ­5.5 7.7 25.8 13.3 6.8 65 NGO 29.2 14.8 14.0 19.1 24.3 11.1 76 Double difference 10.3 20.3 6.3 ­6.7 11.0 4.3 11a Source: Based on Bonu (2006). Note: ANC = antenatal care coverage; BF = children breastfed for at least six months; CCC = Chittagong City Corporation; CPR = contraceptive prevalence rate; FIC = coverage of full child immunization; NGO = nongovernmental organization; QOC = quality of care; SBA = skilled birth attendance; Vit. A = vitamin A coverage among children 6­59 months of age. a. Single difference on a scale from 0­100. Description of Evaluated Contracting Experiences · 135 Table C.5 Changes in Coverage (Percentage Points) among the Poorest 50 Percent, 2000­03 Area ANC CPR SBA FIC Vit. A BF CCC (government) 23.5 ­5.0 3.4 22.7 15.7 8.1 NGO 36.4 15.2 17.9 19.0 23.0 12.0 Double difference 12.9 20.2 14.5 ­3.7 7.3 3.9 Source: Based on Bonu (2006). Note: ANC = antenatal care coverage; BF = children breastfed for at least six months; CCC = Chittagong City Corporation; CPR = contraceptive prevalence rate; FIC = coverage of full child immunization; NGO = nongovernmental organization; SBA = skilled birth attendance; Vit. A = vitamin A coverage. a. Single difference on a scale from 0­100. Bank and other development partners, has expanded the contracts with NGOs to cover more urban areas. (More information is available in Bonu 2006 and Mahmud, Ullah, and Ahmed 2002.) C IXD NEP Case 4. Bolivia: Urban Primary Health Care AP Background. Bolivia's maternal and infant mortality rates, at 370 per 100,000 and 67 per 1,000, respectively, are among the worst in the Americas. In response to these high rates, in 1996 the government began a concerted effort to provide a basic package of services to the entire population. The package aimed at addressing the most common causes of maternal and child deaths and at controlling communicable diseases. Part of this effort also included working with nongovernmental provid- ers to expand service delivery and improve the quality of care. Description of Intervention. The intervention took place in one district of El Alto, Bolivia's fourth largest city with a population of 570,000, which suffered from limited human resources and physical health infrastructure, considerably less than the national average. In the late 1990s, assistance from the Dutch government and the World Bank led to the building of eight health centers and strengthening of the district hospital. Despite these investments, service delivery was of poor quality, and use was low during this period (the six months prior to July 1999, called phase 1). To improve management of the hospital, 136 · Performance-Based Contracting for Health Services in Developing Countries an NGO was given a management contract starting in August 1999 (called phase 2). The contract did not include specific indicators of success or objectives and did not provide the NGO with authority over the staff of the hospital. Out of concern that the original contract was having little demonstrable effect, the nature of the contract with the NGO was changed starting in February 2000 (called phase 3) to include clear objectives, responsibility for managing the health cen- ters, and more control of human, physical, and financial resources. Contracting Arrangements. The contract was managed by a "management contract committee" comprising representatives from the municipality, MOH, and two community representatives. The NGO was responsible for all aspects of management of the hospital and eight health centers. Evaluation Methodology. The effectiveness of the contract with the NGO was evaluated using a before-and-after comparison with a similarly sized control district, also in El Alto, in which management C had remained in the public sector. The data were obtained from the IXD routine recording and reporting system and were available on only a NEP few indicators. The data for each phase were for a six-month period. Results. As can be seen in table C.6, the NGO was able to increase AP the number of deliveries by 41 percent compared with 20 percent in the control district (a double difference of 21 percentage points). Both achieved similar improvements in bed occupancy rates for their gynecological and obstetric services. The number of outpatient vis- its increased 55 percent in the NGO district. It appears that in the NGO-managed district the introduction of a new contract providing the NGO with more autonomy and authority resulted in real improve- ment. There was a 14 percent increase in deliveries (compared with 5 percent in the control area) and a 24 percent increase in outpatient visits between phases 2 and 3. Comments. A large and rapid change in service delivery was seen in the NGO-managed district, and the new contract, which provided clear objectives and greater autonomy, further increased performance. No cost data are available. Although this study involved a controlled before-and-after design, confidence in the conclusions is weakened by reliance on reporting system data of unknown accuracy, availability of data on only a few indicators, short periods of observation, and com- parison of only two districts. (For more information see Lavadenz, Schwarb, and Hendrik 2001.) Description of Evaluated Contracting Experiences · 137 Table C.6 Changes over Time in NGO and Publicly Managed Districts in El Alto Percentage change phase 3 minus NGO district Public sector district phase 1 Phase Phase Phase Phase Phase Phase Public Parameter 1 2 3 1 2 3 NGO sector Deliveries 1,252 1,552 1,766 944 1,081 1,135 41 20 Bed occupancy 74 70 87 65 76 77 13a 12a rate (%) Outpatient visits 25,086 31,378 38,885 -- -- -- 55 -- Source: Based on Lavadenz, Schwarb, and Hendrik (2001). Note: -- = not available. a. Change in percentage points. Case 5. Afghanistan: Rural Primary Health Care C Background. When the Taliban fell in 2001, Afghanistan had some IXD of the worst health indicators in the world. The estimated under-five NEP mortality rate was 256 per 1,000 births, and the total fertility rate was AP 6.8 children per woman. Very serious inequities were also seen be- tween the urban and rural areas for outcomes and availability of ser- vices. The maternal mortality ratio was found to be 15 times higher in one province than in the capital of Kabul, and diphtheria/pertussis/ tetanus (DPT3) immunization coverage was 58 percent in urban areas but only 19 percent in rural ones. There was a serious shortage of health services in much of the country, and what few rural clinics ex- isted were run mostly by NGOs. There was a serious shortage of fe- male health workers, particularly in remote areas. To assert its stew- ardship role and rapidly improve services, the Ministry of Public Health (MOPH) decided to contract with NGOs to deliver a basic package of health services (BPHS). The latter included basic preven- tive, promotive, and curative services, such as immunization, prenatal care, skilled birth attendance, tuberculosis control, well-child care, and curative services for diarrhea and pneumonia. Description of Intervention. The MOPH, with funding from the World Bank, recruited international and local NGOs to provide the BPHS in eight provinces with about 4.1 million population. These service delivery contracts, known as performance-based partnership 138 · Performance-Based Contracting for Health Services in Developing Countries agreements (PPAs), focused on the delivery of services and offered performance bonuses to the NGOs if they achieved specified improve- ments in the quality of care and the coverage of services. The PPAs gave the NGOs considerable autonomy in how to spend the funds they received, but the NGOs were responsible for complying with the technical standards in the BPHS. In three other provinces, the MOPH signed management contracts with individuals to strengthen the health services provided by the MOPH itself. The MOPH also carefully assessed performance and provided performance bonuses based on actual accomplishments. In most other parts of the coun- try, other development partners, predominantly the U.S. Agency for International Development (USAID) and the European Union, pro- vided grants to NGOs to deliver the BPHS. In a few areas, services were provided by the MOPH with modest external support or by NGOs who did not follow the BPHS. Contracting Arrangements. The PPA NGOs were competitively C recruited based on the quality of their personnel, track record, quality IXD of their plans, and cost. The contracts were managed by a grants and NEP contracts management unit (GCMU) in the MOPH made up of com- petitively recruited, local consultants. The GCMU, despite managing AP other projects for the ministry, was able to pay the NGOs in a timely fashion and carry out regular monitoring visits. Evaluation Methodology. Health facility assessments were car- ried out each year in more than 600 facilities nationwide by a third party. Based on these assessments, the third party formulated an index reflecting many different aspects of the quality of care, including, for example, (1) availability of staff, equipment, and medicines; (2) qual- ity of patient-provider interaction; (3) staff knowledge; and (4) patient satisfaction. The maximum score on the index was 100. Results.Healthfacilitiesthatwerecontractedshowedlargeimprove- ments in quality of care (nearly an 18 percentage point improvement in two years), whereas facilities managed by the government (without contracting in of management) appeared to have experienced a worsen- ing in the quality of services they provided (nearly a 7 percentage point decline over the same period). The double difference on quality of care was 24.3 percentage points. There was a smaller difference between facilities with performance contracts compared with grants to NGOs with a double difference of 5.3 percentage points (see figure C.3). The change observed in the index of quality among the performance-based Description of Evaluated Contracting Experiences · 139 Figure C.3 Scores on a Broad Index of Quality of Care 80 70 score index 60 care of 50 quality 40 2004 2005 2006 other contracts performance contracts Ministry of Public Health itself Source: Author. Note: Maximum = 100. contracts was more than one baseline standard deviation, which repre- C sents a very large improvement. IXD Comments. Under difficult circumstances, performance contracts NEP were able to make a large improvement in the quality of care. Other AP types of contracts and grants were slightly less successful, but the government facilities performed least well. The contracting effort in Afghanistan now covers more than 20 million people out of a popu- lation of about 25 million. (For more information on this case, see Waldman, Strong, and Abdul 2006.) Case 6. Rwanda: Performance-Based Payments for Primary Health Care Background. After the 1994 genocide, Rwanda faced a difficult chal- lenge in rebuilding its health services. It readopted the district health system model in which provincial and district health management teams in the public sector were given complete responsibility for run- ning health services. In 2001, the annual public budget for health was $3 per capita, of which it is estimated only $1 made it to the frontline facilities, which generated 60­80 percent of their revenues from user fees. In 2001, a series of contracting efforts was started in different parts of the country by international NGOs. These NGOs acted as 140 · Performance-Based Contracting for Health Services in Developing Countries intermediaries ("fund-holders") and shifted the resources given to health centers from a fixed amount to one based on performance. Two of the contracting examples are described here. Description of Intervention. In early 2002, an international NGO working with funds from a bilateral agency began a "performance ini- tiative" in two health districts of Butare province with a combined population of 388,000 served by 19 rural health centers. These health centers had previously received a fixed budget, and per diems were paid to health workers regardless of their productivity (a "top-up"). Management committees in each health center signed contracts with the NGO that provided the health centers specified amounts based on the provision of five key services. The amount provided for each were as follows: · Institutional delivery, almost $5.00 · Antenatal care (as measured by tetanus toxoid, at least two doses), C $0.50 IXD NEP · Family planning new acceptor, $2.00 · Immunization (completion of measles and DPT3), $1.00 AP · Curative care consultations, $0.10 for each new one. The management committees had considerable autonomy in how they used the funds they received. The performance initiative cost an additional $0.24 per capita per year, of which 62 percent went to the health centers, 27 percent went in incentives to the district and provin- cial staff, and 11 percent covered transaction costs of the new project. In early 2003, a similar effort was also begun in Cyangugu province, which has a population of about 620,000 and is situated in the moun- tainous southwest of the country. The health centers were mostly gov- ernment owned, but about 40 percent were run by faith-based orga- nizations. In Cyangugu, the administrative costs of contracting were estimated to be around 25 percent of the total contracting costs. Contracting Arrangements. The approach is described as using a four-phase model in which during the first phase the health centers developed business plans and the international NGO purchased essen- tial services through a written contract with individual health centers or hospitals. During the second phase, the health centers and hospitals delivered services with considerable autonomy in personnel decisions Description of Evaluated Contracting Experiences· 141 and how to spend the funds they received. The third phase involved monitoring and evaluation by the district health management team, the international NGO, and the community. In the fourth phase, the contract was renewed (on a regular basis) based on feedback regarding outputs, quality of care, and patient satisfaction. Evaluation Methodology. The evaluation of the first contracting effort in Butare province was based on routine reporting system data, which was compared with similar data from the rest of the country. In Butare, district health staff and independent monitors examined registers in the health centers, and contractors faced stiff punishment if they exaggerated their reporting. The use of reporting system data allowed for a controlled before-and-after analysis, comparing inter- vention areas in Butare with the rest of the country (that is, rural areas). The published data on the Cyangugu case come from before- and-after household surveys carried out in early 2003 and late 2005. Results. As can be seen in table C.7, large improvements were achieved in the intervention areas of Butare province in almost all the C services designated in the contracts. The median double difference IXD was 12.9 percentage points, and the only double difference that was NEP negative was related to the number of new family planning acceptors. AP Table C.8 displays the results of before-and-after household surveys in Cyangugu province. In less than three years, the proportion of deliveries occurring in institutions with skilled providers more than doubled, consistent with the results from Butare. Comments. This case combined a controlled, before-and-after study (in Butare) based on health center records with a before-and- after study (in Cyangugu) based on household surveys. The results are consistent and demonstrate large improvements in health system performance through a creative contracting mechanism that involved NGOs that controlled the money (fund-holders), greater autonomy for health facility managers, and individual performance bonuses for health workers. It is possible that other interventions, in particular the advent of prepayment projects (mutuelles), influenced the outcomes. Also, it is not clear what happened to other indicators that were not included in the pay-for-performance contracts. The use of contracting and, more broadly, pay for performance has expanded considerably in Rwanda and will soon cover the entire country. (More details on this case are available in Meessen and others 2006 and Soeters, Habineza, and Peerenboom 2006.) 142 · Performance-Based Contracting for Health Services in Developing Countries Table C.7 Reported Data on Service Delivery in Butare Intervention Area Compared with the Rest of the Country (Rural Areas) Rest of the Rest of the Butare Butare country country Double Parameter 2001 2004 2001 2004 difference New consultations per 0.40 0.67 0.22 0.29 0.20 capita per year Deliveries at facility per 73.1 241.1 114.8 176.2 106.7 health center New family planning 22.2 88.4 80.5 146.9 ­0.2 acceptors per health center TT2+ coverage (%) 39.7 67.6 43.0 54.0 16.9 DPT3 coverage (%) 71.6 80.5 73.0 73.0 8.9 Measles coverage (% ) 69.1 77.6 67.0 69.0 6.6 Source: Author's calculations based on Meessen and others (2006). C Note: DPT3 = third dose of diphtheria/pertussis/tetanus vaccine; TT2+ = two or more doses of IXD tetanus toxoid among pregnant women. a. Single difference on a scale from 0 to 100. NEP AP Table C.8 Before-and-After Results of Household Surveys in Cyangugu Change Indicator Jan. 2003 Oct. 2005 (2005­03) Out-of-pocket expenditure per capita ($) 9.05 3.43 ­62% (7.3­10.8) (2.5­4.3) Deliveries occurring in institutions (%) 25 61 +36% (15­35) (49­71) Contraceptive prevalence rate (%) 5.4 11.6 +6% (3­8) (9­14) Unmet need for family planning (%) 23 11 ­12% (19­27) (8­14) Percentage of respondents knowing the risk of HIV 35 58 +23% transmission through skin piercing (29­41) (53­63) Source: Based on Soeters, Habineza, and Peerenboom (2006). Note: Numbers in parentheses represent 95 percent confidence intervals. Description of Evaluated Contracting Experiences· 143 Case 7. Costa Rica: Cooperatives Delivering Primary Care Background. Costa Rica established a social security institute (Centro Costaricense de Seguridad Social, CCSS) in the 1940s to provide uni- versal access to health services. Although the CCSS helped the country achieve some of the best health status indicators in Latin America, by the late 1980s the system faced serious challenges. Expenditures had risen to nearly 9 percent of the gross domestic product (GDP), waiting lists for treatment were often 12­18 months, and a changing epidemio- logical profile underscored the need to expand coverage and strengthen the primary care model. One of the reforms introduced to try to address these challenges was the establishment of worker-controlled health care cooperatives in facilities previously managed directly by the CCSS. Description of Intervention. In three primary care clinics that covered between 50,000 and 100,000 people, the health workers formed cooperatives that were autonomous legal entities that assumed respon- sibility for managing the existing clinic. They were obliged to provide C the same services as the publicly managed clinics, were not allowed to IXD charge fees, and could obtain inputs (such as drugs and medical sup- NEP plies) from the CCSS or from the market. Legally, all the workers in AP the cooperatives became shareholders, and the cooperatives had full autonomy in the use of funds received from the CCSS, contracting, and management of personnel and resources. In return for funding from the CCSS, the cooperatives were required to follow the guidelines and policy objectives of the CCSS and the Ministry of Health. They were responsible for providing general outpatient primary care, including pharmaceuticals, laboratory and radiology, minor surgery, some spe- cialized medical services, dental care, and emergency services. Contracting Arrangements. The cooperatives received payments based on the population of their catchment areas. This then shifted to the use of historical budgets, and in the end the cooperatives functioned under management contracts in which targets for service production and coverage were set. The cooperatives were obliged to provide annual financial reports to the CCSS for audit purposes but could retain all earnings and assumed all losses from their operations. The coopera- tives were required to evaluate their own performance, but the CCSS also appointed independent auditors to monitor the cooperatives. 144 · Performance-Based Contracting for Health Services in Developing Countries Evaluation Methodology. Data were available for 21 CCSS- managed clinics and three cooperatives from the annual statistical bulletin of the CCSS from 1990 to 1999. Demographic data were available from the census and from regular studies carried out by the CCSS. Hence, this was a time series analysis comparing the cooper- atives since their establishment with all the CCSS-managed clinics. Results. The cooperatives provided more general, specialist, and emergency visits per capita than the CCSS-managed clinics (see table C.9). At the same time, the cooperatives provided fewer medicines per visit, which likely reflects improved quality of care in a region where polypharmacy is seen as a widespread problem. A regression analysis that controlled for the socioeconomic conditions in the community demonstrated that the cooperatives provided 22 percent more general visits and 42 percent more dental visits per capita, while reducing the annual per capita expenditure by about $11. Comments. When worker-owned cooperatives took over CCSS- C managed clinics, this appears to have increased the level of service IXD delivery while reducing costs, implying a large increase in efficiency. NEP This pilot was done on a relatively small scale involving 150,000 to 300,000 beneficiaries. Unfortunately, limited data are available on tech- AP nical quality of care or patient perceptions. Nonetheless, the approach provides an interesting model for improving the performance of pub- licly financed health services. As a result of the experience of using Table C.9 Comparison of Cooperatives and CCSS-Managed Clinics, 1990­99 Indicator Clinic type 1990 1995 1999 Cooperative 1.45 1.77 1.84 General practitioner visits per capita CCSS 1.41 1.33 1.02 Cooperative 0.57 0.45 0.48 Specialist visits per capita CCSS 0.54 0.55 0.32 Cooperative 0.39 0.39 0.47 Emergency visits per capita CCSS 0.16 0.32 0.41 Medicines provided per Cooperative 1.67 2.03 2.08 patient visit CCSS 3.04 3.11 3.08 Source: Based on Gauri, Cercone, and Briceno (2004). Note: CCSS = Centro Costaricense de Seguridad Social. Description of Evaluated Contracting Experiences· 145 cooperatives, the CCSS has now transferred responsibility for other health facilities and health zones to other NSPs. (More information on this case is available in Gauri, Cercone, and Briceno 2004.) Case 8. Global Fund Support for AIDS, Tuberculosis, and Malaria Control Services Background. By the end of 2006, the Global Fund to Fight AIDS, Tuberculosis, and Malaria (GFATM) had provided $6.8 billion worth of grant financing to countries to control these diseases. Based on proposals received from country-coordinating mechanisms (CCMs), the Global Fund uses an independent technical review panel to assess whether the proposal should be funded. The most common "princi- pal recipients" of Global Fund resources are government ministries or national programs, church groups, NGOs, or the United Nations De- velopment Programme. Given the large amount of money disbursed C and the large number of grants the Global Fund has financed, the IXD performance of the principal recipients can be assessed. NEP Description of Intervention. The Global Fund uses a grant-mak- AP ing approach in which a broad array of groups can apply based on their perceptions of need. The CCM mechanism allows NGOs an oppor- tunity to submit proposals so that governments are not the only enti- ties that can apply. Although Global Fund financing should be seen as more of grant than a contract, there is a specific agreement negotiated on what is supposed to be achieved and how success will be measured. Also, the CCM ensures that there is a broad discussion of the needs of the specific country. Contracting Arrangements. Most of the grants are for five years, but after two years the Global Fund classifies the performance of the grants into four categories: (A) meeting or exceeding expectations, (B1) adequate, (B2) inadequate but potential shown, or (C) inadequate. The board of the Global Fund decides, based on the category, whether to continue the funding as is, continue the funding if the recipient meets certain conditions, continue the funding only if targets and budgets are substantially modified, or discontinue the grant. The categorization is based on (1) an appraisal of results comparing services delivered with targets specified in the grant agreement, (2) an assessment of progress on procurement and monitoring and evaluation, and (3) a review of the 146 · Performance-Based Contracting for Health Services in Developing Countries country context, including conflict and natural disasters. The Global Fund usually uses a local fund agent (LFA) to carry out financial man- agement, and the LFA is usually the local office of an international auditing firm. Evaluation Methodology. In 2006, the Global Fund had evaluated the first two years of 134 grants to individual countries. The perfor- mance of the grants was assessed according to the classification made by the Global Fund. A multivariate analysis was carried out examining a large number of predictor variables, including whether the principal recipients were a government or a nongovernmental entity, size of the grant, context of the country such as size of the economy, performance of the health system in the country, and a host of other variables. Results. Of the 134 grants studied, 21 percent were classified as A, 54 percent were classified as B1, and 25 percent were classified as B2 or C. Taking into account all the other factors studied, grants in which the principal recipients (PRs) were government ministries or C national programs were 16.7 percent less likely to be classified as A IXD than those in which the PRs were nongovernmental entities (p < 0.05). NEP Conversely, grants in which the PRs were government entities were 16.8 percent more likely to be classified as B2 or C (p < 0.01). The AP finding that programs with government principal recipients receive weaker evaluation scores is consistent with previous studies regarding the speed of disbursement. Comments. This case is essentially a cohort study of Global Fund proposals that were actually funded. Its strength comes from the rel- atively large number of grants examined. The authors of this study conclude that the results "should encourage countries to facilitate pro- grams with nongovernment actors alongside government programs." Although the Global Fund grants left program design up to the NSPs submitting proposals, the other aspects of the grants look like con- tracts as they are governed by specific agreements. (More details on this case are available in Radelet and Siddiqi 2007.) Case 9. Guatemala: Rural Primary Health Care Services Background. As part of peace accords ending many years of civil war, the government of Guatemala (GOG) was obliged to improve the de- livery of health services rapidly to the indigenous people who make Description of Evaluated Contracting Experiences· 147 up about half the population. The indigenous communities, many in mountainous areas, suffered from infant mortality rates that were much higher than the national average and had limited access to PHC services. The GOG concluded that it would be difficult to expand the government health care system rapidly to meet its obligations. This, combined with the indigenous community's lack of trust in the gov- ernment and the presence of a large number of NGOs working in the rural areas, led to a large-scale effort by the GOG to work with NGOs to improve PHC. Description of Intervention. The GOG, using funds from the Inter-American Development Bank, contracted with NGOs to deliver a defined package of PHC services focused on maternal and child health as well as communicable disease control. The services were to be provided to the mostly indigenous populations living in mountain- ous areas in the north of the country and used three models: (1) a service delivery type of contract (called the "direct" model) in which NGOs provided services themselves although they sometimes worked C with other NGOs who provided administrative support, (2) a man- IXD agement contract (or "mixed" model) in which an NGO administered NEP the health services within the existing MOH system, and (3) a control AP group (the "traditional" model), which was essentially the traditional mode of delivery via publicly operated clinics. The contracting effort began in 1997 and was considerably scaled up beginning in 1998 and continued to expand. It now covers about 27 percent of the entire pop- ulation, more than 3.5 million people. The contracts with the NGOs each covered an area with about 10,000 people and were fixed-price at $6.25 per capita per year. Contracting Arrangements. The initial groups of NGOs were selected based on their previous experience and presence in target areas, and without much competition. However, over time the process became more competitive and based on the quality of the technical proposals submitted by NGOs. By 2002, there were 160 contracts with 88 NGOs, and contract monitoring was done mostly at the district level. Payment of contractors was done centrally and, despite efforts to improve it, has remained an issue. A few NGOs decided not to partici- pate in subsequent rounds of contracting because of delayed payment. Evaluation Methodology. No baseline data were available for this intervention; however, three years after the contracting process began (in 2000), household surveys were carried out in randomly selected 148 · Performance-Based Contracting for Health Services in Developing Countries areas implementing the three different approaches. Unfortunately, the areas implementing the service delivery contracts appear to have been more isolated and had less physical access to services than the other experimental groups (see table C.10). The management contract and control areas sampled appeared to be quite similar. Results. Preliminary results from the household survey found that the management contracts achieved the best results in prenatal care, immunization, and receipt of oral rehydration salts (ORSs) by those young children with diarrhea (table C.11). The management contracts resulted in a median 11 percentage point (range from 5 to 16 per- centage points) difference in the coverage of services when compared with the control group (that is, traditional government provision of services). The success of the service delivery contracts is difficult to evaluate because they were implemented in more remote and difficult areas compared with the two other groups, and no baseline data are available by which to judge changes over time. C Comments. Contracting with NGOs using management contracts IXD appears to have been somewhat more successful than government pro- NEP vision of services; however, the lack of baseline data makes it difficult to be sure of the actual effect size. The effectiveness of the service AP delivery contracts is difficult to assess because they were implemented in more remote areas and no baseline is available. The rapid expan- sion of the contracting effort and its continuation despite a change of government suggest that the contracting approach is sustainable in Table C.10 Characteristics of the Areas Implementing the Different Approaches Used in Guatemala (Percent) Management Service delivery Household characteristic Control contract contract Mayan speaking 96 95 98 Illiterate 60 65 62 Households with MOH facility 70 80 44 less than 10 km away Households with MOH facility 11 8 24 more than 21 km away Source: Based on Danel and La Forgia (2005). Note: MOH = Ministry of Health. Description of Evaluated Contracting Experiences · 149 Table C.11 Results of the Different Approaches to Service Delivery in Guatemala (Percent) Management Service contract Difference delivery Parameter Control (MC) MC-control contract Coverage of prenatal care 75 87 12 78 Coverage of tetanus toxoid among 63 68 5 57 pregnant women Coverage of DPT3 immunization 69 80 11 63 among children Coverage of measles immunization 54 61 7 51 among children Coverage of oral rehydration 39 55 16 36 therapy among children with diarrhea Median difference from control 11 ­3 Source: Based on Danel and La Forgia (2005). C Note: DPT3 = third dose of diphtheria/pertussis/tetanus vaccine. IXD NEP this context. Contracting was also successful despite difficulties with AP contract management. (More information is available in Danel and La Forgia 2005 and La Forgia, Mintz, and Cerezo 2005.) Case 10. Haiti: Performance Bonuses for NGOs Providing Primary Health Care Background. After conflict and neglect had seriously diminished the government of Haiti's ability to deliver health services, in 1995 USAID began support to NGOs providing primary health care in rural areas. However, household surveys conducted in 1997 found a very large variation in the coverage of basic services provided by NGOs. For example, vaccination coverage varied from 7 to 70 percent of children 12­23 months of age, and expenditure per visit, which ranged from $1.35 to $51.93, was not correlated with performance. The unevenness in success on the ground led to a consideration of different approaches to paying NGOs, which had previously been paid on the basis of in- puts. Nongovernmental organizations had been reimbursed for actual 150 · Performance-Based Contracting for Health Services in Developing Countries expenditures up to a negotiated amount regardless of what they actu- ally accomplished. Description of Intervention. Nongovernmental organizations were offered the opportunity to receive 95 percent of their original contract amount in exchange for the opportunity to receive per- formance bonuses worth 10 percent of the contract amount if they achieved agreed-on targets for service delivery. The performance- based contracts were based on lump-sum payments rather than reim- bursements of actual expenditures. Seven specific indicators of service delivery were chosen, including vaccination coverage, coverage of pre- natal care (at least three visits), and use of oral rehydration therapy (ORT). A proportion of the bonus was provided to the NGO if it met the agreed-on target for each indicator (for example, 25 percent of the bonus if it achieved the target for immunization, or 20 percent if it achieved the prenatal care target). The targets were negotiated based on results of a baseline survey. Two indicators, one related to C waiting time in clinics and another to community participation and IXD coordination with the MOH, were subsequently dropped because NEP they either were not a valid indicator of performance or were difficult to measure. AP Contracting Arrangements. Over the course of a few years, most of the NGOs providing primary health care with USAID financing shifted to performance-based contracts. The change to performance- based contracts resulted in less administrative burden for the NGOs. It is not clear whether contract management was easier, but it cer- tainly meant a shift from tracking inputs to tracking outputs through improved monitoring. Payment of the NGOs was timely. Evaluation Methodology. Baseline data on coverage rates were available from a demographic and health survey in 2000. Subsequent data on the key indicators were obtained from the HMIS but were independently verified through a data audit. Results. As the percentage of NGOs being paid on a performance basis (offered performance bonuses if they achieved specified results) increased, the coverage of these services also increased (see table C.12). Comments. The effects of performance bonuses on NGO achieve- ments appear to have been quite positive. That there was a "dose response," that performance increased as the number of NGOs paid by performance-based financing increased, provides good evidence of the Description of Evaluated Contracting Experiences · 151 Table C.12 Performance of NGOs in Haiti as Use of Bonuses Increases (Percent) Indicator 2000 2002 2003 2004 2005 Children fully vaccinated 34 65 91 92 100 Women received at least three prenatal visits 29 50 41 48 60 Deliveries assisted by skilled attendant 58 64 57 63 77 NGOs paid on performance basis 0 35 37 44 93 Source: Based on Eichler and others (2006). Note: Data from 2000 from Demographic and Health Survey; data for 2002 to 2005 from audited management information system. effect of this approach. The change to lump-sum contracts could also have contributed to the improved performance. Performance-based con- tracts have now been expanded to the point where they cover about 1.3 million people. (For more information see Eichler and others 2006.) C IXD Case 11. Eight Countries: Tuberculosis Control Using Private Providers NEP AP Background. National tuberculosis programs (NTPs) in many devel- oping countries have concentrated on the provision of TB diagnosis and treatment in the public sector. However, a large proportion of patients, including the poor, seek care for TB symptoms from pri- vate health care providers. There has been concern that these private practitioners provide care of questionable quality at high cost to the patient. In an effort to improve TB diagnosis and management, an increasing number of NTPs are implementing "public-private" ini- tiatives, whereby they provide TB drugs to private practitioners in exchange for the latter complying with national standards of TB care. In half the 14 such initiatives evaluated thus far, NGOs have been used by the NTPs as intermediaries. The efforts of the NTPs provide an opportunity to look at two types of contracts: the so-called soft contracts with private providers and the more formal agreements that NTPs sometimes have with NSPs to act as intermediaries. Description of Intervention. In all the initiatives evaluated, pri- vate providers were trained in the NTP guidelines on diagnosis, treat- ment, and patient management. In 13 of the initiatives, drugs were 152 · Performance-Based Contracting for Health Services in Developing Countries provided free from the NTPs to the private practitioners on the con- dition that they (1) follow recommended diagnostic procedures and treatment regimens, (2) dispense the drugs free of charge to patients, and (3) follow national standards for referral, recording, and report- ing. In one initiative, drugs were provided through an NGO at a sub- sidized rate to the for-profit providers. In those situations in which NTPs used NGO intermediaries, the latter were usually responsible for sensitization, training, supervision, and monitoring of the individ- ual providers. The intermediaries included national branches of inter- national NGOs, a charitable hospital, a medial association, research institutions, and a national TB association. Contracting Arrangements. These "drugs for performance con- tracts" were always verbal in nature, although certificates or signposts stating that the provider had been "accredited" by the NTP were used in some cases. Direct monetary incentives were used in one initiative, and none of the initiatives used formal competitive tenders. The con- C tract could be ended either by the private provider or by the NTP (or IXD the intermediary NGO) withholding further drug distribution. In all NEP 7 of the 14 evaluated initiatives that involved NGOs as intermediaries, a formal agreement with the NTP defined the division of roles and AP responsibilities. Two had contractual agreements detailing payments made by the NTPs to the not-for-profit organizations, and the not- for-profit organizations contributed their own financial resources in four other cases. Evaluation Methodology. The 14 initiatives used different meth- odologies, but most involved before-and-after designs with informa- tion available from the routine recording and reporting system. In a number of cases, these data were reviewed by independent experts. Information was available on the number of new sputum smear posi- tive (NSSP) tuberculosis cases, NSSP treatment success, NSSP default rate, and change in the NSSP case detection rate. Results. Combining all the studies, the private practitioners were able to achieve a treatment success rate (that is, NSSP cases were treated in accordance with successfully completed directly observed therapy­short course [DOTS] treatment) of 89.6 percent (range 75 percent to 96 percent). The change in case detection rates attribut- able to the collaboration with private practitioners was evaluated in eight studies and ranged from 9 to 36 percent with a median of 16.5 percent (see table C.13). Description of Evaluated Contracting Experiences · 153 Table C.13 Results from 14 Evaluated Initiatives Number NSSP NSSP of NSSP treatment default Change in Involvement cases success rate NSSP case of NGO Name of project evaluated (95% CI) (95% CI) detection intermediary Damien Foundation, 14,035 90 NE NE Yes Bangladesh (90­90) Hyderabad, India 908 96 5 NE Yes (95­97) (4­6) Yogyakarta, Indonesia 386 75 NE NE No (71­79) Katmandu Valley, Nepal 309 92 1 15% Yes (89­95) (0­2) Mumbai, India 296 81 12 11% Yes (77­85) (8­16) Dhaka, Bangladesh 263 93 4 10% No (90­96) (2­6) Delhi, India 168 81 14 36% Yes C (75­87) (5­23) Mandalay, Myanmar 114 90 5 28% No IXD (88­93) (3­8) NEP Ho Chi Minh City, 107 61 34 18% No Vietnam (52­70) (25­43) AP Yangon (SQH Fran- 99 84 8 9% Yes chise), Myanmar (77­91) (3­13) Kannur, India 85 92 5 19% No (86­98) (0­10) Nairobi, Kenya 55 84 5 NE Yes (74­94) (0­11) Makati, Philippines 37 84 5 NE No (72­96) (0­12) Yangon (Shwepyitha), 32 88 3 NE No Myanmar (77­99) (0­9) Source: Based on Lonnroth, Uplekar, and Blanc (2006). Note: CI = contracting in; NE = not evaluated; NSSP = new sputum smear positive. Use of an NGO Intermediary. The seven initiatives in which NGO intermediaries were used achieved a NSSP treatment success rate of 90.1 percent (95 percent, CI: 89.6­90.5) compared with an 82.0 per- cent (95 percent, CI: 79.4­84.2) in those cases in which the NTP itself managed the collaboration with private providers. Similarly the default rate was significantly lower in the cases with NGO intermediaries than 154 · Performance-Based Contracting for Health Services in Developing Countries those cases without (6.4 percent versus 9.4 percent, respectively). There was not a statistically significant difference when it came to changes in case detection rates, but the data on this indicator cannot be calculated from all the studies. Comments. The results of this review indicate that working with private practitioners can achieve high rates of treatment success and increase case detection rates. Using an NGO intermediary appears to increase the chances of success and yields a higher treatment success rate. The 14 examples of working with private providers have been in place for between 3 and 12 years, and 12 of the initiatives have been continued. The only two that were not sustained were those run by NTPs themselves. More than 40 such initiatives are now being imple- mented in 15 countries, which suggests that the approach is spreading. (More information is available in Lonnroth, Uplekar, and Blanc 2006 and Murthy, Frieden, and Yazdani 2001.) C IXD Case 12. Pakistan: Rural Primary Health Care Services NEP Background. There is a widespread feeling in Pakistan that first-level AP facilities known as basic health units (BHUs) are providing only a lim- ited amount of services to the rural population despite the investment of large amount of resources in their construction, staff, equipment, and supplies. Based on ideas from a well-placed "champion" in the government of Punjab, the district government in Rahim Yar Khan (RYK) decided to have its BHUs managed by a local NGO. Description of Intervention. In a poorly performing district of Punjab, an NGO, the Punjab Rural Support Program, was given a management contract to run all the BHUs and considerable auton- omy to implement changes in organization and management. The NGO was also given the same amount of budget as had previously been allocated for the BHUs. The NGO quickly introduced innova- tions, including (1) recruiting managers at market salaries and holding them accountable for results; (2) addressing the shortage of doctors and high rates of absenteeism by increasing the salaries of selected doctors by 150 percent, who then had to cover three different BHUs instead of one; and (3) improving the supply of drugs available in the BHUs without increasing the budget. The RYK district comprised 104 BHUs and about 3.3 million people. Description of Evaluated Contracting Experiences· 155 Contracting Arrangements. The NGO was well known in the Punjab and signed a memorandum of understanding with the district government. It received a monthly tranche of funds based on the same annual allocation as had previously been provided to the local health authorities. The NGO had to provide the government with the same standard reports as providers in other districts. Evaluation Methodology. Before data collection, the indepen- dent evaluation team formed a hypothesis that curative services in the BHUs had improved as a result of the experiment but that preventive services would be little affected. The evaluation used a retrospective, controlled, before-and-after design, with a nearby, similar district, Bahawalpur (BWP), serving as the "control" area. An independent firm carried out health facility assessments in 20 randomly selected BHUs in RYK and another 20 BHUs in BWP. Household surveys were also carried out in the village closest to each BHU and in another village randomly selected from the BHU's catchment area. Results. Based on household survey results and HMIS data, it C appears that use of the BHUs was more than 50 percent higher in IXD RYK than in BWP (see figure C.4). Women and children under five NEP who were sick in the last month were significantly more likely to use AP BHUs in RYK than in BWP, and there was less use of unqualified providers (7 percent and 16 percent, respectively). There was no dif- ference in use of BHUs in the two districts among the poorest third of the population. Levels of community satisfaction also appear higher in RYK, with 40 percent of survey respondents indicating there had been an improvement in BHU services compared with the two years before the survey (before the NGO took over), whereas the comparable fig- ure was only 12 percent in BWP. The health facility survey found that the physical condition of BHUs in RYK was much better than those in BWP. Out-of-pocket expenditures for BHU services were found to be lower in RYK than in BWP. The coverage of preventive services, such as prenatal care and immunization, is low in both districts, and the rates of progress appear to be similar. This was expected because the management of female health workers, vaccinators, and other out- reach workers was not given to the NGO. The intention had been that the Punjab Rural Support Program (PRSP) would be given the same budget to operate the BHUs as had been allocated the previous year. In fact, the budget allocations between the year before PRSP took over (2002­03) and the year after 156 · Performance-Based Contracting for Health Services in Developing Countries Figure C.4 Use of BHU by Respondents Reporting Illness in the Last Month 40 37% 35 34% (%) 31% 30 first 25 24% BHU 22% 20 19% using 15 people 10 sick 5 0 all men women Rahim Yar Kahn Bahawalpur Source: Loevinsohn and others (2006). Note: All differences between districts are statistically significant (p 0.05); the differences between men and women in a district are never significant. C IXD NEP it took over (2003­04) were quite similar. The cost per patient visit in RYK was less than half of what it was in BWP. AP Comments. The original hypothesis, that the contracting approach would improve BHU services, appears to be confirmed. Contracting resulted in an improvement in BHU use, patient satisfaction, physi- cal condition of the BHUs, and reduced out-of-pocket expenditure for BHU services. These improvements were accomplished at lower absolute cost and a considerably lower cost per visit than in BWP. The RYK model has already been expanded to 12 other districts in Pun- jab at the request of their district governments (see figure C.5) with good results. The approach is now being expanded to all the other provinces in Pakistan. (More information on this case is available in Loevinsohn and others 2006.) Case 13. Madagascar and Senegal: Community Nutrition Services Background. Because of worsening economic conditions, the rates of severe and moderate malnutrition were increasing in Senegal and Madagascar, as in much of the rest of Sub-Saharan Africa in the 1990s. Both countries had seen very little involvement by the government in Description of Evaluated Contracting Experiences · 157 Figure C.5 Change in the Average Number of Outpatient Visits to BHUs per Month, Compared with Same Month of Year before in NGO (PRSP)-Managed Districts and Government-Managed Districts 600 500 400 of BHU 300 to number 200 visits 100 average in 0 outpatient ­100 change monthly ­200 ­300 ­400 -05 -05 C Aug-04 Sep-04 Oct-04 Nov-04 Dec-04 Jan-05 Feb-05 Mar Apr May-05 Jun-05 Jul-05 IXD PRSP districts non-PRSP districts Source: Loevinsohn and others (2006). NEP Note: For any given month, the number of visits in the same month of the year before is subtracted AP from the number of reported visits to calculate the change. PRSP = Punjab Rural Support Program. addressing nutrition issues, and the limited experience there to date had not been successful. To address worsening nutritional status, both governments wanted to work with NGOs to implement community- based nutrition interventions. Description of Intervention. In Madagascar and Senegal, the governments used World Bank and World Food Program resources to contract with NGOs to deliver community-based nutrition inter- ventions. The projects were launched in poor periurban areas of Sen- egal and rural areas in Madagascar and reached 457,000 and 490,000 women and children, respectively. Both projects provided monthly growth monitoring, weekly nutrition education to mothers, food sup- plementation for malnourished children, and referral to health ser- vices for unvaccinated children, pregnant women, and severely mal- nourished children. The cost per direct beneficiary was $48 in the Senegal project and $15 in Madagascar. In Senegal, the community 158 · Performance-Based Contracting for Health Services in Developing Countries nutrition activities were undertaken by more than 300 small groups of unemployed youths who received training and supervision from an NGO. In Madagascar, 50 NGOs were directly responsible for service delivery, and they hired local community nutrition workers to help in the villages. Contracting Arrangements. In Senegal, overall management of the project was carried out by Agence d'Exécution des Travaux d'Intérêt Public (AGETIP), a parastatal organization (although legally an NGO) with considerable experience in contract management in a number of sectors. AGETIP contracted with local NGOs, which trained the youth groups that provided the community services. Supervisors were hired mainly from among unemployed medical doc- tors. In Madagascar, the NGOs contracts were managed by a body in the office of the president. Both projects had to comply with MOH standards and policies. It appears that contract management was done well in both countries and that NGOs were paid on time. C Evaluation Methodology. In Madagascar, only project records IXD on participants were available. In Senegal, the project was evaluated NEP using a before-and-after design based on household surveys. There were also nonproject (control) communities in Senegal; however, these AP neighborhoods were "contaminated" by a large proportion of children who participated in the program. An independent study examined the coverage of monthly growth monitoring in one Senegalese city. Results. Before-and-after surveys in one project community in Sen- egal found that severe malnutrition among children 6­11 months old declined from 6 percent to 0 percent in 17 months, and moderate mal- nutrition among children aged 6­35 months declined from 28 to 24 per- cent. The independent study showed that monthly growth monitoring attendance was 72 percent in project neighborhoods compared with 35 percent in nonproject neighborhoods. The project records in both Sen- egal and Madagascar found that there was a 20 to 30 percentage point reduction in malnutrition rates among cohorts of project participants. Comments. The results of the before-and-after evaluation in Sen- egal suggest that the project may have had a modest effect on mal- nutrition rates as a result of effective implementation of community nutrition interventions. However, the absence of a control group makes it difficult to be certain about the actual effect size. From a con- tracting point of view, the NGOs appeared to have done a good job in Description of Evaluated Contracting Experiences· 159 implementing the program as designed; however, it is unknown how well the government would have done. Both governments continued the NGO nutrition efforts, albeit in different formats. One impor- tant aspect of the Senegal experience was the use of a large number of small groups of unemployed youths to deliver community services in their own communities. (More details on this case are available in Marek and others 1999.) Case 14: India: Improving Quality of Care from Private Providers in Bihar Background. Bihar suffers from the third-highest level of under-five mortality among the states in India. Much of it is due to common childhood diseases, such as acute respiratory tract infections (ARIs) and diarrhea. It is believed that 85 percent of cases of such childhood diseases are treated by private practitioners, many without formal training, and there is widespread concern that the quality of care pro- C vided by these practitioners is very poor. The government of India, IXD with assistance from the World Health Organization (WHO), devel- NEP oped guidelines for management of common childhood illnesses. A AP major challenge is how to get private practitioners to employ these guidelines and therefore improve the quality of care they provide. Description of Intervention. With funding from USAID, three local NGOs were recruited to identify local private practitioners who were treating childhood illnesses and provide them with training and follow-up aimed at improving case management based on WHO guidelines. The intervention took place in 110 villages with a popu- lation of 54,000 located in two districts of Bihar. The female liter- acy rate was less than 20 percent. Using community health workers (CHWs) to interview parents, 67 private practitioners were identified and provided support using a model called information feedback con- tracting and ongoing monitoring (INFECTOM), which comprised INformation on case management, FEedback on their current prac- tices, agreeing to an informal ConTract by which they would comply with the WHO case-management guidelines, and Ongoing Monitor- ing of their practices by the CHWs. The private providers received no remuneration or expenses for participating in the intervention. The cost of the intervention is estimated to be $15 per capita per year. 160 · Performance-Based Contracting for Health Services in Developing Countries Contracting Arrangements. Two types of contracts were used, one with the NGOs who provided training, outreach, and monitoring, and another series of "subcontracts" between NGOs and providers for changing their practice behaviors. The NGOs submitted applications for financing to USAID and were reimbursed on the basis of expenses. The "contracts" with the private practitioners were managed by the CHWs and the NGOs. Evaluation Methodology. This study used a before-and-after design in which household surveys, called verbal case reviews (VCRs), were undertaken with the parents of children under five who had ARI, diarrhea, or fever in the previous two weeks. At baseline, 600 VCRs were carried out by CHWs, and seven months after implementation of the INFECTOM intervention, a follow-up survey of 300 parents using the VCR was carried out. Results. A large improvement was seen in the management of childhood illnesses by private practitioners in the area covered by the C intervention, ranging from 25 to 57 percentage points on selected indi- IXD cators. As can be seen in table C.14, improvements were achieved in NEP both those behaviors private practitioners did well and those in which they did not before the training. AP Comments. This relatively small-scale before-and-after study tested an interesting approach in using NGOs to change the behav- iors of private practitioners. Although the changes were large and likely of practical importance, the short time frame, seven months, makes it difficult to know whether these behavior changes were main- tained for a significant period. (More details on this case are available in Chakraborty and Frick 2002.) Table C.14 Change in Private Provider Behaviors Before After Practitioner behavior intervention (%) intervention (%) Change (%) Used watch or timer to measure 14 71 57 respiratory rate Recommended ORS for diarrhea 16 48 32 cases Asked about history of illness 60 93 33 Touched child as part of examination 71 96 25 Source: Based on Chakraborty and Frick (2002). Note: ORS = oral rehydration salt. APPENDIX D Terms of Reference for a Third-Party Evaluation of the Contracting Effort in Country X Background This example is based on the contracting effort described in appendix A, which would serve as the background for these terms of reference. Pilot Test of Public-Private Partnership (PPP). The Govern- ment of Country X (GOX) is interested in evaluating whether this D type of PPP can achieve the following objectives: IXD · Significantly strengthen primary health care in primary health care NEP centers (PHCCs) and their associated catchment areas to ensure the AP widespread delivery of a standard package of preventive, curative, and promotive services that will help improve the health and well- being of Country X. · Dramatically improve the (1) coverage and use of services, (2) qual- ity of care, and (3) equity of access to services by geographical areas, income levels, and women and children. · Ensure that patients and communities are increasingly involved and satisfied with the publicly financed health services and facilitate the community's participation in the design, delivery, and evaluation of health services. 161 162 · Performance-Based Contracting for Health Services in Developing Countries A third-party firm or organization is required to design, conduct, and analyze the baseline and follow-up studies that will evaluate the PPP pilot test. Objectives and Design of the PPP Effort Indicators of Success. Achievement of the above objectives will be assessed by the indicators and targets described in table A4.1. By the end of the four-year period covered by PPP, it is expected that the sig- nificant progress will have been made toward the targets in table A4.1. These targets may be revised as data become available and are meant to be indicative rather than exact. What matters is significant progress along these parameters. Evaluation Methodologies. The achievement of the indicators in table A4.1 and certain other aspects of health care (such as community satisfaction, health expenditures, use of unqualified providers) will be assessed using the following methodologies: · Household surveys. In each of the 10 PPP lots and 10 comparison areas, household surveys will be carried out. The baseline survey will be done as soon as possible, with a midterm survey in July 2011, and the endline survey in June 2014. D IXD · Health facility assessments. An in-depth health facility assess- NEP ment will be carried out annually among a random sample of health facilities in PPP districts and comparison areas. The assessment AP will examine the quality of care, broadly defined, and other related indicators such as staff morale. · Health management information system. Health management information system data will be used by the provincial health de- partments to monitor, on a monthly basis, the performance of the PPP and the comparison PHCCs. Objectives of the Consultancy This consultancy aims to (1) design, conduct, and analyze the differ- ent types of studies needed for judging the effectiveness of the PPP in Terms of Reference for a Third-Party Evaluation of the Contracting Effort · 163 improving health services in Country X; (2) design the overall evalu- ation such that it provides the strongest evidence for decision makers; and (3) build the capacity of the Ministry of Health (MOH) to con- duct, analyze, and use such information. Table D.1 Key Performance Indicators and Targets Means of Approximate measuring Indicator Baselinea target indicator Number of consultations per person 0.3 1.0 HHS and per year provided by the PHC and its HMIS outreach activities Contraceptive prevalence rate: couples 9% 15% HHS of reproductive age currently using a modern family planning method TB case detection rate (number of 26% 45% HFA and sputum-positive cases detected as % of HMIS target based on estimated prevalence, that is, case finding) Children 6­59 months who have received 36% 55% HHS and vitamin A supplement within last 6 HMIS months Vaccination: measles immunization 28% 50% HHS and coverage before 12 months of age among HMIS D children 12­23 months of age IXD Coverage of antenatal care: pregnant 22% 50% HHS and women receiving at least one antenatal HMIS NEP care visit from a skilled provider AP Proportion of births attended by skilled 14% 25% HHS and attendants (includes institutional deliv- HMIS ery but excludes trained TBAs) Score out of 100 on an index of qual- 46 70 HFA ity of care as judged by third party, which includes the adequacy of waste management Improved equity: ratio of poorest to 0.42 0.60 HHS richest income quintiles (based on asset index) on number of consultations Source: Author. Note: HFA = health facility assessment; HHS = household survey; HMIS = health management information system; PHC = primary health care; TB = tuberculosis; TBA = traditional birth attendant. a. Baseline data are approximate and will have to be updated. 164 · Performance-Based Contracting for Health Services in Developing Countries Scope of Services The selected firm/organization (the consultant) will work closely with the GOX and under the direction of the GOX's task manager to carry out the following: · Carefully discuss the PPP pilot with the GOX staff and provide an evaluation design that provides the information necessary to judge the effectiveness of the PPP intervention. · Develop household survey instruments that build on international best practice in measuring use of health services, health outcomes, satisfaction with care, cost to consumers, and quality of care. · Also building on international best practice, develop the instru- ments and methods for a health facility assessment that examines quality of care broadly defined, and other related indicators, such as patient satisfaction and staff morale. · In consultation with the GOX, determine the sample sizes required to attain reasonable statistical power. · Ensure that the household and health facility instruments are dis- cussed with the GOX and subjected to peer review, after which they will be field-tested and modified accordingly. D IXD · Maintain the schedule agreed on during negotiations for carrying NEP out the baseline surveys expeditiously. AP · Ensure that surveyors are properly trained and supervised. Design and implement a quality assurance mechanism for data collection and entry acceptable to the GOX. · Enter and clean the data using best-practice techniques and prepare the data for analysis, including getting the data in a state where the data could be made publicly available on a Web site. · Analyze the baseline data and prepare a concise and user-friendly report for the GOX and other stakeholders, including the private sector partners. Carry out such additional analysis as the stakehold- ers reasonably request to make use of the data. · Repeat the household survey and conduct a health facility survey 18­24 months after the baseline evaluation using the same meth- Terms of Reference for a Third-Party Evaluation of the Contracting Effort· 165 odology and instruments. Although questions may be added to address key issues that arise during the study, only in unusual cir- cumstances would questions in the baseline surveys be dropped. · Analyze the endline data and prepare a concise and user-friendly report evaluating the whole experiment for the GOX and other stakeholders, including the private sector partners. Carry out such additional analysis as the stakeholders reasonably request to make use of the data. · Develop an explicit plan, acceptable to the GOX, for building the capacity of GOX staff and decision makers in the design, conduct, and analysis of household and health facility surveys. · Implement the capacity-building plan with due attention to quality and timeliness. · Present the data during meetings and workshops set up by the GOX to disseminate the results of the study. Prepare the results for publi- cation in the international health literature. · Carry out such tasks as the GOX's task manager reasonably requests to ensure efficient, high-quality, and expeditious design, collection, analysis, and dissemination of the data. D Location and Duration of Services IXD NEP The successful firm/organization must maintain a presence in Coun- AP try X during the baseline and endline evaluations. The exact districts have yet to be determined. It is expected that the consultancy will have a total duration of about 26 months. However, limited work will be needed between baseline and endline assessments. Qualifications and Experience The successful firm/organization will have the following qualifica- tions and experience: · A minimum of five years of experience in evaluating health or other social services, with extensive experience in developing countries 166 · Performance-Based Contracting for Health Services in Developing Countries · A successful track record of carrying out high-quality evaluations in developing countries · Staff who have published articles in peer-reviewed journals on eval- uation of health services · Staff with facility in English, both written and verbal · A demonstrated ability to get along easily with governments and NGOs and deal successfully with disagreements · A facility with computers and a variety of software · Knowledge of country X, which would be an advantage. Services and Facilities Provided by the GOX The GOX will take the following actions: · Provide the firm/organization with relevant information related to the consultancy, such as previous similar evaluations and existing survey instruments and · Ensure that its staff are available for periodic meetings as needed. D IXD NEP Consultant's Responsibilities AP It is the responsibility of the consultant to have its own computers. The consultant will also be expected to provide electronic and hard copies of all materials developed during the consulting assignment. Electronic files should be presented in formats used by common-use software. APPENDIX E Terms of Reference Examples Reader: This appendix provides examples of detailed terms of reference that can be used in contracts. The two specific examples deal with delivering primary health care and HIV prevention services among female sex workers. Example 1: Primary Health Care Background Health Status. The health status of the approximately 21 million citizens of Country Y is among the worst in the world. The infant mortality rate is thought to be 140 per 1,000 live births, and a recent study estimated the maternal mortality ratio to be 1,600 per 100,000 live births. The same study indicated that about 74 percent of mater- nal deaths could have been prevented and that distressingly few of the women who died had access to basic reproductive health services. E Only 12 percent had any prenatal care, 11 percent had received tetanus IXD toxoid, 5 percent were delivered by a skilled attendant, and 4 percent had ever used family planning. The nutritional status of children and NEP women is also very poor. AP Limited Access to Basic Health Services. A recent study of all existing health facilities indicated that most of the population of Country Y does not have access to the basic services that could make a large difference to their health. This has led to low levels of cover- 167 168 · Performance-Based Contracting for Health Services in Developing Countries age for important preventive, promotive, and curative services. For example, routine immunization coverage (diphtheria/pertussis/teta- nus) is estimated to be only 19.5 percent. Even that part of the popu- lation with access to health facilities is not receiving adequate services because of a lack of trained staff, particularly female staff, and equip- ment. Seventy-five percent of existing health facilities do not have trained female staff, and the uneven distribution of health workers can be extreme, with one doctor per 100,000 population in Province A and one per 1,000 population in the capital. Government Collaboration with NGOs. The Ministry of Health (MOH) recognizes and appreciates that nongovernmental organiza- tions (NGOs) are already helping deliver much of the primary and hospital health care available to the citizens of Country Y's popula- tion. The MOH would like to work collaboratively with NGOs and other private sector organizations to expand rapidly the availability of basic health services, improve the quality of care, build the capac- ity of citizens of Country Y health workers and managers to deliver the basic package of services (BPS), ensure greater equity in access to health services, ensure smooth coordination between the government and NGOs as well as among NGOs working in the same areas, and strengthen the citizens of Country Y. Performance-Based Agreements. To achieve these objectives the government, through the MOH, would like to enter into perfor- mance-based agreements (PBAs) with reputable NGOs, private sector firms, and other organizations (hereafter referred to, for convenience, as service providers) to deliver the BPS. The idea of the PBA is that the MOH will specify a set of basic services (the BPS) that the win- ning organization will be responsible for delivering in a specific geo- E graphical location, Province B. The performance of the organization IXD will be judged against progress on specific, measurable indicators that NEP will be evaluated regularly. Failure to make progress, and ultimately achieve these objectives, will be grounds for remedial action including AP termination of the contract, whereas there will be bonuses for good performance. The MOH aims to encourage successful innovation. Hence, exactly how these objectives are achieved will mostly be up to the organization selected so long as it complies with the technical guidelines, standards, and laws of the government. Terms of Reference Examples· 169 Objectives The service provider will be responsible for achieving the following objectives: · Foremost, ensure that the provision of the BPS in Province B is significantly expanded. The service provider is free to provide other services it feels are important (using funds from other sources) in the province, but it must provide all components of the BPS. · Ensure that the health services provided comply with the quality- of-care standards established by the MOH. · Pay careful attention to the needs of patients and communities and ensure that they are increasingly involved and satisfied with the publicly financed health services that are available. Facilitate the participation of communities in the design, delivery, and evaluation of health services. · Ensure that the managerial capacity of MOH staff to successfully coordinate, plan, implement, supervise, monitor, and evaluate the BPS at the provincial level is strengthened. · Build the technical capacity of citizens of Country Y health staff working in the province to deliver the BPS. Provide preservice training to health workers in line with the human resource devel- opment policy of the MOH. · Support the strengthening of the citizens of Country Y by helping ensure that the community views the services as being provided through government efforts (that is, that credit for service delivery is shared with the MOH). E · Ensure effective and efficient coordination of health activities in IXD Province B through the Provincial Public Health Coordination NEP Committee (PPHCC). AP · Deliver services throughout the province, in coordination with other partners through the PPHCC, to ensure equity of access be- tween social classes and among ethnic groups, and increase access to women and children under five. 170 · Performance-Based Contracting for Health Services in Developing Countries · Implement innovative interventions identified and agreed to by the MOH (such as diarrhea prevention through the safe water system or novel approaches to health care financing on a pilot basis to test their feasibility in a field setting in Country Y). · Ensure that health services are focused on the community by using community health workers (CHWs) and carrying out needed out- reach and satellite clinic activities. · Rationalize the number and distribution of health workers in the province in line with the MOH's human resource development policy. Indicators By the end of the three-year PBA period, it is expected that the targets described in table E.1 will be achieved. These targets may be revised based on data collected during the baseline household survey (HHS) and health facility assessments (HFAs). Using a standard formula, the various indicators will be combined to yield a single index of overall service provider performance. Table E.2 provides other management indicators for judging success of the service provider. Scope of Services Rapidly Expanding Coverage of BPS. The service provider selected by the MOH will be responsible for implementing the plan developed in concert with the provincial health department, including mobiliz- ing an adequate number of skilled health workers, providing services E to more isolated areas through community-based activities including IXD satellite clinics and outreach activities, and supporting CHWs. NEP Identifying Optimal Location of Health Facilities. As part of its efforts to expand coverage, the MOH will decide on the number of AP new first-referral hospitals, comprehensive health centers (CHCs), and basic health centers (BHCs) that a province requires (its "envelope") based on the number of existing facilities, its population, density, and geography. Within the envelope, the provincial health director (PHD) and the service provider will, using a consultative process, determine where exactly the new health facilities should be located and the prior- ity in which they should be built. Terms of Reference Examples · 171 Table E.1 Targets for the PBA Target for end Means of measuring Indicator Baselinea of project indicator Under-five mortality rate 250/10,000 20% reduction HHS Maternal mortality ratio 1600/100,000 10% reduction HHS Contraceptive prevalence 5.1% 15% HHS rate: women 15­49 years currently using a family planning method (modern) Treatment success rate 80% 85% HFA and HMIS among TB cases detected (cohort analysis) Children 6­59 months who 90% 90% HHS and HMIS have received vitamin A supplement within last 6 months DPT3 coverage among 19.5% 55% HHS and HMIS children 12­23 months Provider knowledge score 51% 70% HFA (% correct answers) Increase in health facilities 24.8% 80% HFA and HMIS with women health workers Number of consultations per 0.23 1.0 HMIS and HHS person per year Source: Author. Note: Targets are meant to be indicative and not exact. What matters is significant progress along these parameters in the project area. DPT3 = third dose of diphtheria/pertussis/tetanus vaccine; HFA = health facility assessment carried out by MOH with assistance from _____; HHS = household survey; HMIS = health management information system; TB = tuberculosis. a. These are nationwide estimates. Estimates for individual provinces have been calculated where possible. E Establishing First-Referral Hospitals, CHCs, and BHCs. IXD According to its plan, the service provider will have to rapidly estab- lish first-referral hospitals, CHCs, and BHCs using whatever build- NEP ings are available and appropriate. This may include renting tempo- AP rary facilities or houses. As part of establishing hospitals, the service provider will be responsible for ensuring that the province has a functioning referral system. Coordinating with MOH, the Community, and Other Service Providers. The PBA service provider will coordinate its activities with all stakeholders present in the province through PPHCC. 172 · Performance-Based Contracting for Health Services in Developing Countries Table E.2 Management Indicators for the PBA Baseline Means of measuring Indicator data Target indicator Coverage of antenatal care: coverage 4.6% 35% HHS and HMIS of all pregnant women receiving at least one antenatal care visit Pregnant women receiving at least 31.5% 60% HHS and HMIS two doses of tetanus toxoid Births attended by skilled attendants 6% 16% HHS and HMIS (excluding trained TBAs) TB case detection rate (number of 23% 50% HFA and HMIS sputum-positive cases detected as percentage of target based on estimated prevalence, that is, case finding) Children 6­18 months who received 67.8% 75% HHS breast milk and appropriate comple- mentary food in the last 24 hours Children 0­6 who were exclusively 82% 90% HHS breastfed in the last 24 hours Proportion of parents able to name -- 50% HHS and HMIS spontaneously the danger signs of diarrhea and ARI and the appropriate response Measles coverage among children 75.6% 85% HHS and HMIS 12­23 months Score out of 100 on the "balanced 44.8 55 HFA scorecard" that summarizes quality of care in BHCs and CHCs Decrease in variation between clus- -- 50% HHS ters in HHS for coverage of basic ser- decrease vices (that is, reduction in variance) from baseline E Number of CHWs per 1,500 population -- 2 HMIS IXD submitting monthly reports NEP Source: Author. Note: Targets are meant to be indicative and not exact. What will matter is significant progress AP along these parameters in the project area. -- = not available; ARI = acute respiratory tract infection; BHC = basic health center; CHC = comprehensive health center; CHW = community health worker; HFA = health facility assessment carried out by MOH with assistance from _____; HHS = household survey; HMIS = health management information system; TB = tuberculosis; TBA = traditional birth attendant. Terms of Reference Examples· 173 Training Health Workers and Ensuring That They Are in Place. In keeping with the MOH's human resource development plan and the BPS, the service provider will be responsible for ensur- ing that training and supervision of community midwives (CMWs) are effectively carried out (training may be done in existing accred- ited CMW schools, but the service provider is responsible for ensur- ing that an appropriate number of women from the cluster are being trained). Training and supervision of CHWs will also be effectively implemented. The service provider will also be responsible for ensur- ing that each health facility has at least one female staff trained to provide maternal and child health (MCH) services. Service provid- ers will, where appropriate, give priority to hiring existing MOH staff. Ensuring Competency of Health Staff. The service provider must ensure that all the health workers it employs have minimum competencies in the BPS as judged by tests carried out during the health facility assessments. The service provider will coordinate with the MOH human resource unit to decide how best to do this and how to carry out continuing education. Building Capacity of MOH Managers. The service provider will be responsible for building the capacity of MOH managers, particu- larly the provincial health director, and selected senior officials under the overall guidance of the central MOH. The service provider will be responsible for ensuring that these selected managers develop, at the least, the knowledge and skills needed for planning, budgeting, data analysis, monitoring, supervising, and evaluating health system performance. Recording and Reporting. The service provider will have to implement the MOH standard recording and reporting system, both E the existing health management information system (HMIS) and new IXD forms developed for particular vertical programs. Records will also have to be kept for community-based activities. The service provider NEP will be responsible for providing assistance to the provincial HMIS AP unit in data collection and analysis, provision of feedback, and transfer of data to central MOH offices. Promoting Special Health Activities. The service provider will have to participate in special MOH activities, such as national immu- nization days and other mass campaigns. In addition, the service pro- 174 · Performance-Based Contracting for Health Services in Developing Countries vider will have to respond appropriately to epidemics and other health emergencies and carry out other such activities in which the MOH is expecting all other provinces to be involved. Procuring Drugs, Supplies, and Basic Equipment. The service provider will be responsible for the procurement of essential drugs (on the MOH's essential drug list) and supplies of acceptable quality from reputable suppliers for the provincial hospital and CHCs and BHCs in keeping with the MOH's emerging pharmaceutical policy. The quality of the drugs might be tested on a random basis dur- ing the health facility assessments. The service provider will also be responsible for procuring the basic equipment (such as examining tables and stethoscopes) needed for the temporary CHCs and BHCs that need to be established in the province. Where MOH specifi- cations exist (for example, for furniture) the service provider will be responsible for ensuring that the procured furniture meets the specifications. Being Sensitive to the Needs of People with Disabilities. The service provider will ensure that disabled people can access health facilities and will make best efforts to recruit disabled people as CHWs and auxiliary midwives. Improving Waste Management in Health Facilities. The ser- vice provider will be responsible for implementing improved waste management procedures consistent with the MOH's environmental management plan. Location and Duration of Services E The above-mentioned services will be delivered to the entire popu- IXD lation in Province B. The services will be provided over four years, NEP starting approximately March 25, 2009. AP Compliance with Technical Guidelines In carrying out the services described above the contractor will com- ply with existing MOH technical guidelines and those that are devel- oped during the life of the PBA. Terms of Reference Examples· 175 Authority and Responsibilities of the Service Provider The Service Provider will enjoy sole discretion in the following areas: · The procurement of drugs, supplies, equipment, and other re- sources needed to meet contractual obligations · The use of resources purchased or provided under the PBA contract (however, the NGO will not refuse reasonable requests for the use of such resources by the PHD needed for implementing the BPS) · Hiring, firing, posting, remuneration, and customary managerial prerogatives over staff who are receiving payments from the NGO. The Service Provider is responsible for helping build the capacity of citizen of country Y health workers, including MOH staff. This may involve training or offering alternative opportunities to existing staff at the discretion of the Service Provider. The Service Provid- ers will have the authority to bar MOH staff from clinics that the Service Provider operates, if the Service Provider feels their presence interferes with effective service delivery. The Service Provider will not provide any payments, benefits, or other material resources to the PHD, his or her staff, or families. Any resources needed by the PHD will be provided through the central MOH. Authority and Responsibilities of Provincial Health Director The PHD has the authority to visit Service Provider's facilities at his or her discretion and to obtain such relevant information as to allow proper monitoring and supervision of the Service Provider and its subcontractors. The PHD may convene a meeting with the man- E agement of the Service Provider at any mutually agreeable time to discuss and resolve issues related to the PBA and its implementation. IXD The PHD also has the authority to review the quarterly report of NEP the Service Provider and provide additional comments to the central AP MOH. The PHD will have sole discretion over resources provided to the provincial health office by the central MOH, such as vehicles and communications equipment. However, the PHD will not refuse rea- sonable requests for the use of such resources by the Service Provider in implementing the PBA. The PHD will not seek from the Service 176 · Performance-Based Contracting for Health Services in Developing Countries Provider any payments, benefits, or other material resources for the PHD, his or her staff, or families. The PHD and key provincial staff may be provided a performance-based bonus, if the Service Provider achieves exceptionally good performance and is rewarded with a bonus under the PBA. Example 2: HIV/AIDS Prevention Services for Female Sex Workers Background. The HIV situation is changing rapidly in Pakistan. Re- cent evidence indicates a concentrated epidemic among injecting drug users (IDUs) across the country and among male sex workers (MSWs) and hijras (male transvestites) in Karachi. Although HIV prevalence among female sex workers (FSWs) is still below 1 percent, the evidence from various cities indicates high-risk behavior among this group, in- cluding low condom use and close interaction and overlap with other high-risk populations, such as IDUs and MSWs. The Sexually Trans- mitted Infection Survey found that nearly one-third of IDUs and 25 percent of MSWs in Lahore and Karachi had visited FSWs in the past month. Mapping studies further categorizes different subgroups among FSWs. Only a small proportion of FSWs are brothel based; a major- ity operate from residential localities (kothikhanas) or undertake street- based work, making access to these population groups more challeng- ing and difficult. Given the evidence of risky behaviors in Pakistan and the experi- ence of other Asian countries, the epidemic is almost certain to spread from IDUs to the commercial sex networks and possibly from there to the general population. The Pakistan Enhanced HIV-AIDS Pro- E gram that was launched in 2003 with assistance from the World Bank, IXD Canadian International Development Agency (CIDA), U.K. Depart- NEP ment for International Development (DFID), and other development partners includes a core component of support to targeted interven- AP tion for FSWs, IDUs, and other high-risk groups. The key challenge facing the Enhanced Program is to scale up HIV prevention services to high-risk populations, including FSWs, early enough to contain the epidemic. The Provincial AIDS Control Program, Department of Health, Government of Pakistan, intends to hire the services of an NGO/ organization for the delivery of a defined package of services for FSWs Terms of Reference Examples· 177 aimed at controlling and preventing the spread of HIV. The five-year provincewide contract will aim to cover all major cities of the prov- ince to ensure coverage of an estimated FSWs based on results of mapping studies or outbreak investigations become available. The contract will include provisions for expansion to other urban centers, thereby enabling a flexible response to the emerging epidemic. This approach will have the advantage of easier management with lower transaction costs, more effective use of technical assistance, easier attribution, and greater accountability. Existing contracts for female sex workers that end by December 2008 will fold into this province- wide contract. The contract will be a lump-sum contract and therefore output based rather then focused on inputs. The selected organization will have considerable autonomy in deciding service delivery mechanisms to achieve project objectives. Payments will be made primarily on the success of the organization in making progress toward the speci- fied targets measured annually by a third party. Other sources of data for judging progress will include the management information system and integrated behavioral and biological surveillance (IBBS). Achievement of results on the ground will be considered of primary importance. Objectives. The objective of this contract is to control and pre- vent the spread of HIV/AIDS in the FSW population in the Punjab. The contractor will deliver a defined package of services for primary health care, sexually transmitted infections (STIs), sexual and repro- ductive health, and behavior change services (described in the sub- sequent paragraphs) that will be provided to all categories of FSWs, including those operating in brothels and residential localities and as street-based workers. The work will be done in close coordination E with the Provincial AIDS Control Program and under the techni- IXD cal guidance of the National AIDS Control Program (NACP) during contract execution. Services will be implemented in accordance with NEP written NACP guidelines. AP The objectives to be achieved by June 2012 are that (1) HIV preva- lence remains below 5 percent among FSWs in the project area (moni- tored through HIV serosurveillance), (2) syphilis infection is reduced by half from baseline (assessed through HIV serosurveillance, and (3) condom use occurs in at least 60 percent of penetrative sex acts in the project area of FSWs. Table E.3 provides a complete list of indicators. 178 · Performance-Based Contracting for Health Services in Developing Countries Table E.3 Impact, Output, and Process Indicators for FSW HIV Prevention Services Contract Means of data Indicator collection Baseline value Approximate targeta Impact indicators HIV prevalence · IBBS 1% < 5% among FSWs Prevalence of syphilis · IBBS To be established Reduced by 50% among FSWs from the baseline Output indicators Sampled sex workers · IBBS To be established 60% using a condom dur- ing last penetrative sex act FSWs who can iden- · IBBS To be established 60% tify that condoms can · Third-party prevent STIs evaluation FSWs who score · IBBS To be established 60% above 80% on a stan- · Third-party dardized test of HIV evaluation knowledge Sex workers carrying · IBBS To be established 60% a condom · Third-party evaluation Process indicators FSWs reached · HMIS To be established 60% through counseling in contract area Sex workers who can · HMIS To be established 60% correctly demon- · Third-party strate the use of evaluation condom E IXD FSWs accessing STI · HMIS To be established 60% services · Third-party NEP evaluation AP Source: Author. Note: FSW = female sex worker; HMIS = health management information service; IBBS = integrated behavioral and biological surveillance; STI = sexually transmitted infection. a. Targets are to be achieved at the end of five years; when signing the contract, break down targets into yearly targets to ensure that the implementation is on track. Terms of Reference Examples· 179 Targets. Based on independent evaluations, FSWs in selected areas of Punjab, by the end of June 2012, will have acquired the following knowledge and skills: · Safer sex behavior: · Sixty percent of FSWs report using a condom during last pen- etrative sex act with client. · Sixty percent of FSWs carry a condom when working and can show it to an interviewer. · Sixty percent of FSWs report using a condom during last pen- etrative sex act with a regular sex partner. · Knowledge. Eighty percent of FSWs surveyed in the project area can spontaneously, without prompting, correctly identify the following: · Two ways that HIV infection is transmitted · Three ways to prevent transmission of HIV infection, including the use of condoms · That the use of condom can prevent the occurrence of other STIs · At least two sites for obtaining condoms · A local clinic or private doctor that provides treatment for STIs · The location of a center for voluntary confidential counseling and testing (VCCT) services. Scope of Services for FSWs. The implementing NGO will pro- vide the following package of services to the FSWs to enable them to improve their knowledge, attitudes, skills, behavior, and health: E · Establish safe and attractive drop-in centers at which training and other activities take place daily, with clinic facilities operating at IXD least twice per week NEP · Increase use of safer sex practices and improve healthcare-seeking AP behavior through behavior-change interventions implemented through a peer outreach program · Provide condom use and negotiation skills and ensure that condoms are widely available to the FSWs 180 · Performance-Based Contracting for Health Services in Developing Countries · Provide a selected set of primary health care and STI services based on syndromic approach using updated national guidelines · Promote empowerment and social development activities among FSWs · Promote an enabling environment to support project implementation · Provide education on sexual and reproductive health including STIs · Continuously strengthen the organization's capacity to deliver the above services. The design of the interventions will be developed on the basis of formative research behavioral studies. Formative research should be accomplished within three to four months and is essential to develop- ment of the project design. The qualitative assessment will provide insights into the life of sex workers and sex work dynamics, includ- ing issues related to personal partners; children; financial, health, and legal issues; and various forms of discrimination and victimization. Regular feedback and inputs from FSWs will be required through participatory design workshops to inform the development and review of the service delivery strategies. · Establish drop-in centers for the sex workers to gather and meet and provide a safe place for the following: · Group discussions, psychosexual counseling, advice, information, and support toward building a sense of community among FSWs. The approach should combine education with entertainment. · Skill building on proper condom use and disposal, negotiation skills, STI knowledge and recognition, interpersonal communi- E cation, and reducing drug and alcohol abuse. IXD NEP · Clinic facilities that operate at least twice a week. · Bringing FSWs together to develop self-help groups and empow- AP erment activities. · Implement behavior change intervention to increase safer sex and health-seeking behavior through an outreach program of peer educators: · Develop a program to meet health and safety needs of FSWs based on formative research with involvement of the FSWs. The Terms of Reference Examples· 181 program should incorporate effective peer education methodolo- gies, interpersonal communication strategies, field staff recruit- ment, and skill building. · Hire and train supervisors and peer educators from different subpopulations of FSWs. Ensure continued training and effec- tive supervision of peer educators. · Train peer workers to build skills of FSWs in proper condom use and disposal, negotiation skills, STI knowledge and recognition skills, personal communication skills with sexual partners, and reducing drug and alcohol abuse. · Ensure that materials and activities include education on condom skills, sexual and reproductive health including family planning and STIs, legal rights, and VCCT. Educational and skill-building material should be drafted with the aid of peer educators. · Review and revise strategy and activities based on project expe- rience and behavioral surveillance results, and in light of issues raised during implementation. · Provide a select set of primary curative services, including STI treatment: · Develop a detailed education and services delivery plan for sexu- ally transmitted infections based on discussions with FSWs. · Provide a select set of primary curative services, including for STIs and provision of condoms to FSWs through clinic services and drop-in centers. · Train local service providers who are used frequently by the FSWs in provision of services for STIs using a syndromic approach. E · Regularly review and monitor the quality of services for STIs used by the FSWs in the project area, using mystery clients, exit IXD interviews, and sex worker clinic guidance committees and sup- NEP port the improvement and maintenance of quality services. AP · Implement a referral system for specialist services. · Provide access to VCCT services: · Establish VCCT services so that FSWs have effective and appropri- ate access to a VCCT service package or refer or accompany them to existing VCCT centers where accessible. 182 · Performance-Based Contracting for Health Services in Developing Countries · Phase in introduction of antiretroviral medicines (ARVs) where VCCTs are available to sex workers. · Provide VCCT training to project staff to ensure accessible and acceptable services to FSWs and partners. · Monitor the experience of FSWs in accessing VCCT services and take remedial action to improve VCCT educational activities and testing facilities. · Refer or accompany HIV-positive cases to ARV treatment centers. · Provide condom distribution and skills in use and negotiation: · Ensure that condoms are easily available in the project area, including primary health care clinics and drop-in centers. · Promote condom use through free distribution of condoms through drop-in centers, peer educators and outreach workers, and local STI services. · Provide skills in condom use and disposal and negotiating con- dom use with clients through drop-in centers and peer education, and include this information in materials developed for behavior change intervention. · Review and revise condom education and distribution activities based on project experience and behavioral surveillance results. · Promote an enabling environment to support program imple- mentation: · Identify potential individuals and groups or others who could hinder progress of the project. Develop a plan to promote a more E positive environment for HIV prevention among FSWs in the IXD project area. NEP · Undertake advocacy and educational activities to promote under- standing of local police officials and other public sector officers AP toward the importance of HIV prevention and in working with FSWs for HIV prevention. · Create an advocacy plan and implementation that should include sex trade managers (madams, pimps) and other gatekeepers as necessary. Terms of Reference Examples · 183 · Monitor harassment and violence against sex workers by police and other local power brokers and take appropriate steps as needed. · Promote empowerment and social development activities among FSWs: · Initiate self-help groups of sex workers around primary social and economic needs, such as literacy and saving projects. · Develop referral systems for other key support activities, such as schooling for children, microcredit, and women's legal rights groups. Staffing. In addition to program staff, the NGO will be required to have at least the following full-time managerial staff on its payroll: project manager; monitoring and evaluation/research officer, financial officer, advocacy officer, and training officer. In addition, field manag- ers will be required at each intervention site. Monitoring Progress. The implementing NGO will provide quarterly progress report within 20 days after the end of each quarter of the project period. The primary means for judging progress will be the independent assessment of the appropriate indicators described above. In addition, and of secondary importance, the client will judge progress toward achieving the targets described above by examining whether the NGO is demonstrating progress toward accomplishing objective semiannual milestones, which are described below. In the case that data on the above-described indicators are not available, the Provincial AIDS Control Program will judge the progress based on information from the management information system, progress toward process milestones (see below), and appropriate field monitor- E ing. Any decision to terminate the contract or take other remedial action specified in the contract will be based on past progress of the IXD NGO; the existence of extraneous constraints, challenges, or impedi- NEP ments; a summary of all available quantitative information; and the AP latest results of behavioral surveillance and serosurveillance. Milestone one by the end of the first six months: · Senior project staff have been recruited and trained in the basic principles of HIV interventions for sex workers. 184 · Performance-Based Contracting for Health Services in Developing Countries · A specific staff member is delegated and trained to conduct advo- cacy for an enabling environment; an advocacy program is begun with police, sex trade managers, or other important gatekeepers. · At least a few active sex workers regularly advise project staff or are included as a staff member in a defined position that contributes to decision making. · Basic infrastructure, that is, transportation and main office, has been completed. · Peer education manuals are drafted, and criteria for recruitment of peer educators and their supervisors are developed. · A specific staff member is delegated and trained for monitoring and evaluation, and needed computer programs are installed and operating. · Baseline research has been undertaken and completed, a draft re- port submitted, and a completed report submitted no later than the end of month 6. Milestone two by the end of the year: · A participatory project design workshop has been held with sex workers (several may be needed if different types of sex workers as well as power holders are included) and options explored and dis- cussed, and the most feasible decided on collectively, including lo- cation of drop-in centers or other safe spaces for meetings and/or training of sex workers. · Staffing is completed based on size estimation results and acces- E sibility of sex workers. IXD · Knowledge and skills in the technical aspects of STI management NEP for FSWs and/or hijras are improved with appropriate technical AP assistance. · Infrastructure--that is, computer programs, clinics, safe spaces, and drug supplies--is secured and operating. · Peer education and peer supervisor training have begun with at least 40­50 peer educators graduated from the first three-week course. Terms of Reference Examples· 185 · All staff are trained in the principles and practices of behavior change interventions and nondiscrimination, including medical staff and auxiliary staff, such as drivers; this training should include issues relating to empowerment and social inclusion. · The process of bringing sex workers together to develop self-help groups has begun and specific "empowerment" activities selected, that is, literacy, savings projects, and microcredit. · Materials (printed, video, audio, musical) used in discussions among sex workers are developed in participatory workshops. · Monitoring and evaluation framework is completed, including in- dicators for coverage, exposure to intervention and changes in safer sex behaviors, STI treatment-seeking behaviors, and quality of STI care and effectiveness and of advocacy for an enabling environment, and at least three months of process data are collated and available. Duration of Contract and Geographical Spread of Services. The Provincial AIDS Control Program will sign the contact with the successful NGO, which will remain effective for a period of five years subject to satisfactory execution of the contract. The executing NGO will provide services to FSWs in major urban centers of the province as identified by the program. Coverage could be extended to other cit- ies in response to mapping studies or discovery of other "hot spots." Existing projects in Lahore and Multan will be incorporated into the project on completion of the contract in December 2008. Compliance with National and Provincial Guidelines. The executing NGO will follow national guidelines (current guidelines or those that will be developed during the period of contract execu- tion) for delivery of services to the FSWs. While procuring essential E drugs, including condoms, the executing firm will ensure that they IXD meet the specifications and standards laid down by the provincial Department of Health. Where such standards or specifications do NEP not exist, the specifications and standards laid down by the World AP Health Organization will govern. Items That Will Be Provided by the Government. The Depart- ment of Health, through its AIDS Control Program, will provide the following facilities to the successful NGO during the execution of the contract: 186 · Performance-Based Contracting for Health Services in Developing Countries · Social assessment study of high-risk groups and results of surveys, including IBBS · Reports of mapping studies of FSWs · Updated national guidelines for management of STIs, VCCT stan- dards, and ethical guidelines · Training of drop-in center staff in HIV rapid testing · Standard recording and reporting formats, to be developed jointly through mutual consultation · Authorization from the government to work with FSWs · Invitation to attend World AIDS Days, AIDS-related training, conferences, and key seminars with expenses for participation to be borne by the NGO · Copies of key reports and related research carried out in Pakistan · Support for in-training and capacity-building activities, whereas the NGO will bear the cost of travel and boarding/lodging for its participants · Access to public sector health services for FSWs registered with the NGO · Access to public sector HIV testing facilities · Access to ARV treatment centers. Recording and Reporting Requirements. The minimum record- ing and reporting requirements will be as follows: E · Maintenance of a daily log of activities by the NGO's staff (includ- IXD ing peer educators or outreach workers) in sufficient detail to allow NEP a review and assessment by the supervisory personnel AP · Maintenance of a log of the number of clients per day using the services and the regularity of clients in using services · Maintenance of stock registers to allow monitoring and reporting of shortages of essential commodities · Maintenance of a register of patients at the drop-in center and for Terms of Reference Examples· 187 VCCT services in sufficient detail to allow data analysis and in- terpretation, but maintaining the confidentiality of records from people not related to program management and implementation · Maintenance of income and expenditure statements of the project proceeds for an external annual financial audit, and provision of a copy of the audit report to the client or its representative within three months after the completion of a fiscal year (July 1­June 30), the financial audit to be used solely for determining whether the organization is financially viable. · Preparation of quarterly progress reports and their submission to the client and the management firm within 20 days after the com- pletion of every quarter, taking the contract signing as the refer- ence date; the quarterly progress report will provide at least the following information: · Progress made against the agreed on work plan · Progress made in achieving the agreed on semiannual process/ output target(s) · Challenges encountered and options used to resolved them · Relations with stakeholders like FSWs and their clubs (if any) · Unions/associations, local police, and community leaders. Accountability and Working Relationship. The NGO will be accountable to the Provincial AIDS Control Program for the satisfac- tory delivery of the services defined here. It will work in close collabo- ration with the National AIDS Control Program, the World Bank, other relevant development partners, and other NGOs working with FSWs. E IXD NEP AP References Abramson, Wendy. 1999. "Partnerships between the Public Sector and Non- Governmental Organizations: Contracting for Primary Health Care Ser- vices." Special Initiative Report 33, Abt Associates, Bethesda, MD. Bhushan, Indu, Sheryl Keller, and Brad Schwartz. 2002. "Achieving the Twin Ob- jectives of Efficiency and Equity: Contracting Health Services in Cambodia." ERD Policy Brief 6. Asian Development Bank, Manila. Bloom, Erik, Indu Bhushan, David Clingingsmith, Rathavuth Hong, Elizabeth King, Michael Kremer, Benjamin Loevinsohn, and J. Brad Schwartz. 2006. "Contracting for Health: Evidence from Cambodia." http://www.brookings .edu/views/papers/kremer/20060720cambodia.pdf. Bonu, Sekhar. 2006. "Contracting Out Primary Health Care to NGOs." Presenta- tion made at Johns Hopkins University. Bryce, Jennifer, S. el Arifeen, G. Pariyo, C. Lanatar, D. Gwatkin, and J. P. Hap- icht. 2003. "Reducing Child Mortality: Can Public Health Deliver?" Lancet 62: 159­64. Chakraborty, Sarbani, and Kevin Frick. 2002. "Factors Influencing Private Health Providers' Technical Quality of Care for Acute Respiratory Infections among Under-Five Children in Rural West Bengal, India." Social Science and Medicine 55 (9): 1579­87. Danel, Isabel, and Gerard La Forgia. 2005. "Contracting for Basic Health Care in Rural Guatemala: Comparison of the Performance of Three Delivery Models." In Health Systems Innovations in Central America: Lessons and Impact of New Ap- proaches, ed. Gerard La Forgia, 49­88. Washington, DC: World Bank. Domberger, Simon. 1999. The Contracting Organization: A Strategic Guide to Out- sourcing. Oxford, U.K.: Oxford University Press. 189 190 · References Eichler, Rena, Paul Auxila, Antoine Uder, and Bernateau Desmangles. 2006. "Going to Scale with Performance Based Payment: Six Years of Results in Haiti." Work- ing Paper 121, Center for Global Development, Washington, DC. http://www .cgdev.org/content/publications/detail/13543/. England, Roger. 1997. Contracting in the Health Sector: A Guide to the Use of Contract- ing in Developing Countries. London: Institute for Health Sector Development. Frick-Cardelle, Alberto José. 2003. Health Care Reform in Central America: NGO- Government Collaboration in Guatemala and El Salvador. Miami, FL: North- South Center Press. Gauri, Varun, James Cercone, and Rodrigo Briceño. 2004. "Separating Finance from Provision: Evidence from 10 Years of Partnership with Health Coopera- tives in Costa Rica." Health Policy and Planning 19 (5): 292­301. Green, Andrew T. 1987. "The Role of Non-Governmental Organizations and the Private Sector in the Provision of Health Care in Developing Countries." Inter- national Journal of Health Planning and Management 2: 40. Hebrang, Andrija, Neven Henigsberg, Viktorija Erdeljic, Sarlota Foro, Vinko Vi- djak, Ante Grga, and Tomislav Macek. 2003. "Privatization in the Health Care System of Croatia: Effects on General Practice Accessibility." Health Policy and Planning 18 (4): 421­28. Karim, R., S. A. Lamstein, M. Akhtaruzzaman, K. M. Rahman, and N. Alam. 2003. "The Bangladesh Integrated Nutrition Project: Endline Evaluation of the Community-Based Nutrition Component." University of Dhaka, Bangladesh; and Tufts University, Boston, MA. Koppel, Agris, Kertsi Meiesaar, Hannu Valtonen, Andrus Metsa, and Margus Lem- ber. 2000. "Evaluation of Primary Health Care Reform in Estonia." Social Sci- ence and Medicine 56 (12): 2461­66. La Forgia, G., P. Mintz, and C. Cerezo. 2005. "Is the Perfect the Enemy of the Good? A Case Study on Large Scale Contracting for Basic Health Services in Rural Guatemala." In Health Systems Innovations in Central America: Lessons and Impact of New Approaches, ed. Gerard La Forgia, 9­48. Washington, DC: World Bank. Lavadenz, Fernando, Nicole Schwarb, and Straatman Hendrik. 2001. "Redes públi- cas, descentralizadas y comunitarias de salud en Bolivia." Pan-American Journal of Public Health 9 (3): 182­89. Lewis, M. 2006. "Governance and Corruption in Public Health Care Systems." Working Paper 78, Center for Global Development, Washington, DC. http:// www.cgdev.org/content/publications. References · 191 Liu, X., D. R. Hotchkiss, and S. Bose S. 2008. "The Effectiveness of Contracting Out Primary Health Care Services in Developing Countries: A Review of the Evidence." Health Policy and Planning 23 (1):1­13. Liu, Xingzhu, and Anne Mills. 2002. "Financing Reforms of Public Services in China: Lessons for Other Nations." Social Science and Medicine 54 (11): 1691­98. Loevinsohn, Benjamin. 2000. Contracting for the Delivery of Primary Health Care in Cambodia: Design and Initial Experience of a Large Pilot-Test. World Bank Institute Flagship Program Online Journal. http://www.worldbank.org/wbi/ healthflagship/journal/index.htm. Loevinsohn, Benjamin, E. T. Guerrero, and S. P. Gregorio. 1995. "Improving Pri- mary Health Care through Systematic Supervision: A Controlled Field Trial." Health Policy and Planning 10 (2): 144­53. Loevinsohn, Benjamin, Inaam Haq, Agnes Cuffinhal, and Aaka Pande. 2006. "Part- nering with NGOs to Strengthen Management: An External Evaluation of the Chief Minister's Initiative on Primary Health Care in Rahim Yar Khan Dis- trict, Punjab." World Bank, Washington, DC. Loevinsohn, Benjamin, and April Harding. 2005. "Buying Results? A Review of Contracting for Health Service Delivery in Developing Countries." Lancet 366: 676­81. Loevinsohn, Benjamin, Rathavuth Hong, and Varun Gauri. 2006. "Will More In- puts Improve the Delivery of Health Services? Analysis of District Vaccination Coverage in Pakistan." International Journal of Health Policy and Management 21: 45­54. Lonnroth, Knut, Mukund Uplekar, and Léopold Blanc. 2006. "Hard Gains through Soft Contracts: Productive Engagement of Private Providers in Tuberculosis Control." Bulletin of the World Health Organization 84: 876­83. Mahmud, Hasib, Amanat Ullah, and Salahuddin Ahmed. 2002. Mid-Term Health Facility Survey: Urban Primary Health Care Project. Dhaka, Bangladesh: Mitra and Associates. Marek, Tonia, Issakha Diallo, Biram Ndiaye, and Jean Rakotosalama. 1999. "Suc- cessful Contracting of Prevention Services: Fighting Malnutrition in Senegal and Madagascar." Health Policy and Planning 14: 382­89. Meessen, Bruno, Laurent Musango, Jean-Pierre Kashala, and Jackie Lemlin. 2006. "Reviewing Institutions of Rural Health Centers: The Performance Initiative in Butare, Rwanda." Tropical Medicine and International Health 11 (8): 1303­17. Mills, Anne. 1998. "To Contract or Not to Contract? Issues for Low and Middle Income Countries." Health Policy and Planning 13: 32­40. 192 · References Murray, Christopher, Bakhuti Shengelia, Neeru Gupta, Saba Moussavi, Ajay Tan- don, and Michel Thieren. 2003. "Validity of Reported Vaccination Coverage in 45 Countries." Lancet 362: 1022­27. Murthy, K. J. R., T. R. Frieden, and A. Yazdani. 2001. "Public-Private Partnership in Tuberculosis Control: Experience in Hyderabad, India." International Journal of Tuberculosis and Lung Disease 5: 354­59. Palmer, Natasha. 2000. "The Use of Private Sector Contracts for Primary Health Care: Theory, Evidence, and Lessons for Low-Income and Middle-Income Countries." Bulletin of the World Health Organization 78: 821­29. Pfeiffer, James. 2003. "International NGOs and Primary Health Care in Mozam- bique: The Need for a New Model of Collaboration." Social Science and Medicine 56 (4): 725­38. Radelet, Steve, and Bilal Siddiqi. 2007. "Global Fund Grant Programmes: An Analy- sis of Evaluation Scores." Lancet 369: 1807­13. Robertson, Susan, and Joseph Valadez. 2006. "Global Review of Health Care Sur- veys Using Lot Quality Assurance Sampling (LQAS), 1984­2004." Social Sci- ence and Medicine 63: 1648­60. Savas, Emanuel 2000. Privatization and Public Private Partnerships. New York: CQ Press. Schwartz, Brad, and Indu Bhushan. 2003. "Reducing Inequity in the Provision of Pri- mary Health Care Services: Contracting in Cambodia." http://info.worldbank .org/etools/docs/library/114527/RTPmaterials/Reaching%20the%20Poor/ Session%206A-Abstracts/Schwartz%20and%20Bhushan%20.pdf. ------. 2004. "Improving Equity in Immunization through a Public-Private Part- nership in Cambodia." Bulletin of the World Health Organization 82 (9): 661­67. Slack, Katherine, and William Savedoff. 2001. Public Purchaser­Private Provider Con- tracting for Health Services: Examples from Latin America and the Caribbean. Publi- cation SOC-111. Washington, DC: Inter-American Development Bank. Soderlund, N., P. Mendoza-Arana, and J. Goudge, eds. 2003. The New Public/Pri- vate Mix in Health: Exploring the Changing Landscape. Geneva: Global Forum for Health Research. Soeters, Robert, and F. Griffiths. 2003. "Improving Government Health Services through Contract Management: A Case from Cambodia." Health Policy and Planning 18: 74­83. Soeters, Robert, C. Habineza, and P. B. Peerenboom. 2006. "Performance-Based Fi- nancing and Changing the District Health System: Experience from Rwanda." Bulletin of the World Health Organization 84: 884­89. References · 193 Sorensen, R., and J. Grytten. 2002. "Service Production and Contract Choice of Primary Physician." Health Policy 66: 73­93. Stansfield, Sally K., Julia Walsh, Ndola Prata, and Timothy Evans. 2006. "Informa- tion to Improve Decision Making for Health." In Disease Control Priorities in Developing Countries. 2nd ed. Ed. D. T. Jamison and others. Washington, DC: World Bank. http://www.DCP2.org. Transparency International. 2006. Global Corruption Report 2006. http://www .transparency.org/publications. Turshen, Meredith. 1999. Privatizing Health Services in Africa. New Brunswick, NJ: Rutgers University Press. Waldman, Ronald, Leslie Strong, and Wali Abdul. 2006. "Afghanistan's Health Sys- tem since 2001: Condition Improved, Prognosis Cautiously Optimistic." AREU briefing paper, Afghanistan Research and Evaluation Unit, Kabul. December. http://www.areu.org.af. World Health Organization. 2000. World Health Report 2000--Health Systems: Improv- ing Performance. Geneva: WHO. World Bank. 2004. World Development Report. Washington, DC: World Bank. Index Boxes, figures, maps, notes, and tables are as step in contracting process, indicated by "b," "f," "m," "n," and "t." 5­6, 55­64, 55t Bolivia, urban PHC services absorptive capacity constraints, contracting in, 75t, 135­37, overcoming, 83 137t accountability and autonomy, budgeting and finance managerial, 5, 55­56, 83 cost concerns about contracting, 84 Afghanistan management of contracts, 54, 115­16 community focus on service delivery monitoring and evaluation, in, 93 budgeting funds for, 44 decentralizing drug procurement P4P,. financial risk in, 33 in, 59b selection process affected by, 50­51 rural PHC services contracting in, 76t, 137­39, 139f Cambodia AIDS. See HIV/AIDS health workers in underserved areas, auditors, independent and private, 57b 62­63 rural PHC contracting in, 74t, autonomy and accountability, 125­30, 127­28f, 129t managerial, 5, 55­56, 83 capacity building absorptive capacity constraints, Bangladesh overcoming, 83 NTPs serviced by private providers for contractors, 63 in, 153t economies of scale in, 35 rural nutrition services contracting, for health workers, 63 74t, 130­33, 132t NSPs, uncertain capacity of, 36 urban services contracting, 75t, CBOs (community-based 133­35, 145­45t organizations). See bidding process nongovernmental carrying out, 6, 64­66, 66t organizations drafting of documents for CCMs (country-coordinating stakeholders, sharing drafts mechanisms) used by with, 21 GFATM, 145 195 196 · Index champions, identifying, 21 Costa Rica checklists cooperatives delivering PHC in, 24b, contracting, xiii, 67­68 77t, 143­45, 144t SCs, 42 government provision of services collection of data, 4, 8­39t, 37­44, in, 81 40­41b costs. See budgeting and finance community country-coordinating mechanisms addressing concerns of, 25 (CCMs) used by GFATM, 145 service delivery, focus on, 93 Croatia, government services compared as stakeholder, 20 to contractors in, 73b community-based organizations Cuba, government provision of services (CBOs). See nongovernmental in, 81 organizations comparison/control groups for data data collection process, 4, 8­39t, 37­44, collection and analysis, 43­44 40­41b competition, constructive, 82 decentralization of procurement, 59b, competitive selection process, 4, 44­46 68­59 computerization of management defining services to be provided, 9 function, 54 equity, 30­31, 32 concerns about contracting, 83­93, objectives of contract 86­87t defining, 26­30 cost concerns, 84 focusing purchaser and equity, 84­85 contractor on, 33­34 foreign NSPs, influence of, 92 P4P, 31­33, 32b government, role of, 90­92, 91t performance indicators, 27­29t, long-term prospects, 88­90 27­30, 37, 38t management concerns, 90 PHC services, TORs for, 170, 174 NGO direction and focus, 92­93 quality, 30­31, 32 postconflict situations, 92­93 in sample contract, 110­12 quality of care, 93 in sample plan, 100­101 scale of contracted services, 84 scope of services. See scope of sustainability, 85­88 services, defining Congo, Democratic Republic of, size and location of contract lots, governmental personnel defining, 34­36, 165, 170, management in, 57b 174, 185 contracting. See performance-based as step in contracting process, 3­4, health services contracting 26­36, 26f contracting parties definition of contracting, xi, 1­2, 9­11 authority of, clarifying, 56 delivery of services. See services to be protecting interests of, 5­6 provided responsibilities of. See Democratic Republic of Congo, responsibilities of contracting governmental personnel parties management in, 57b control/comparison groups for data development partners (DPs) and donors, collection and analysis, 20, 25­26 43­44 differentiation of contracting from corruption, 22, 35, 82 other methods of organizing cost issues. See budgeting and finance health services, 2, 9­10 Index · 197 directly observed therapy-short course female sex workers (FSWs), health (DOTS) for TB, 152 care services for, 13­15, dispute resolution mechanisms, 61, 116 176­87, 178t documents financial issues. See budgeting and drafting of finance stakeholders, sharing drafts foreign NSPs, concerns over influence with, 21 of, 92 as step in contracting process, FSWs (female sex workers), health 5­6, 55­64, 55t care services for, 13­15, sample contract, 107­18 176­87, 178t sample plan, 97­106 donors and DPs (development partners), GFATM (Global Fund to Fight AIDS, 20, 25­26 Tuberculosis, and Malaria), DOTS (directly observed therapy-short 77t, 145­46 course) for TB, 152 Ghana, health care services for FSWs double difference, 71, 95 in, 13­15 DPs (development partners) and donors, Global Fund to Fight AIDS, 20, 25­26 Tuberculosis, and Malaria drafting of contract and bidding (GFATM), 77t, 145­46 documents global study of contracting. See whether stakeholders, sharing drafts with, 21 to contract, global study of as step in contracting process, 5­6, government 55­64, 55t coordination with, 173 duration of contract, 59­60, 165, key issues, contracting allowing 174, 185 focus on, 93 local. See local government ECO (Electronic Collections Online), MOHs, 53­54 71 political and bureaucratic opposition Econ-Lit, 71 to contracting by, 21­24, 94­95 economic issues. See budgeting and in postconflict situations, 92­93 finance provision of services by, 72, 73b, 81 economies of scale, 34­35 role of, concerns over, 90­92, 91t Electronic Collections Online, (ECO), government health officials 71 addressing concerns of, 21­23 equity, 30­31, 32, 84­85 in management structure, 53­54 Estonia, government services compared personnel management by, 57b to contractors, 73b as stakeholders, 20 evaluated contracting experiences, case government health workers studies in. See whether to addressing concerns of, 23­24 contract, global study of management of, 56­58, 57b evaluation of contracted services. See as stakeholders, 20 monitoring and evaluation grants process differentiated from contracting, 2, evaluation of potential contractors. See 13­15 selection process GFATM support for control services, 77t, 145­46 faith-based organizations (FBOs). Guatemala, rural PHC services See nongovernmental contracting in, 78t, 146­49, organizations 148­49t 198 · Index Haiti, performance bonuses for NGOs monitoring and evaluation process, providing services in, 32b, 78t, designing (step 3), 4, 37­44, 37t 149­51, 151t performance indicators. See health care services performance indicators performance-based contracting for. plan, development of (step 5), 5, See performance-based health 54­55 services contracting selection process, determining (step PHCs. See primary health care 4), 4­5, 44­51, 44t (PHC) services stakeholders, dialogue with (step 1), types of services contracted for, 1, 9 2­3, 20­26, 66 health facility assessments (HFAs), 40­42 steps in, 2­6, 5f, 20f health management information (HMI) hybrid selection mechanisms, 51 systems, 37­38, 40b, 112 health workers independence capacity, building, 63 of auditors, 62­63 governmental. See government of selection process, 47­48 health workers India management of, 56­58, 57b, 63, government provision of services 173, 183 in, 81 in underserved areas, 57b NTPs serviced by private providers HFAs (health facility assessments), in, 153t 40­42 quality of care from private HHSs (household surveys), 38, 41b providers, improving, 80t, HIV/AIDS 159­60, 160t contracting for services related to, indicators of success. See performance 1, 11, 12 indicators GFATM, 77t, 145­46 Indonesia, NTPs serviced by private grants differentiated from providers in, 153t contracting, 13­15 infrastructure, responsibility for, 64 LOAS and, 41b interests of contracting parties, selection process for contracting protecting, 5­6 services for, 5 TORs for FSW services, 176­87, 178t Kenya, NTPs serviced by private HMI (health management information) providers in, 153t systems, 37­38, 40b, 112 household surveys (HHSs), 38, 41b length of contract, 59­60, 165, 174, 185 how to contract, xiii, 2­6, 19­20 LOAS (lot quality assurance bidding process. See bidding process sampling), 41b defining services (step 2), 3­4, local government 26­36, 26f. See also defining addressing concerns of, 23 services to be provided in management structure, 53 drafting documents as stakeholders, 20 stakeholders, sharing drafts location and size of contract lots, with, 21 defining, 34­36, 165, 170, as step in contracting process 174, 185 (step 5), 5­6, 55­64, 55t long-term prospects, concerns about, management of contracts 88­90 arranging for (step 5), 5, 52­55, 52t lot quality assurance sampling carrying out (step 7), 6, 64­66, 66t (LOAS), 41b Index · 199 lots in contract, defining location and data collection process, 4, 8­39t, size of, 34­36, 165, 170, 174, 185 37­44, 40­41b lump-sum contracts, 58 designing, 4, 37­44, 37t economies of scale in, 34­35 Madagascar, community nutrition HIV/AIDS services for FSWs, services contracting in, 80t, TORs for, 183­85 156­59, 157f management by, 65 malaria control services, GFATM performance indicators. See support for, 77t, 145­46 performance indicators management of contracts in sample contract, 110 arranging for, 5, 52­55, 52t in sample plan, 101 auditors, independent and private, Myanmar, NTPs serviced by private 62­63 providers in, 153t autonomy and accountability, maximizing, 5, 55­56, 83 national tuberculosis programs (NTPs), budgeting and financial 151­53 management, 54, 115­16 Nepal, NTPs serviced by private capacity of contractors and staff, providers in, 153t building, 63 new sputum smear positives (NSSPs) in carrying out, 6, 64­66, 66t TB cases, 152­53 computerization, 54 nongovernmental organizations concerns about, 90 (NGOs) dispute resolution mechanisms, addressing concerns of, 24­25 61, 116 community participation, concern duration of contract, 59­60, 165, with, 25 174, 185 concerns about direction and focus economies of scale in, 34 of, 92­93 financial management, 115­16 grants differentiated from government structure, clarifying, contracting, 2, 13­15 52­54 as NSPs. See nonstate providers lump-sum contracts, 58 NTPs, as private providers for, monitoring visits, 65 151­53 payment provisions, 60­61 performance bonuses for health care personnel, 56­58, 57b, 63, 173, 183 services provision in Haiti, PHC services, TORs for, 174t 32b, 78t, 149­51, 151t physical infrastructure, in selection process, 45 responsibility for, 64 as stakeholders, 20 procurement, contractor TORs for collaboration with, 170 responsibility for, 59b, 68­59 nonstate providers (NSPs) reporting obligations , defining, 8, defined, 1, 10 63, 113, 173, 186­87 foreign, concerns over influence of, responsibilities, defining, 52­54 92 in sample plan, 105 grants differentiated from sanctions and terminations, 61, contracting, 13 117­18 uncertain capacity of, 36 SCs, 42 NSSPs (new sputum smear positives) in Ministries of Health (MOHs), 53­54 TB cases, 152­53 monitoring and evaluation process NTPs (national tuberculosis programs), budgeting funds for, 44 151­53 200 · Index objectives management of. See management defining, 26­30 of contracts focusing purchasers and contractors P4P, relationship to, 2, 10, 16, 17t. on, 33­34 See also pay for performance HIV/AIDS services for FSWs, plan for. See plan for contracting TORs for, 179 procurement methods, xivt in sample contract, 108 contractor responsibility for, 59b, in sample plan, 98 68­59 third-party evaluators, TORs for, World Bank approach, xivt, 162­63 119­24, 120­22t obligations of contracting parties. See reviewing contract and contract responsibilities of contracting plan, 66 parties success of. See success of contracting outcome and output indicators, TORs. See terms of reference 28­30, 29t types of services contracted for, 1, 9, 11­12 P4P. See pay for performance whether to. See whether to contract, PAIS (Public Affairs Information global study of Service), 71 performance indicators Pakistan HIV/AIDS services for FSWs, data collection in, 40b TORs for, 178t rural PHC services contracting, 79t, how to set up, 27­29t, 27­30, 37, 38t 154­56, 156f PHC services, TORs for, 170, 171­72t pay for performance (P4P) in sample contract, 108, 109t defining services to be provided, in sample plan, 99t 31­33, 32b third-party evaluators, TORs performance bonuses for NGOs for, 163t providing services in Haiti, personnel management, 56­58, 57b, 63, 32b, 78t, 149­51, 151t 173, 183 relationship of contracting to, 2, 10, PHC. See primary health care (PHC) 16, 17t services in sample plan, 100 Philippines, NTPs serviced by private TORs for PHC services, 170­72 providers in, 153t payment provisions, 60­61, 102, 115­16 physical infrastructure, responsibility performance-based health services for, 64 contracting, xi­xv, 1­8 plan for contracting, xivt checklist, xiii, 67­68 development of, 5, 54­55 concerns about. See concerns about review of, 66 contracting sample plan, 97­106 defined, xi, 1­2, 9­11 schedule, 105, 106t differentiated from other methods of politics. See government organizing health services, 2, postconflict situations, 92­93 9­10, 12­15, 14t PPPs. See public-private partnerships duration of, 59­60, 165, 174, 185 price of services, 60­61, 102 global study of. See whether to primary health care (PHC) services contract, global study of performance-based contracting for. how to. See how to contract See performance-based health improvement of service delivery services contracting by, 71­72 TORs for, xivt, 167­76, 171­72t Index · 201 private sector responsibilities of contracting parties addressing concerns of, 24­25 addressing, 6 auditors, 62­63 HIV/AIDS services for FSWs, PPPs. See public-private partnerships TORs for, 185­87 as stakeholders, 20 management responsibilities, TB services provided via, 46b, 79t, defining, 52­54 151­53, 153t PHC services, TORs for, 175­76 procurement methods, xivt for physical infrastructure, 64 contractor responsibility for, 59b, in sample contract, 113­15 68­59 third-party evaluators, TORs World Bank approach, xivt, 119­24, for, 165­66 120­22t reviewing contract and contract protecting interests of contracting plan, 66 parties, 5­6 RFP (request for proposals), provision of services. See services to be formulating, 8 provided Rwanda, P4P in PHC services Public Affairs Information Service contracting in, 76t, (PAIS), 71 139­42, 142t public-private partnerships (PPPs) defined, 10­11 sample contract, 107­18 TB treatment, 46b, 79t, 151­53, 153t sample plan, 97­106 third-party evaluators, TORs for, sanctions, imposition of, 61, 117­18 161­66 scale, economies of, 34­35 PubMed, 71 scale of contracted services, 84 purchaser, defined, 10 scope of services, defining, 36, 110­12 HIV/AIDS services for FSWs, quality TORs for, 179­83 concerns about contracting and, 93 PHC services, TORs for, 172, 176 in defining services to be provided, third-party evaluators, TORs for, 30­31, 32 164­65 improvement of private provider care SCs (supervisory checklists), 42 in India, 80t, 159­60, 160t search and review methodology, 70­71 selection process, as factor in, 50 selection process carrying out, 48­51 recordkeeping determining, 4­5, 44­51, 44t data collection process, 4, 8­39t, in sample plan, 102­5 37­44, 40­41b Senegal, community nutrition HIV/AIDS services for FSWs, services contracting in, 80t, TORs for, 186­87 156­59, 157f HMI systems, 37­38, 40b, 112 services to be provided PHC services, TORs for, 173 community focus on service routine administrative records as delivery, 93 part of HMI, 37­38, 40b defining. See defining services to be in sample contract, 112 provided reporting obligations, 8, 63, 113, 173, differentiating contracting from 186­87 other methods of service request for proposals (RFP), provision, 2, 9­10, 12­15, 14t formulating, 8 improved service delivery by resolution of disputes, 61, 116 contracting, 71­72 202 · Index methods of service delivery, 2, 9­10, third-party evaluatorsm TORs for, xivt, 12­15, 14t 161­66 types of services contracted for, 1, 9, TORs. See terms of reference 11­12 transparency of selection process, 47­48 typology of service delivery, tuberculosis (TB) control services 12­13, 14t GFATM, 77t, 145­46 settlement of disputes, 61, 116 private providers used in NTPs, 46b, single-source selection (SSS), 51 79t, 151­53, 153t size and location of contract lots, defining, 34­36, 165, 170, underserved areas, attracting health 174, 185 workers to, 57b Sri Lanka, government provision of services in, 81 Vietnam SSS (single-source selection), 51 government provision of services staffing management, 56­58, 57b, 63, in, 81 173, 183 NTPs serviced by private providers stakeholders, dialogue with, 2­3, in, 153t 20­26, 66 success of contracting whether to contract, global study of, context affecting, 93­95, 94t xiii, xivt, 6­8, 69­96. See also evaluating. See monitoring and specific countries evaluation process applicability of findings, 73 global study of. See whether to biases in study, 81 contract, global study of concerns raised. See concerns about indicators of. See performance contracting indicators context reasons for, 82­83 success of contracting affected by, supervisory checklists (SCs), 42 93­95, 94t sustainability of contracting, 85­88 type of contracting affected by, 10, 15t TB. See tuberculosis (TB) control country-by-country results, 74­80t services double difference, calculating, 71 technical guidelines, compliance exclusion criteria, 70 with, 174 government services compared to technical proposal evaluation criteria, 47 contractors, 72, 73b, 81 termination of contract, 61, 117­18 limitations of, 72­73, 72t terms of reference (TORs) methodology used to study, 70­73, community participation often 72t specified in, 25 reasons for success of contracting, P4P, 32 82­83 HIV/AIDS services FSWs, results of study, 71­72 176­87, 178t World Bank obligations and responsibilities procurement methods for clients and addressed by, 6 staff, xivt, 119­24, 120­22t for PHC services, xivt, 167­76, selection of contractors, guidelines 171­72t for, 4, 45 for third-party evaluators, xivt, WorldCat, 71 161­66 E C O - A U D I T Environmental Benefits Statement The World Bank is committed to preserving Saved: endangered forests and natural resources. · 11 trees The Office of the Publisher has chosen to · 508 pounds of solid waste print Performance-Based Contracting for Health Services in Developing Countries: A Toolkit on · 3,957 gallons of waste water 50-pound Exact Opaque, a recycled paper · 953 pounds of net made with 30 percent post-consumer waste, greenhouse gases in accordance with the recommended stan- · 8 million British thermal dards for paper usage set by the Green Press units of total energy Initiative, a nonprofit program supporting publishers in using fiber that is not sourced from endangered forests. For more informa- tion, visit www.greenpressinitiative.org. espite the presence of effective interventions, many developing coun- D tries are not on track to achieve the Millennium Development Goals for health due to the inadequate delivery of health services to the pop- ulation. Contracting, particularly with nonstate providers, can improve the sit- uation. This Toolkit draws on a review of 14 real-life examples of health services contracting in different developing countries to conclude that performance- based contracting can rapidly secure improvements in the coverage and quality of publicly financed health services. Contracting is a mechanism for a financing entity to procure a defined set of services from private or nongovernmental sources. Performance-based contracting hinges on a clear set of objectives and indicators, systematic data collection to assess contractor performance, and some consequences for the contractor (rewards or sanctions) based on performance. The systematic approach described in the Toolkit provides guidance on how to effectively contract, and it addresses issues, such as how to Have a constructive dialogue with all stakeholders; Define health services, in terms of what services are to be delivered, where, the number of beneficiaries to be served, equitable access, and quality of care; Design appropriate monitoring and evaluation systems; Select contractors fairly and transparently; Draft contracts and bidding documents; Carry out the bidding process; and Arrange for effective contract management. This practical "how-to" guide for successful contracting of health (and similar) services will be particularly useful to staff of government agencies, insurance companies, social insurance funds, nongovernmental organizations, faith- based organizations, and private health care providers, as well as international development partners. ISBN 978-0-8213-7536-5 SKU 17536