93871 RBF A Smarter Approach to Delivering More and Better Reproductive, Maternal, Newborn, and Child Health Services ACRONYMS AND ABBREVIATIONS ANC Antenatal Care CHW Community Health Workers CPA Complementary Package of Activities CPG Country Pilot/Program Grant DHS Demographic and Health Survey DLI Disbursement-Linked Indicators DMO District Medical Officers HCC Health Center Committees HMIS Health Management Information System HNPGP Health, Nutrition and Population Global Practice HRITF Health Results Innovation Trust Fund IDA International Development Association IE Impact Evaluation K&L Knowledge and Learning Grant LGA Local Government Area MCH Maternal and Child Health MDGs Millennium Development Goals MPA Minimum Package of Activities M&E Monitoring and Evaluation NSHIP Nigeria State Health Investment Project NGO Non-Governmental Organization PBF Performance-Based Financing PNC Post-Natal Care PHC Primary Health Centers PforR World Bank’s Program for Results RBF Results-Based Financing RMNCH Reproductive, Maternal, Newborn, and Child Health TA Technical Assistance WBG World Bank Group DISCLAIMER The findings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. ©World Bank Group. September 2014 1 The Challenge: 5 Numbers You Should Know 8 2 New Evidence from Africa and Latin America 10 Zimbabwe Argentina Democratic Republic of Congo Nigeria Zambia 3 Making Smarter Investments Through RBF 38 RBF at the Frontline of Service Delivery RBF Gaining Momentum with Governments and Donors 4 The Evolving Role of the Health Results Innovation Trust Fund 48 Building Country Capacity and Knowledge Supporting the Design and Implementation of RBF Assisting the Evaluation of RBF programs Disseminating RBF Knowledge and Evidence 5 About the Health Results Innovation Trust Fund 66 Background Funding Streams Financial Update Mothers and pregnant women attend a health education seminar In Eastern Zambia Mary Chibanga returns to a clinic for her last post-natal checkup Letter from the Director Dear Friends and Partners, The 2010 launch of the G8 Muskoka Initiative on Maternal, Newborn and Child Health and the United Nations Secretary General’s Every Woman Every Child Global Strategy signaled a global consensus that ending preventable maternal, newborn, and child deaths is not only our collective responsibility, but it is possible in our lifetime. We know which interventions are the most cost-effective, and we know the high-burden countries and communities that must be targeted with extra help to solve longstanding barriers to care. Although a large number of countries, particularly in Africa and South Asia, will miss the 2015 Millennium Development Goals (MDGs) targets to improve maternal and child survival rates, there is reason for cautious optimism. Across the globe, we are seeing a smarter service delivery model transforming health systems and saving the lives of women and children. More and more countries are shifting their focus from paying for inputs to paying for verifiable results. This strategic shift is improving access to more and better quality reproductive, maternal, newborn, and child health services – and accelerating progress toward MDGs 4 and 5. This report shares exciting new evidence—from Argentina to Zimbabwe—that paying for results and empowering health workers and managers at the frontline drives transformational change in access to quality health services and saving lives. Using this results-based financing (RBF) approach: In Argentina, the probability of in-hospital neonatal mortality of babies of mothers enrolled in Plan Nacer dropped by 74 percent In Nigeria, the modern contraceptive prevalence rate in RBF areas was approximately twice the rate of non-RBF areas at 21.5 and 10 percent respectively In Zimbabwe, pregnant women in RBF districts were 13.6 percentage points more likely to deliver in a facility and were 11.6 percentage points more likely to receive post-natal care within two-months of delivery as compared to those in non-RBF districts The quality of care also improved in these countries Results-Based Financing Progress Report / 5 The evidence and lessons emerging from the Health Results Innovation Trust Fund (HRITF) demonstrate that by strengthening frontline service delivery through the introduction of structured incentives, countries can accelerate improvements in both coverage and quality of health services for women and children. Over the past five years, US$420 million in HRITF linked to US$2.4 billion in financing from the International Development Association (IDA), the World Bank Group’s fund for the poorest countries, has supported RBF programs in 32 countries. This means that every dollar the trust fund invests in RBF for women’s and children’s health services leverages an additional five dollars in IDA financing. This is proven value for money. As we countdown to the December 2015 MDG deadline and look to 2030, the urgency is high and the timing is right to step up investments in solutions. Results- based, smarter service delivery can help countries take their maternal and child health services to scale. It’s also time to ensure that countries can mobilize additional, and sustainable, domestic financing for women’s and children’s health as their economies grow. We have the momentum, and we have the evidence. Let’s seize this opportunity and realize once and for all the promise to end preventable maternal, newborn, and child deaths in this generation. TIM EVANS Senior Director, Health, Nutrition and Population World Bank Group 6 / Health Results Innovation Trust Fund “ The future should be brighter for every woman and every child. As the Lancet Commission on Investing in Health shows, a global convergence on maternal, newborn, and child health is possible within a generation—that is, if governments and donors invest sufficiently and smartly. And these investments will not only save lives, they will drive economic growth and prosperity. Shifting our focus from inputs to paying for results has been proven to be extremely effective in getting high quality, essential health services to women and children.” JIM YONG KIM PRESIDENT, WORLD BANK GROUP A mother and child get a check up at a local medical health clinic in Afghanistan The Challenge “Innovative approaches to Although steady progress has been made— financing are urgently required to meet the health needs of the maternal and child deaths almost halved world’s women and children. between 1990 and 2013—many low- and lower- Results based financing can improve the quality and middle-income countries, particularly those efficiency of services and, just struggling with resource and governance as important, enhance equity.” constraints, will still fall short of the Millennium Development Goals (MDGs) to BAN KI-MOON, UN SECRETARY GENERAL reduce child mortality and improve maternal health 8 / Health Results Innovation Trust Fund 5 Numbers you should know about this challenge 1 In 2012, 6.6 million children under five years died around the world. 2 Of all child deaths, 44 percent will occur within the first month of life. 3 The child mortality rate in a low-income country is over 15 times that of a high-income country. 4 The maternal mortality rate in a low-income country is nearly 30 times that of a high-income country. 5 The poor are disproportionately affected: in 81 low- and middle-income countries, the median coverage of births attended by skilled health professionals was only 56 percent in the poorest quintile. Health is a major contributor to poverty—people with low incomes tend to have higher rates of many illnesses; ill health leads to impoverishment because of out-of- pocket payment for health care, and not being able to earn income and save money. Worldwide estimates are that out-of-pocket health spending forces 100 million people into extreme poverty every year and inflicts severe financial hardship on another 150 million people. Health is also recognized as a form of human capital because better health increases the productivity of workers, family income, and therefore, the economic growth of a country. In addition, healthier children have higher rates of school attendance and better cognitive development, contributing to a better qualified labor force, productivity and economic growth. The 2013 Lancet Commission on Investing in Health shows that between 2000 and 2011 improvements in health may have accounted for as much as 24 percent of growth in full income in low- and middle-income countries. With appropriate mobilization of resources, the Commission suggests that low- and middle-income countries could improve their maternal and child survival rates to equal those found in upper middle-income countries within a generation. As part of the global effort to accelerate progress on this issue, the World Bank Group and the World Health Organization (WHO) aim to ensure that all people (no matter how rich or poor, regardless of where they live or their gender) have at least 80% access to essential health services, like child immunization and delivery by skilled staff by 2030. RBF approaches that focus on the delivery of high-impact, low-cost interventions are proving to be successful and may be one of the smartest investments that countries and development partners can make towards that goal of Universal Health Coverage. Results-Based Financing Progress Report / 9 New Evidence “We now have concrete Emerging evidence from Africa and Latin evidence that results-based financing approaches can have America indicates that shifting the focus from a profound impact on health inputs to results can help get high impact, outcomes and on the quality, provision, and utilization of quality health care to poor women and children health services. This is possible around the world, and give them a chance to thanks to strong technical assistance, autonomy in the use survive and thrive of the funds, and mechanisms to strengthen the capacity of The first phase of the World Bank Group’s RBF program, which started in 2008, has provided solid evidence that RBF can improve health outcomes by increasing access health centers to to better quality and more equitable services, and promoting greater efficiency in deliver services.” even the poorest countries. MARTIN SABIGNOSO NATIONAL COORDINATOR OF PROGRAMA SUMAR New evidence highlights key lessons from RBF programs in Argentina, the Democratic AT ARGENTINA’S MINISTRY OF HEALTH Republic of Congo (DRC), Nigeria, Zambia, and Zimbabwe. While the RBF designs in these countries are context-specific, they all explicitly link financing to results based on the delivery of select reproductive, maternal, newborn, and child health (RMNCH) services following rigorous verification. To enable providers to attain these results, RBF increases autonomy, strengthens accountability, and empowers frontline providers and health facility managers. The results suggest that RBF programs can have positive impacts on health care utilization, and also highlight the importance of careful attention to design and implementation. 10 / Health Results Innovation Trust Fund Nursing staff from surrounding clinics came to Holme Eden Clinic in Zimbabwe for the RBF program’s launch Zimbabwe How RBF systematically changes the health system to improve the quantity and quality of maternal and child health services The Program Zimbabwe’s RBF program aims to improve service delivery at the health facility level through a fee-for-service scheme combined with a balanced scorecard to assess facility quality. On the demand-side, the program removes the financial barrier of user fees to maternal and child health (MCH) services in all RBF districts. The program was launched in July 2011. After nine months piloting in two districts, RBF was scaled-up in an additional 16 rural districts serving a total population of 3.5 million, approximately 30 percent of the population. Health facilities are paid based on the quantity and quality of MCH services they provide, including 16 high-impact services at the primary level, and six emergency and obstetric care services at the secondary. Results are verified quarterly before RBF payments are made to health facilities. A portion of these payments can be used for staff incentives. 12 / Health Results Innovation Trust Fund “The eight days I stayed at the hospital, I just focused on my child to get well. I knew somebody was going to be there paying for me, RBF.” TECLAR GOHORI, ZIMBABWE Community members attend an event to announce the nationwide scale-up of RBF in Zimbabwe, at Holme Eden Clinic RBF is now being scaled-up across the nation “Results-Based Financing has Additional investments in health system governance are provided through incentives for District Health Executives (DHEs), who are responsible for providing supportive a transformational effect on supervision, including performance coaching of health facility teams. the health system. There is greater emphasis on In Zimbabwe, the RBF program is jointly implemented by the government and accounting for resources and its international partner, Cordaid. The government’s leadership and ownership of RBF has grown in Zimbabwe, and is now embedded in the National Results- results by health providers Based Management Strategy and the Ministry of Finance’s Results-Based Budgeting and managers at various approach. The RBF approach has also influenced the way development partners levels of the health system.” support Zimbabwe’s health sector. For example, RBF is now being implemented in the remaining 42 rural districts using funds from the Health Transition Fund managed by BRIGADIER GENERAL, DR. GERALD GWINJI UNICEF. PERMANENT SECRETARY MINISTRY OF HEALTH AND CHILD CARE ZIMBABWE The Evaluation The impact evaluation (IE) covers the two-year period from 2012 to 2014. The RBF program is evaluated by matching the 16 intervention districts with 16 comparison districts. Changes in service delivery and other key outcomes in the RBF districts are then compared to changes in outcomes in the comparison group. This “difference- in- difference” strategy assumes that the changes in the comparison districts represent what would have happened in the RBF group—the counterfactual—without the RBF intervention. The fact that the trends in the assessed indicators before 2012 are identical across the two groups –based on the most recent national Demographic and Health Survey (DHS)—lends credibility to this difference-in-difference strategy. It is especially important to account for counterfactual trends in key outcomes in the Zimbabwe case because: a. During the RBF period Zimbabwe as a whole witnessed strong gains in population health indicators as the nation continued to recover from the economic crisis of 2008-2009 b. The districts selected for the RBF pilot were disadvantaged in relation to the average rural district in Zimbabwe, and had significantly lower rates of service coverage at baseline. 14 / Health Results Innovation Trust Fund Women in RBF districts are more likely to deliver in a health facility and attend post-natal care The Results Findings show strong gains in service utilization and quality of service as a result of the RBF program: Pregnant women in RBF districts experienced a relative gain of 13.6 percentage points in the likelihood to deliver in a facility compared to pregnant women in non-RBF districts. The quality of ANC care, as reported by beneficiaries, also increased at a faster rate in RBF districts. Women in RBF districts seeking antenatal services experienced a 14.8 percentage point increase in the likelihood of having their urine tested and a 7.4 percentage point increase in the likelihood of receiving a tetanus shot. The same women also experienced a relative increase in the total number of tetanus shots (0.31 more) during ANC care compared to those in non- RBF districts. Postnatal care (PNC) within two-months of delivery also increased by a relative 11.6 percentage points in RBF districts. Other MCH indicators, such as the rate of ANC coverage, saw no significant relevant change between RBF and comparison districts. These results represent the effect of all RBF activities, including the removal of user fees on select services financed by the RBF project. Further analysis seeks to distinguish between the gains in utilization due to user-fee suspension versus gains in other aspects of the RBF program. Since the evaluation contrasts the measured trends between RBF and comparison districts, any movement of clients from non-RBF to RBF districts due to select user-fee removal in RBF districts is not large enough to obscure the identification of positive impacts, which could occur if women in comparison districts opt to receive care in RBF districts. Subsequent analysis will attempt to control for the possibility of client shifting and may suggest even larger program impacts. HRITF also financed a process monitoring and evaluation study in 2013 in five districts. Learning from frontline health workers, beneficiaries, and local authorities, the study aimed to understand how RBF works and how socio-economic and cultural factors may affect it. It highlighted the importance of contextual factors (like women’s health-seeking behavior, geographical access to health facilities, staff availability, the role of traditional healers and community leaders) in boosting or hindering the potential of RBF. This study informed necessary changes to the design and implementation of RBF. Results-Based Financing Progress Report / 15 The analysis of RBF operational data allowed for the identification of high, medium and low performers in service delivery. Facilities in the five selected districts were included to reflect these various levels of performance. Qualitative research was then conducted among healthcare providers, district and provincial authorities, and beneficiaries. The results are multi-dimensional and shed light on how RBF affects service delivery on the ground: 1 2 3 Improves staff morale and Improves health facility Improves local authorities’ promotes teamwork infrastructure and drug and management and supportive equipment availability supervision Health facility staff greatly appreciated For example, health facilities used RBF Four districts reported that they the team-based performance incentives funds for improvements in the physical received quarterly supervision, technical introduced by the RBF program. Health structures including the installation of expertise and coaching by the district workers noted that team incentives reliable, clean water supplies connected health executives (DHE). This boosted prompted collective recognition for team to delivery wards, expectant mothers’ both health workers’ morale through efforts by service providers. shelters, secure drug storage rooms, receiving feedback on their performance toilets, renovation of staff housing, and advice on ways to improve future and purchasing of generators and performance. A nurse-in-charge incinerators. The delivery rooms were illustrated the reinforcement of incentives improved through acquisition of new at district level: “Due to the nature of beds, mattresses, bed linen, chairs and RBF being dependent on well-structured tables. Medical equipment, including supervision of health facilities by the delivery packs, dressing trolleys, DHE, the district health executive weighing scales, BP machines, and now makes an effort to conduct the autoclaves, were purchased with the supervision quarterly. Even if there is no provided funds. The RBF funds were also fuel, effort is put to get the fuel, even if utilized for transportation for referrals. there is no vehicle, effort is again put to get a vehicle for this purpose.” A nurse in Chiparawe, Zimbabwe stands next to a wall of statistics 4 5 6 Strengthens relationships Improves beneficiary Fosters innovations and between health care providers satisfaction with waiting entrepreneurship among and communities and time, service delivery and health workers improves access to services staff ethics at the community level Client satisfaction has been positively RBF decentralizes decision-making, RBF fosters accountability between influenced by the removal of user- allowing health facilities to use RBF health facility staff and community fees, greater availability of drugs funds to address local challenges to representatives in the catchment area. and equipment at health facilities, service utilization. The evaluation Health center committees (HCCs) and improved staff attitudes due to highlighted some of the following local mobilize communities to access services the introduction of RBF. Community innovations. In one clinic, mothers of and express their views and concerns members in almost all health facilities children with up-to-date vaccinations to health facility staff. According to expressed satisfaction with the reductions received prizes. Some HCCs prioritized participants in the qualitative evaluation, in waiting time and the provision of renovating shelters to accommodate the inclusion of local leadership and services at health facilities. Communities women who live far from the facility, and village health workers in HCCs improved served by high performing facilities ensure their access to care at delivery. communication between communities revealed that staff attitudes had become The provision of free food at waiting- and health facilities. It also allowed more positive. These facilities cited staff mother’s shelters also incentivized some communities to express their perceptions reporting for work on time and being pregnant women to deliver at the health of healthcare delivery. RBF funds were willing to serve the community even after facilities. Facilities in two remote districts also used to carry out community clinic hours. The communities reaffirmed kept mothers for three to seven days outreach activities, and provided village the health workers’ willingness to post-delivery to ensure that traveling health workers with weighing scales conduct outreach services and to listen long distances would not prevent them for growth monitoring in an effort to to the communities’ perspectives. from accessing PNC and child boost the RBF subsidies. Because RBF immunizations. To promote timely facilitated outreach programs and the submission of HMIS statistics to the decentralization of work to village health district office, one facility used patient workers, households were able to access ambulances to transport their data. In growth monitoring services, which used three districts, the authorities publicly to be out of reach. announced facility subsidies to spur competition among health staff. Argentina How an innovative program, that combines incentives at multiple levels, cost-effectively improves healthcare coverage, quality of care, and health outcomes for this country’s poor women and children The Program In Argentina, Plan Nacer, which in Spanish translates to “Birth Plan,” has improved access to maternal and child health care to the country’s poorest populations. Plan Nacer targets the two million uninsured pregnant women and children (up to six), who are typically poor relative to the general population. Coverage under the Plan has been scaled-up since its launch in 2004; it initially focused on the target population in nine provinces in northern Argentina, and in 2007 gradually started to expand to all of Argentina’s 23 provinces. By 2012, the Plan was operating nationally and covering a high percentage of the targeted population. 18 / Health Results Innovation Trust Fund “Thanks to Plan Nacer, we provide an equal opportunity to all children, regardless of age and health coverage” NATALIA FRANCHI, REFERENCE CENTER DIRECTOR CHACO PROVINCE, ARGENTINA A new mother cradles her infant son in an Argentine health center The Flow of Results-Based Financing in Argentina ment Paym Enroll e nt Capitation Tr a c e e nt rs Pa y m Argentina Provinces Fee for Sevice Health Care Provider Use of Funds Health care provider Staff Incentives Staff Hiring decides on the final Supplies use of funds Investments Mainenance Source: Argentina National Ministry of Health (2011) Plan Nacer integrates health insurance and RBF “The results show that Plan Nacer is interesting because it integrates health insurance and RBF, allocating Plan Nacer is a successful funding to provinces based on beneficiary enrollment, and adding performance incentives based on the use and quality of MCH services and health outcomes in the program for ensuring that province. In practice, a maximum of US$ 8 (based on the 2005 capitation payment uninsured pregnant women amount, which may not reflect current exchange rate values) per person per month and children under the age is paid to the provinces, and US$ 5—or 60 percent of the maximum per capita of six have access to quality payment—is paid for every eligible individual enrolled in the program. Up to an additional US$ 3 per capita —or 40 percent of the maximum payment—is paid if health services, resulting in health targets for the eligible population are achieved. These targets are measured significant improvements in with ten tracer indicators. Provinces then use the RBF payments to pay health clinics health outcomes.” and hospitals for priority maternal and infant health services on a fee-for-service basis (see diagram on page 20. PAUL GERTLER, LI KA SHING PROFESSOR IN THE BUSINESS AND PUBLIC HEALTH SCHOOLS AT THE The Evaluation UNIVERSITY OF CALIFORNIA, BERKELEY HRITF co-financed Plan Nacer’s IE with the Government of Argentina and the Strategic Impact Evaluation Fund. The IE was carried out between 2004 and 2008 in seven provinces in Argentina’s northeast and northwest. The IE used a unique dataset that combined information from administrative sources with information from birth certificate records kept at public maternity hospitals. It used a difference- in-difference analysis to measure the program’s impact on the provision of prenatal care, quality of care, birth outcomes and neonatal mortality. Plan Nacer’s cost- effectiveness was also assessed. Results-Based Financing Progress Report / 21 Plan Nacer increased the quantity, quality and health outcomes of maternal and child health services RESULTS OF PLAN NACER fig 1 The Results As measured by the number of visits and the probability of receiving a tetanus vaccine, the study found that the program increased the use and quality of prenatal care. It also: Reduced beneficiaries’ probability of low birth-weight by an estimated 19 percent. Reduced the probability of in-hospital neonatal mortality of babies of mothers enrolled in Plan Nacer by 74 percent. Approximately half of this reduction came from preventing low birth- weight, and half from better postnatal care.2 Reduction in the beneficiaries’ probability of low birth-weight by this amount. Plan Nacer also had positive spillover effects for non-enrolled babies. Regardless of whether or not they were enrolled in the program, babies in Plan Nacer-enrolled ......... . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . clinics had 22 percent lower neonatal mortality than those in non-enrolled clinics. Plan Nacer is also very cost-effective, using a relatively small amount of resources (2 to 4 percent of total provincial health expenditures) to provide incentives to provinces and providers. The evaluation further found that the cost of saving a disability-adjusted life year through the program was US$814, which compared with Argentina’s US$6,075 gross domestic product per capita over the period, is highly cost-effective. Reduction in the probability of in-hospital neonatal mortality of babies of mothers enrolled in Plan Nacer. 1 1 Approximately half of this reduction came from preventing low birth weight, and half from better postnatal care. Due to partial data availability, the impact on in-hospital neonatal mortality was calculated on a reduced sample, totaling around 2 Notethat due to partial data availability, the impact on in-hospital neonatal mortality was calculated on a reduced 40 hospitals representing 45 percent of total births. sample, totaling around 40 hospitals representing 45 percent of total births. 22 / Health Results Innovation Trust Fund Health workers take care of newborns in Argentina Democratic Republic of Congo How the design and implementation of an RBF program can make (or break) its possibility for success—lessons learned from the Haut-Katanga pilot The Program Between 2009 and 2013, an RBF pilot and IE was implemented in Haut-Katanga, a district in the DRC’s south-east. It was planned for performance payments to health facilities to be based on the quantity of services provided, using a fee-for-service mechanism. In practice, however, the payment became based on the quantity of services provided by the facility with a fixed total budget, using a points system. This alteration was made to better control program costs and ensure equal payment levels between treatment and comparison groups of the IE. A comparison group of facilities received a predictable, monthly transfer based on the number of workers on the payroll with a preset payment to staff. Both groups received an average monthly facility payment of US$550 equating to US$0.43 per capita per year. Two things to note: i) payments did not depend on the quality of service; and ii) community verification, to confirm whether or not patients received reported services, occurred more sporadically than planned (only 6 out of 28 planned verifications). 24 / Health Results Innovation Trust Fund Community health workers help document the weight and growth of children at the Marechal Health Center in Kinshasa, DRC RBF and the Science of Delivery The Evaluation The program was evaluated using a randomized controlled trial, which compared the results of the two payment mechanisms. Different sources of data were used to evaluate the effect of these mechanisms across the intervention and comparison groups. Data sources included household and health facility surveys (baseline and endline), operational data on the quantity of services provided, and a qualitative survey administered to a sub-sample of 30 health facilities. The Results The design of the DRC’s RBF program was expected to increase service utilization, but in the end, had no significant impact on utilization. The financial incentives linked to quantity did not lead to significant changes in the coverage of services provided, nor were any unintended consequences found on the quality or availability of non-targeted health services. The evaluation found that the payment mechanism in the intervention group led to higher payment volatility at the facility level than the fixed-payment mechanism in the comparison group. As payments based on the point system were adjusted relative to the performance of other facilities in the intervention group, the ambiguity of expected revenues may have led to feelings of insecurity and difficulties in financial planning among staff in these pilot facilities. Despite the lack of impact on service utilization, the flexibility provided to health facility managers in the intervention group led to a more egalitarian distribution of payments to staff, with a higher proportion of non-technical staff receiving bonuses in the intervention group than in the comparison group. Such findings underline the premise that increasing management autonomy for health facilities can lead to greater transparency and equity in resource allocation among staff, where distribution criteria are based on individual performance and not administrative rules, hierarchy or grade. 26 / Health Results Innovation Trust Fund Haut Katanga impact evaluation contributes to the global evidence on RBF “….at the World Bank Group The introduction of the financial incentives in the intervention group led to concrete changes in health worker behaviors and the service delivery models they adopted. we are placing a priority Facility staff was found to be present at intervention facilities more often than in the on what we are calling the control group, and they introduced different strategies to increase demand and science of delivery, which for utilization of health services by the population, such as decreasing user-fees prices us means a more rigorous and for patients and increasing community outreach activities. While the financial incentive payment mechanism led to an overall increase in activities at the systematic focus on outcomes intervention facilities, the strategy to reduce the cost of health services for service —and how to achieve them. users led to lower revenue at intervention facilities. For example, the evaluation We need to understand why found that, the intervention group had 42 percent less total resources than the development approaches comparison group. This may have also reduced the ability of intervention facilities to restock essential supplies and drugs, as suggested by the significantly lower quantity succeed in one country or and quality of equipment in the treatment group as compared to the comparison context and fail group. It is also important to note that several months after the pilot’s end, staff in another.” attendance was found to be lower in the intervention facilities than in the comparison group, and thus may have contributed to a reduction in motivation in the short-run JIM YONG KIM, PRESIDENT after the payments stopped. WORLD BANK GROUP The Haut Katanga impact evaluation has contributed to the global evidence base on RBF and has yielded valuable lessons on the importance of RBF design and implementation in achieving desired results. Furthermore, it has informed the redesign of the program which will be extended under a new health project in DRC. Aspects of the redesign include using a fee-for-service mechanism, increasing the output budget to US$3 per capita, improving supervision, coaching and verification mechanisms, and linking payments to quality of care assessments. Results-Based Financing Progress Report / 27 Nigeria Lessons from a pilot on how RBF can improve service utilization and perceptions of health facility quality The Program The Nigeria State Health Investment Project (NSHIP), which operates in three states, aims to increase the delivery and utilization of high impact MCH, reproductive, and disease-control health services, with a specific focus on the poor. NSHIP also looks to improve quality of care provided, particularly in publicly financed health facilities. NSHIP uses performance-based financing (PBF) to provide participating facilities with financial incentives based on performance. Using verified operational data on the quantity and quality of services provided, facilities receive a quarterly performance payment. Facilities have autonomy to use resources for operational costs and payments to individual health workers. Services are purchased from facilities run by public as well as private providers, including private for-profit as well as faith- based organizations. 28 / Health Results Innovation Trust Fund “A woman’s chance of dying from pregnancy and childbirth in Nigeria is 1 in 13, compared to 1 in 5000 in developed nations” DR. MUHAMMAD PATE, FEDERAL MINISTRY OF HEALTH, GOVERNMENT OF NIGERIA An expectant mother visits her local health clinic for antenatal care Piloting RBF is important before scaling-up “Before, when we went to the In December 2011, PBF was introduced as a pilot in one local government area (LGA) health center, there was no staff in each of the three project states, Fufore (Adamawa), Wamba (Nasarawa) and Ondo East (Ondo), in 36 health facilities. These included 33 primary health centers (PHCs) and no medicine…. Now there and three general hospitals. The PHCs were expected to provide a minimum package is medicine in the health centre of activities (MPA) including: and plenty of staff. We come and they treat us very well.” Outpatient curative care Immunization and growth monitoring for children ROSE MADAKI, PATIENT AT ONE OF THE RBF Maternal health (antenatal, institutional deliveries, postnatal care) HEALTH FACILITIES IN WAMBA LGA (NASARAWA) Family planning services Treatment of tuberculosis and sexually transmitted infections Preventive services for malaria and HIV General hospitals provided a complementary package of activities (CPA), which in addition to MPA included major surgery, inpatient care and treatment for HIV. The Evaluation The analysis of the pilot used data from the baseline survey for the impact evaluation of the NSHIP pilot scale-up in the three states. The survey was conducted in December 2013, two years after the pilot was rolled out to the 36 health facilities. The pilot did not include a baseline; the analysis thus matched households in pilot LGAs to similar households in LGAs not included in the first phase of NSHIP, and then compared outcomes of interest between the two samples. Comparison of households in pilot LGAs with households in LGAs that did not receive the PBF interventions suggests that smaller and poorer LGAs were systematically chosen for inclusion in the NSHIP pilot.2 2 The matching algorithm employed in computing these results accounts for any observed differences in the selection of pre-pilot LGAs, but it does not control for any unobserved differences, for instance political ties, that may plausibly have been significant determinants of where the pilot was rolled out and where it was not. 30 / Health Results Innovation Trust Fund RBF pilot doubled utilization of modern contraceptive methods and improves patient perception of the quality of care The Results Households in LGAs that received the PBF intervention were found to have a significantly greater incidence of at least one antenatal care (ANC) visit prior to birth and a higher prevalence of the use of modern contraceptives. In pilot LGAs, the prevalence rate of the use of modern contraceptives was 21.5 percent, compared to only 10 percent in control LGAs, which represents an 11.5 percentage point difference. Indeed, eight out of the ten priority indicators examined for this analysis suggest a positive trend in pilot LGAs relative to the controls although the small size of the pilot means that this study lacks the “power” to detect the statistical significance of many of these differences. Many RBF programs, including the NSHIP pilot, also aim to increase the quality of services delivered. In Nigeria, clients in PBF pilot LGAs perceived the quality of care to be higher than those in non-PBF LGAs. This difference in perception of quality may explain the higher rate of service utilization in the pilot areas. Only 12.5 percent of respondents in pilot areas reported not using the nearest health facility for reasons of poor service quality, compared with 36.5 percent in control areas. Even starker is the perceived difference in supply constraints; none of the respondents in pilot areas listed drug stock outs or the lack of an on-site laboratory as a reason for not going to the nearest facility. This is compared to control areas where a respective 26 percent and 24 percent of respondents cited those reasons. Results-Based Financing Progress Report / 31 Zambia RBF proves to be more effective in increasing the utilization of reproductive health services than the country’s “business as usual” and “additional financing” models The Program Zambia’s RBF program was launched in April 2012. The program was designed to address key service delivery challenges, including insufficient and poorly motivated staff, limited autonomy in decision-making at decentralized levels of the health system, weak monitoring and evaluation systems, poor service quality, and low levels of service utilization. It is being implemented in 11 districts across 10 provinces, and covers a total of 203 health centers with a population of 1.5 million. Facilities are paid according to the quantity of services delivered, with an additional payment for quality scores over 50 percent, which are measured using the RBF program quality checklist. In addition, District Medical Offices (DMOs) are rewarded if they fulfill a set of supervision and management functions. The program’s fee-for- service approach links performance payments to the delivery of nine facility-based MCH and HIV/AIDS services (see figure 2, on next page). 32 / Health Results Innovation Trust Fund “Staff bonuses through RBF have motivated staff. We are getting more done than before. We come to work clean, in uniforms, with improved morale. I am very proud.” BRIDGET BANDA, REGISTERED MIDWIFE Dr. Kouakoussui reviews a patients medical records in preparation for a curative consultation SOME OF THE RBF INCENTIVIZED PHC SERVICES fig 2 Infant Antenatal / Curative Immunization Postnatal Care Consultations Facility- Family based Delivery Planning The Evaluation The Zambia RBF program was deliberately designed for rigorous evaluation against two alternative models of health service delivery. One alternative is the “business-as- usual” model, or the way the health system currently functions, and the other is the “additional financing” model in which the system receives additional but unstructured funding in an amount equivalent to the RBF program. Of the 11 RBF districts, 10 have been matched for evaluation to 10 ‘business-as-usual’ districts and 10 ‘compensated financing’ districts. Data for the impact evaluation is currently being collected to assess the program’s quantitative impact on certain indicators related to population health (i.e. health- seeking behavior patterns), overall health systems (i.e. quality of care for basic MCH services), and the intervention cost and cost-effectiveness. Two additional sources of data make it possible to draw interim lessons on program performance and identify early successes and potential challenges. The first is a qualitative process evaluation (PE), which monitors factors that may influence program implementation. The PE also includes in-depth interviews with frontline health workers and their district supervisors in nine health centers in three districts. The second is administrative data, which is generated through the health management information systems (HMIS) provided by facilities. 34 / Health Results Innovation Trust Fund More women come for early ANC, delivery and family planning services in the RBF facilities “RBF has helped with The Results attracting more men to escort Changes in service utilization have been analyzed using administrative data from women to the health facility. January 2011 until December 2012. It captures the impact of the first year of RBF Now when they come together implementation, and compares the RBF model with two alternative models, they can make an informed “additional financing” and “business-as-usual.” choice together.” The analysis found that RBF has had a statistically significant impact on several BRIDGET BANDA, REGISTERED MIDWIFE targeted services: The number of deliveries by skilled personnel significantly increased when RBF facilities were compared with the non-RBF facilities. The magnitude of this increase is equivalent to a 7 percentage point increase in the skilled delivery rate for women living in rural Zambia. An additional 12 first ANC visits (within 20 weeks of pregnancy) were made in each RBF facility when compared to the facilities in the other models. This is the equivalent of a 12% increase compared to the baseline of the RBF facilities. In contrast to the “business-as-usual” facilities, 24 additional pregnant women received the third dose of preventive malaria treatment from each RBF facility (a 20% increase compared with pre-program reporting) and 18 more returned to each RBF facility for postnatal care within six days of delivery (a 14% increase). Total family planning attendance per RBF facility also increased by almost 140 women (a 13% increase) when compared to clinics in the “additional financing” districts. Problems in the vaccine supply chain across the country impacted access in RBF and non-RBF districts. This may explain the relative decline in the number of fully immunized children. Since the supply issue was resolved, immunization has increased in RBF districts. No change was found in reported total outpatient visits. The lack of RBF’s impact on non-incentivized indicators—such as provision of Vitamin A and the completion of tuberculosis treatment—suggests that the gains in targeted indicators did not come at the expense of non-incentivized activities. Results-Based Financing Progress Report / 35 Community health workers sing a song about RBF and the difference it has made in their community RBF positively influences the functioning of the health system “I came to the clinic so I will A particularly noteworthy finding is the lack of measured impact from the “additional financing” model when compared with the “business-as-usual” model. know the status of the baby. If This suggests that an infusion of unstructured resources to districts is less effective in the baby has HIV I will seek improving desired outcomes than the RBF model, where financing is explicitly tied to treatment for her. If she does results and the delivery of pre-agreed services. In this regard the results are similar to not, I will find out how to what was found in the Rwanda RBF pilot, where additional unstructured resources were also less effective than the RBF program. prevent it.” MARY CHIBANGA, EASTERN ZAMBIA Qualitative results form the process evaluation found that RBF yielded wider positive effects, such as: Improvements in health worker motivation and reduced absenteeism The perception of improved quality of services delivered improved partly due to better use of established protocols Increased participation of community members in health planning and implementation Empowerment of health facility teams to make their own financial and managerial decisions, including how to allocate health center resources Enhanced communication and knowledge sharing among health centers across staff at different health centers, and between health center staff and DMOs. The PE also identified certain challenges, including a significant increase in civil servants’ salaries ranging from 4 to 200 percent over the past year, which may affect the power of the RBF incentives to achieve further gains going forward. Collectively, the lessons from these five countries are important as RBF programs are scaled-up and increasingly adopted across the globe as a model to transform health systems to deliver more and better quality health services for women and children in the communities they serve. Results-Based Financing Progress Report / 37 Making Smarter Investments through RBF “We work knowing that the Shifting the focus from inputs to paying for harder we work, the more bonuses we earn. That is why results has proven to be effective in getting there is a difference. This [PBF] high quality, essential health services to has been at the origin of the women and children good customer care we are witnessing today. On both the RBF is defined as a cash payment or nonmonetary transfer made to a national or patient and staff side, sub-national government, manager, provider, payer or consumer of health services everyone is satisfied.” after predefined results have been attained and verified. Results-based financing is an umbrella term that encompasses various types of interventions that target DR. BENIGNE NKUNZIMANA DIRECTOR OF THE KIGARAMA HEALTH CENTER beneficiaries (e.g. conditional cash transfers), providers (e.g. performance-based BURUNDI financing) and country governments (e.g. cash on delivery). 38 / Health Results Innovation Trust Fund Distributing in-kind incentives in Rwanda’s RBF program RBF is a Health Systems Intervention Linking Payments to Results Autonomy Contracting Verification Different types of RBF approaches are used Examples of these different types of RBF approaches are found throughout the HRITF/ IDA funded portfolio. Some countries, like Nigeria and Ethiopia, use Disbursement Linked Indicators (DLIs) to provide funding to the state or national level with funds disbursed based on, for example, improvements in service utilization. Most HRITF/ IDA-funded RBF programs focus on improving the supply-side before addressing the demand-side by changing the way the health system functions. These programs usually include incentives at the health facility and district/provincial level. However, more and more countries are including community and/or demand- side components. Countries, such as Rwanda and Cameroon, have included community health workers (CHWs) as part of their RBF programs to strengthen the link between health facilities and the communities they serve. In these cases, CHWs are paid for referrals to health facilities or the provision of services, like immunization, or for household visits to deliver health promotion activities. Incentives can also be paid to the community, as is the case in the Gambia, where the Village Development Committee and Village Support Group are paid based on indicators linked to percentage of households with latrines and pregnant women and mothers who have adequate knowledge of breastfeeding. Other countries, like Zimbabwe and Rwanda, have added demand-side incentives to initially supply-side focused RBF programs, either by providing in-kind incentives—like an umbrella distributed to women delivering in the health facility in Rwanda or the use of vouchers for the urban poor in Zimbabwe RBF program. The RBF approach (see diagram on the left) enables a results-oriented focus through the following: Linking payment to results (whether demand- or supply-side) based on context- specific health priorities A contract or agreement to clarify the respective responsibilities of all stakeholders Autonomy for those contracted to be able to use RBF funds to attain the pre- agreed results; Verification of the results to ensure they are accurate; and Enabling the use of data for planning purposes and for those contracted to enhance the results. In the RBF projects funded by IDA and HRITF, there is often also a level of community involvement, which acts as an accountability mechanism. Results-Based Financing Progress Report / 41 RBF at the Frontline of Service Delivery ESTIMATED COVERAGE OF RBF DELIVERIES fig 3 RBF empowers frontline health workers and decision-makers to set priorities that respond to local health needs. RBF provides facilities with the resources to implement activities that strengthen their service delivery, as well as improve worker motivation through incentives. This was confirmed by the qualitative research highlighted in Section 3. The examples from Zimbabwe and Zambia illustrate that certain elements of the RBF interventions were of great importance. For example, supportive supervision (provided by district health executives) boosted health worker morale and promoted relationship building between health providers and district or provincial health supervisors. The process evaluation concluded that this supportive supervision should be continued and strengthened by adding written feedback to health providers after supervision visits, which would help to further improve health facility performance. ‘ ‘ ‘ RBF programs collect and use operational data to provide information on the coverage, quality and equity of service delivery, with the added advantage that data Afghanistan Cameroon Zambia are available real time. RBF encourages, supports, and benefits from the collection Benin Nigeria Zimbabwe of real time data, both operational and qualitative, which can be used to assess program results and improve program implementation, making course corrections where required. This gives program managers and providers the opportunity and This graph shows the estimated capacity to monitor their results and develop solutions of their own in order to coverage of institutional deliveries with improve the quantity and quality of maternal, newborn and child health services skilled attendants, under RBF programs in delivered (see figures 3 left, and figure 4 on the next page). six different countries. The figure suggests an overall improvement in coverage. Some programs, however, are performing better than others. In Zimbabwe, for example, RBF has greatly improved the coverage of institutional births in less than two years. 42 / Health Results Innovation Trust Fund RBF Gaining Momentum with Governments and Donors This graph illustrates how the overall OVERALL QUALITY OF CARE IN HEALTH CENTERS AND HOSPITALS fig 4 quality score (a composite measure of technical competence and patient satisfaction) for seven countries has improved under RBF. The increases in the overall quality score in Nigeria have been particularly striking—after two years it has increased from 29 to 73 percent. Burkina Fa. Benin Cameroon Kenya Nigeria Zambia Zimbabwe Note: Each bar represents a quarter of implementation For governments looking for a way to use limited resources more efficiently and to meet the obligation to provide essential services to its most vulnerable citizens, RBF mechanisms can serve to clearly signal health priorities to all levels of the IMPROVING EQUITY BETWEEN health system. This approach can also help Ministries of Health to focus efforts on REMOTE AND NON-REMOTE HEALTH producing tangible results on the ground, and monitoring them stringently, while at CENTERS IN ZAMBIA fig 5 the same time empowering those closest to the communities they serve to set priorities according to local needs. Beyond quantity and quality, RBF programs can also promote equity of service utilization in a variety of ways to ensure that women and children have the access they need to RMNCH services. One way is to pay facilities an equity bonus that is based on an assessment of their operational context—for instance, if facilities are located in remote and rural areas or if they serve a particularly vulnerable population. Another way is to pay higher unit cost to facilities that serve hard to reach areas or facilities that serve poor patients. Operational data suggest that these measures to improve equity have benefited the most vulnerable and typically underserved populations. RBF in Zambia, for example, has made great general progress throughout the country. This progress has been observed in both remote and non-remote areas, and it is much stronger in the former Remote HC Non-remote HC (see figure 5). Results-Based Financing Progress Report / 43 Progress through more and better services Development partners, increasingly concerned with value for money in international aid, appreciate the rigorous verification that is a core component of RBF programs. Verification plays a pivotal role in RBF because it ensures that operational data—the very results on which incentive payments are based—are accurate. Verification helps identify any misreporting, including under- and over-reporting. “Patient tracing” is one type of verification method. In this method, verifiers contact a sample of patients that providers have identified as having received health services to confirm that they have actually received them. Many countries use this opportunity to also ask patients about their satisfaction with the service provided. This information then becomes part of the quality score that is again linked to the RBF payment to providers. As such, it is a mechanism to ensure the voice of the patient is heard. Two years after implementing verification in RBF, overall error rates in reporting on quantity indicators have largely been reduced in countries like Afghanistan, Burundi and Argentina. This reduction appears to result from the fact that providers are now paying more attention to the accuracy of operational data registries (often using their national HMIS data) as there is an incentive linked to it. Operational data, confirmed through the verification process, also inform program management at different levels and provide key information on incentive payments and on the overall coverage, quality, and equity of service delivery. The picture on the right was taken after a joint field visit to the north of the DRC. The basket was bought to symbolize an agreement to align and harmonize the respective development partners’ interventions in the Equateur and Bandundu provinces through a virtual basket fund in order to offer an integrated expanded package of quality primary health care services to a wider population. Through PBF this alignment will contribute to making health services more accessible to the poor. Left to right: Mark Troger, DRC portfolio manager at Global Fund; Barbara Bentein, UNICEF country representative; Hadia Samaha, Senior Operations Officer, World Bank Group. 44 / Health Results Innovation Trust Fund A platform for better in-country harmonization RBF programs are also a platform for better in-country harmonization for the implementation of a comprehensive package of key MCH services that respond to a country’s health priorities. In Rwanda, donors have contributed to specific RBF ser- vices based on strategic interests and priorities. The Government of Rwanda directs harmonization and alignment efforts under the RBF program, and invites donors to provide support for specific services or indicators. When PBF was scaled-up nation- wide, the Government of Rwanda purchased 16 general health services, and the United States President’s Emergency Plan for AIDS Relief (PEPFAR) initially purchased 15 services related to HIV/AIDS (this was later reduced to 10 to better balance the country’s different priorities). The Global Fund to fight AIDS, Tuberculosis and Malaria (GFATM) agreed to purchase the reproductive health, HIV/AIDS, and tuberculosis indi- cators. Eight other development partners have since purchased indicators, six through PEPFAR. These harmonized efforts have lowered the government’s transaction costs, improved efficiency in the reporting and verification for partners, and allowed for a broader package of services supported through RBF. In some countries, like Zimbabwe and Burundi, RBF can help harmonize support from different donors around a comprehensive package of services. We also see more domestic financing for RBF, including in Zimbabwe and the Republic of Congo, where governments contributing to the RBF program. According to a United Nations Development Program review of Burundi’s progress towards the MDGs, the country, which historically has struggled with high maternal and child mortality rates, now has a chance to achieve the health MDG targets, if additional efforts are made. Interestingly, this small nation has become a leader in using RBF as a platform for harmonizing health financing for MCH services. The national PBF program in Burundi is the government’s mechanism for implementing its free health care basic package. The government contributes almost 45 percent of PBF payments to health facilities through a virtual harmonized pooling system. Some donors choose to partner on an RBF program through a “joint basket” system. In Benin, for example, a joint basket system is used to manage the PBF programs, which are identical across districts. This arrangement allows partners to purchase health results in a particular geographic area where they may already have a presence. The Ministry of Health manages this system and the partners—the World Bank Group, the GAVI Alliance, and the GFATM—contribute an equal share for RBF operating costs based on the number of districts in which the joint basket works. The co-financing arrangement has made it possible to scale-up PBF throughout Benin, and it is now operational in 85 percent of the country’s districts. Partnering with the World Bank Group reduced the fiduciary risk associated with the implementation of PBF, which made supporting its scale-up more attractive to partners. Results-Based Financing Progress Report / 45 Making an Impact around the World Country programs are at the core of the RBF portfolio at the World Bank, with 80 percent of HRITF funding allocated to the design and implementation of RBF programs. As of June 30, 2014, HRITF is funding 38 approved and pipeline projects in 32 countries. Kyrgyz Republic Armenia Chad Nigeria Haiti Benin Burkina Faso Senegal The Gambia Sierra Leone Liberia Ghana Cameroon Congo Central African Republic CPG Approval Year by HRITF * FY12 (or earlier) FY13 FY14 Pending *A circle denotes a country with multiple RBF projects, with related year(s) of approval indicated by the corresponding color. Tajikistan Afghanistan Pakistan India Myanmar Yemen Lao People's Democratic Republic Djibouti Ethiopia Kenya Rwanda Burundi Democratic Republic of Congo Tanzania Zambia Zimbabwe Lesotho The Evolving Role of the Health Results Innovation Trust Fund “Evidence shows that It has been a year of growth and diversification results-based financing has a significant impact—saving for the HRITF portfolio. As more Country lives and expanding access Program Grants (CPGs) are yielding results, to quality, essential health services for the poorest women more impact evaluations are being completed, and children in developing and more countries are discussing the scale-up countries,” of successful pilots, HRITF’s role is evolving JIM YONG KIM, PRESIDENT WORLD BANK GROUP To meet the growing demand for financing, technical assistance, tools, and resources on RBF, HRITF’s efforts focus on four key areas—its four pillars of work: 1 Building country capacity and knowledge 2 Supporting the design and implementation of RBF 3 Assisting the evaluation of RBF programs 4 Disseminating global RBF knowledge and evidence 48 / Health Results Innovation Trust Fund A health worker speaks with a mother in Eastern Zambia about her daughter’s development 1 Building Country Capacity and Knowledge “We have new networks now— HRITF supports activities to build RBF when we go back home, we can easily communicate with awareness, capacity and knowledge through friends and colleagues all learning opportunities that range from over the world.” technical training workshops to peer learning, as well as the development of knowledge MARIAM ALLY, ASSISTANT DIRECTOR FOR POLICY AND PLANNING, MINISTRY OF HEALTH AND SOCIAL WELFARE, TANZANIA projects and other tools build capacity / knowledge disseminate RBF pilot RBF knowledge RESULTS evaluate 50 / Health Results Innovation Trust Fund 155 participants from 29 countries gather Annual Results and Impact Evaluation Workshop for RBF to showcase their RBF results, share implementation and impact evaluation experiences, and learn from their peers In March 2014, Buenos Aires, Argentina, played host to 155 participants from 29 and experts in the RBF community countries—from Afghanistan to Zimbabwe—that were brought together to share lessons learned and challenges faced from implementing RBF around the globe. Over eight days, 30 sessions and field trips to Argentina’s Plan Nacer health facilities, country teams showcased their RBF results, shared implementation and impact evaluation experiences, and learned from peers and experts in the RBF community. A rich agenda was developed through consultations with country teams and drawing on the results of operational portfolio reviews. The 2014 workshop helped respond to the needs of a large number of teams at very different stages in their programs and evaluation design or implementation. This presented a rare opportunity for more advanced teams to share lessons and thereby help anticipate issues down the line for less advanced teams who are still at the design stage. For the first time, scaling-up in the most advanced countries was also a reality and issues of financial and institutional sustainability could be concretely explored. An innovation this year was the introduction of country team Action Plans—six-month strategies formulated by country teams to overcome the weaknesses and bottlenecks of their respective RBF program and impact evaluation. RBF Task Team Workshop The trust fund also delivered the flagship Task Team workshop to provide learning for World Bank staff who were intending to develop, or who had already supported the development of, an RBF-for-Health program. The workshop provided participants with background knowledge on RBF, information on tools to assist in decision-making and actual RBF design and implementation of programs. The workshop was also an opportunity for new and experienced RBF practitioners to share experiences and lessons learned on the design and implementation of RBF projects within the Bank. Results-Based Financing Progress Report / 51 A health care worker checks her supplies at Chiparawe Rural Clinic in Zimbabwe Making Knowledge and Learning on RBF Accessible OVERALL USEFULLNESS OF THE COURSE fig 6 RBF e-Learning Course Percentage distribution of responses An RBF E-Learning Course was successfully piloted as part of the ongoing effort to make knowledge and learning on RBF accessible to practitioners around the world. Ensuring the appropriateness of RBF for a particular context is vital and at the core of the e-learning course content. The course is based on materials that were developed over the course of several years for the delivery of face-to-face RBF learning events. The Task Team workshop learning materials generated a rich repository of knowledge products that have been translated into e-learning modules to reach a critical mass of health practitioners globally. The course aims to equip participants with the specific knowledge and tools needed to work on RBF in countries where health system strengthening and poverty-reduction strategies are being implemented. The course is designed for: Health and Financial Practitioners/Professionals at the national and district levels MINIMUM MAXIMUM Non-Governmental Organizations (NGOs) engaged in RBF Development Partners supporting RBF schemes Academia involved in health system and other relevant research Policymakers 75 Participants received a certificate of completion for the pilot course and 97% of the respondents rated the overall usefulness of the course either a 4 or 5 on a scale of 1 through 5—5 being the maximum usefulness, and 1 being the minimum usfefulness (see figure 6, left). Performance-Based Financing Toolkit The Performance-Based Financing (PBF) Toolkit was successfully launched this year, in response to the rapid expansion of this supply-side intervention in health systems, particularly in Africa. The Toolkit contains the experiential knowledge of PBF pioneers who have designed, implemented and evaluated this approach in several countries and contexts. The book and its companion online portal aim to answer the most pressing issues related to supply-side RBF programs—the most common approach in the HRITF portfolio. Results-Based Financing Progress Report / 53 2 Supporting the Design and Implementation of RBF HRITF technical assistance (TA) for the design and implementation of RBF at country level is even more critical as RBF approaches are increasingly used to operationalize the World Bank Group’s Health, Nutrition, and Population strategy build capacity / knowledge disseminate RBF pilot RBF knowledge RESULTS evaluate 54 / Health Results Innovation Trust Fund Discussing appropriateness of RBF and Some of the technical and operational assistance activities supported by HRITF, in its possible approaches during HRITF collaboration with country teams, are highlighted below: supported workshop in Nigeria helped inform the RBF design currently being piloted in three states At the design phase: Country visits to raise awareness and explore the utility of RBF as part of an IDA-funded health project; review and feedback on CPG proposals; participation in project appraisal missions, reviewing and commenting on terms of reference for implementing agencies and individuals to support RBF programs. During implementation: Monitoring and evaluation of ongoing RBF projects by analysis of operational data and supporting country teams to perform qualitative assessments. Working with Bank cross-cutting teams: To broaden the understanding of RBF and its implications for Bank projects especially the fiduciary, safeguards and risk-management aspects. This year, a review of eight CPGs offered a unique opportunity to assess fidelity to RBF core principles in project design, to learn from country implementation successes and challenges, and to offer suggestions for enhancing project achievements. The programs reviewed varied in size and from small pilots to larger, countrywide programs. All programs primarily targeted the utilization of reproductive, maternal and child health services, and some also included nutrition, HIV, malaria and TB indicators. Results-Based Financing Progress Report / 55 Technical assistance plays a vital role in RBF and its corresponding IE implementation Learning lessons to improve RBF implementation The eight countries selected for the review were: Afghanistan, Benin, Burundi, Cameroon, Nigeria, Sierra Leone, Zambia, and Zimbabwe. Their collective implementation experience was reviewed in order to achieve the following: Identify key implementation challenges and factors for success Measure the effectiveness of RBF implementation by exploring implementation progress with several core principles of RBF Develop generic lessons and related recommendations for the wider portfolio Many of the projects have already demonstrated positive results on output indicators such as improved antenatal care, delivery by skilled birth attendants, postnatal care, and childhood vaccinations, all known to contribute to the MDGs to reduce child mortality and improve maternal health outcomes. While not all the recommendations will be useful for every project, the findings can help strengthen the overall RBF program at the World Bank. These lessons were shared at the annual Results and IE workshop for RBF in Argentina. Beyond informing the design and implementation of IDA/HRITF-funded CPGs, the lessons can contribute to the body of knowledge on RBF best practices. The Top Ten Lessons as Identified by Portfolio Review Participants 1 RBF payments should be timely 2 Measurement of quality should be dynamic to continuously improve 3 Keep on learning and adapting 4 To be sustainable, programs should link to the wider health financing strategy 5 Ensure data availability and utilization in the management of RBF 6 Consider demand- and supply-side challenges to improve health care utilization 7 Identify poorly performing regions/health facilities and take action 8 The political environment significantly influences the sustainability of RBF 9 Monitor RBF implementation to ensure design phase assumptions remain valid 10 For IE, go for simple, clear research questions Results-Based Financing Progress Report / 57 3 Assisting the Evaluation of RBF Programs HRITF provides support to country programs for evaluating RBF, as part of its mandate to build the evidence base. More and more, both quantative and qualitative research methods are used build capacity / knowledge disseminate RBF pilot RBF knowledge RESULTS evaluate 58 / Health Results Innovation Trust Fund Carrying out the household survey for A well-funded impact evaluation portfolio underpins HRITF’s comprehensive the baseline of the impact evaluation in learning agenda, which is complemented by operational data and learning from the country of Rwanda implementation studies. Figure 7 shows the growth of the portfolio, which aims to capture a diversity of lessons and insights on RBF through a rich set of evaluations and analytical methods. The portfolio is very broad geographically and in terms of content (see map on page 62) and has evolved from looking at the impact of RBF to also learning from specific program design elements. A move towards greater integration between quantitative and qualitative methods, through mixed methods design (such as that scheduled to start in Yemen in 2015) is further strengthening the work. Some countries, including Afghanistan, have successfully integrated the impact evaluation design with learning during implementation from administrative data. GROWTH OF IE PORTFOLIO fig 7 HRITF Funding Baseline data Follow-up Completed Approved collected Number of Impact Evaluations FY FY FY Results-Based Financing Progress Report / 59 Using quantitative and qualitative research methods to learn about RBF Many of the RBF programs are complex, health-system interventions,3 and experience from the field suggests that multiple factors and their interactions affect implementation. RBF programs prompt behavioral changes that need to be understood in order to show a logical sequence between intervention and outcome. Information on intermediate causal steps is thus essential to attribute the observed outcomes in an impact evaluation to the intervention.4 Understanding these causal steps is not merely an academic exercise—it is of vital importance to implementers and policymakers. If an RBF intervention shows success across a variety of quantitative indicators, it is important to understand the details of how the intervention was implemented and the context (i.e. the political economy, community, and health system) in which it was implemented. Such details are particularly useful for practitioners or policymakers considering scaling-up or replicating the intervention. Questions of “why” and “how” will typically surface if the intervention has had mixed, null, or negative results—if it deviates from what was expected. Policymakers deciding whether or not to scale-up or replicate a program, will likely wonder whether the RBF intervention had consequences or impacts (either positive or negative) that were unforeseen at its outset. HRITF IEs are increasingly including qualitative research methodologies to learn more about how RBF mechanisms work and the intermediate components (versus steps) in the causal pathway. When combined—or “mixed”— with intervention design and evaluation, qualitative research, can broadly be categorized as a formative research, process evaluation and outcome evaluation.5 A “mixed-methods” approach, in which formative qualitative research was conducted alongside household surveys, was used in the Gambia to better identify barriers to service utilization.6 Focus group discussions were conducted with pregnant women, mothers of children under-five, community agents, and community-based organizations; in-depth interviews were conducted with adolescent girls and boys, health providers and regional supervisors. 3 Complex interventions can be defined as interventions that have multiple components that interact, multiple behaviors required by those receiving the intervention, multiple groups or levels impacted by the intervention, multiple of outcomes of interest, and flexibility in design and implementation (Craig et al, 2008). 4 Ssengooba F, McPake B, Palmer N. 2012. Why performance-based contracting failed in Uganda -- An “open- box” evaluation of a complex health system intervention. Social Science & Medicine 75 (2012) 5 Chandler, C.I.R., Reynolds, J., Palmer, J.J., & Hutchinson, E. (2013) ACT Consortium Guidance: Qualitative Methods for International Health Intervention Research. 6 A Rapid Assessment of the Gambia PHC and Community Health & Nutrition Structures: A Mixed-methods approach 2014; A Rapid Assessment of Household Health Expenditures and Health Seeking Behaviors 2014 60 / Health Results Innovation Trust Fund Where We are Learning A geographically broad impact evaluation portfolio, which has evolved from looking at the impact of RBF to also learning from specific program design elements, is at the core of the HRITF’s rigorous learning agenda. Denotes countries in the Impact Evaluations portfolio Identifying barriers and challenges in seeking health care to inform RBF design This research revealed that individuals and households face a variety of barriers in seeking health care, including cultural obstacles, cost, inconvenience, and inadequate understanding about the risks and benefits of seeking care or adopting a particular behavior. For example, when asked about frequency and timing of antenatal care, one woman responded: “[Antenatal visits should be made] nine times before delivery. She should start in her third month of pregnancy.” WHO recommends a minimum of four antenatal visits, “… spaced at regular intervals throughout pregnancy, commencing as early as possible in the first trimester.” Recommendations of the study have informed the design of the new project, which includes RBF mechanisms at the health facility and community levels. These RBF mechanisms are intended to address both the provision of services, as well as demand for services and the adoption of healthy behaviors. During the Annual Results and IE Workshop in Buenos Aires, country teams identified and discussed the common challenges in implementing and carrying out quality impact evaluations. They also shared experiences on the issues surrounding the measurement of priority health care outcomes, including utilization and quality. HRITF also organized several workshops to help program teams take advantage of the learning opportunities presented by baseline data. Second phase teams, from countries further along in their RBF journey, were assisted in developing robust IE plans. In addition, targeted workshops were held for IE country teams including Burkina Faso, Rwanda, and Nigeria. These workshops brought together the Bank team, principal investigators leading the respective impact evaluations, local government stakeholders, survey teams, and IE specialists to work together and agree on a plan for the analysis. HRITF provided technical assistance on analytical design, methodology and report writing at each workshop. All the participating countries have completed their baseline data collection. And Rwanda has completed its endline data collection. The evidence base grows more and more robust with each new program that is designed, implemented and evaluated. New evidence from Africa and Latin America is presented in Section 3 on page 10. 62 / Health Results Innovation Trust Fund A pregnant woman visits her local health clinic for one of several antenatal care appointments that will help her enjoy a safer delivery for her and her baby 4 Disseminating RBF Knowledge and Evidence HRITF continues its efforts to strengthen its online presence and capacity to effectively and efficiently reach and engage with our global audiences build capacity / knowledge disseminate RBF pilot RBF knowledge RESULTS www.rbfhealth.org evaluate @rbfhealth facebook.com/rbfhealth 64 / Health Results Innovation Trust Fund The redesigned RBF for Health website Embracing the Internet and Social Media is the primary platform for learning and knowledge sharing on Results- based Financing This year, the website, which is the primary platform for learning and knowledge sharing, has been redesigned to ensure that it meets the expectations of audiences in the Internet age. In addition to being more interactive and engaging, the new website will make content more accessible and streamlined. Given the number of CPGs in Francophone countries, the site will also increase French content, particularly for flagship knowledge products and tools. User feedback will be captured in a follow-up survey, and adjustments to the new website will be made accordingly. Social media is playing an increasingly important role in helping HRITF meet the challenges of sharing knowledge with audiences by providing options for where, when and how they consume information. HRITF is taking advantage of data and analytics from digital channels to better understand audience segments. North America and Western Europe still top the list for website traffic, whereas South Asia and Africa are dominant on Facebook. This social media channel continues to be a lively point of engagement, and the RBF Health Facebook page now counts almost 40,000 fans. HRITF debuted on Twitter this year, and its handle, @RBFhealth, has allowed the trust fund to engage with global health and development influencers, think tanks, policy makers, and donors. HRITF also hosted and participated in a variety of events, including the Annual Results and IE Workshop for RBF, the RBF Seminar Series, and high-level conferences and meetings. These events have provided different stakeholders with opportunities to discuss the latest evidence on RBF and to share lessons learned. Technology enhanced the reach and impact of some of these events, which were live-streamed through WebEX, allowing for virtual attendance anywhere in the world. A new Slideshare account made it easy to access content from the Annual Results and IE Workshop, and received over 2,500 visits. Content related to the Workshop increased traffic across all online channels, including the website, Twitter, Facebook. By taking advantage of diverse offline and online channels, HRITF is well positioned to meet its mandate to gather and share RBF knowledge with audiences around the world. Results-Based Financing Progress Report / 65 About the Health Results Innovation Trust Fund “This approach to delivering HRITF has been supporting RBF approaches aid aims to ensure every penny we spend on life- since 2007 to help resource-constrained saving health programs countries accelerate progress towards the produces real results. It is good for donors, good MDGs, particularly those that focus on for taxpayers and, most maternal and child health and nutrition7 importantly, good for the millions of people across Background the developing world who desperately need access to HRITF is supported by the governments of Norway and the United Kingdom, with a better health care.” total commitment of US $537 million equivalent through 2022, and managed by the World Bank as a multi-donor trust fund. A unique benefit of HRITF is that grants are 8 LYNNE FEATHERSTONE co-financing IDA. Each HRITF dollar leverages five dollars of IDA. INTERNATIONAL DEVELOPMENT MINISTER OF THE UNITED KINGDOM 7 The MDGs of particular interest are 1C (nutrition), 4 (child mortality), and 5 (maternal health). 8 Donor commitments are converted to U.S. dollar equivalents every six months, taking into account disbursements and cash contributions received. As of July 1, 2014, contributions are US$348 million from the Government of Norway and US$188 million from the Government of the United Kingdom. 66 / Health Results Innovation Trust Fund Funding Streams HRITF has three funding streams to support countries at various stages of the RBF journey—from exploring the concept of shifting the focus from inputs to results, to designing, implementing and evaluating the right RBF approach for their context. 1 2 3 Knowledge and Country Program Impact Evaluation Learning Grants Grants Grants Knowledge and Learning (K&L) grants CPGs are at the core of the RBF IE grants support learning from support technical dialogue and learning program at the Bank, with 80 percent successful and unsuccessful RBF about RBF design and implementation of funding allocated to the design and experiences around the world, while in IDA-eligible countries. Recipients can implementation of the CPG portfolio. contributing to local and global use the grants to explore and analyze Low- or lower-middle-income countries evidence-based policymaking. This whether RBF is the right approach for with a new or existing IDA-financed funding stream for which middle- their country context and health system project that includes an RBF component income countries are also eligible, challenges. K&L grants also help build in the health sector can apply for up includes financing for standalone impact RBF capacity of interested countries to to US$20 million. Funding is typically evaluations of non-HRITF-financed design, implement and evaluate RBF awarded to governments, through programs to contribute to global RBF programs. dollar-to-dollar matching of the knowledge. proportion of the IDA grant or credit. Grants are prepared and supervised within the World Bank’s operational framework, ensuring regional and country oversight of management and support for rigorous CPG design, implementation, monitoring, and evaluation. A health worker at Lundazi Urban Clinic in Eastern Zambia reviews vaccination plans with a child’s mother. The health facility holds these regular consultations in a structure financed by RBF Financial Update To date, HRITF has committed US$420 million, or 78% of the total contributions, for 38 RBF programs in 32 countries which is linked to US$2.4 billion financing from IDA. Between the fund’s inception and the end of June 2014, US$152.6 million has been disbursed for eligible activities under the three funding streams. As the portfolio matures, disbursements over the last two years have accelerated significantly and, as illustrated in figure 8, are expected to increase further as more CPGs advance their implementation. CURRENT DISBURSEMENTS AND FUTURE DISBURSEMENT PROJECTIONS fig 8 Disbursements in US milllions HRITF IDA FY FY FY FY FY FY FY FY FY FY FY FY Note: Fiscal Year is July 1 thru June 30. Projections for FY15-FY20 are based on total commitments and the expected closing date of the projects. 69 / Health Results Innovation Trust Fund Results-Based Financing Progress Report / 69 Financial Update (continued) Although 76% of CPG funding benefits countries in Africa, the overall portfolio of approved grants and those in the pipeline is diverse and includes IDAeligible countries from all Bank regions (see figure 9). COUNTRY PROGRAM ALLOCATION BY REGION fig 9 4% Europe and Central Asia 5% Latin American & Carribean 4% Middle East & North Africa 8% South Asia 3% East Asia & The Pacific 76% Africa 70 / Health Results Innovation Trust Fund Get in Touch www.rbfhealth.org rbfhealth@worldbank.org @rbfhealth facebook.com/rbfhealth Design: kngraphicdesign.com Photography: All photographs are the exclusive property of the World Bank Group. All rights reserved.