Document of The World Bank Report No: ICR00003119 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-49100 IDA-49110 TF-11049) ON A CREDIT IN THE AMOUNT OF SDR 79.5 MILLION (US$125 MILLION EQUIVALENT) TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR A FLOOD EMERGENCY CASH TRANSFER PROJECT October 17, 2014 Social Protection and Labor Global Practice South Asia Region i CURRENCY EQUIVALENTS (Exchange Rate Effective 00000000) Currency Unit = PKR PKR 1.00 = US$0.01 US$1.00 = PKR 59.90 FISCAL YEAR July 1 – June 30 ABBREVIATIONS AND ACRONYMS CDCP Citizen’s Damage Compensation program CPS Country Program Strategy DCO District Coordination Officer DFID Department for International Development MOU Memorandum of Understanding NADRA National Database and Registration Authority PDMO Provincial Disaster Management Offices PDO Project Development Objective RDMO Regional Disaster Management Offices USAID United States Agency for International Development Vice President: Philippe H. Le Houerou Country Director: Rachid Benmessaoud Sector Manager: Pablo Gottret Project Team Leader: Kelly Johnson ICR Team Leader: Kelly Johnson ii COUNTRY Islamic Republic of Pakistan CONTENTS DATA SHEET .................................................................................................................................. A. Basic Information...................................................................................................... iv B. Key Dates .................................................................................................................. iv C. Ratings Summary ...................................................................................................... iv D. Sector and Theme Codes ........................................................................................... v E. Bank Staff ................................................................................................................... v F. Results Framework Analysis ..................................................................................... vi G. Ratings of Project Performance in ISRs ................................................................. viii H. Restructuring (if any) .............................................................................................. viii I. Disbursement Profile ................................................................................................. ix MAIN TEXT 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes .......................................................................................... 11 4. Assessment of Risk to Development Outcome......................................................... 14 5. Assessment of Bank and Borrower Performance ..................................................... 14 6. Lessons Learned ....................................................................................................... 16 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 17 ANNEXES Annex 1. Project Cost and Financing………………………………………………….18 Annex 2. Outputs by Component ................................................................................. 19 Annex 3. Economic and Financial Analysis ................................................................. 21 Annex 4. The main elements of the Memorandum of Understanding on changes to the CDCP process, agreed between the Government and the Bank ................... 23 Annex 5. Bank Lending and Implementation Support/Supervision Processes ............ 24 Annex 6. Beneficiary Survey Results ........................................................................... 26 Annex 7. Borrower’s ICR and/or Comments on Draft ICR ........................................ 29 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders ...................... 37 Annex 9. List of Supporting Documents ...................................................................... 39 Annex 10. MAP ………………………………………………………………………40 iii DATA SHEET A. Basic Information Pakistan Flood Country: Pakistan Project Name: Emergency Cash Transfer Project IDA-49100,IDA- Project ID: P125105 L/C/TF Number(s): 49110,TF-11049 ICR Date: 10/17/2014 ICR Type: Core ICR ISLAMIC REPUBLIC Lending Instrument: ERL Borrower: OF PAKISTAN Original Total XDR 79.50M Disbursed Amount: XDR 79.50M Commitment: Revised Amount: XDR 79.50M Environmental Category: C Implementing Agencies: National Database and Registration Authority (NADRA) Cabinet Division Cofinanciers and Other External Partners: Department for International Development (DFID) USAID Government of the Italian Republic B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 01/27/2011 Effectiveness: 06/28/2011 01/09/2013 Appraisal: 02/14/2011 Restructuring(s): 04/30/2013 Approval: 03/29/2011 Mid-term Review: 05/16/2012 05/18/2012 Closing: 06/30/2013 03/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Moderately Satisfactory iv C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Highly Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Highly Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Other social services 92 92 Public administration- Other social services 8 8 Theme Code (as % of total Bank financing) Natural disaster management 100 100 E. Bank Staff Positions At ICR At Approval Vice President: Philippe H. Le Houerou Isabel M. Guerrero Country Director: Rachid Benmessaoud Rachid Benmessaoud Practice Pablo Gottret Mansoora Rashid Manager/Manager: Project Team Leader: Kelly Johnson Andrea Vermehren ICR Team Leader: Kelly Johnson ICR Primary Author: Hjalte S. A. Sederlof v F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project Development Objective was to support the recovery of flood affected households by assisting the Government of Pakistan in strengthening the implementation of its Citizen's Damage Compensation Program Revised Project Development Objectives (as approved by original approving authority) The PDO was not revised during implementation (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years At least one million flood affected eligible households as per the re-targeting Indicator 1 : criteria benefit from cash grants Value quantitative or 0.00 at least 1 million 1,204,624 Qualitative) Date achieved 03/16/2011 03/31/2014 08/09/2013 Comments (incl. % achievement) Targeting efficiency: average baseline poverty score of CDCP II households will Indicator 2 : be lower than average baseline poverty scores of non-CDCP families in each province Value quantitative or 0.00 10 percent 25.3 percent Qualitative) Date achieved 03/16/2011 03/31/2014 08/09/2013 Comments Targeting efficiency was calculated using the poverty scorecard database. The (incl. % average PSC for a CDCP II beneficiary was 19.27 while the average score for a achievement) non-beneficiary was 24.14 a 25.3% difference. Indicator 3 : At least 70% of beneficiaries express satisfaction with the program delivery. Value quantitative or 0.00 70.00 77.50 Qualitative) Date achieved 03/16/2011 03/31/2014 05/29/2014 Comments There is no one question on overall satisfaction. However, the follow-up survey (incl. % asks beneficiaries: Would you say your overall experience with the payment achievement) process was? 77.5% of beneficiaries stated good or very good. vi (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : program evaluated, and results reported Value partial: a rapid evaluation Full program Full program (quantitative of CDCP I was evaluation evaluation or Qualitative) undertaken Date achieved 06/30/2011 03/31/2014 03/31/2014 Comments (incl. % achievement) Public Information Campaign carried out as per the agreed communication Indicator 2 : strategy Value (quantitative No Yes Yes or Qualitative) Date achieved 06/30/2011 03/31/2014 06/09/2013 Comments (incl. % achievement) Management Information System strengthened, as measured by MIS modules for Indicator 3 : targeting, enrollment, etc. developed, MIS training provided, computer systems in place, etc., according to project activity Value MIS Fully MIS Fully (quantitative Partial Operational Operational or Qualitative) Date achieved 06/30/2011 03/31/2014 03/06/2014 Comments (incl. % achievement) Functioning Grievance mechanism for retargeting and payments established and Indicator 4 : accessible Value (quantitative Partial Full Full or Qualitative) Date achieved 06/30/2011 03/31/2014 03/31/2014 All card centers had a grievance desk and trained staff. Support from IOM to the Comments district coordinators resulted in improved operations and better record keeping. (incl. % 1,179,956 appeals were received, reviewed, and resolved. 29 percent were achievement) included. Indicator 5 : At least 85 percent of families pick up cash grants within stipulated timelines Value (quantitative 0 percent 85 percent 99.3 percent or Qualitative) Date achieved 03/16/2011 03/31/2014 03/31/2014 vii Comments The target was exceeded by 17 percent, as 99.3 percent of participants withdrew (incl. % their complete payments within 72 hours of disbursement of their payments achievement) Mechanism agreed and established to enable GOP to effectively respond to Indicator 6 : potential future disaster recovery Value (quantitative Partial Full Full or Qualitative) Date achieved 03/13/2011 03/31/2014 03/06/2014 Comments An action plan was approved by the Prime Minister. Several provinical (incl. % workshops for dissemination were held to raise awareness. the CDCP II model achievement) was used as a blueprint for 2011 and 2012 flood responses at the provincial level Operational Audit regularly carried out, and recommendations incorporated into Indicator 7 : program implementation Value (quantitative No Audi Yes Yes or Qualitative) Date achieved 03/13/2011 03/21/2014 03/16/2014 The operational audits were applied during the entire time there were field Comments activities and grievances were being reviewed. The consultant presented a final (incl. % analysis and recommendations that will be incorporated into the disaster recovery achievement) action plan G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 09/21/2011 Satisfactory Satisfactory 40.00 2 02/05/2012 Satisfactory Moderately Satisfactory 45.67 3 07/07/2012 Satisfactory Satisfactory 122.68 4 04/04/2013 Satisfactory Moderately Satisfactory 122.43 5 10/19/2013 Highly Satisfactory Moderately Satisfactory 122.78 6 04/05/2014 Highly Satisfactory Moderately Satisfactory 122.78 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Re-allocation of funds from Component 1 to Component 2 01/09/2013 S S 122.07 to cover additional TA costs. Potential need for this outlined viii ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions in minutes of negotiations. Extension of closing date to 04/30/2013 S MS 122.43 03/30/2014 I. Disbursement Profile ix 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. The project was introduced as an emergency response to massive flooding in the north and north-western regions of Pakistan and in the province of Sindh in the southeast when, over the course of the monsoon season in July and August 2010, the country experienced the worst floods in its history. The floods covered some 100,000 square kilometers over 78 districts, and affected some 20 million people, or nearly 10 percent of the population. Approximately 1.6 million homes were destroyed, and livelihoods were severely affected: 60 percent of households described themselves as being without a main means of earning a living, and 53 percent of households reported a 76 to 100 percent decline in household income. The floods disproportionately affected the poorer districts of the country, worsening a poverty situation that had been steadily improving. 2. Relief operations were launched immediately by the Government, assisted by various UN agencies. They focused on immediate relief: keeping people alive and safe, and ensuring that their basic needs were met. However, Pakistan did not have the mechanisms, notably the safety nets, to stabilize basic consumption and recapitalize assets, and begin to restore livelihoods. Social protection had only just arisen as an important agenda item, a national protection strategy had been put in place, and work had started on developing a national cash-based safety net program, with assistance from the Bank (the Benazir Income Support Program, BISP). But that program was not meant to address emergencies directly, although its poverty database is certainly a valuable tool for that purpose as well. Instead, the Government initiated a special cash grant program for flood-affected families, The Citizens’ Damage Compensation Program (CDCP) Phase I, using existing federal and regional-level institutions for the design, financing and implementation of the program. Applying mainly geographic targeting, the program transferred some US$400 million to some 1.8 million beneficiary households (US$200per beneficiary household). The program was coordinated by the Cabinet Division of the Prime Minister’s office in partnership with the provincial governments through their respective Provincial/Regional Disaster Management Offices (PDMOs, RDMOs). The National Database and Registration Authority (NADRA) that maintains the database of Pakistani citizens and issues them computerized national identity cards, led systems’ development and operations. 3. The CDCP covered immediate needs of flood-affected households, as cash benefits were used for basic consumption. Less was spent on recapitalizing assets, recovering livelihoods or repairing housing. To that intent, the Government sought to launch a second phase of the program, CDCP Phase II, with additional payments targeting some 1.1 million households, and it was seeking donor assistance. The Bank was well-placed to support Pakistan in extending and strengthening the CDCP. It had already been assisting the Government in its flood response through a post-disaster needs assessment and US$300 million in IDA resources for critical imports, introduced through additional financing under the Earthquake Emergency Recovery Credit Project (P099110). It had substantial regional experience in disaster recovery 1 and reconstruction through interventions in Sri Lanka and the Maldives following the 2004 tsunami. The Bank also had considerable experience in crisis response in Pakistan from its involvement in the 2005 post-earthquake crisis to help finance the design and delivery of livelihood support cash grants to the poor. It had been supporting social protection sector reforms in Pakistan through a technical assistance project. And potential donors were looking to the Bank to take the lead in helping the Government provide further support to the CDCP. 4. Moreover, the project was consistent with the Bank’s Country Partnership Strategy (CPS) for FY 2010-2013. The CPS recognized the need to reduce vulnerability through effective safety nets as well as improved disaster risk management under Pillar 2 of the strategy, entitled Accelerating delivery of human development and social protection services. 5. Additional donor financing was necessary at the time. Pakistan faced significant fiscal constraints, and economic growth was at a modest 2.5 percent in 2010, in large part as a result of the flood. A coalition of development partners, led by the World Bank, was established, initially with participation by the Government, USAID and IDA. Later DFID, through a Bank-administered trust fund, and the Italian Government (who parallel financed the program) also participated in financing the second phase of CDCP. The overall amount covered under the World Bank Project for CDCP II was roughly US$216.4 million (IDA: US$125 million, DFID: US$91.4 million). 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 6. The project development objective (PDO) was to support the recovery of flood affected households by assisting the Government of Pakistan in strengthening the implementation of its Citizens Damage Compensation Program (CDCP). 7. The PDO was to be achieved through the provision of cash grants to targeted households, and through technical assistance to CDCP to strengthen its management, transparency and accountability. Key indicators were the following: 1. Coverage: At least 1 million flood affected, eligible households benefit from cash grants; 2. Targeting efficiency: Average baseline poverty score of CDCP II families will be lower than average baseline poverty scores of non CDCP families in each province; 3. Efficient delivery: At least 70 percent of beneficiaries express satisfaction with the program’s delivery system 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 8. Neither the PDO nor the key indicators were revised during implementation. 2 1.4 Main Beneficiaries 9. The project was designed to provide cash payments to an estimated 1 million poor households with housing damages caused by the 2010 flood. The beneficiaries could use the cash as they wished, but an underlying assumption was that cash transfers to households affected by housing damage would be taken as an opportunity for asset building and livelihood restoration, as the initial cash payment under CDCP I had overwhelmingly gone to meet consumption needs. The implementing agencies – the Cabinet Division and NADRA, as well as the Provincial Disaster Management Agencies (PDMAs) and the District Disaster Management Agencies (DDMAs) - benefited from technical assistance that strengthened their abilities to fulfill their roles in program implementation, albeit temporarily, since the need for such knowledge dispersed with the end of the project, reflecting its emergency nature. Permanent fallout from the project would prove to be a strengthened civil registry process in NADRA, as the CDCP, including the Bank-supported project, provided an opportunity to update and expand coverage of the national database and indicated means to strengthen national registry maintenance. As one feature of eligibility was to be in the database, this served to underscore to the population the importance of being on the national registry. 1.5 Original Components (as approved) 10. Component 1: Effective delivery of cash grants to flood affected families (total estimated cost – US$115 million). The component provided cash grants to eligible households in two installments through the commercial banking system. Provision of grants included the processes for identifying beneficiaries, delivering benefits and addressing complaints. Facilitation centers were established to support households through the processes. 11. The processes were as follows: • Coverage and Targeting. The program was to provide payments to at least 1 million households in the flood-affected areas, identified through provincial housing damage data, and verified by NADRA through triangulation with the beneficiary database from the first phase of the program, as well as by independent spot checks. • Payments. All eligible beneficiaries were to receive a total of PKR 40,000, irrespective of the degree of damage to their houses, as long as damage could be determined. Payments were to be made in two installments of PKR 20,000 each. A 10 percent contingency was included to address grievances of eligible beneficiaries. Payments could be drawn through ATM machines at regular bank branches using ATM debit cards, or through use of debit cards at point-of-sale facilities provided by participating banks at temporary payment centers, and based on beneficiary lists provided by NADRA. Information campaigns were carried out and help lines set up. • Grievance Redressal System: Depending on the grievance, different stakeholders were to respond: o Grievances related to personal identity cards. NADRA was responsible for ensuring that personal identity cards (necessary for getting ATM debit cards) 3 contained the correct information about the beneficiary; and NADRA was to regularly update the beneficiary database, as updated lists were received from the Provincial authorities (the Provincial Disaster Management Authorities, PDMAs). NADRA also operated help lines for beneficiaries to address grievances related to identity card details. o Grievances related to eligibility/targeting. District Review Committees (DRC), including representatives of civil society and the district administration, initially reviewed eligibility-related grievances, forwarded their recommendations to provincial appeals secretariats at the PDMA/RDMA level who in turn provided their final recommendations to NADRA for verification, amendment of eligibility rosters, establishment of payment lists and issuance of payment cards. Where District administrations were unable to effectively manage grievances, the PDMA/RDMA provided necessary support. o Grievances related to payments. The partnering commercial banks also instituted grievance offices and dedicated hot lines to respond to complaints related to payments. o The ombudsman process. As a last resort for grievances, federal and provincial ombudsmen would be available to the public to settle disputes. 12. The component included a social accountability and transparency element, focusing on spreading information about the program as broadly as possible. Eligibility criteria and beneficiary lists were published on the NADRA website and in notices placed in union councils (at the village and sub-village level). NGOs with presence in the flood-affected areas were enrolled to help inform about the program. A mobile-based text messaging system was set up by NADRA. Web-based updates on program progress were provided by NADRA on a dedicated website. A call center was also provided by NADRA for lodging complaints relating to service delivery issues, as well as for providing information to beneficiaries on the status of their grievances. 13. Component 2: Strengthen program management, monitoring and evaluation (total estimated cost US$10 million). This component was to provide technical assistance to strengthen program management, transparency and accountability at the federal, provincial and local levels through capacity building, communication and public information, monitoring and evaluation, as well as coordinating of advisory services for beneficiaries. The main areas of technical assistance included: • Operational Audits to review beneficiary targeting, payments, grievance redressal, public information, and program management and oversight. • Communications and Public Information where, to inform beneficiaries and minimize public grievances, support was provided to NADRA to develop an effective training and communication strategy. • IT software development and hardware provision was provided to complement the existing management information system (MIS) in NADRA, develop additional modules (e.g. for grievance redressal), and decentralize use of the MIS. 4 • Strengthening of the Cabinet Division through short and long-term consultancies to support it in its coordination and policy preparation functions, as well as in strategic questions on future disaster management with regard to cash transfers. Assistance was also provided to strengthen the financial management capacity of the Cabinet Division. • Strengthening of PDMAs/DCOs through technical assistance, training and hard and software acquisitions to support their playing an active role in this and future emergency programs, in particular in operations, outreach, communication and monitoring of the program. • Impact Evaluation. Using appropriate ex-post impact evaluation techniques to provide information on the short to medium term effects of Phase I and Phase II cash transfers on beneficiary households’ consumption, labor and investment outcomes and coping strategies. Negative or positive spillover effects on the local economy and community were assessed, Exclusion errors were assessed • Fiduciary and program management. Individual consultants including international and national technical experts and fiduciary consultants were to be contracted as needed. • Incremental operating costs to cover expenditures for the management and coordination of project activities which would not have been incurred in the absence of the project, excluding the salaries of the recipient’s civil servants. Financing. 14. Against the estimated US$580 million equivalent needed to meet the Government’s target of covering at least 1 million beneficiary households, a Memorandum of Understanding was signed on July 9, 2011 between the Government, USAID, and IDA, confirming IDA financing of US$125 million, USAID financing of US$190 million, and a Government contribution of US$100 million. Subsequently, DFID would participate with US$91.35 million equivalent and Italy with US$74 million equivalent. 1.6 Revised Components 15. Components were not revised during implementation. 1.7 Other significant changes 16. There were two level-2 restructurings of the project during implementation. The first, approved on January 9, 2011 was to reallocate US$5 million from component 1 of the project to component 2 to cover additional technical assistance costs. During appraisal the estimated cost of component 2 was US$15 million but during negotiations the amount was reduced and the minutes documented that if additional technical assistance costs would be required they could be reallocated at that time. The second restructuring occurred April 30, 2013 and extended the closing date of the project from June 30, 2013 to March 30, 2014. The additional months were required 5 to oversee final payments to beneficiaries. Other than these minor changes there were no significant changes to the project during implementation, but some elements supporting implementation were reinforced. They are indicated in the section on “Implementation”. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 17. Project preparation. As noted above, the Bank had considerable prior experience of crisis interventions in the region as well as in Pakistan. In supporting a second phase of the CDCP, the Bank was able to bring to bear this experience, as well as considerable international experience in designing cash transfer schemes, and this allowed improvements in program design and implementation that were likely to lead to better outcomes when focusing on recovery rather than immediate relief. 1 A technical assessment of the first phase of the CDCP, including a third party evaluation of it, was undertaken, and consultations were held with relevant stakeholders at the federal and provincial levels to provide a design for the program that was likely to be both relevant and implementable, taking into account circumstances on the ground. 18. As a result, the emphasis of the project was placed on recovery; and a number of changes were introduced in the design of the initial CDCP, which were spelled out in a Memorandum of Understanding (MoU) between the Government and the Bank. Agreement on the MoU would trigger Bank involvement in the second phase of the program. The measures are detailed in the PAD (page 11) and Annex 4 of the ICR, and summarized in the discussion on design below. To guide implementation, an Operating Manual (OM) was developed, setting out the roles and responsibilities of program stakeholders, as well as policies, processes and procedures for program administration and results monitoring. In order to help increase implementation capacity, the Bank team developed a risk assessment framework in collaboration with the counterparts; and it carried out a Risk and Opportunities Workshop before project start-up. The main risks defined during the workshop included: (i) uneven capacity at the provincial level and potential conflicts with the Federal government; (ii) potential difficulties in improved targeting when distribution was universal in flood affected areas under the first CDCP; and (iii) fraud and corruption, as well as inclusion/exclusion errors in identifying beneficiaries. 19. Processes were adjusted to respond to identified risks. Interested development partners were consulted on project design from an early stage, and this would subsequently help firm up their engagement in the program. The project was prepared in three months, but it would take an additional five months until initial disbursements were made, as implementation capacity at provincial and district levels 1 Post-disaster response distinguishes between immediate relief and recovery. Immediate relief focuses on keeping people alive and safe and ensuring that their basic needs are met; while recovery emphasizes restoring livelihoods and rebuilding assets. 6 had to be built up, and cumbersome government procedures slowed down disbursements. 20. Design. In developing the project, the Bank chose to engage in an ongoing program, reflecting the emergency nature of the intervention, as well as the absence of suitable alternative means to intervene in the crisis. At the same time, the nature of the emergency was changing - it was passing from early relief to recovery. Early relief had been addressed in the immediate aftermath of the disaster and in the first phase of the CDCP; and the second phase (the project) was expected to increasingly emphasize recovery. Government and donors, including the Bank, recognized this, and the MoU brought in recovery aspects into the design. They also recognized that recovery still would have to include relief; and .unconditional cash transfers were maintained for the second phase to provide recipients with the opportunity to themselves determine their specific relief or recovery needs 21. With the shift in emphasis towards recovery, the targeting method was changed. Under the first phase, mainly geographic targeting had been used; under the second phase, more rigorous targeting was applied, based on housing damage as the main determinant of need, and consequently eligibility. Beneficiaries were then identified through provincial housing damage data verified by NADRA through triangulation with the beneficiary database from Phase 1 of the program, as well as through independent spot checks; female-headed households and households headed by disabled persons were automatically included. The targeting mechanism was backed up by a grievance redressal mechanism to capture errors. It was further underpinned by requiring household members to be present at assessment. While this may have made the process more cumbersome, it did help ensure the legitimacy of the assessment. 22. The financial and technical assistance that was introduced under the project aimed at directly supporting a more effective implementation of the CDCP. The MoU between the Government and the Bank was introduced as a condition of effectiveness. It served to introduce a number of changes to the design of the CDCP, involving in particular targeting and a robust grievance redressal mechanism; clarification of institutional roles; improved public communications and increased transparency in implementing the CDCP; third party program monitoring and evaluation; formal contracting of participating commercial banks; and the elaboration of an Action Plan for the development of a more effective future disaster recovery response through cash transfers that would include appropriate mechanisms for asset recovery. The elements of the MoU are summarized in Annex 4 of the ICR. Each one of the changes introduced through the MoU would bring more rigor into the program; it was also recognized that they would challenge implementation capacity. Consequently the design included an important management, monitoring and evaluation component (component 2) that incorporated capacity building in the main implementing agencies, communications campaigns to support coherent messaging and information dissemination at province levels, and a rigorous grievance process to correct errors, and contain fraud and corruption. 23. Especially challenging elements of project design were, first of all, the new processes that were put in place – targeting, beneficiary selection, payment systems, grievance 7 mechanisms, that all were largely new activities that were unfamiliar to all the involved stakeholders, with the possible exception of NADRA, which at least had the experience of maintaining a database of citizens and issuing them identity cards as its main function. Other stakeholders, the Cabinet Division, the Provincial Disaster Management Authorities, and the District administrations had no experience of the kind; nor did the eventual beneficiaries. Realizing this, the Bank team assessed the overall risks as “high”, and introduced mitigating measures into project design: capacity building in the main implementing agencies, communications campaigns to support coherent messaging and information at province levels, and a rigorous grievance process to correct errors, and contain fraud and corruption. 24. Also realizing that the project involved multiple stakeholders, a number of coordinating bodies were set up. A high-level Strategic Review Committee would provide overall leadership and coordination; and operational issues would be addressed in an Operational Steering Committee. In addition, the three main entities, the Cabinet Division, NADRA and the participating provinces signed a tripartite agreement on implementing the project. 2.2 Implementation 25. The efforts that were made during project preparation to strengthen beneficiary selection and payment processes and address potential risks would pay off during implementation and allow the project to reach all its key objectives, and even exceed the target of one million beneficiary households. 26. Implementation challenges would mainly relate to targeting, and payment delays due to cumbersome government procedures. The grievance appeals system was seen as a key element of successful project execution and would prove particularly important in beneficiary selection. Housing damage as a determinant of eligibility, including the presence of a household member at the site, would initially prove awkward and result in a significant number of eligible households initially being excluded. Results were affected by the following elements: (i) the housing damage survey was conducted almost a year after the initial floods; (ii) in some cases, flood victims had been displaced, or they had already repaired damages to their houses; (iii) initially, communications to flood-affected households were weak, and many were unaware of the assessment; this was aggravated by the requirement that a household member had to be present at the assessment of damages to a house. All three elements initially led to significant errors of exclusion, but these were subsequently addressed through the grievance mechanism. The new targeting mechanism was cumbersome for good reason – with fairly large transfers, the focus had to be on reducing errors of inclusion, and then addressing the exclusion challenge through a rigorously applied grievance appeals system that came to play a critical role in determining legitimate beneficiaries and drawing up beneficiary lists. The communications program was subsequently strengthened with donor support. 27. Grievance appeals are standard features in safety net systems, and can be particularly important in emergency-type interventions, where often simple and straightforward selection procedures are applied within relatively short periods of time, as the case 8 was in CDCP II. Anticipating grievances, NADRA undertook a capacity assessment to define needs at the province and district levels to handle grievances; and subsequently provided training, and logistical and operational support to district registration centers. Other measures were also taken: where capacity gaps were obvious, the International Office for Migration (IOM) was contracted to help staff registration centers; and grievance redressal coordinators were stationed in every PDMA. The Cabinet Division developed filters for managing the caseload. Systematic information and communications campaigns, aimed at encouraging beneficiaries to file appeals or withdraw their funds, guided potential beneficiaries through the system. 28. As these measures were applied, they made for substantially improved targeting. Whereas initial beneficiary identification (the baseline) had resulted in the estimated exclusion of 61 percent of potentially eligible flood-affected households, with modified processes, and especially a fully operational grievance redressal system, errors of exclusion were reduced to 32 percent. Where exclusion did occur, it was driven by difficulties in capturing people without identity cards (who would be automatically excluded), those without permanent address, very isolated households and those living in insecure areas. Enrollment difficulties were encountered in particular in Balochistan and Gilgit-Balistan, where low administrative capacity in both areas, and stone-built, less flood-sensitive houses in the latter, resulted in few households being verified. Still, improved processes would reduce exclusion errors in Balochistan to a low 30 percent, compared to a baseline of 73 percent. 29. Risk management was taken very seriously during implementation: As acknowledged in the DFID assessment of the program (see Annex 9), whereas many of the risks considered high initially did not materialize (e.g. corruption, fraud), implementing agencies and provincial governments reported that they valued the additional rigor brought by World Bank fiduciary controls. The central risk management forum was the Operational Steering Committee (OSC) which partners felt was an effective forum for discussing and acting on emerging risks. 30. The establishment of targeting and payment systems that would, and did, minimize implementation risks, notably with regard to exclusion, fraud and corruption, did not immediately allow for a smooth roll-out. As noted above, the first disbursements under the CDCP II started in December, 2011, and some 37 percent of beneficiaries had not yet received their full allotment of two benefits by the time of the impact evaluation surveys in mid-2013. This reflects both the slow roll-out as systems were put in place and adjusted during implementation; awkward budget transfer processes at the central government level; and delays in releasing some donor funds (see p. 5, paragraph on Financing). All donor funds have now been released, and remaining payments are being made beyond project closing. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 31. Design. Monitoring arrangements built on the management information system (MIS) that had been put into place for the first phase of the CDCP. Project monitoring reports, administrative data on processes generated by NADRA, and a 9 program impact evaluation towards the end of the project were to allow project management and donors to monitor progress towards PDO targets. NADRA was to draw on its database of individual identity numbers to track individual beneficiary status and benefit receipt. Monitoring reports and administrative data from the field were to provide information on the implementation of key processes. These were to be underpinned by third party verification (spot checks) of the quality of damage assessments; and third party operational audits to assess how implementing agencies, notably provincial PDMAs and DRCs, were applying the processes outlined in the OM. The impact evaluation was to draw lessons about the CDCP program, its approach and impact on those affected by the flood. 32. Implementation. The M&E design was successfully implemented. Supported by field visits by Cabinet Division and NADRA staff, it was able to monitor the key processes (targeting, enrollment, benefit payment, grievance redressal) and signal problems as they arose. The supporting spot checks and operational audits were carried out as intended. The impact evaluation was implemented drawing on follow- up surveys to assess the effects of the cash benefit on beneficiary households. 33. Utilization. The information coming from the M&E system was used to inform decision makers on progress and to take action where issues arose. As a result, damage assessment methods were adjusted, communications’ channels and messages refined, targeted technical assistance, training and incentives introduced to counteract weaknesses at the provincial and district administration levels. Lessons learned through the process, and the findings of the operational audits, were actively applied to adjust the program while it was being implemented; the longer term impact of the lessons may be felt to the extent that they are incorporated into disaster management, for instance the disaster response action plan that was being developed under the project. Useful lessons were learned in implementing an emergency operation, but ultimately, the CDCP was a temporary expedient put in place to deal with a particular issue. It was not intended to become a permanent tool for tackling emergencies. 2.4 Safeguard and Fiduciary Compliance 34. There were no deviations or waivers from the Bank safeguards and fiduciary policies and procedures during the implementation of the project. No safeguard policies were triggered by the project. 35. Financial management. Cabinet Division was in charge of the financial flows relating to the cash transfers; NADRA was in charge of managing the technical assistance. Both had adequate internal controls in place for processing payments and accounting purposes. While Cabinet Division initially faced a problem of staff turnover, no substantive financial management issues arose in either Cabinet Division or NADRA during project implementation. An overall satisfactory financial performance was moderated by delays in the disbursement of funds by the Government. 36. For the external audit, Cabinet Division’s financial statements were audited on an annual basis by the Auditor General of Pakistan, and NADRA’s by an independent 10 external auditor. Audit reports reported no material weaknesses, but pointed to control weaknesses in accounting in both agencies. 37. Procurement. No procurement issues arose during implementation. Contract award was completed in record time, and contract management was handled without problems. 2.5 Post-completion Operation/Next Phase 38. The CDCP, on which the project built, was conceived as an emergency operation, designed to provide a temporary safety net in an environment where there were no adequate mechanisms. While the structure seems to be dismantling as the emergency is no longer there, it has underscored the need for one – in an environment susceptible to environmental shocks, ad hoc arrangements may not be the most cost-effective approach. In the meantime, the Benazir Income Support Program (BISP), a targeted, unconditional cash benefit, has become fully operational. The systems and poverty score card underpinning the program could now be used for emergencies. This is in keeping with the future disaster response action plan that was designed under the project drawing on the experience of CDCP. 39. The current CPS (for the period of 2015-2019) contains provisions to build flexibility for the possible reallocation of resources within the country’s IDA portfolio to enable a rapid response in crisis and emergency situations, if needed. The foundation for building a suitable institutional environment including a future disaster relief instrument was laid through the institutional support provided under the CDCP II, and was codified in the future disaster respond action plan, which was approved by the Prime Minister. As such it could be re-mobilized by drawing together existing disaster relief agencies, and the experience gathered under CDCP, and BISP. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 40. The project was designed to address a particular event – the 2010 floods – and the PDO was relevant to dealing with that event. It was intended to be a disaster recovery effort, albeit one that could offer relief, or support for basic consumption, as well as help restore assets and livelihoods. Actual spending would, of course, reflect households’ actual needs and preferences. Project implementation was relevant to the recovery effort. Risk of further crises remains, and preparedness and shared approaches for managing disaster risk remains an explicit element in the most recent Bank’s CPS (2015-2019). 3.2 Achievement of Project Development Objectives 41. The project was to support recovery of flood affected households, that is, protect basic consumption, recapitalize assets and restore livelihoods. Mostly, it was expected that a majority of recipients would use their benefit to restore assets. Here, the impact evaluation provides a more nuanced picture of what the project achieved. 11 It appears that most beneficiaries used their benefit for house repairs, and on food and debt repayment, suggesting that people were still to some extent in a relief mode, and still needed to meet basic requirements. The program’s most significant effect seems to have been that it lowered the likelihood of reduced food consumption in the household. When spending on recovery type activities, on building productive assets, the focus was not unexpectedly on agriculture and on livestock management. Altogether, asset restoration (house repairs, household assets and livestock) totaled 35 percent of spending. (Please see more detailed discussion of use and impact of cash transfers in Annex 3: Economic and Financial Analysis, and in Annex 6: Beneficiary Survey Results). 42. All three PDO indicator targets were met, and in fact exceeded. (i) The project was to provide cash benefits to at least 1 million flood-affected, eligible households. This was done: it provided full cash benefits of US$400 to close to 1.2 million households, and partial payments to another 0.6 million.2 The majority was in Punjab, Sindh and Khyber Pakhtunkhwa, although there also are significant numbers of beneficiary households (about 10 percent of the total) in Balochistan, despite the relatively high errors of exclusion. (ii) The targeting efficiency objective (poverty targeting) was achieved: the average baseline poverty score of CDCP recipients was some 25 percent lower than the average baseline poverty scores of non-CDCP recipients in each of the participating provinces. (iii) The project quality indicator was exceeded: some 78 percent of beneficiaries expressed satisfaction with the program, compared to a target of 70 percent. 43. In summary, the project supported the effective distribution of cash benefits to almost all poor households in the flood area. It helped maintain the relief effort, and it promoted the beginnings of recovery through asset building. While it was focused on ensuring that the immediate emergency was addressed, it also launched a longer tem institution by helping formulate a national disaster response action plan. Consequently, the outcome is rated satisfactory. 3.3 Efficiency 44. The ICR assesses efficiency against project design and implementation, and costs involved in achieving the project. 45. The following features of design and implementation influenced project efficiency. International experience, including in South Asia, and the trend in current best practice, suggest that post-disaster income support is best provided in the form of cash. This allows recipients to prioritize their spending, and the experience is that they do so sensibly. A number of criteria introduced through the MOU (Section 2.1. on Design) contributed to increasing efficiency: while stronger governance and 2 The exact number is 1,208,302 households. With an average household size of 6.5, a total of 7,853,963 people benefited from the program. This number includes households that had not yet received their full allotment at project closing. As noted in Section 2.2, some 37 percent of approved households had not yet received their full allotment (two payments) at the time of the impact assessment. Payments were continuing beyond project closing. 12 institutional streamlining, and more rigorous assessment and grievance redressal processes increased the efficiency of the beneficiary identification, selection and payment processes and, ultimately, allowed wider coverage of needy households than had been the case in the initial phase of the CDCP. Efficiency was somewhat reduced by a slow startup of the project (Triggering the need for an extension of the closing date by nine months), and there is some evidence that the provision of the benefit in separate tranches rather than all at once may have reduced the impact on livelihood recovery (see also Annex 3). 46. Administrative costs were low, estimated at between 7 and 10 percent, compared with 8-9 percent for other well executed cash and near-cash programs. This was largely a consequence of the relatively small number of large transfers involved, and the use of already existing supporting mechanisms for the project. 47. The ICR did not undertake quantitative analysis (cost effectiveness, benefit-cost analysis) of the program, noting its mainly short-term consumption effects on beneficiary households. Fiscal sustainability of the program is not an issue, as it supported short-term recovery of flood-affected households rather than the provision of a permanent safety net. 48. Overall, efficiency is rated substantial. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 49. The overall outcome of the project is rated satisfactory. The project objective – to help in the recovery of flood-affected households - was relevant, and in practice the project would come to support both relief and recovery: a significant number of households still felt a need to apply the funds to consumption-related expenditures three years after the flood. Project design was relevant in strengthening key elements of CDCP procedures, which was to bring more rigor into the program cycle, while at the same time maintaining low project costs. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 50. The targeting of female (and disabled) headed families that was introduced into the targeting process contributed directly to promoting economic empowerment of women, and inclusion of vulnerable groups. A key project outcome indicator addressed poverty targeting, measuring the average baseline poverty score of CDCP II households compared to non-CDCP II households, drawing on BISP’s poverty scorecard database. The average score for CDCP II households was 19.27, while the average score for non-CDCP II households was 24.14. (b) Institutional Change/Strengthening 51. The project had institutional implications: (i) it underlined the importance of getting a national safety net mechanism, and may have underlined the importance of the Benazir Income Support Program as a national safety net with the flexibility to scale 13 up its operations when necessary; and (ii) it signaled the need for a strong, coherent disaster relief system, and the importance of proceeding with disaster relief action plan. (c) Other Unintended Outcomes and Impacts (positive or negative) 52. Drawing on the Impact Assessment (Annex 6), unintended outcomes and impacts included the following: • The demand for labor in flood affected communities contracted over the period after the flood, and workers have been moving away from farm wage labor to work on own farm as the relative returns between each activity have changed. • The CDCP II transfer appears to have reduced the level of child labor, although child labor still remains an important livelihood strategy. • Migration is becoming an increasingly important livelihood strategy for flood affected households, with both the likelihood of a household having a member who has migrated and the total number of migrating members increasing over the period of the evaluation. The most common reason given for migrating is for employment with respondents citing, low wages and a lack of employment opportunities as important factors. This correlates with findings of a contraction in local labor markets. • The CDCP II transfer did not impact child nutrition, and child malnutrition was found to be at critical levels. Child nutrition is dependent on a number of factors other than economic access, which the CDCP II transfer did not address. These include the general levels of hygiene and sanitation in a community, education levels of parents, particularly the mother, and the morbidity of children. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 53. See Annex 6. 4. Assessment of Risk to Development Outcome Rating: Low 54. At the time of evaluation for the ICR, institutions specifically set up for the project, such as arrangements within the Cabinet Division and NADRA, or the district review committees, are being dismantled. This is not an unlikely outcome, given that the project served a specific purpose at a specific point in time. To that extent, the project risk is assessed as low. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 55. The Bank fielded an experienced team that could draw on prior work on social protection reform and emergency in Pakistan. It also brought broader international experience to bear. The team was able to prepare the project over a short three-month 14 period, and it became effective three months after preparation had begun. The changes in design of the CDCP that it insisted on introducing improved the rigor of the system, but also slowed down implementation in the face of capacity constraints in implementing agencies, notably at province and district levels, that had to be addressed during implementation. The team introduced a monitoring and evaluation system that allowed bottlenecks in implementation to be tackled, and lessons to be learned at the end of the project. The thorough work the team undertook during preparation provided the necessary comfort for other donors to participate in a Bank- administered multi-donor trust fund. The introduction of the action plan for the development of a future disaster recovery mechanism provided a valuable addition to the project, opening up the possibility to draw lessons from an operation where lessons otherwise easily might have been lost. Choosing to support an unconditional cash transfer proved to be sensible in a situation where individual household needs showed significant variation. (b) Quality of Supervision Rating: Highly satisfactory 56. Realizing that the CDCP for the second phase was different than the first one, the Bank team worked closely with the main implementing agencies (the Cabinet Division, NADRA) throughout the project. Attention was paid to the implementation issues that did arise, essentially process ones, and the team stood ready to advise and allocate resources, mainly technical assistance, to reduce their impact, or recommend other solutions. No safeguard policies were triggered, nor did any fiduciary issues arise. Generally, supervision was hands-on, and this was appreciated by the implementing agencies. The fact that the team was a mix of headquarters and country staff facilitated this. Performance reporting on project progress was thorough, and ratings fairly reflected the degree of seriousness of issues. Missions focused on advancing solutions to the main concerns of implementation – strengthening appeals and review processes at province and district levels (where the main capacity concerns lay, as already identified during preparation); streamlining the grievance process, and completing damage validation and eligibility for payment. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 57. The overall rating of Bank performance is satisfactory. It is based on the quality at entry and supervision ratings. A higher rating was preempted by insufficient consideration of sub-national capacity, which did affect the project throughout implementation. 5.2 Borrower Performance (a) Government Performance Rating: Moderately satisfactory 58. Throughout preparation and much of implementation, Government support for the project was strong. With a good track record from the first CDCP, it nevertheless agreed with the design changes for the second phase, including more rigorous 15 targeting to identify beneficiaries; this also gave the project a greater poverty focus. At the same time, delays in the release of Government funds were a persistent problem for the flow of funds in the project, having some adverse effect on implementation and are to some extent reflected in the lag in benefit payments. Decisions on implementing the future disaster action plan still remain to be taken. This was a moderate shortcoming, and government performance is rated moderately satisfactory. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 59. The implementing agencies, the Cabinet Division and NADRA, performed well. Their concern about capacity at the province and district levels throughout the project gave it a top-heavy emphasis, but was well-placed. Both applied a rigorous and active approach to implementation, focused on making it a success. Fiduciary responsibilities were met without problems. Some weaknesses in implementation might have been avoided, notably overcrowding at grievance redressal, with better- formulated information and communications campaigns. That is, however, counterweighed by negotiating a successful payment system with selected commercial banks and a nuanced approach to the diversity inherent in dealing with many provinces and districts with different capacities. Implementing agency performance is rated satisfactory. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 60. Overall Borrower performance is rated moderately satisfactory. While there were delays in the release of counterpart funds by the government, the project did exceed its key outcome targets, reflecting an overall strong implementation effort. 6. Lessons Learned 61. Some lessons to be drawn are the following: • Targeting. In an emergency program, targeting methods should be as simple and well understood and based on disaster impact rather than wealth. This places heavier reliance on grievance redress mechanisms, which become an important feature; this is not necessarily so in a normal project where more time can be spent on design than is appropriate in an emergency operation. Moreover, when significant amounts are being transferred, the system still requires rigor, and the focus should be on reducing inclusion errors. This may initially also lead to significant exclusion, but this can subsequently be addressed through the grievance mechanism. Requiring household members to be present at damage assessment in some instances proved to be cumbersome, but may have added to the legitimacy of the assessment process. • Phasing the response. While a post-disaster cash transfer program needs to be prepared more quickly after a disaster, applying a two-phase response may be effective – a first phase that is purely disaster targeted and provides immediate relief, focusing on keeping people alive and safe, and ensuring that their immediate needs 16 are met; and a second phase focused on recovery, not only protecting basic consumption but also recapitalizing assets and restoring livelihoods. • Identification cards. If national identification cards are to be used as an element to verify beneficiaries, they need to have broad coverage and be up-to-date. In the project, this did pose a risk and may have led to some degree of exclusion. On the other hand, it did offer incentives for both NADRA and flood victims to seek registration. Ultimately, maintaining ID cards up-to-date on a continuous basis should be a routine concern of NADRA or any other similar agency, as well as the population. That does, of course, require continuous communication about the importance of registration, especially in environments where civil registry is not recognized as an obligation. • Public awareness and well-prepared communications are essential to ensure smooth processes. This was not the case under the project; this created confusion and contributed to pressures on the grievance process. It points to the importance of carefully developing public awareness campaigns that draw on traditional (mosques, teachers) as well as modern methods of communications. • Cash payments. Recipients valued cash above other post-crisis assistance. Their needs continued to be a combination of relief and recovery, and cash gave them the flexibility to apply spending sensibly. • Payment in tranches. In the project, payment was in two tranches. In relief, where daily needs are being addressed, payment in tranches may be preferable. If the emphasis is placed on recovery, where single spending may be in larger amounts, a single, larger amount may be preferable. The use of debit cards and ATM machines is likely to reduce the risk of storing large amounts at home. • Impact Evaluation. The impact evaluation was accurate in capturing the impact of the two transfers under CDCP II on beneficiary households. However it was unable to capture the extent to which those transfers mitigated against the impact of the floods due to the lack of a pre-flood baseline. Preparing a pre-emergency baseline would help better understand the adequacy of transfer amounts and program design elements in mitigating against the impact of a disaster. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies N.A. (b) Cofinanciers N.A. (c) Other partners and stakeholders N.A. 17 Annex 1. Project Costs and Financing 3 (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD million) Appraisal million) 1. Cash Grants 570.00 514.22 90.2% 2. Program management, 10.00 15.00 150.0% monitoring and evaluation Total Baseline Cost 580.00 529.22 91.2% Physical Contingencies 0.00 0.00 Price Contingencies 0.00 0.00 Total Project Costs 580.00 529.22 91.2% Front-end fee PPF 0.00 0.00 Front-end fee IBRD 0.00 0.00 Total Financing Required 580.00 529.22 91.2% (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (USD million) (USD million) Government Budget 100.00 84.62 84.6% US: Agency for International Grant 190.00 190.00 100% Development (USAID) UK: British Department for Grant 91.35 91.35 100% International Development (DFID) IDA Grant 125.00 125.00 100% Italy Grant 74.00 38.25 4 51.7% Total 580.00 529.22 91.2% Note : Gains in financing were made due to significant fluctuations in the exchange rate of the PKR favoring the grants. 3 This includes financing by the Government of Pakistan, IDA funding, DFID TF managed by the Bank, and funds provided by other donors. For details see (b). 4 An Italian share of US$21.6 million was still outstanding at closing, tied to delays in the procurement of an auditor for the previous Italian disbursements. 18 Annex 2. Outputs by Component Component 1: Effective delivery of cash grants to flood affected victims (total actual cost US$514.22 million) The component was to provide cash grants to eligible households in two installments through the commercial banking system. Beneficiaries were identified through provincial housing damage data, and verified by NADRA through its population data base and spot checks in the field. All eligible beneficiaries were to receive, irrespective of the degree of damage to their houses, a total of PKR 40,000 (about US$390 equivalent) in two (?) installments of PKR 20,000 each. A grievance redressal system was to address grievances related to national identity cards, eligibility, and payments. To ensure transparency, eligibility criteria and beneficiary lists were to be published on the NADRA website and in hard copy at the union council level by district authorities. The component provided 1,204,624 people with cash benefits. Total complaints received till the reporting period were 10,813. Analysis revealed that about 7,828 (71 percent) of the complaints were lodged against the Banks including either there was no cash at the ATM or the ATM was over-crowded or due to load shedding the ATM was out of service. Most of such cases were happened in Baluchistan and Sindh. A total of 441 corruption cases were lodged. Component 2: Strengthen program management, monitoring and evaluation (total actual cost US$15.00 million) The component was to provide technical assistance to enhance program management, transparency and accountability at the federal, provincial and local levels through capacity building, communication and public information, monitoring and evaluation. The main areas of technical assistance included: • Operational Audits to review the performance of the program throughout the project cycle including beneficiary targeting, payments, grievance redressal, public information, management and oversight. • Communications and Public Information. To inform beneficiaries of key information and minimize public grievances, NADRA developed an internal and external training and communication strategy to inform beneficiary populations of the program and payment schedule. • IT software development and hardware complemented the existing MIS and develop additional modules (e.g. for grievance redressal) including decentralized utilization of the MIS. • Strengthening of the Cabinet Division. Short and long-term consultancies supported the Cabinet Division in coordination and policy preparation functions. • Strengthening of PDMAs/district authorities through capacity building by technical assistance Capacity will be built through technical assistance. • Impact Evaluation. Using appropriate ex-post impact evaluation techniques to provide statistically robust information on the short to medium term effects of Phase I 19 and Phase II cash transfers on beneficiary households’ consumption, labor and investment outcomes and coping strategies. • Fiduciary and program management. Individual consultants including international and national technical experts and fiduciary consultants were contracted. 20 Annex 3. Economic and Financial Analysis (including assumptions in the analysis) An economic and financial analysis was not done (see Section on Efficiency). The following summarizes the impact of the project on beneficiary households and their behavior, drawing on the findings of an impact evaluation report for the project (2013). In terms of losses, livestock loss affected 35 percent of treatment households and many households had other productive and household assets damaged by the floods. 28 percent of households had at least one durable good damaged beyond repair, and an additional 17 percent experienced a repairable damage to a durable good. Further to the direct damage caused by the flood, negative coping strategies adopted in direct response to the flooding led households to further deplete their stock of productive assets. The average livestock owning family incurred losses of livestock averaging approximately PKR 50,000. In addition, the majority of households who are crop agriculture dependent, lost crucial harvest crops from the spring Rabi harvest of 2010, as well as the loss of vital seed and fertilisers required to restart in the winter Kharif season. To this should be added the loss of income for households dependent on agricultural jobs. In assessing the value of the transfer compared to the damages, the total damage to the average flood affected household was PKR 57,402. Flood affected households may have received up to a total of PKR 60,000 if they benefited from all three tranches that were provided in Phases I and II. Thus for many households even the full value of both Phases of the CDCP transfer would not be sufficient to compensate for the full value of the loss caused by the flooding. In other households who were less severely affected by the flooding it is likely that the value of the transfer was more than losses. This suggests that in the future, consideration should be given to the possibility of flexibility in the value of transfer. While this would require extra effort in the targeting stage, it would ensure that the transfer more accurately reflected the needs of the beneficiaries and would free up extra resources for those who are the most severely affected by future flooding. While beneficiary households could use the benefit as they wished, an underlying driver was to support asset building and livelihood restoration. In line with this, benefits were mainly spent on repair & other house building related expenditures (22 percent). Expenditures on food took up 21 percent of the value of the transfer. Some 15 percent was used for debt repayment, mainly of debt accumulated to cope with flood-generated shocks, as borrowing from relatives was the most commonly used coping strategy. Some 14 percent was spent on agriculture and livestock, household assets and non-farm business investment. The emphasis of spending on asset building may reflect the lag between the shock and receipt of the cash. Where cash is being provided as emergency relief, the majority of the funds are often spent on immediate consumption. When the situation is less acute, cash can help to stimulate productive investment. The CDCP II transfers reached a large number of households more than a year and a half after the flooding, when they had already applied their coping strategy; selling livestock or increasing their debt levels. 21 Their needs were therefore to invest in productive assets, as immediate needs, such as basic essentials (i.e. food) has already been satisfied or dealt with. Annex Table. Project Transfers by Expenditure Type 5 % of Total Value by Expenditure Type Repair & other house 22 building related Food 21 Debt Repayment 15 Medicines and health 13 services Agriculture and 10 livestock purchases Household assets 3 Education 1 Non-farm business 1 investment Savings 1 Other 13 100 There is some evidence that efficiency was somewhat diluted by a slow startup of the project, and delays in benefit payments. The first benefits were distributed almost a year after project approval, or some two years after the flood. Three years after the flood, 63 percent of eligible households had received both tranches, 24 percent had received one tranche, while 13 percent had not yet received a benefit. With such delays, beneficiary households are likely already to have addressed immediate needs, influencing how project moneys would be spent. A one-off cash transfer given in the early stages of emergencies is likely to be spent on priority basic needs, such as food and emergency health care. Transfers given later in the recovery stage may be better suited for investments and recovery of livelihood assets. CDCP II presents a mixed picture, with the bulk of benefit spending going to housing repairs, food and debt repayment. In part this may reflect the emphasis of selection criteria on the assessment of housing damage and the less well-off. In part it may also reflect the design of benefit delivery. Cash transfers were delivered in two separate tranches rather than all at once. In the impact evaluation, several beneficiary focus groups indicated that PKR 20,000 was a small amount relative to requirements for indivisible asset purchases. 5 Source: Impact Evaluation Report (2013) 22 Annex 4. The main elements of the Memorandum of Understanding on changes to the CDCP process, agreed between the Government and the Bank MoU points Agreed Actions Adopt improved Housing damage, captured through provincial surveys in flood affected areas, would targeting target beneficiary households for Phase II. The Bank and donors agreed subject to the following: 1) satisfactory validation of the survey methodology and data to ensure that it is objective and accurate; 2) exclusion of the well-off; 3) equal payment for beneficiary independent of the extent of housing damage; 4) inclusion of legitimate flood affected vulnerable households from Phase I not eligible through the targeting criteria in Phase II. Clarify the roles and Institutional roles clarified by assigning the Cabinet Division and NADRA the financing by Federal respective roles that they had performed in the first phase, appropriately enhanced with and Provincial levels: technical assistance. The Government co-financing for the CDCP Phase II was confirmed. Disburse the payments Beneficiaries were to receive two installments of PKR 20,000 each. in installments Strengthen the The grievance redressal and public information campaign in coordination with grievance redressal PDMA’s, for Phase II was to be improved. The Government agreed mechanism and to settle all legitimate grievances of Phase I institute a robust public information campaign Deploy robust A third-party rapid implementation evaluation of the first phase of the CDCP evaluation systems was deployed in January 2011. A more rigorous impact evaluation was to be carried out in Phase II. Operational audits were also to be fielded. Formal contracts with Were to be negotiated participating commercial banks Strengthen governance The project would emphasize governance through effective controls and accountability of the program mechanisms (above) as well as public disclosure of program performance, hotlines for grievances, and strengthening of oversight institutions e.g. Cabinet Division. Develop advisory NGOs were to provide advisory services to beneficiaries once the payments are made. services for the poor to This was to be done through targeted communication and coordination with the main assist in responsible use service providers such as Rural Support Program Network (RSPN), Pakistan Poverty of funds Alleviation Fund (PPAF), Strengthening Participatory Organization (SPO), and others. Time-bound action plan The Government understood the need to develop more sustainable mechanisms for the for the development of a recovery phase of disasters. Cabinet Division was tasked with this endeavor. The future disaster recovery Action Plan was to be a condition of disbursement of category 2 (cash transfers) of the response through cash proposed Bank project. transfers that include appropriate mechanism for asset recovery Source: PAD 23 Annex 5. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Andrea Vermehren Senior Social Protection Specialist SASSP (Co-Task Team Leader) Iftikhar Malik Senior Social Protection Specialist (Co- SASSP Task team leader) Cem Mete Senior Economist SASSP Maria Gracheva Senior Operations Officer SASHN Naoko Ohno Operations Officer SASHN Nistha Sinha Senior Economist PRMGE Yue Man Lee Young Professional SASHD Kelly Johnson Extended term Consultant OPCFC Amjad Zafar Khan Extended Term Consultant SASSP Martin Serrano Senior Counsel LEGES Akiko Ogawa Counsel LEGES Asif Ali Senior Procurement Specialist SARPS Shahzad Sharjeel Senior Communications Officer SAREX Khalid Bin Ajum Procurement Specialist SARPS Hasan Saqib Sr. Financial Management Specialist SARFM Chau-Ching Shen Sr. Financial Officer CTRFC Anwar Bhatti Financial Analyst SACPK Chaohua Zhang Sr. Social Sector Specialist SASDS Samina Musarrat Islam Social Development Specialist SASDI (Consultant) Javaid Afzal Senior Environment Specialist SASDI Francisco Ayala Consultant ECSHD Alison Barrett Consultant SASHD Quanita Ali Khan Consultant SASSP Mohammed Aslam Malik Team Assistant SASSP Elfrida Vincent Program Assistant SASSP Supervision/ICR Iftikhar Malik Senior Social Protection Specialist (Co- SASSP Task team leader) Cem Mete Senior Economist SASSP Maria Gracheva Senior Operations Officer SASHN Naoko Ohno Operations Officer SASHN Kelly Johnson Operations Officer (Co-Task Team SASSP Leader Amjad Zafar Khan Social Protection Specialist SASSP Martin Serrano Senior Councel LEGES Shahzad Sharjeel Senior Communications Officer SAREX Khalid Bin Ajum Procurement Specialist SARPS Hasan Saqib Sr. Financial Management Specialist SARFM Chau-Ching Shen Sr. Financial Officer CTRFC Anwar Bhatti Financial Analyst SACPK 24 Shahzad Sharjeel Senior Communications Officer SAREX Khalid Bin Ajum Procurement Specialist SARPS Hasan Saqib Sr. Financial Management Specialist SARFM Chau-Ching Shen Sr. Financial Officer CTRFC Anwar Bhatti Financial Analyst SACPK Quanita Ali Khan Consultant SASSP Mohammed Aslam Malik Team Assistant SASSP Elfrida Vincent Program Assistant SASSP Mohammad Khalid Khan Program Assistant SASSP (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) USD Thousands Stage of Project Cycle No. of staff weeks (including travel and consultant costs) Lending 111.10 749,316.83 Total: 111.10 749,316.83 Supervision/ICR 101.31 427,145.39 Total: 101.31 427,145.39 25 Annex 6. Beneficiary Survey Results 6 The beneficiary survey compares the situation of beneficiary and non-beneficiary households at the time of their selection into the programme (baseline), with their situation 12 months later. Over that 12 month period beneficiary households were expected to receive PKR 40,000 in two tranches of PKR 20,000. Experience with the CDCP Phase II transfer Despite the launch of an extensive grievance redressal mechanism, 32 percent of flood affected households who attempted to become Phase II beneficiaries were unsuccessful in their applications. Exclusions errors were partly explained by high exclusions in Balochistan which as significant accessibility, security and administrative challenges. Based on the initial survey prior to launching the grievance process, errors of exclusion were at 61 percent. Almost 40% of households entered a grievance case because: 1) they were excluded due to completed housing repairs; or 2) housing damage was not assessed; or 3) assessment findings were considered incorrect by the household. Beneficiaries experienced delays in receiving payments, with just 63% of beneficiaries having received both tranches of the transfer and 13 percent of households having received neither tranche, more than a year after the inauguration of Phase II and almost three years after the 2010 flooding. Despite these delays beneficiaries were in general satisfied with the enrolment and payment mechanisms. The majority of households expressed that their experience with both of these mechanisms was either good or very good. There was less satisfaction expressed with the grievance redressal mechanisms. Whilst this is likely a reflection of the proportion of households who had a grievance case rejected, there was some evidence from the qualitative research of a lack of responsiveness and transparency in the process from the point of view of the applicants. The majority of the CDCP Phase II transfer was spent on house repair, food and debt repayment. Suggesting that many households are still recovering their basic requirements following the flooding, though there was also evidence of some expenditure on productive assets, particularly for agriculture or livestock. Livelihoods and household productive assets Flood affected households are predominantly dependent on agriculture. However, the demand for labour in flood affected communities has contracted and workers are moving away from farm wage labour to work on own farm. 6 The text is drawn from the CDCP Impact Evaluation Report 26 There appears to be little impact on the proportion of the adult population that is inactive, allaying fears that a cash grant may discourage adult work. Instead, there is a reduction in the level of child labour particularly in Punjab and Balochistan. There is strong evidence that migration is becoming an increasingly important livelihood strategy for flood affected households, with both the likelihood of a household having a member who has migrated and the total number of migrating members increasing over the period of the evaluation. The CDCP Phase II transfer reduced the number of migrating members in Punjab, suggesting that at least in the short term the transfer has protected beneficiary households from adopting migration as a livelihood strategy. The CDCP Phase II transfer has had a positive impact on livestock ownership in all provinces bar Punjab. This is likely a result of households rebuilding herds lost to the flooding. The transfer has had an impact on the ownership of some productive assets including blacksmithing tools and motorcycles. Economic and material welfare and poverty Despite impact on purchases of livestock and productive assets, there is no impact on consumption expenditure. This finding is consistent with the fact that the CDCP Phase II transfer is a one-off cash grant, rather than a regular and reliable source of income which could be used by to consistently increase their level of consumption. Despite the CDCP Phase II transfer, poverty rates did not change in most provinces and in fact increased in Sindh and Balochistan. Many households in Sindh and Balochistan have experienced multiple rounds of flooding and it is likely that this is contributing to the increase in poverty observed. However the transfers did reduce the likelihood that households reduced food consumption as a coping mechanism suggesting they prevented further increases in poverty. The CDCP Phase II transfer does not have an impact on household amenities including access to clean drinking water, sanitation and electricity. Food security There was no positive impact of the CDCP Phase II transfer on household food security across multiple measure of food security. In line with the findings on poverty, there was evidence of increased food insecurity for flood affected households in Sindh and Balochistan in terms of overall food expenditure and dietary diversity. The CDCP Phase II transfer does not impact child nutrition, and child malnutrition was found to be at critical levels. Child nutrition is dependent on a number of factors other than economic access that the Phase II transfer does not address. These include the general levels of hygiene and sanitation prevalent in a community, education levels of parents particularly the mother and the morbidity of children. 27 Access to services There was no impact on savings and the proportion of households with savings remains low. However, financial penetration remains low particularly in flood affected communities which are predominantly rural. Furthermore evidence of impact on livestock and other productive assets could indicate that households are using other informal savings methods. There was no impact on expenditure on health and only a small impact on education expenditure in Sindh. The lack of impact on health expenditure is likely related to households viewing health spending as a necessity which must be ‘afforded’ in the face of a health shock. The lack of a significant impact on education expenditure is likely related to the nature of the cash grant. Given that it is one off households will not rely on it for the continued education of their children. Vulnerability to shocks The CDCP Phase II transfer significantly reduces the likelihood of reducing food consumption in the face of an exogenous shock suggesting that the Phase II transfer has had a role to play in protecting households against shocks to their welfare. 28 Annex 7. Borrower’s ICR and/or Comments on Draft ICR Genesis of CDCP Over the course of the monsoon season in July and August 2010, Pakistan experienced the worst floods in its history. Heavy rainfall caused flash and riverine floods in the north and north-western regions of Pakistan (parts of Khyber Pakhtunkhwa [KP], Gilgit Baltistan [GB], Balochistan, and Azad Jammu and Kashmir [AJK]) that combined to create a moving body of water equal in dimension to the land mass of the United Kingdom traveling southwards. The floods affected 78 districts and nearly 10% of Pakistan’s population over a vast geographical area. The National Disaster Management Authority (NDMA) has estimated that in the affected districts, the floods covered over 100,000 square km affecting more than 20 million people3, with over 1,980 reported deaths and nearly 2,946 injured. Approximately 1.6 million homes have been destroyed, and 2.4 million hectares of crops damaged. The Multi-Cluster Rapid Humanitarian Needs Assessment (McRAM) conducted in August 2010 found that farm and non- farm livelihoods were severely affected with 60% of households describing themselves as without a main source of livelihood post floods, and 53% of households reporting 76- 100% decline in household incomes. The overall recovery and reconstruction cost associated with the floods is estimated to be between US$8.7 and US$10.8 billion, which includes costs for relief, early recovery, and medium to long term reconstruction. International experience and evidence largely endorses the use of cash transfers at the recovery stage of post disaster response. In light of the lessons from international experience on best practice, the Citizen’s Damage Compensation Program has been reasonably designed and implemented. The floods were likely to have worsened The initial payment of PKR 20,000 helped the poverty situation in Pakistan. Support households to cover immediate needs. to flood affected households was However, it was unlikely to be sufficient essential in mitigating the adverse effects for recapitalization of assets given the of income shocks and increased poverty extent of damage and loss. The rapid and vulnerability. Pakistan’s existing evaluation of Phase I found families social safety nets were unable to cope spending the grants mostly on food, with the scale of support needed at the healthcare and medicines, repair of recovery stage. Therefore, a rapid housing, and paying off debts. The GoP response cash transfer program focused sought to provide total additional on supporting flood affected families was payments of PKR 40,000 to flood affected households. The selection of beneficiaries was based on their housing damage as a proxy for eligibility. In addition, particularly vulnerable flood affected households (female and disabled headed households) from Phase I were included. Beneficiaries were expected to use these funds to not only cover basic consumption but to also recapitalize assets, recover their livelihoods and repair housing. The GoP sought to provide these additional payments to an estimated 1.1 million households. As there were, on average, 6.9 people per household in flood affected areas in Pakistan, 29 the program was expected to reach between 7.5 and 8.3 million people. Program Objectives The objective of the program was to support the recovery of flood affected households by assisting the Government of Pakistan in strengthening the implementation of its Citizen’s Damage Compensation Program. This was achieved through the provision of cash grants to selected households through the banking system (Component 1), and through technical assistance to enhance program management, transparency and accountability at the federal, provincial and local levels through capacity building, communication and public information, monitoring and evaluation, as well as coordination of advisory services for beneficiaries (Component 2). Key performance indicators The following set of quantitative and qualitative indicators was used to assess project performance: Coverage: At least 1 million flood affected, eligible households benefit from cash grants. Targeting efficiency Average baseline poverty score of CDCP II families will be lower than average baseline poverty scores of non CDCP families in each province. Efficient delivery: At least 70% of beneficiaries express satisfaction with the Program’s delivery system. Achievements of CDCP Component 1 Component 1 comprised of effective delivery of cash grants to flood affected families. The component provided cash grants to eligible households in two installments through the commercial banking system. Following are the details highlighting the CDCP achievements; Enrolment The program provided payments to an estimated 1.1 million households identified through the provincial housing damage data, and verified by NADRA through triangulation with the beneficiary database of Phase I of the program, as well as independent spot checks according to mutually agreed parameters. The Program included beneficiaries which were registered in the notified flood- affected areas including the legitimate vulnerable. Province wise Breakup – Watan Card Facilitation Centers S. No. Province Total Affected Total WCFCs Closing Date Districts Launched 1 Khyber pakhtunkhwa 26 26 Jan 31, 2013 30 2 Baluchistan 13 14 Jan 31, 2013 3 Sindh 14 17 Jan 31, 2013 4 AJ&K 10 10 Mar 31, 2013 5 Gilgit Baltistan 07 05 Jan 31, 2013 6 Punjab 11 21 Dec 31, 2012 Beneficiaries’ Biometric Verification and Debit Card Processing S. No. Province Total Caseload Processed Processed (NADRA) (Banks) 1 Khyber pakhtunkhwa 312,477 302,314 302,112 2 Baluchistan 124,537 117,295 116,966 3 Sindh 433,180 416,081 415,644 4 AJ&K 14,951 14,309 14,302 5 Gilgit Baltistan 12,823 12,445 12,423 6 Punjab 351,253 345,859 345,816 TOTAL 1,249,221 1,208,303 1,207,439 Payments All eligible beneficiaries received, irrespective of the degree of damage to their houses, a total of PKR 40,000. Payments were made in installments of PKR 20,000 each, and were staggered within and between districts and provinces in accordance with the logistic plan agreed with partner Banks. The program included provisions for a 10% contingency to attend the grievances of eligible beneficiaries. Payments were delivered using the same system as Phase I, wherein beneficiaries were paid using ATM/debit cards issued to them during Phase I. Phase II beneficiaries were re-certified by NADRA if necessary and informed of the amount, payment period, and payment points as well what documentation to bring to the payment center via a communications campaign. Informed by NADRA of the list of beneficiaries, Partner Banks set up Point of Sale and ATM machines at payment centers, created virtual accounts for payment and activated debit cards enabling beneficiaries to withdraw their funds. Help lines were set-up by NADRA and the Partner Banks to support beneficiaries who encounter difficulties with withdrawing their funds, lost their debit cards or PIN or experienced other issues related to payment. Sr. No. Province Total Amount Disbursed (PKR) 1 Khyber pakhtunkhwa 12,049,940,000 2 Baluchistan 3,474,560,000 3 Sindh 16,161,740,000 4 AJ&K 570,000,000 5 Gilgit Baltistan 496,720,000 6 Punjab 13,813,760,000 TOTAL 46,566,720,000 Grievance Redressal System Grievance redressal centers were opened and staffed by NADRA and District officials (DCOs). These were responsible for addressing grievances related to eligibility, and inform appellants as to why they were excluded. Depending on the nature of the 31 grievance, different stakeholders were responsible for providing solution to the grievances: Grievances related to Computerized National Identity Cards (CNIC) NADRA ensured the updating of CNICs, as appropriate, including changes in family status, updating address, family name etc. NADRA also regularly updated the beneficiary database, as per the updated lists received from the PDMAs. NADRA operated help lines for beneficiaries to address grievances related to CNIC details. Sr. No. Province Updates Filed Updates Resolved Pending 1 Khyber pakhtunkhwa 1,104 1,096 08 2 Baluchistan 1,004 986 18 3 Sindh 524 500 24 4 AJ&K 123 123 0 5 Gilgit Baltistan 10 10 0 6 Punjab 3,055 3,047 8 TOTAL 5,820 5,762 58 Grievances related to eligibility/targeting The District Administration was the key local body collating the beneficiary lists based on household damage. The beneficiaries whose names did not appear in the lists were first checked with their local authority at the moaza/goth/village level and requested for an update. This information was then passed on the District Administration, who subsequently informed the PDMAs to update the beneficiary lists. These updated beneficiary lists were then sent to NADRA, who verified and included eligible beneficiaries into the formal lists for generation of payments and issuance of Watan cards. Where the District Administrations failed to effectively manage these grievances, the respective PDMA was responsible for ensuring the capacity and transparency of the process. Province Appeals Resolved by Approved by Passed Lodged DRC DRC through Business Rules Punjab 301,802 301,802 35,754 31,266 Sindh 436,810 433,957 214,733 184,556 KPK 284,125 277,263 53,876 32,398 GB 13,499 13,452 9,384 8,380 AJK 25,034 24,892 9,984 7,780 Baluchistan 118,686 110,613 88,661 74,473 TOTAL 1,179,956 1,161,979 412,392 338,853 Grievances related to District Administration, NADRA or Partner Banks 32 The partner commercial banks responded to grievances related to payments, such as lost Watan cards, forgotten PIN numbers etc., the Partner Banks operated dedicated hotlines. These help-lines were used to provide guidance about the different grievance processes. Total Closed Complaints Status DA NADRA Alfalah UBL HBL Closed Complaints 2213 711 126 1727 5944 Total Complaints 2250 715 132 1752 5944 %age 98.4 99.4 95.5 98.6 100.0 Component 2 It comprised of strengthening program management, monitoring and evaluation (total estimated cost – US$10 million). This component provided technical assistance to enhance program management, transparency and accountability at the federal, provincial and local levels through capacity building, communication and public information, monitoring and evaluation, as well as coordinating of advisory services for beneficiaries. This component was implemented by NADRA. The main areas of technical assistance included: Operational Audits The objective of the audit process was to review the performance of the program throughout the project cycle including beneficiary targeting, payments, grievance redressal, public information, management and oversight. Communications and Public Information To inform beneficiaries of key information and minimize public grievances, NADRA developed an effective internal and external training and communication strategy. This also informed existing organizations providing advisory services to beneficiary populations of the program and payment schedule allowing them to provide timely support to beneficiaries. IT software development and hardware IT software and hardware was developed / procured to complement the existing Management Information System (MIS) and develop additional modules (e.g. for grievance redressal) including decentralized utilization of the MIS. Strengthening of the Cabinet Division Short and long-term consultancies were hired to support Cabinet Division in its coordination and policy preparation functions (for NODMC), as well as in strategic questions on future disaster management with regard to cash transfers. Assistance was also provided to strengthen the financial management capacity of the Division through staffing and provision of improved office facilities. Strengthening of PDMAs/DCOs Capacity was enhanced through technical assistance, training and possible hard and software acquisitions to support their playing an active role in this and future emergency 33 programs. Assistance was given to enhance their role in operations, outreach, communication and monitoring of the program. Impact Evaluation Impact evaluation was carried out by using appropriate ex-post impact evaluation techniques to provide statistically robust information on the short to medium term effects of Phase I and Phase II cash transfers on beneficiary households consumption, labor and investment outcomes and coping strategies. Negative or positive spillover effects on the local economy and community was assessed. It also explored the extent to which any households have been inappropriately excluded from the program and if so whether there were any common factors leading to their exclusion. Following are the key findings of impact evaluation exercise; • The payments system seems to have been a success with beneficiary households satisfied with the way in which payments were delivered to them with more than 80% of beneficiaries reporting that they thought the payments system was very good or good. • Encouragingly it was found that the CDCP transfer reduced the incidence of child labor by 9% in Punjab. • Evidence was found that the CDCP induced an increase in the incidence of livestock ownership by 4% nationally with statistically significant impact in Sindh (impact of 8%) and Khyber Pakhtunkhwa (impact of 16%). Further, CDCP Phase II transfer enabled beneficiary households to purchase larger animals such as cows and buffaloes. • In addition, the CDCP appears to have had an impact on the ownership of productive assets). In particular the transfer has caused an increase in the ownership of blacksmithing tools (7%), sewing/knitting tools (5%), food preparation tools (7%) and motorcycles (7%). • CDCP transfer may eventually have an impact on consumption expenditure and poverty through its indirect effects on livelihood recovery. • It was found that the CDCP transfer impacted on the proportion of households whose main coping strategy was to consume less or lower quality food. Lessons Learnt The program has generated following lessons learnt during its implementation. Enrolment & Payment • Electronic ticketing can be used through the SMS Server for more effective/planned mobilization in any future programs. • At the time of issuance of ATM Cards at WCFCs, the ATM card number should be saved in the NADRA database against the particular CNIC. This will help eliminate chances of malpractices and aid in data reconciliation activities. • The partner banks must provide real-time electronic feedback to the implementing agency for each transaction and payment to beneficiaries. This will provide the stakeholder with the opportunity to track the progress of payments made to the beneficiaries. 34 • Installation of Point of sale (POS) to be ensured for facilitating payments to beneficiaries at the centers; adopting one window operation mechanism. • Provision of a valid mobile number should be mandatory for all beneficiaries of any future programs. This number can be used by all the stakeholders for contacting the beneficiary and in particular by the partner banks for informing the beneficiaries about card activation and amount withdrawal. Grievance Redressal • GR Registration time period should be identified at the Planning stage of the program and a cut-off date should also be communicated to the beneficiaries accordingly. This will ensure that the beneficiaries are fully aware of the program Grievance mechanism and duration at the time of registration. • A mechanism should be officially recognized by all stakeholders that enables the District Administration to provide a schedule for mobilization of the GR appellants in registration of their appeals. This would help organize the queue management at the facilitation centers. • At the time of Data Entry of GR, the CNIC validation and verification should be made part of the MIS. This will ensure the appellant is valid and further filter out any fraudulent or invalid CNICs registration; hence reducing the invalid entries in the system and ensuring a more accurate and transparent system. • A provision to resolve/review the Appeal Cases using the real-time MIS system. This will help track the appeals within the system, eliminating the issue of missing appeal cases. It will also enable NADRA to generate real-time reports for Appeals. Monitoring & Evaluation • For future programs, the Monitoring & Evaluation function should be strengthened to cater the program needs at the time of inception. • Operational Audit function has been instrumental in providing external monitoring to the project. For future preparedness, Operational Audit firms should be identified and reserved (with MOUs) by the stakeholders so that the quick mobilization of operations is underway in case of any future calamity/disaster response. • Instructional Capacity of PDMAs should be strengthened for Monitoring & Evaluation. Public Information Campaign • Before launching any future programs, a work shop may be held at Divisional level for briefing the concerned stakeholders about the awareness of the program as well as the Campaigns and their role in beneficiary facilitation. • Metrics and methodology to determine the accurate effectiveness of the media campaigns should be devised in the Planning stage of any future programs. This would highlight the accurate representation of the effectiveness of the Campaigns. • Social Mobilization also plays a major part in any PIC for information dissemination to the beneficiaries. It is suggested that the traditional methods (Announcements from the Masjid, School Teachers and Patwaris) in rural areas 35 are found to be most effective for social mobilization and special funds may be dedicated for this purpose. Borrower’s Rating of Performance Overall, the performance of CDCP can be rated as “Satisfactory” as the program has achieved its objectives of covering 1.2 million flood affected households out of which more than 80% were satisfied with the cash assistance and delivery mechanism. There were significant improvements in a number of indicators such as livelihood recovery, assets, economic & material welfare, food security, access to services and vulnerability to shocks (reference Impact Evaluation Report). Bank’s Performance The World Bank and development partners were very much proactive in providing necessary support for preparation and implementation of Program operations. 36 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders The following draws on comments in DFID’s assessment of the program. It largely expresses concerns or views that also were expressed by IOM, USAID and the Italian representative. It is a strongly abridged version of DFID’s conclusions. Impact. CDCP II has met expectations in terms of the number of flood-affected people who benefited from the transfer and meets targets on women beneficiaries. Few other interventions following the 2010 floods had such wide coverage and feedback from the Government and targeted communities has been positive. CDCP II allowed beneficiaries to meet some of their immediate food and health needs and to rebuild their home or paid debts from earlier rebuilding efforts. The Impact Evaluation demonstrated it enabled beneficiaries to replenish some of their livestock. Yet the payments did little for the poverty rate, childhood nutrition or food security. Part of this is due to the size of the transfer and the serious payment delays. Exclusion errors reduced its potential coverage to more flood-affected households. Targeting. While beneficiaries were satisfied with the transfers, CDCP II had serious delays, starting with the initial targeting system. Several aspects were at odds with the nature of the floods: (i) the household damage survey required a member of the household be physically present to provide CNIC information, even though the floods had displaced many households for months; (ii) it was clear villages in flood plains had a very high proportion of their households affected and individual household damage surveys were unnecessary; (iii) while household damage was designed as a proxy for flood affectedness, it underemphasized wider livelihoods impact (such as damage to agriculture land/crops, livestock); (iv) assessments based on household damage were time-consuming and costly to administer. Government systems, the operational audit, the third party validation and the support to grievance redressal worked effectively to [avoid inclusion errors]. In contrast, there was relatively little focus on exclusion errors. Grievance. Thirty percent of the ultimate beneficiaries came through the grievance mechanism and this added to efforts, corruption risks, administration costs and delays. Although the grievance process was well managed, relying on proactive complaints through the grievance mechanism was likely to exclude the vulnerable and marginalized. . Communications. Beneficiaries had varying degrees of awareness about the source of funds. Some knew it was from the government, but still others did not, and some of the groups surveyed believed it was direct from God, from Benazir Bhutto from the grave or from Turkey. Beneficiary feedback and the high number of grievances suggest the communication campaign was insufficient in informing people about eligibility criteria. There was too great a focus on formal media (e.g. newspapers). Through IOM, DFID provided NADRA with important resources for communications, but the campaign terminated early, leaving communities with little information on the closure of centers: some were still waiting further payments. 37 Payments. The use of debit cards and national ID card verification contributed to a transparent implementation with minimal fraud and financial mismanagement. The majority of recipients were satisfied with the use of debit cards, though they would have preferred a single, large transfer to be able to invest in larger assets. Risks. Many of the risks considered high in the initial WB project document did not materialize. The risk of improper use of funds was minimized by the World Bank’s fiduciary controls, the use of bank accounts, mag-stripe cards combined with biometric identification, and through Third Party Validation. Implementing Agencies and Provincial Governments reported they valued the additional rigor brought by World Bank systems. The central risk management forum was the Operational Steering Committee (OSC) which most partners felt was an effective forum for discussing and acting on emerging risks. There was evidence that some of the OSC was not receptive to some of the challenges identified by PDMAs, such as the speed at which the initial household damage survey could be completed. The Operational Audit undertaken by Deloitte was an important source of information on practical challenges and the experience of beneficiaries. Governance. World Band technical assistance and the trust fund appears to have provided a good coordinating center and fiduciary reassurances, however there is room to make better use of technical expertise in social protection. Sustainability. There has been some good capacity building and sharing of experiences amongst provinces, but with significant expertise held by individuals, there is a risk of knowledge being lost. Provincial Disaster Management Authorities are functional and active at provincial level, but have limited actual presence in the districts. District Disaster Management Authorities are only partially functional and district-level capacity to monitor or support CDCP payments or complaints is limited. Value for Money. CDCP II was very “cost-efficient” with low program costs relative to the transfers to beneficiaries. 38 Annex 9. List of Supporting Documents CDCP – Building Resilience – Case Study, June 2013, World Bank CDCP Impact Evaluation Report, Oxford Policy Management, August 2013 DFID – Project Completion Review – CDCPII, February, 2014 Emergency Project Paper – Pakistan Flood Emergency Cash Transfer Project, March 16, 2011 Final Operational Audit Report – CDCP II; Deloitte, December 13, 2013 Implementation Status and Results (ISR) 2,3,5,6 Memorandum of Understanding between Pakistan, the US, the World Bank and Additional Development Partners, June 9, 2011 Restructuring Paper – Pakistan Flood Emergency Cash Transfer Project, April 30, 2013 Supervision and Evaluation Mission Reports 12/2010, 02/2011, 08/2011, 29/2011, 03/2012, 07/2012, 12/2012, 05,2013, 10/2013, 05/2014 39 Annex 10. MAP 40