Document of The World Bank Report No: ICR0002893 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-4412 CE) ON A CREDIT IN THE AMOUNT OF SDR 13.8 MILLION (US$ 22.6 MILLION EQUIVALENT) TO THE DEMOCRATIC SOCIALIST REPUBLIC OF SRI LANKA FOR A PUBLIC SECTOR CAPACITY BUILDING PROJECT June 11, 2014 Poverty Reduction and Economic Management Network South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective May 30, 2014) Currency Unit = Sri Lankan Rupee (LKR) LKR 1 = USD .01 USD 1 = LKR 130.40 SDR 1 = USD 1.54 FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS ADP Accelerated Data Program AG Auditor General AGD Auditor General’s Department CAAT Computer Aided Audit Techniques CAS Country Assistance Strategy CBSP Central Bank Strengthening Project CFAA Country Financial Accountability Assessment CoPA Committee on Public Accounts CoPE Committee on Public Enterprises CPS Country Partnership Strategy DAG Deputy Auditor General DCS Department of Census and Statistics DECDG Development Data Group ERTA Economic Reform Technical Assistance Project FA Financial Audits FY Financial Year GDDS General Data Dissemination System GDP Gross Domestic Product GoSL Government of Sri Lanka IA Investigative Audits ICRR Implementation completion and results report ICT Information and Communication Technology IDA International Development Association IDF Institutional Development Fund IDP Institutional Development Plan IEG Independent Evaluation Group (of the World Bank) IFB Invitation for Bids IO Intermediate Outcome IRD Inland Revenue Department ISR Implementation Status and Results Report (World Bank) LKR Sri Lankan Rupee LJRP Legal and Judicial Reforms Project MoFP Ministry of Finance and Planning MTR Mid Term Review NDC National Data Committee PA Performance Audits PAD Project Appraisal Document PDA Portable Data Assistants PDO Project Development Objective PMU Project Management Unit PRSC Poverty Reduction Support Credit PSCB Public Sector Capacity Building Project PUF Public Use Files RF Results Framework ROSC World Bank Accounting and Audit Report on the Observance of Standards and Codes SDDS Special Data Dissemination Standard SMP Statistical Master Plan SOE State Owned Enterprise TA Technical Assistance TTL Task Team Leader USD United States Dollars Democratic Socialistic Republic of Sri Lanka Public Sector Capacity Building Project Implementation Completion and Results Report CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Result Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Profile 1. Project Context, Development Objectives and Design ...............................................................................1 2. Key Factors Affecting Implementation and Outcomes ..............................................................................4 3. Assessment of Outcomes ............................................................................................................................8 4. Assessment of Risk to Development Outcome ........................................................................................18 5. Assessment of Bank and Borrower Performance .....................................................................................19 6. Lessons Learned .......................................................................................................................................22 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners...........................................24 Annex 1. Project Costs and Financing..............................................................................................................25 Annex 2. Outputs by Components ....................................................................................................................26 Annex 3. ICRR Methodology Note: PSCB ‘Causal Chain’ and Indicator Interpretation ................................31 Annex 4. Technical Assistance Before or During PSCB..................................................................................34 Annex 5: Additional data for assessment of results for Component 2 .............................................................35 Annex 6. Bank Lending and Implementation Support/Supervision Processes.................................................36 Annex 7. Calculations on the Overall Outcome Rating ...................................................................................38 Annex 9. Beneficiary Survey Results ...............................................................................................................40 Annex 10. Stakeholder Workshop Report and Results ....................................................................................41 Annex 11. Borrower Comments on Draft ICRR .............................................................................................42 Annex 12. Comments of Co-financiers and Other Partners/Stakeholders........................................................46 Annex 13. List of Supporting Documents and Information .............................................................................47 MAP .................................................................................................................................................................49 LIST OF TABLES Table 1: Public Sector Capacity Building Project (PSCB) Key Indicators ........................................................2 Table 2: Component 1 Efficacy Per Key Target Indicator ...............................................................................12 Table 3: Component 2 Efficacy Per Key Target Indicator (original PDO) ......................................................16 Table 4: Component 2 Efficacy Per Key Target Indicator (revised PDO) .......................................................16 Table 5: Original PDO Efficacy Rating Per Key Target Indicator ...................................................................16 Table 6: Revised PDO Efficacy Rating Per Key Target Indicator ...................................................................17 Table 7: Overall Outcome Rating (August 2012 Restructuring) ......................................................................18 Table 8: PSCB ‘Causal Chain’ * .......................................................................................................................31 Table 9: ICRR Clarification of Key Target Indicators (when applicable)........................................................32 Table 10: PSCB Original PDO ICRR Split Ratings .........................................................................................38 Table 11: Project’s Overall Outcome Rating (August 2012 Restructuring) if Efficacy Rounded Up in the Original PDO ....................................................................................................................................................38 Table 12: Project’s Overall Outcome Rating (if restructured in December 2011) ...........................................39 Table 13 : Project’s Overall Outcome Rating (if based on actual expenditure at August 2012) ......................39 A. Basic Information Public Sector Capacity Country: Sri Lanka Project Name: Building Project ID: P097329 L/C/TF Number(s): IDA-44120,TF-54897 ICR Date: 06/11/2014 ICR Type: Core ICR DEMOCRATIC Lending Instrument: SIL Borrower: SOCIALIST REPUBLIC OF SRI LANKA Original Total USD 22.60M Disbursed Amount: USD 13.75M Commitment: Revised Amount: USD 15.27M Environmental Category: B Implementing Agencies: Auditor General’s Department (AGD) Department of Census and Statistics (DCS), Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 07/21/2005 Effectiveness: 10/15/2008 10/15/2008 10/21/2010 12/19/2011 Appraisal: 04/10/2007 Restructuring(s): 07/23/2012 12/14/2012 Approval: 06/05/2008 Mid-term Review: 02/01/2011 Closing: 12/31/2011 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Moderately Unsatisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Unsatisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project at No Quality at Entry (QEA): None any time (Yes/No): Problem Project at any time Quality of Supervision Yes None (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 100 100 Theme Code (as % of total Bank financing) Administrative and civil service reform 17 17 Economic statistics, modeling and forecasting 17 17 Managing for development results 33 33 Public expenditure, financial management and procurement 33 33 E. Bank Staff Positions At ICR At Approval Vice President: Philippe H. Le Houerou Praful C. Patel Country Director: Francoise Clottes Naoko Ishii Sector Manager: Alexandre Arrobbio Simon C. Bell Project Team Leader: Farah Zahir Tatiana Nenova ICR Team Leader: Simon Carl O'Meally ICR Primary Author: Simon Carl O'Meally F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To enhance the effectiveness, efficiency and productivity of two key public sector agencies (Department of Census and Statistics [DCS] and Auditor General's Department [AGD]) through an investment package that includes organizational strengthening, capacity building, information management, communication improvements, physical and information technology infrastructure, and Information and Communication Technology (ICT) support. Revised Project Development Objectives (as approved by original approving authority) The project PDO was revised at restructuring of July 23, 2012 and was given effect through an amendment to the Financing Agreement conveyed through Bank's letter dated August 7, 2012 and effective upon GoSL signing from August 27, 2012. The PDO was revised as follows: 'to enhance the effectiveness, efficiency and productivity of the Recipient's AGD'. (a) PDO Indicator(s) Original Target Actual Value Formally Values Achieved at Indicator Baseline Value Revised Target (from approval Completion or Values documents) Target Years COMPONENT 1: Evidence of the improved effectiveness, efficiency, and productivity for Indicator 1 : the key public sector agency DCS Strong evidence of Value No evidence of effectiveness, Modest to substantial quantitative or effectiveness, efficiency, efficiency and evidence Qualitative) and productivity productivity Date achieved 03/18/2008 12/31/2011 12/31/2011 Comments Target partially achieved. Evidence is patchy due to lack of quantitative (incl. % baseline/indicators. However, taking into account overall evidence, it is judged that there achievement) is modest to substantial evidence of improved effectiveness, efficiency, & productivity. COMPONENT 1 (HIGHER LEVEL OUTCOME INDICATOR): Evidence that the DCS Indicator 2 : is relied upon as the main source of national statistical data for the country, as reflected in the National Data Committee (NDC) reports • NDC formed (2010) Value National Accounts data not Quarterly/ • 5 NDC meetings quantitative or harmonized within the biannual NDC held Qualitative) national statistical system meetings • NDC functions partially fulfilled Date achieved 03/18/2008 12/31/2011 12/31/2011 Comments Target substantially achieved. As per text, this is assessed in terms of: (i) formation of (incl. % NDC; (ii) regularity of NDC meetings; and, (iii) identified key functions of NDC being achievement) fulfilled. COMPONENT 2: Evidence of the improved effectiveness, efficiency, and productivity for Indicator 3 : the key public sector agency AGD Strong evidence of Strong evidence Value No evidence of improved improved of improved quantitative or effectiveness, efficiency effectiveness, effectiveness, Strong evidence Qualitative) and productivity efficiency and efficiency and productivity productivity Date achieved 03/18/2008 12/31/2011 12/31/2012 12/31/2013 Comments Target substantially achieved. Taking into account overall evidence base, it is judged that (incl. % there is strong evidence of the improved effectiveness, efficiency, and productivity of the achievement) AGD under both the original and revised PDO as per defined targets. COMPONENT 2 (HIGHER LEVEL OUTCOME INDICATOR): Evidence that improved public audit functions by AGD have increased accountability and transparency in the use Indicator 4 : of public funds, as reflected in the annual public audit cycle submissions by AGD to Parliament. Value AGD submissions to AGD submissions to AGD • 68% audits within quantitative or Parliament indicate that Parliament indicatesubmissions to annual plan (FY12). Qualitative) Audits are 2-4 years late and that maximum audit Parliament • 68% SOEs 50% of SOEs covered by indicate that delay is 2 years and submitted accounts audit. 100% of SOEs maximum audit timely (FY12). covered by audit. delay reduced to • Some improvements maximum one in transparency and year and 100% accountability. of SOEs covered by audit. Date achieved 02/05/2008 12/31/2011 12/31/2012 12/31/2013 Comments Target partially achieved. The target was to be measured by a combination of the target (incl. % values for IO5 – 100% audits within annual plan – and IO6 – 100% of SOEs audited achievement) (timely). (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Indicator 1 : COMPONENT 1 - Decreased processing times for completion of key data series • TA delivered. • Some hardware Value 48 month processing time National census provided. (quantitative (National Census); 18 reduced to 24 • Processing took or Qualitative) months (household surveys) months place post-component closure. Date achieved 03/18/2008 12/31/2011 12/31/2011 Comments Target modestly achieved. Target is for 'data processing time' to 24 months. Target not (incl. % achieved as processing took place in 2012. However, project provided knowledge and achievement) infrastructure contributing to the reduced processing time in 2012 (to 7 months). COMPONENT 1 - Data processing of next HIES survey conducted according to HIES Indicator 2 : pilot data entry methodology • Field offices sending HIES completed data electronically Value according to new 58% (from 25%). (quantitative Fully centralized data entry data collection • Data processing or Qualitative) scheme reduced to 5 months (from 18). Date achieved 03/18/2008 12/31/2011 12/31/2011 Comments Target substantially achieved. Slight ambiguity regarding the target and results achieved, (incl. % but this is clarified in the ICRR text on the relevant target, as well as in Annex 3. achievement) Indicator 3 : COMPONENT 1 - Percentage of technical staff trained on relevant statistical training • 52% Value • Training plan (quantitative Insufficient training 50% revised and or Qualitative) implemented. Date achieved 03/18/2008 12/31/2011 12/31/2011 Comments Target exceeded. (incl. % achievement) COMPONENT 1 - Improved access to data and dissemination of data outputs; facilitating Indicator 4 : user education through workshops and website Review of new data • User survey sharing policy based undertaken. on feedback from Value Data-sharing policy has been • New data sharing users; regular user (quantitative articulated; inadequate user policy not (re)drafted. education or Qualitative) education. • 2 user education workshops; feedback workshops. from users on data • Data improved. access. Date achieved 03/18/2008 12/31/2011 12/31/2011 Comments (incl. % Target partially achieved. Some ambiguity in the indicator, which is clarified in Annex 3. achievement) Indicator 5 : COMPONENT 1 (OUTPUT INDICATOR): DCS Building Value • Design completed. (quantitative No Building One Building • Foundation or Qualitative) Completed. Date achieved 03/18/2008 12/31/2011 12/31/2011 Comments Target partially achieved. Building foundations completed; bidding documents for (incl. % superstructure prepared; design consultants recruited for foundation and superstructure. achievement) COMPONENT 2 - Improved audit quality and timeliness at AGD - percent of audits Indicator 6 : delayed (original PDO) to Timeliness of audit reports – percent of audits delayed (revised PDO) • 68% audits completed within annual plan (FY12). • Backlog of audits Value 100% of audit 100% of audits cleared (2009). (quantitative Audit 2-4 years late within annual within annual plan • Audit reports to or Qualitative) plan Parliament increased. • Annual AGD Reports submitted to Parliament. Date achieved 03/18/2008 12/31/2011 12/31/2012 12/31/2013 Comments Target partially achieved. Due to ambiguity in the target, this is measured through: (i) (incl. % clearing the backlog of audits; (ii) audit reports (based on accounts rendered) issued by achievement) AGD by September 30; and (iii) tabling of audit reports in Parliament. Indicator 7 : COMPONENT 2- Widened public audit coverage - percent of SOEs audited • 68% SOEs submitted accounts timely (FY12). Value • Mandate to audit all (quantitative 50% 100% 100% SOEs unchanged or Qualitative) (Audit Act not tabled in Parliament for enactment). Date achieved 03/18/2008 12/31/2011 12/31/2012 12/31/2013 Comments Target partially achieved, measured through: (i) timely submission of accounts by SOEs (incl. % for audit (i.e. by Feb 28); and, (ii) expanding mandate, in practice, of AGD SOE audit achievement) coverage (via Audit Act). Progress in this area was partly beyond AGD control. Indicator 8 : COMPONENT 2 - Percent of technical staff trained Value 50% (reduced to 50% (increased (quantitative 0% 40% at restructuring to 75% by 31- More than 75% or Qualitative) of August 2012) Dec-2013) Date achieved 03/18/2008 12/31/2011 12/31/2012 12/31/2013 Comments (incl. % Target exceeded. achievement) COMPONENT 2 - Increasing Percent of investigative and performance audits over total Indicator 9 : audits (original PDO) Number of investigative and performance audits (revised PDO) Value • 17 PAs completed. (quantitative 0% 25% 50 (no) • 42 IAs completed. or Qualitative) Date achieved 03/18/2008 12/31/2011 12/31/2012 12/31/2013 Comments Target partially achieved (original PDO); target exceeded (revised PDO). In June 2011, (incl. % the indicator was revised as the target of 25% of Investigative Audits (IA) and achievement) Performance Audits (PA) to total audits was impractical and not feasible. Indicator 10 : COMPONENT 2 (OUTPUT INDICATOR): AGD Building • Main building One building Value completed. (plus additional (quantitative No Building One Building • 80% of additional training or Qualitative) training building building) completed. Date achieved 03/18/2008 12/31/2011 12/31/2013 12/31/2013 Comments Target exceeded (original PDO); target substantially achieved, not met (revised PDO). (incl. % Main headquarters building completed (with additional floor) and 80% of additional achievement) training building completed (with savings from the project). G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 12/31/2008 Moderately Satisfactory Moderately Satisfactory 0.12 2 06/28/2009 Moderately Satisfactory Moderately Satisfactory 0.72 3 12/30/2009 Moderately Satisfactory Moderately Satisfactory 1.61 4 10/11/2010 Moderately Satisfactory Moderately Satisfactory 4.83 5 11/20/2010 Moderately Satisfactory Moderately Satisfactory 4.83 6 06/15/2011 Moderately Unsatisfactory Moderately Unsatisfactory 8.32 7 02/18/2012 Moderately Unsatisfactory Satisfactory 9.81 8 09/29/2012 Satisfactory Satisfactory 13.30 9 01/01/2013 Satisfactory Satisfactory 13.30 10 06/18/2013 Satisfactory Satisfactory 13.30 11 12/25/2013 Moderately Unsatisfactory Moderately Satisfactory 13.75 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring in Changes Made Change DO IP USD millions • To allow any person, other than the AG, nominated by the 10/21/2010 N MS MS 4.83 Government of Sri Lanka (GoSL) acceptable to IDA to be the Project Director. • Extension of closing date from Dec 31, 2011 to Dec 31, 2012. • Cancellation of IDA (SDR 4.75 12/19/2011 N MU MU 9.41 mn or $ 7.33 mn) under Statistical component on Dec 21, 2011. • No activities under Statistical component beyond Dec 31, 2011. • Revision in PDO and results framework • Inclusion of support to regional 07/23/2012 N MU S 13.30 training center at Ratnapura (as specified in the Restructuring Paper but not in the Financial Agreement amendment). • Extension of closing date from 12/14/2012 N S S 13.30 Dec 31, 2012 to Dec 31, 2013. I. Disbursement Profile 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. At the time of appraisal, the country’s vision for development was enshrined in Sri Lanka’s national development strategy, the Mahinda Chintana. It focused on pro-poor, pro-growth income improvement and redistribution policies with the complementary participation of a socially responsible private sector and a strong public sector. In 2007, the country’s annual per capita income surpassed the US$1,600 mark. The investment target was 30 percent of gross domestic product (GDP) in order to achieve an annual growth rate of 8 percent in the medium term. 2. Sound economic policies toward higher growth, increased investment, and poverty reduction depended on the efficiency and productivity of the public and private sectors. Regulatory risks and complex administrative procedures needed to be addressed to ensure Sri Lanka’s compatibility with other emerging economies. Improved transparency and financial accountability were also identified as important to promote a healthier business environment. Further, public sector capacity strengthening measures were seen as important to enhance the administration’s ability to implement complex re forms as well as to develop appropriate economic policies. The armed conflict posed additional challenges to macroeconomic management and to the development agenda. 3. As such, several measures to strengthen public sector capacity had been introduced prior to appraisal. This included the fields of tax administration, governance, procurement, trade, legal and judicial, and investment policy, and encompassed a wide range of key institutions. The Bank had supported these reform efforts for a number of years through the Economic Reform Technical Assistance Project (ERTA), the Legal and Judicial Reforms Project (LJRP), the Central Bank Strengthening Project (CBSP), and the Poverty Reduction Support Credit (PRSC). In addition, specialized agencies to deal with the nation’s unique challenges had been established; for example, the Reconstruction and Development Agency, which focused on post-tsunami and post-conflict resolution issues. 4. In this context, one key engagement had focused on the country’s agency for data provision, the Department for Census and Statistics (DCS). Demand for high quality data was growing within the public sector, the private sector, and donor community in Sri Lanka in order to both design policy and monitor programs. Yet, coordinating and improving the statistical system so that it was in line with international standards had been challenging. A variety of non-lending TA from the Bank over the three years prior to appraisal had helped to enhance the capabilities of DCS and helped them develop useful tools to inform policy. These gains needed further reinforcement. In this regard, a Statistical Master Plan (SMP) was prepared with financial assistance from the International Development Association (IDA) to form the basis of a long-term DCS strengthening strategy. The strategy was reinforced by the DCS Implementation Plan, which established a demand-driven core data collection and development program. 5. Another key engagement focused on strengthening the public audit function carried out by the Auditor General’s Department (AGD). The Government of Sri Lanka (GoSL) had undertaken several initiatives directed at strengthening the public audit function. These included the Country Financial Accountability Assessment (CFAA) undertaken by the GoSL and the World Bank, the World Bank Accounting and Audit Report on the Observance of Standards and Codes (ROSC), the drafting of the National Public Audit Act, and the development of the AGD Institutional Development Plan (IDP). With these initial building blocks in place, AGD was keen to use the reform momentum and to continue its transformation into a modern Supreme Audit Body. 6. The rationale for Bank support was to consolidate and build upon the progress made in these two areas of national statistical capacity and public sector auditing practices. The initiatives, by their 1 closing dates, had resulted in well-developed plans and initial reform steps. However, additional support was required to consolidate the gains and implement new activities. The Bank was therefore asked to provide support to strengthen the DCS and AGD. Providing technical assistance (TA) as well as building public sector capacity were key areas of Bank comparative advantage, and the Bank had maintained a continuous dialogue on improving statistical capacity and strengthening public auditing. Moreover, capacity building, improved governance, macroeconomic stability, and pro-poor service delivery were core elements of the Bank’s support, as outlined in the Country Assistance Strategy (CAS) (2004 – 2007). 1.2 Original Project Development Objectives and Key Indicators (as approved) 7. The original Project Development Objective (PDO) in the Project Appraisal Document (PAD) was stated as follows: “To enhance the effectiveness, efficiency and productivity of two key public sector agencies (Department of Census and Statistics [DCS] and Auditor General's Department [AGD]) through an investment package that includes organizational strengthening, capacity building, information management, communication improvements, physical and information technology infrastructure, and Information and Communication Technology (ICT) support”. The PDO in the original legal agreement is stated as: “to enhance the effectiveness, efficiency and productivity of the Recipient’s DCS and AGD”. The PDO that is assessed in this ICRR is that which is expressed in the legal agreement. 8. The progress towards the PDO was to be measured via the Results Framework outlined in the PAD, Annex 3. Based on information available at project appraisal, a results framework was developed and it encompassed the key indicators outlined in Table 1. Table 1: Public Sector Capacity Building Project (PSCB) Key Indicators Hierarchy of Objectives Key Indicators Project Development Objective  Evidence of the improved effectiveness, efficiency, and To enhance the effectiveness, productivity for each of the key public sector agencies (DCS efficiency and productivity of two and AGD). key public sector agencies (DCS and  Evidence that DCS is relied upon as the main source of national AGD) through an investment statistical data for the country, as reflected in the National Data package that includes organizational Committee reports. strengthening, capacity building,  Evidence that improved public audit functions by AGD have information management, increased accountability and transparency in the use of public communication improvements, funds, as reflected in the annual public audit cycle submissions physical and information technology by AGD to Parliament. infrastructure, and ICT support. Intermediate Results (Component 1 –  Decreased processing times for completion of key data series. Upgrading Statistical Capacity)  Data processing of next HIES survey conducted according to HIES pilot data entry methodology.  Percentage of technical staff trained on relevant statistical training.  Improved access to data and dissemination of data outputs; facilitating user education through workshops and website. Intermediate Results (Component 2 –  Improved audit quality and timeliness at AGD (percent of audit Improving Auditing Standards) delayed).  Widened public audit coverage (percent of State Owned Enterprises (SOE) audited).  Percent of technical staff trained.  Percent of investigative and performance audits over total audits. Source: PSCB Project Appraisal Document, Table A3.2 1 2 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and Reasons/Justifications 9. The Board approved a PDO revision (Level 1 Restructuring) on July 23, 2012. This change was made effective through a signed amendment to the Financing Agreement on August 27, 2012. The revised PDO was: ‘To enhance the effectiveness, efficiency and productivity of the Recipient's AGD’. 10. The PDO was revised because ‘Component 1: Upgrading Statistical Capacity’ closed. Component 1 closed on December 31 2011 and the GoSL requested the Bank to cancel USD$7.33 million of IDA funds allocated to that component. Component 2 was granted two time extensions up to December 31 2013, as noted below. This could be considered ‘corrective restructuring’ insofar as the weaker performing component ceased, as per GoSL request. 11. In terms of the key indicators, the PDO restructuring engendered the dropping of all indicators related to Component 1 and the revision of two of Component 2’s intermediate outcome indicators. All key indicators related to Component 1 were dropped. Two of Component 2’s IO indicators were revised. First, the IO indicator, ‘Increasing percentage of investigative and performance audits conducted over total audits by AGD’, was dropped and a replacement indicator was provided as follows: ‘Number of Investigative and Performance Audits’. The Restructuring Paper of July 2012 noted that this new indicator was used because the target for the original indicator was considered impractical of being achieved. Second, the IO indicator ‘Improve audit quality and timeliness of AGD’ was reworded as ‘Timeliness of Audit Reports’ without any change in baseline or end -line target. The reason for this change was not mentioned in the restructuring paper or other project documentation. 1.4 Main Beneficiaries 12. The PAD identifies the primary beneficiaries as the two public sector agencies (DCS and AGD) of the GoSL. The general public was also expected to benefit through the provision of better public services. Under the revised PDO (effective August 2012), the primary beneficiary is the AGD. 1.5 Original Components 13. The PDO was intended to be achieved through the implementation of two project components, which represented an allocated IDA financing of US$22.6 million. The project was divided into two components:  Component 1: Upgrading Statistical Capacity (US$14.05 million of which US$12.05 million was allocated from IDA; cancelled US$ 7.33 mn).  Component 2: Improving Auditing Standards (US$12.05 million of which US$l0.55 million was allocated from IDA). Component 1: Upgrading Statistical Capacity 14. Component 1 objective: to help develop tools for demand-driven core data collection and dissemination. 15. Component 1 activities: under this component, planned areas of support included skill development, information and technology, client relations, and physical infrastructure development. The sub- components were: (a) Organizational Development; (b) Statistical and IT Infrastructure and Client Services; (c) Data Development; (d) Physical Infrastructure; and, (e) Project Management.2 3 Component 2: Improving Auditing Standards 16. Component 2 objective: to address weaknesses in the financial management of public resources, the effectiveness of the public audit function, and the availability of public access to information. 17. Component 2 activities: under this component, planned activities were intended to focus on HR development, communications and external relations, auditing methodologies, and IT/physical infrastructure. The sub-components were: (a) Audit Methodology; (b) Human Resource Development; (c) Communications and External Relations; (d) IT and Equipment/Fittings; (e) Physical Infrastructure; and, (f) Project Management.3 1.6 Revised Components 18. Following the revision of the PDO, Component 2 formally remained the same although two additional activities were included in the project. On account of cost savings achieved under this component, two additional activities were included in the project as per the restructuring paper of December 17, 2012. These pertained to the construction of an AGD regional residential training center in Ratnapura, Sri Lanka4 and the other was to host an international seminar of the Asian Organization of Supreme Audit Institutions (held in June 2013). 19. Component 2 was also subject to two time extensions. The first extension occurred on December 19 2011 and extended the project from December 31 2011 to December 31 2012. This was applicable only to Component 2 given Component 1 closure was effective from December 31 2011. The reason for this extension was to enable AGD to fully utilize the allocated resources under Human Resources, and Communications and External Relations sub-components. The second time extension occurred on December 14 2012 and extended the completion date from December 31 2012 to December 31 2013 to enable AGD to complete the two additional activities (outlined above) and to further institutionalize enhancements to the effectiveness, efficiency and productivity of the AGD. 1.7 Other Significant Changes 20. On October 21 2010 a level 2 restructuring was undertaken. This was to allow any person other than the Auditor General (AG), nominated by the GoSL and acceptable to IDA, to be the Project Director of Component 2. As a result, the ex-AG, who retired during project implementation, was retained as the Project Director to ensure continuity in Component 2 project leadership. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Key Strengths at Entry 21. The project emerged from longer term engagements between the Bank and GoSL which focused on enhancing the functioning of public sector institutions. Prior to the PSCB, the Bank had channeled TA to GoSL through a range of initiatives such as ERTA or LJRP (see Annex 5). These engagements had resulted in well-developed plans and initial reform steps. Seeing the high impact of the ERTA initiative, it was decided to develop a follow on project, which eventually led to the PSCB.5 22. The project design integrated a range of key findings from the Bank’s prior TA. On the auditing side, a number of key studies were undertaken. 6 For instance, the CFAA undertaken in June 2003 recommended: (i) strengthening the oversight function provided by the public accounts and public 4 enterprises committees; and, (ii) strengthening the AGD by providing more independence through a National Audit Act, by building its capacity via an IDP and by providing more relevant audit information to Parliament. The IDP was developed and a number of IDP actions were integrated into the PSCB.7 In terms of statistical capacity building, the Bank had contributed to DCS work, such as through the development of a poverty mapping methodology and support to the development of the SMP (SMP, 2007).8 A number of actions identified in the SMP were integrated into project design. 23. In order to achieve the PDO, the PSCB proposed a mixture of physical infrastructure and organizational strengthening, with a relatively simple design. The range of functional and capacity strengthening activities was needed as per the SMP (statistics) and IDP (auditing). The main physical infrastructure – new DCS and AGD buildings – were also important to achieving the PDO because: (i) the 18 divisions of DCS were accommodated in eight separate locations in and around Colombo which created significant challenges in coordination and communication, as well as costly, unsafe, time- consuming transport of physical data to storage;9 and, (ii) the building that AGD was then occupying was to be demolished under an environmental plan of the government, and using funds to re-wire an old building was considered inefficient, whereas the new construction permitted complete modernization. Key Challenges at Entry 24. However, the PDOs were quite ambitious, given the three-year timescale and the capacity of the implementing agencies. This was recognized at mid-term review (MTR), which pointed out that it was arguably unrealistic to expect the defined results across the board in just three years, especially given that the implementing agencies had limited experience of managing large-scale projects, as noted further below. Furthermore, the PDOs and related activities could have been more relevant if the institutional analysis of AGD (2003 CFAA) and DCS (2007 SMP) had been explicitly updated and integrated into project design at the time of appraisal. 25. Moreover, the risk analysis and mitigation plan could have been more robust. While there is a good risk analysis in the PAD, a greater emphasis could have been placed on providing more intensive supervision; on risk mitigation planning to deal with building construction; and, on the risk that the implementing agencies may not be able to manage the reforms while responding to their day-to-day commitments. 26. There was also no mechanism in the project design to ensure overall project co-ordination. The two components were combined as a result of a request from the GoSL. However, the project design did not elaborate a clear set of mechanisms to ensure overall co-ordination and tracking of the operation. 2.2 Implementation 27. The project management units (PMUs) experienced some implementation delays, especially in the first year or so, which slowed progress; but the pace largely picked up thereafter. There were initial challenges, especially with procurement and turnover of staff. In Component 1, implementation was hampered by delays in hiring and retaining PMU staff, especially procurement staff. In Component 2, there was a delay in recruiting the consultant for developing manuals and providing training. Implementation largely accelerated after start-up challenges in Component 2 but took longer in Component 1. Two key challenges affected Component 1’s implementation progress over the life of the project: (i) the DCS PMU had to hire four procurement specialists over the project period, which slowed down a range of procurements; and, (ii) the fulfillment of day-to-day requirements of collecting key data sources and preparing for the Census, engaged much of the DCS resources in late 2010 and the first three quarters of 2011. Three key challenges affected Component 2’s progress, which were beyond AGD control: (i) there were delays of submission of SOE statements for auditing, which in turn delayed AGD audit completion; (ii) the AGD faced challenges in retaining staff, which they could not influence due to 5 a lack of administrative independence; and, (iii) the requirement to translate audit reports into three languages before tabling parliament added to delays (see also Section 4). 28. Limited supervision support also contributed to implementation delays up to 2010, mainly for Component 1, however the support intensified thereafter and helped accelerate progress. The components were often supervised separately with Bank missions sometimes fielded separately. Bank supervision for Component 1 was limited and there were shortcomings in ensuring adherence to implementation arrangements and in processing procurement clearances. However, Component 1 supervision support improved in 2010. Support to the AGD component was much more pro-active from inception (see Section 5). 29. At MTR, steps were taken to address some key design and implementation weaknesses. Implementation and monitoring arrangements were further clarified with Component 1 counterparts. Steps were put in place to enhance Component 1 supervision support. The baseline was collected and certain indicators were elaborated under Component 1. It was also noted that, under Component 2, a longer time period (beyond the 3 years) would be needed to allow full implementation of the new audit technologies on an enterprise-wide basis. 30. In terms of the physical infrastructure, the Component 1 building experienced long delays and, by December 2011 – component closure – DCS had completed the foundations. The construction of the foundation took longer than foreseen for a number of reasons, such as: (i) challenges of DCS retaining procurement staff; (ii) some delays in Bank clearance of procurement up to 2010; (iii) the need for DCS to address the issue of land squatters and boundary administration bottlenecks; (iv) Ministry of Finance and Planning’s (MoFP) request to have additional space in the new building considering the land value, which led to the re-design of the building plans; and, (v) rains causing flooding of the site. 10 The superstructure was delayed for the following reasons. The contract for the superstructure was tendered in 2011. However, the Bank was unable to clear the awarded contracts because the bidding process did not conform to Bank guidelines. 11 The Bank provided timely support in attempting to resolve this and offered two options to address the issue: (i) re-invite bids from the 13 bidders who had submitted bids in response to the first Invitation for Bids (IFB); and, (ii) invite fresh bids following National Competitive Bidding (NCB) procedure and cancel the previous tender. The DCS elected to proceed with their own bid evaluation. Towards the project closure date, this bottleneck had not been overcome and a decision was taken to cancel 7.33mn USD from the DCS component following a letter from the GoSL to the Bank dated September 5 2011.12 31. The AGD building (Component 2) experienced smoother implementation and was completed 3 months after the originally planned project completion date. The AGD was able to complete its planned office building by March 2012 and, as a result of project savings, it was able to build 80% of an additional training centre building at Ratnapura during the time extension period (from January to December 2013). 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Monitoring and Evaluation Design 32. The PSCB M&E framework included a number of critical performance indicators to monitor progress towards the PDO. The M&E results framework (hereon, RF) contained a number of key indicators that were identified by the AGD’s CFAA and IDP and the DCS’s SMP. It is thus judged that the set of indicators, for the most part, were broadly relevant to identifying progress towards the PDO. 33. However, in certain areas the baseline information was incomplete and vague and a number of targets were not specific enough for measuring progress. For the statistical upgrading component, 6 certain indicators did not have a baseline until 2010, as noted at MTR, and there are a number of indicators in the PAD that are not quantifiable. Also, some target indicators were unclear and did not fully outline the means by which progress could be measured (see Annex 3). 34. Equally, the RF did not include the buildings. Given that the buildings represented a substantial proportion of the loan; it may have made sense to include them in the RF. This could have helped, at design stage, in breaking the construction down into intermediate steps and focused the project more heavily on construction risk mitigation planning. M & E Implementation and Utilization 35. M&E implementation and utilization was delayed in the earlier stages of project implementation but improved considerably. While monitoring and reporting was delayed in the initial stages (and delayed for Component 1 up to January 2010); the implementing agencies, on the whole, provided regular monitoring and reporting via six monthly status reports. 36. The flaws in some areas of M&E design surfaced during project implementation and needed to be addressed during the MTR. For example, for Component 1 the MTR noted that the limited guidance in the PAD on interpreting certain indicators, and the lack of baseline data limited the extent to which results could be measured. So the Bank, in consultation with the client, outlined additional sub- indicators to elaborate on some of the existing IO indicators. While these sub-indicators were not formal revisions to the IOs, they usefully expanded on some of the IO indicators, and are used to inform parts of the ICRR assessment, as noted in Annex 3. 2.4 Safeguard and Fiduciary Compliance Environment 37. The operation was classified as category B throughout the project life and the adherence to environmental safeguards was rated above MU throughout. OP 4.1 was triggered given the building construction, leading to an Environmental Assessment (EA) and Environmental Management Action Plan (EMAP). The EMAP and reporting was largely adhered to for the DCS and AGD buildings. The Bank team periodically reviewed the implementation of mitigation measures and there were no major construction stage environmental impacts, which were irreversible. In terms of the construction of the training centre at Ratnapura, there were two minor issues: (i) reporting and monitoring outlined in the EMAP was not up to date; and, (ii) following a site visit in November 2013, although no major environmental impacts were observed, there were a few breaches in terms of sight management, safety to the workers and drainage. The implementing agency took steps to resolve these issues. Social 38. The only social safeguard issue related to the triggering of OP 4.2 on involuntary resettlement to deal with a land squatter on the DCS building site. The buildings are located on government owned land without involving acquisition of privately owned land or displacement of people. However, OP 4.2 was triggered due to a squatter on the DCS building site. This was resolved and led to compensation in order to relocate the squatter. Procurement 39. Procurement performance varied and was rated as satisfactory in the last ISR. In terms of the auditing activities, the AGD required hand holding in selection of some consultants so the Bank organized training sessions. AGD procurement plans and progress were periodically reviewed by the 7 Bank and the PMU managed all major procurements and contracts well. On the statistical component, procurement experienced challenges, especially in recruiting and retaining a procurement specialist; although they did manage a number of major procurements as well as the piling of the building. DCS experienced significant delays in the tendering of the building superstructure, as noted above. Due to the delays experienced under the DCS Component, procurement was rated as “moderately unsatisfactory” in 2011. Subsequently Procurement was rated as “Satisfactory” by project close. No mis-procurement was noted on the part of both agencies. Financial Management 40. Financial management was also largely satisfactory. In spite of some initial delays, financial reporting was by and large timely and adequate. Agreed internal and external audits were conducted and reports were shared with the Bank. Both AGD and DCS did well in maintaining financial management systems and staff, even if DCS ran into some difficulties in retaining a qualified FM specialist for the PMU. Following the PDO restructuring, some delays in submission of withdrawal claims and documentation of expenditure against the balance in Designated Account resulted in rating downgrade to moderately satisfactory (during Apr-May 2013 mission), but this was upgraded to satisfactory at project close. 2.5 Post-completion Operation/Next Phase 41. Under the auditing activities, certain transitional steps have been taken to maintain the project benefits. The AGD has ensured that operation and maintenance costs for the building are sufficient, and it has institutionalized the project activities into the day-to-day operation of the agency, such as the establishment of dedicated units for training and performance and investigative audits; and, the building of a regional training center. 42. As for next steps, the AGD has identified some areas for consolidating and building on the project achievements, and has had some early dialogue with the Bank. Potential next steps include the following: applying the Supreme Audit Institution performance measurement framework as a basis for updating the AGD strategic development plan; increasing training, especially in post-graduate and overseas training; introducing internal competition awards to incentivize staff; linking all branches with the head office network server; and, setting up a citizen hotline for grievances. 43. Similarly, under Component 1, certain post-operational steps have been taken and steps have been identified for future action. A number of results were built on post-operation. For instance, the DCS building is making progress, at around 80% completion, with government funds allocated to ensure its completion; and the NDC continues to expand its functions. As for a next phase, DCS have identified the need for certain equipment under its ICT plan (developed under the project), such as the provision of laptops to all field staff. It also aims to achieve other activities that were identified as critical during the project, such as the preparation of a Master Sampling Frame or the design of an Official Statistics Plan (OSP). These activities may require additional funding. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives Sub-Rating: High. 44. The PDOs remain very relevant to, and consistent with, Sri Lanka’s current development priorities and the Bank’s country and sectoral assistance. The objectives are consistent with the GoSL national development strategy (i.e. the Mahinda Chintana) and the Bank’s country support, as 8 highlighted in the current and past two CAS documents dating back to 2004. In terms of the statistical component, the Mahinda Chintana identifies a need for good quality statistics and the current Country Partnership Strategy (CPS) (2012-2016) notes “a clear need for Bank engagement beyond the STATCAP [PSCB] project to continue to support the statistical system in the country.”13 The IMF Article IV, while stating that statistics in Sri Lanka are ‘broadly acceptable’, report s that there are still areas in which improvement is needed. Similarly, the strengthening of public audit function is included in the current CPS pillar, ‘Increasing fiscal space and efficiency of public spending’, which seeks to achieve the Mahinda Chintana goal of, ‘Improved flow of budget information including reliable and timely information on budgets and outcomes’. 45. More broadly, the objectives are aligned with the current CPS focus on Sri Lanka’s transition from a low- to middle-income country and pro-poor service delivery. Having modern and effective public statistical and audit functions are a key characteristic of middle-income countries. Indeed, the DCS has indicated that it would like to move from the IMF’s ‘General Data Dissemination System’ (GDDS) to the more challenging ‘Special Data Dissemination Standard’ (SDDS) classification, which is more in line with aspiring and current middle-income countries.14 Moreover, the DCS provides statistics for measuring improvements in delivery, especially to the poor; and the PA’s conducted by AGD can identify key areas for service delivery improvement. Relevance of Project Design and Implementation Sub-Rating: Substantial 46. The project design was largely consistent with the PDOs and targeted important areas for achieving the objectives. The selection of sub-activities was notably relevant insofar as they drew on the CFAA diagnostic study and IDP (for AGD) and the SMP (for DCS) and resulted from a longer term engagement of technical support (see Section 2). The planned provision of capacity support, functional reforms and buildings was also consistent with identified needs. 47. However, there are a few issues that potentially limited the design’s relevance. First, explicit and detailed updating of the SMP (2007) and CFAA (2003) during appraisal could have ensured the highest relevance of design. Relatedly, the design would have been more relevant if measurable baselines and targets were defined across the board. Second, it would have made sense to have designed a clear mechanism of overall project co-ordination and oversight. Third, the targets, mainly under Component 1, were arguably overly ambitious, especially given a heavy day-to-day DCS workload and implementing agency capacity. 48. The actual implementation of the project was largely relevant to the achievement of the PDO. Project implementation was largely consistent with the PDOs and responded to changing circumstances. For example, the Bank responded to a request from GoSL to close Component 1 and extend Component 2. Moreover, the MTR sought to rectify some of the design flaws in the M&E framework. However, although the project did respond to changing circumstances via Level 1 restructuring, this took place 8 months after Component 1 closure. Equally, implementation may have been more relevant if supervision was, overall, stronger (see Section 5). Overall Relevance Rating Relevance Rating (Original PDO): Substantial Relevance Rating (Revised PDO): Substantial 49. The relevance rating for the original and revised PDO is judged to be substantial, even if the rating against the revised PDO is at the higher end of the ‘substantial’ band. The relevance of the objective for both the original and revised PDO is high because they both reflect proper diagnosis of development issues that remain highly relevant today. The design and implementation relevance can be rated as substantial insofar as they were often consistent with objectives and changing circumstances. 9 However, relevance was undermined by the absence of some up-to-date and measurable baselines and targets, the lack of a single project co-ordination mechanism and the failure to process restructuring at an earlier stage. In terms of the revised PDO, when one removes any shortcomings associated with Component 1, the overall revised PDO rating is slightly higher but still substantial. 3.2 Achievement of Project Development Objectives Attribution and Results Measurement 50. In order to assess the extent to which the PSCB project achieved the highest-level PDO, the project causal chain and RF indicators were scrutinized, as outlined in Annex 3. The ICRR considers only those results that can be reasonably attributed to the project activities. It is judged that the RF indicators are, in many ways, appropriate measures of progress towards the PDO. As such, the below assessment is based on the following core target indicators: 2 higher-level outcome indicators, 8 IO indicators and 2 key output indicators. Given that the PDO was restructured, results are judged against the original and revised PDO. 51. However, certain RF indicators lack clarity and the evidence is patchy in certain areas, so the ICRR has made its indicator interpretation explicit in Annex 3. Where necessary, the ICRR team has clarified the indicator and employed relevant sub-indicators based on project documentation. Please see Annex 3 for a full discussion of the ICRR methodology. Component 1 Results (December 2011): Upgrading Statistical Capacity Intermediate Outcome (IO) Target 1: National census (data processing time) reduced to 24 months 52. Summary of Results: prior to the project, the processing time for surveys was slow, manual and inefficient, taking up to 48 months. The project contributed to this target by: (i) assisting a pilot census (in September to November 2010) by simplifying pilot census questionnaires, helping test scanning methods and supporting computer-aided editing and data coding; (ii) consultancy training on using scanning methodologies and TA for preparation of the tender to out-source scanning; and, (iii) providing some IT equipment to district offices.15 53. Achievement sub-rating: modest. This rating balances a number of points: (i) the target was not achieved as the data processing took place in 2012; (ii) however, when the processing actually took place (in 2012) it took around 7 months; and, (iii) the project contributed to this reduction by building the DCS knowledge on data processing, even if full attribution to the PSCB does not apply, as the actual census scanning was outsourced and funded by non-project funds. IO Target 2: HIES survey completed according to a new data collection scheme (to reasonable extent) 54. Summary of Results: prior to the project, collection was predominantly centralized and manual. The project planned to support enhanced efficiency by decentralization of collection via district offices. Under the project, training and modern ICT infrastructure was provided (such as internet connections to District offices and upgraded computers) to improve decentralized district data transmission. 55. Achievement Sub-Rating: substantial. This rating is based on the following: (i) the percent of field offices sending HIES data electronically increased from 25% of district offices (6 of 24) to 58% (14 of 24) by December 2011;16 and, (ii) in addition, the data processing period for 2009/2010 HIES reduced from 18 months to 5 months.17 It was ambitious to expect the new scheme to be fully implemented by the 2009/2010 HIES given the project was effective towards the end of 2008 and the next HIES was 10 scheduled after project completion (2012/2013). Thus, also taking into account what is ‘reasonably’ achievable by 2009/2010, this marks substantial progress. IO Target 3: 50% of technical staff trained in relevant areas. 56. Summary of Results: prior to the training program was somewhat limited. The project supported the development of a strengthened Training Program and Strategy, which enabled DCS to organize relevant training opportunities based on international good practices. The program became a “rolling” annual program which was re-assessed each year in order to best tailor to needs. It was the first time a comprehensive program was designed in order to best respond to the DCS staffing needs. 57. Achievement Sub-Rating: high (52%; target exceeded). Also, a revised and thus strengthened training plan was implemented. IO Target 4: Review of new data sharing policy based on feedback from users; regular user education workshops; feedback from users on data access 58. Summary of Results: this aimed to further the PDO by improving public access to data and encouraging public outreach. Results include the following: (i) a user satisfaction survey was conducted; (ii) two user education workshops were completed; (iii) data sharing procedures improved insofar as 18 Statistical Procedure Manuals were prepared, however, the micro-data access procedure remained cumbersome; 18 and, (iv) a new data sharing policy was not (re)drafted during the project, although it was reviewed internally by DCS, and it is currently (2014) being reviewed. 59. Achievement Sub-Rating: modest. This is based on the fact that the new policy was not (re)drafted or explicitly put in place; and user education was not biannual, on average. However, data access procedures improved, feedback from users was collected (one-off) and 2 education workshops were undertaken. Output Indicator Target 1: Construction of a DCS office building. 60. Summary of results: this aimed to contribute to the PDO by bringing all DCS units together in one location in a modern building. The DCS was able to complete the piling (foundation of the building). It also prepared the bidding documents for the superstructure and recruited design/project management consultants for the foundation and superstructure. 61. Achievement sub-rating: modest (superstructure incomplete). Higher-Level Outcome Indicator 1: Quarterly/biannual NDC Meetings 62. Summary of Results: prior to the project, there was significant duplication in the production of statistical data, even if DCS was mandated to be the lead agency. The NDC was designed to address this.19 The NDC was established, albeit only by October 2010, and met 5 times up to December 2011. Other activities include (see Annex 2): (i) 3 out of 7 planned functional co-ordination committees were established to help coordinate inter-office work; (ii) no MoUs were signed with line agencies by December 2011; (iii) a committee was set up under NDC to review the Statistical Legislation (the legislation is, to date, still pending which is beyond DCS control); and, (iv) baseline assessments were completed for agriculture, ICT and national accounts.20 63. Achievement sub-rating: substantial. The main target, establishing NDC and meeting biannually (on average since its establishment), was achieved and there was also progress against some sub-indicators. 11 Other progress and results (relevant but not explicitly captured in the RF) 64. Another area through which the efficiency and effectiveness of the DCS was sought to be enhanced was through ICT improvements. An assessment of the DCS ICT system was undertaken and an IT strategy was designed. Certain activities in the ICT strategy were achieved as of project close. However, delays were incurred due to difficulties in retaining a project Procurement Specialist, and some IT equipment was not purchased because it depended on the completion of the new building.21 65. The project also sought to improve DCS effectiveness through additional consultancies funded under the project. Additional consultancies included TA to staff on National Accounts or the development of a web-based Library catalog (see Annex 2). Highest-Level PDO Indicator Target 1: strong evidence of improved efficiency, effectiveness and productivity of the DCS. 66. Summary achievements: taking into account all results, the following key areas underline higher-level outcomes related to improvements in effectiveness, efficiency and productivity of the DCS: • The target for improving the data entry processing of the census was not achieved but some gains were made. The target was not achieved until after project closure and was only partially attributed to the project. • It managed to roll-out elements of the new HIES data collection methodology and reduce processing time. However, the new methodology was not fully rolled out by component closure. • It made strong progress in improving the number and relevance of training. It updated its training plan and, for the first time, rolled out training based on staff needs and good practices. • The office building was a long way from completion at component close. However, the foundation was completed and the building work has continued since. • The DCS was not able to produce a new revised data sharing policy but it took some steps in this direction, by also strengthening data access and user education. Further steps could be taken going forward to strengthen its activities in this area. • The DCS took some important steps to establishing itself as the main source of national statistical data. It formed the NDC even if it did not manage to fulfil all the NDC functions. • ICT-based efficiencies were also realized in some areas. Its ICT strategy was revised during the project period, even if its implementation during the project was limited. 67. Achievement sub-rating: modest. This rating balances three points: (i) there is some evidence of an improved DCS via the results registered in the project; (ii) there was limited achievement of the targets relating to the census, data sharing and the building; and, (iii) there is patchiness of the evidence base in certain areas related to baselines, project activities and attribution. 22 In addition, table 2 below summarizes the above ratings by indicator. Table 2: Component 1 Efficacy Per Key Target Indicator High Substantial Modest Negligible No. of Target 1 2 3 0 Indicators % (rounded) 17% 33% 50% 0% Component 2 Results (December 2013): Improving Auditing Standards IO Target 5: 100% of audits within annual plan 68. Summary achievements: increasing the percentage of audits completed on time represents gains in the efficiency, productivity and effectiveness of the AGD. A number of activities were undertaken to 12 achieve this target such as efficiency improvements, use of auditing software and improved methodologies, shorter reporting cycles and greater ease with which evidence from previous audits was carried forward to aid the next audit. 69. Achievement sub-rating: • Original PDO: modest. There was partial achievement of the objective based on three considerations: (i) AGD cleared the backlog of audits by 200923; (ii) 68% of audits completed within audit plan in FY2012 (i.e. prior to September 30 2012), as compared to 44% in FY 2009; and, (iii) submission of audit reports to Parliament progressed well over the project period, see Annex 5, even if this aspect is partly beyond AGD control, as described under IO target 6.24 The ICRR also recognizes the related audit quality improvements, which are considered in the overall component PDO rating below. • Revised PDO: as above. IO Target 6: 100% of SOEs audited 70. Summary achievements: this target aimed to enhance the AGD’s effectiveness by recognizing its mandate to audit all SOEs and to ensure that 100% of the SOEs submitted accounts for audit in time. As noted above, the AGD cleared the backlog of audits in 2009, improved its record of completing timely audits and increased audit reports tabled in Parliament. This contributed to creating an environment for demanding SOEs to submit accounts timely.25 More SOEs rendered accounts timely for audit over the project period compared to the baseline. However, the Annual Reports of the AGD provide instances of SOEs either not rendering accounts or doing so with considerable delay. 71. Achievement sub-rating: • Original PDO: modest. This rating is based on: (i) the observed increase in the number of SOEs that submitted accounts timely; and, (ii) partial achievement of the target by FY2012, 68%, against a baseline of 50%. However, this target rating (and IO5 to a lesser degree) is given less weight in the overall Component 2 highest-level PDO indicator rating below because: (i) timely submission of SOE accounts is beyond AGD control (submission was also delayed by the first time implementation of new reporting standards); and, (ii) 94 government companies continued to remain outside the AGD purview, hence, 100% public audit coverage was not achievable (the pending National Audit Act proposes to include these companies).26 • Revised PDO: as above. IO Target 7: 50%27 of technical staff trained (in 2013, the target was ‘informally’ revised to 75%). 72. Summary achievements: this aimed to provide skills improvement in line with international good practices. A range of relevant trainings, including pilots, in modern audit methodologies, accounting standards, soft and IT skills was provided based on an AGD Training Plan. The skills acquired were applied to various types of audits, with evidence of increased application. Training contributed to expansion of business lines and increased number of modern FAs, including using audit management software TeamMate (introduced in 2011) and CAAT.28 73. Achievement sub-rating: • Original PDO: high (75%; target exceeded). • Revised PDO: high (75%; formal target exceeded). IO Target 8: Original PDO: 25% (% of investigative and performance audits over total audits by AGD); Revised PDO: 50 (number of performance and investigative audits). 13 74. Summary achievements: this aimed to further the PDO by increasing AGD’s emphasis on meaningful audits, such as PAs, which have the potential to improve accountability and public service delivery. Separate manuals for PAs and IAs were prepared (in 2010) and training imparted to core staff and then rolled out. In 2010, separate units for PAs and IAs were created at the head office. By project close, the AGD had conducted 17 PAs and 42 IAs; and, 5 PAs were in progress and 6 PA and 1 IA reports had been placed in Parliament.29 75. Achievement Sub-Rating: • Original PDO: modest (4%). • Revised PDO: high (59; target exceeded). Output Indicator Target 2: Original PDO – Construction of an AGD office building; Revised PDO – Construction of an AGD office building plus a training centre. 76. Summary of achievements: AGD built a fully operational head office with separate divisions for PA, IA and audit training, a large lecture hall, and two computer labs. Taking advantage of project cost savings, AGD also established an extra floor in the head office building for a training division, which was additional to the original target. Using cost savings, it also constructed a significant amount (around 80%) of a regional residential training center at Ratnapura, designed to institutionalize and sustain gains in staff training. Note, as such, the revised PDO target – to build a second building – represents a notable increase in the ambition in this target area. These achievements have clearly contributed to the PDO by providing, for example, an improved working environment, in-house training capacity and modern infrastructure.30 77. Achievement Sub-Rating: • Original PDO: high (target achieved; office completed plus additional floor by March 2012). • Revised PDO: substantial (target nearly achieved but not exceeded; 80% of training centre built by project close). Higher-Level Outcome Indicator Target 2: AGD’s submissions to Parliament indicate that maximum audit delay reduced to one year and 100% of SOEs covered by audit. 78. Summary achievements: according to the PAD, this target would confirm the improved effectiveness of AGD, as well as its promotion of enhanced transparency and accountability. As noted above, the backlog of audits was cleared, 68% of audits were completed within the annual plan (in FY12) and more SOEs rendered accounts timely to AGD. Progress towards increased accountability was reflected in increase in timely submission of reports to Parliament and increase in reports available to the constitutional Parliamentary committees of oversight (Committee on Public Enterprises (CoPE) and Committee on Public Accounts (CoPA). However, by project closure, 32% of audits were not completed within the annual plan, not all the SOEs had submitted their financial statements timely, and there were peaks and troughs in submission of reports to Parliament (Annex 5). Steps were also taken to improve transparency via enhancement of the capacity of AGD to communicate with Parliament, auditees and the general public. The AGD developed a new website and uploaded audit reports and Annual Reports.31 Latest reports of the CoPE and CoPA are also in the public domain. These reports are discussed in Parliament and the findings are increasingly being picked up by the media and civil society. 79. Achievement sub-rating: • Original PDO: modest. As per the PAD, this rating combines the two IO targets’ (Nos. 5 and 6) ‘modest’ rating. • Revised PDO: as above. 14 Other results (relevant but not explicitly captured in the RF) 80. Another area through which the efficiency and productivity of the AGD was sought to be enhanced was through ICT investments. The project funded a range of IT investments; including hardware, audit software and networking of the head office with the regional centers (see Annex 2). This contributed to the PDOs, for instance, by improving audit efficiencies and quality. 81. The HR Strategy envisaged in project design was developed but was still not approved by the Salaries and Cadre Commission at project closure. The strategy was necessary to attract and retain professionally qualified staff and to develop AGD’s capability in deployment of multi-disciplinary audit teams. However, its approval is pending and this was beyond the control of the AGD. Highest-Level PDO Indicator Target 2: Strong evidence of improved efficiency, effectiveness and productivity of the AGD. 82. Summary achievements: taken as a whole, the following key project results underline the higher-level outcomes related to effectiveness, efficiency and productivity of the AGD: • Expanding the number of audits completed. This includes clearing the backlog of audits and rolling out audits on SOEs. • Improved timely completion of the public audit cycle. This is evidenced from increased submission of audit reports to Parliament, even if timeliness needs to be further improved. • Enlargement of AGD business lines and functions alongside adoption of improved audit methodologies. This has included adoption of modern FA methodology and expanding into ‘value for money’ audits. In terms of improved audit quality, more time is now spent at AGD on audit planning to consider audit risks, to review internal controls and to design audit procedures.32 • Strengthening business processes. This has included using audit management software and computer-aided audit techniques, combined with improved IT capabilities. This has resulted in improvements in efficiency, positively impacting audit coverage. • Improved transparency and accountability in certain areas. There was an increase in audit reports reviewed by the Parliamentary oversight bodies and improved transparency of audit information. • Completion of a modern central building and near-completion of an additional training centre. The head office has contributed to improved efficiency and effectiveness of the AGD. • Building of staff capacity via training. Training in technical and soft skills, alongside the strengthening of in-house training capacity via institutionalization and training-of-trainers. • Institutionalization of certain reforms in the organizational structure. This included the creation of dedicated units for training, PAs and IAs. 83. Achievement sub-rating: • Original PDO: substantial. Considering the cumulative reform interventions, the achievements provide relatively “strong” evidence of improved effectiveness, efficiency and productivity of the AGD, in line with international good practices. However, the AGD made more modest gains in consistently ensuring timeliness within the annual plan (IO5 and IO6) and the Audit Act and HR Strategy were not passed; even if some of these results are partly beyond AGD control. In addition, Tables 3 and 4 summarize ratings by indicator. Note that, in table 3 (original PDO), two of the ‘modest’ indicators are given less weight in the overall rating because one was largely beyond AGD control (IO6, 100% SOEs audited) and the other (IO7, % of PAs/IAs) was deemed impractical and later revised. • Revised PDO: as above. 15 Table 3: Component 2 Efficacy Per Key Target Indicator (original PDO) High Substantial Modest Negligible No. of target 2 0 4 0 indicators % (rounded) 33% 0 67% 0 Table 4: Component 2 Efficacy Per Key Target Indicator (revised PDO) High Substantial Modest Negligible No. of target 2 1 3 0 indicators % (rounded) 33% 17% 50% 0 Overall Achievement of Objectives Rating (Combining Components 1 and 2) Achievement of PDO Rating (Original PDO): Modest Achievement of PDO Rating (Revised PDO): Substantial 84. In sum, the overall achievement of objectives (i.e. efficacy) under the original PDO is split and is rounded downwards to modest. This combines the high-level PDO objective of ‘modest’ for Component 1 and ‘substantial’ for Component 2. While this presents a difficult judgment and in spite of substantial results in the project (Annex 8), this is rounded downwards due to the protracted delays in civil works under Component 1 and the eventual cancellation of funds. Table 5 also provides a summary of the ratings by indicator. Table 5: Original PDO Efficacy Rating Per Key Target Indicator Target High Substantial Modest Negligible Component 1 National Census (data processing time) reduced to 24 months X HIES survey completed according to a new data collection X scheme (to reasonable extent) 50% of technical staff trained in relevant areas X Review of new data sharing policy based on feedback from X users; regular user education workshops; feedback from users on data access Quarterly/biannual NDC Meetings X Construction of a DCS office building X Component 2 100% of audits within annual plan X 100% of SOEs audited* X % of investigative and performance audits over total audits by X AGD** 50% of technical staff trained X AGD’s submissions to Parliament indicate that maximum X audit delay reduced to one year and 100% of SOEs covered by audit Construction of an AGD office building X Total 3 2 7 Percentage of Key Target Indicators 25% 17% 58% 0% *Note: this IO target should be given less weight as largely beyond AGD control. ** Note: IO target was deemed highly impractical and was later revised. 85. In sum, the overall achievement of objectives (i.e. efficacy) under the revised PDO is judged to be substantial. This rating captures the fact that Component 2, when judged against the revised PDO had minor shortcomings and met a number of targets, as outlined above. Table 6 summarizes by target. 16 Table 6: Revised PDO Efficacy Rating Per Key Target Indicator Target High Substantial Modest Negligible 100% of audits within annual plan X 100% of SOEs audited* X 50 (no. of PAs/IAs) X 50% of technical staff trained X AGD’s submissions to Parliament indicate that maximum X audit delay reduced to one year and 100% of SOEs covered by audit Construction of AGD office building plus a training centre** X Total 2 1 3 Percentage of Key Target Indicators 33% 17% 50% *Note: this IO target should be given less weight as largely beyond AGD control. ** Note: the revised PDO building target represents a notable increase in ambition. 3.3 Efficiency Efficiency Rating Original PDO: Modest Efficiency Rating Revised PDO: Substantial 86. Precise, quantitative rates of return cannot be determined but one can judge overall efficiency to be modest under the original PDO and substantial under the revised PDO. While precise rates of return cannot be calculated for this type of project, the project efficiency can be judged based on a qualitative analysis of available data.33 In terms of component 1, the DCS achieved a number of project targets via efficient use of resources and investments. However, efficiency is judged as modest because: (i) it was unable to complete certain activities within the agreed timeframe; (ii) there were protracted delays in building construction; and, (iii) the component did not spend its allocated resources, with USD 7.33 million cancelled. In terms of component 2, there was substantial efficiency insofar as: (i) a number of project targets were achieved; (ii) it nearly fully disbursed by the extended project closing date; and, (iii) it made a saving of USD 2 million, which was then utilized to undertake activities with direct relevance to achieving the PDO. However, Component 2 required two time extensions in order to fully disburse the allocated resources, albeit completing two new activities. In sum, under the original PDO (DCS and AGD), the rating is modest on account of the significant delays and fund cancellation under component 1 as well as the time extensions under component 2. Under the revised PDO, the efficiency was substantial. 3.4 Justification of Overall Outcome Rating Overall Outcome Rating: Moderately Unsatisfactory 87. Based on the original PDO, the overall outcome rating emerges as moderately unsatisfactory. This aggregates the ratings for Relevance (substantial), Efficacy (modest) and Efficiency (modest), which are outlined above. Based on the Independent Evaluation Group (IEG) guidelines the combination of the Relevance, Efficacy and Efficiency rating leads to a moderately unsatisfactory rating (Annex 7). Overall, it is judged that there were moderate to significant shortcomings in the operation’s achievement of its objectives, in its efficiency and its relevance against the original PDO. 88. Based on the revised PDO, the overall outcome rating is satisfactory. This combines the rating for Relevance (substantial), Efficacy (substantial) and Efficiency (substantial), outlined above. Overall, it is judged that there were only minor shortcomings in the operation’s achievements of its objectives, in its efficiency and its relevance against the revised PDO. 17 89. Overall, based on the method for arriving at a rating for projects with a revised PDO, the ICRR team has arrived at a rating of moderately unsatisfactory. Table 7 summarizes the methodology. This ICRR also recognizes the challenges of applying an average rating to the observed results (see lessons learned and Annex 7). Table 7: Overall Outcome Rating (August 2012 Restructuring) Original Revised Rating PDO PDO Overall Outcome Rating 3 (MU) 5 (S) % Disbursed 97% 3% Weighted value (by % 2.91 0.15 3.06 = disbursed) 3 (MU) 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development. Not applicable. (b) Institutional Change/Strengthening. Given that the project focused heavily on institutional strengthening, all relevant points have been considered above. (c) Other Unintended Outcomes and Impacts (positive or negative) 90. A positive unintended outcome is that the training provided to DCS was received not only by DCS but also by members of other organizations. Overall, 690 officers of DCS and 124 officers of other organizations were trained under this Project, so one can assume that some benefits of the training accrued to some other organizations. 91. Another unintended outcome is that the PSCB arguably contributed, to a degree, to the Parliamentary oversight bodies (CoPE and the CoPA) being more active. Timely conduct of audit reports led to an increase in the number of reports to the CoPE and CoPA for review leading to larger number of SOEs being examined. Number of SOEs examined by CoPE was 81, 229 and 197 (up from 24 in 2009) and by CoPA were 39, 44 and 53 (up from 22 in 2009), respectively for 2010, 2011 and 2012. 4. Assessment of Risk to Development Outcome Risk Rating: Substantial 92. There are various reasons to be relatively optimistic about the prospects of maintaining the project’s development outcomes, especially in the shorter term. The government and implementing agencies have shown strong commitment towards the reforms targeted under the project, and, as outlined above, both components of the project remain relevant. There are also good reasons to judge that a number of the gains will be maintained and not reversed, especially as a number of post-operational steps have already been undertaken (see Section 2.5). 93. However, based on available information, one can identify certain risks that could potentially limit the prospects for the gains to be maintained over the medium term. Five main risks have been identified and assessed for their likelihood and potential impact: (i) There is a risk that AGD will not be able to retain the necessary staff . This is identified in the AGD’s Corporate Plan (2014-2015), and there has been a decline in AGD’s technical staff since 2010. An HR Strategy was developed under the project allowing the AGD to recruit independently and directly. However, this Strategy is pending approval by the Cadre and 18 Salaries Commission and it is not clear when it will be approved. Without an approval, AGD’s ability to maintain all areas of project progress may be curtailed, as also inferred from comments received (Annex 11). (ii) There is a risk that the enactment of the National Audit Act will be further delayed . Although enactment of the new audit law was not one of the target activities under the PSCB, it is one of the building blocks for the longer term effectiveness of the public audit function.34 At the time of the ICRR, the Act has not been tabled in Parliament and it is not clear when it will be tabled. (iii) There is a risk that efficiency gains made via the adoption of Teammate auditing software will not be maintained at an optimal level. In order to ensure the very latest software via updates, there is a large annual licensing fee, and there is a reasonable likelihood that the AGD would not have the resources to pay that fee. In addition, the requirement to translate audit reports into three languages before tabling in parliament contributes to delays in submission, as there is a weak pool of adequately training Tamil translators. (iv) There is a risk that some of the gains with regards to statistical capacity building will not be maintained unless an updated Statistical roadmap is developed and further resources allocated. The DCS has shown a high level of motivation and leadership. However, its SMP and Implementation Plan could be updated to ensure future work remains focused on strategic gap- filling. Adequate funding would also be needed to maintain the new statistical infrastructure under the project. More broadly, the adoption of revised statistical legislation would reduce longer-term risks to the achievement of DCS’ important roles as the lead statistics agency. (v) There is a small risk that the DCS building may be further delayed in the short term . While this risk is relatively low as construction progress has progressed significantly, the building remains incomplete; which impacts on DCS’s efficiency, productivity and effectiveness. 94. Taken as whole, the risk is judged to be substantial, even if the risks could be reduced if some pending actions are taken. The rating recognizes that a number of the gains made under the intermediate indicators are likely to be maintained. However, there are certain risks, as noted above, which are substantial when one balances the likelihood of them materializing against their potential impact over the medium term. These risks could be mitigated to some degree with the passing of the Audit Act and implementation of the HR Strategy, and through an updated SMP and Statistical Legislation. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Sub-Rating: Moderately Unsatisfactory 95. There were strengths in ensuring quality at entry. The project emerged from longer term engagement between the Bank and GoSL, the design was informed, to a good degree, by robust analytical work and the objectives were and remain quite highly relevant to development priorities. 96. However, it is judged that there were some significant shortcomings in quality at entry leading to a moderately unsatisfactory rating. These shortcomings relate to: (i) the lack of up-to-date and quantifiable baseline information alongside the limited clarity in certain M&E indicators and targets;35 (ii) the failure to establish a robust mechanism for ensuring co-ordination and oversight of both components; (iii) the overly ambitious targets in some areas, which failed to take into account the lessons learned on similar reform projects (e.g. most statistical capacity development projects are carried out over a 5 year period); and, (iv) greater attention could have been paid to risk mitigation, such as mitigating the risks associated with construction, addressing limited implementing agency capacity, and 19 strongly factoring in that the agencies would need to balance ambitious project reforms with a demanding day-to-day workload (such as the DCS with the national census). (b) Quality of Supervision Sub-Rating: Moderately Unsatisfactory 97. There were often two separate Bank teams supporting the AGD and DCS component, which makes it more complex to reach a judgment on overall Bank supervision. The components were often supported by two separate Bank teams, and one ISR notes that the project ‘suffered from this stand- alone approach’. However, as the project progressed, there was greater co -ordination between the two teams. 98. Component 2, on auditing, enjoyed relatively proactive support from the outset . The Bank provided regular supervision and timely support to the AGD counterparts from the outset via distance interaction and field missions. For example, the Bank team set up training to project staff on consultant selection and provided timely procurement approvals. The team was also responsive to changing client needs. At MTR, the task team identified the need for consultant support for another audit cycle and facilitated AGD requests for taking up additional activities in the time extension period. 99. However, the Bank supervision support, most notably for Component 1, was sub-optimal for nearly the first two years of implementation. There were major shortcomings. First, in 2010, an ISR (No. 4) noted that the project documentation, including record of communication with government and Aide Memoires, was very limited. This applied mainly to Component 1, even if there was also some limited documentation under Component 2; as well as some inconsistent documentation across the board, as noted in the ICRR. Second, Bank ISRs in 2009 and 2010 flagged that support and responsiveness to DCS counterparts was insufficient. There were delays on the Bank’s side for reviewing and clearing procurement requests. Moreover, the Bank support team did not include DECDG experts who had experience in running statistical capacity development projects.36 Third, for Component 1, reporting and implementation arrangements were weak up to 2010, and there was no record that the Bank team was following up on these arrangements. In sum, this lack of Bank support was documented as one contributing factor to delays in project progress up to 2010.37 100. Supervision to Component 1 nonetheless considerably improved from late 2010 with attempts to rectify project problems, which resulted in improvements. Following a series of Bank management comments and a change in team composition, a number of positive steps were taken. These included: ensuring the DCS and AGD were meeting all implementation requirements; the setting up of a stronger technical supervision support team; ensuring more regular documentation via ISRs and A-Ms; the identification of a baseline and strengthened M&E indicators for Component 1 (at MTR); acceleration of Bank clearance time for procurement; and, pro-active support to address procurement delays in the building of the DCS superstructure. This improved support helped Component 1 to achieve a 'moderately satisfactory' rating in PDO achievement and overall implementation in 2011. 38 Nonetheless, given that the procurement of the DCS superstructure was not resolved, the GoSL communicated to the Bank to close the Component in December 2011 and cancel 7.33million USD. 101. The Bank was also responsive to changing circumstances, although there was a time-lag in restructuring the PDO. It undertook a Level 2 restructuring on October 21 2010 to ensure continuity in AGD PMU leadership as well as two one year time extensions. It also undertook a Level 1 (PDO) restructuring to respond to the closure of Component 1 and continuation of Component 2. However, the Level 1 Restructuring of the PDO was not effective until August 2012 (7 months later).39 As noted in Annex 8, this has an impact on the overall outcome rating. 20 102. Overall, it is judged that the supervision support is moderately unsatisfactory, even if an overall judgment is difficult to reach as support varied over time and by component. This rating balances a number of considerations. It acknowledges the major shortcomings in supervision support in the first two years to Component 1, the limited documentation in the project, the lack of overall project co-ordination and the delayed restructuring of the PDO. However, this should be balanced against the fact that the supervision support was pro-active for the AGD component, from the outset, and that the support for DCS markedly improved from late 2010. However, these improvements perhaps came too late within the planned 3-year implementation period to entirely turn the project around. (c) Justification of Rating for Overall Bank Performance Overall Bank Performance Rating: Moderately Unsatisfactory 103. The overall rating for Bank performance is moderately unsatisfactory. This combines the moderately unsatisfactory quality of entry with the moderately unsatisfactory supervision, and also considers the impact of this performance on the overall outcome rating. Taking an overall view of the project, the Bank could have been more proactive and responsive from the outset with support to the two components in equal measure. 5.2. Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 104. The government showed commitment to the project throughout. The GoSL ensured implementation readiness, timely resolution of implementation issues and complied with the Bank fiduciary requirements. Adequate and timely counterpart funding via the GoSL’s land contribution was available. Transitional arrangements are also in place with government resources allocated to the completion of the Ratnapura and DCS buildings, as well as allocation for operation and maintenance. 105. More specifically, the government supported activities under Component 1 – statistical upgrading – even if there were some minor challenges. In addition to supporting overall implementation, the MoFP is supporting DCS in becoming the main source of statistical data by chairing the NDC meetings and it has also provided its own funds to complete the DCS building. There were, however, some delays in the DCS building construction that were related, in part, to relatively slow government approvals and to the fact that the MoFP required floors of the DCS so the building plans had to be re-designed.40 106. The government, moreover, actively supported the Component 2 auditing activities, even if it perhaps could take further steps in strengthening the auditing enabling environment. The GoSL provided support for the achievement of the auditing PDOs. The Government also reiterated its instructions to SOEs to address the audit findings and the Parliamentary oversight bodies submitted their reports in 2011 after a hiatus of 4 years. However, the government did not complete two aspects that shape the enabling environment for AGD effectiveness: (i) the National Audit Act has not yet been presented in Parliament; and, (ii) the new HR strategy is still pending approval. While the Audit Act is important, it is given less relative weight in reaching the rating here as it was not a stated project pre- condition for achieving the project PDOs. (b) Implementing Agencies Performance Rating: Moderately Satisfactory 107. The performance of the AGD was, overall, strong. Commitment to achieving the PDOs was evident throughout. In the earlier stages, there were some delays in implementation readiness, but this 21 markedly improved. Overall, the AGD illustrated strong performance on all accounts, ensuring smooth implementation and clearing all impediments and challenges as they occurred. AGD ensured a well- functioning PMU. The presence of a qualified procurement specialist throughout the life of the project helped the construction of the office building within the project time frame. The Project Director actively supervised implementation progress and provided leadership, strategic direction and timely guidance. The AGD was able to exceed certain targets by using cost savings, mainly within the time extension period, to undertake key additional activities outlined above. 108. The performance of DCS was good but was also hampered by challenges. The DCS showed commitment and leadership throughout. However, frequent turnover of DCS PMU staff – especially procurement staff – hampered implementation and contributed to slower progress. This contributed to the fact that the DCS office building was partially completed by the end of 2011. Other challenges hindered its progress such as: (i) the demands of the national census and other surveys on staff time; (ii) delays resolving land boundary and squatting issues; (iii) delays in Bank clearances (noted in the first two years of implementation); and, (iv) the inability to recruit superstructure contractors as per Bank guidelines. Following improved Bank support in 2010 and the recruitment of international consultants, DCS was able to further improve its performance. 109. When one combines the performance of both implementing agencies, the rating is moderately satisfactory. This balances the fact that AGD and DCS showed good performance, even if there were some notable shortcomings, principally associated with completing the building under Component 1. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 110. The overall rating of borrower performance is moderately satisfactory. This is a combination of government and implementing agency performance. It balances the commitment and implementation progress of both sets of actors against shortcomings related to implementation delays and the under- achievement of some targets. 6. Lessons Learned 111. Civil works in public sector reform programs can be risky so it would be advisable to finance them selectively and support them appropriately. This operation shows how combining civil works with institutional strengthening can present risks and mixed results. On the one hand, the civil works component can hinder progress and divert project teams’ attention from focusing on core instit utional results. On the other hand, buildings can be successfully completed as a critical input to institutional strengthening. Two major lessons emerge from this. First, the mixed results in this project could be explained by the differences in the implementing agencies’ leadership, capacity and skill mix to deliver on civil works; and, by the differing degree of Bank supervision support by Component. Second, the following key aspects would need to be considered to decide whether or not to include civil works in public sector reform programs: (i) there are no alternative funding sources for the construction work; (ii) buildings are demonstrated to be critical in achieving the PDOs; (iii) the building plans are at advanced readiness and subject to a robust risk mitigation plan; and, (iv) appropriate staffing (including construction experts) are members of the core task team throughout. If civil works are included, the focus needs to remain on core institutional results alongside disbursement. This focus could be strengthened by using results-based lending instruments. 112. Different components could be combined into one project where there is a compelling case to do so and where there are strong mechanisms for overall coordination and oversight. The 22 combining of quite different components into one project can add to design and implementation challenges. These include challenges for project monitoring, supervision support (as each component can have different objectives and capacities), sharing of relevant expertise across components and for rating overall performance. This also risks increasing transaction costs. As such, this combining would only need to be considered where there are compelling reasons to do so. In cases where it is deemed necessary, it would be beneficial if overall institutional implementation arrangements are strong and if synergies between the different components are maximized. For instance, the PSCB could have jointly overseen the construction of the two buildings, thus potentially reducing transaction costs. 113. It is important that projects have more realistic and measurable targets based on the implementation context and relevant international experience. The project targets were, in places, overly ambitious and not fully calibrated to the challenges of local context and implementing agency capacities.41 For example, Statistical capacity building programmes are usually complex and require ample time – often five years or more – to achieve results, especially where the implementing agency has limited experience and capacity. 42 Moreover, the effective monitoring of progress and resolution of bottlenecks was undermined by weak baselines and difficult-to-measure targets. In sum, a robust results framework needs to be in place by the point of project appraisal. 114. Strong Bank supervision is required from the outset, especially when working with lower capacity agencies. The degree of supervision support in the project was important in influencing progress towards the PDOs. In fact, the variable quality of supervision explains, in part, the variable performance by component. Moreover, improvements in supervision over time – pro-active management follow-up, changes in team composition and strong mid-term review – also contributed to improved outcomes, notably under the statistical capacity building (Component 1). This underlines the key importance of strong supervision throughout. However, given that Bank supervision budgets are constrained and not based on the complexity of the project, concerted efforts should be made to optimize supervision with existing resources and to mobilize additional resources from other sources. 115. There is a need to respond to changing project circumstances as quickly as feasible. This operation reflects how public sector change is often complex and non-linear. As such, timely and adaptable support is needed in order to ensure results are maximized over time. On the one hand, the project showed good adaptation by facilitating time extensions and enabling the implementation of additional activities, which strengthened the PDO results. On the other hand, the project could have restructured the PDO at an earlier stage. 116. Implementing agencies’ leadership and ability to retain project staff are key drivers of results. The components performed differently in the same operating environment. This can, in part, be explained by differences in the implementing agencies’ ability to retain staff and drive results in the face of challenges. On the one hand, the AGD PMU had strong and pro-active leadership; alongside a dedicated and well-functioning project management team with longer term staff. The AGD project completion report notes that technical leadership combined with the soft skills of negotiation, conflict management and team building were critical in driving results. On the other hand, the DCS experienced challenges in retaining key staff, such as a procurement specialist.43 As such, project design teams would be wise to give systematic attention to assessing the leadership capabilities of their counterpart implementing agencies and should tailor the project targets and supervision accordingly. 117. Future projects could adopt, and build upon, the project’s capacity building model. The AGD adopted a multi-stage approach with a core focus on building in-house capacity and informed by its longer-term plan for institutional development. This focused on a ‘training-of-trainers’ method as follows: first, consultants trained a select group of staff in the new audit methodologies; second, this select group’s knowledge was consolidated via pilot audits which were adapted to AGD needs; and, third, there was a roll-out of enterprise-wide training led by the in-house teams.44 Finally, these gains 23 were further institutionalized via the physical establishment of an in-house training division and the building of a residential training centre. 118. Future statistical capacity development operations could build on international experience and actively engage a wider range of stakeholders. There is an increasing body of international evidence of what works in statistical capacity development. It is critical to ensure that the design team of future projects have access to this evidence and specialist advice; for instance by engaging DECDG specialists. A number of these lessons were reflected in the PSCB, such as the importance of adopting a longer-term timeframe. In addition, two other key lessons can be flagged. First, it is important that the key performance indicators are defined through consultation with a range of stakeholders to ensure that they are measurable and widely ‘owned’; and to ensure that all stakeholders are moving in the same direction. Second, an SMP has greater prospects of effective implementation if it is actively supported by higher- level bureaucratic and political ‘champions’. This can help ensure that all government agencies support its implementation.45 119. The effectiveness of public sector institutions could be further strengthened if investments are accompanied by stronger policy dialogue and reform. The successes in this project illustrate the important results that can be achieved while working within the existing reform space. However, the public institutions’ long-term effectiveness may be undermined if broader reforms in the enabling environment are not also undertaken, such as revised Statistical Legislation or the enactment of an Audit Act. In future operations, mechanisms for stronger policy dialogue could be built in through, for example, a component on information sharing in investment lending or through a Development Policy Loan or a results-based lending instrument. More specifically, policy reform processes could be strengthened via evidence-based policy dialogue, the building of reform-minded coalitions and engaging with a broader range of societal stakeholders. For example, one could seek to strengthen the impacts of accountability mechanisms by strengthening parliamentary oversight and follow-up; and by fostering more pro-active transparency of information on audit and statistical issues.46 120. An average outcome rating may not always give the fullest account of results achieved on the ground in such complex projects. Given that the project combined two disparate components and there was significant variation in targets and results by component, it is challenging to arrive at a representative overall outcome rating. For example, the current outcome rating, alone, does not capture the substantial achievements of Component 2. Furthermore, the methodology for rating based on PDO restructuring may not always adequately capture the actual results on the ground, as noted in Annex 7. In the future, one might consider minor modifications in the ICRR guidelines in order to provide final split ratings by component as opposed to a combined average. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies. Not applicable. (b) Cofinanciers. Not applicable. (c) Other partners and stakeholders. Not applicable. 24 Annexes Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Components Appraisal Revised at Actual/Latest %age of Estimates Restructuring in Estimate Appraisal Aug 2012 1. Upgrading Statistical Capacity 12.05 4.72 3.79 31% a. Goods, Consultancy, Training and 6.05 1.50 3.12 52% Operating Costs b. Civil Works – physical infrastructure 6.00 3.22 0.67 11% 2. Improvement in Audit Standards 10.55 10.55 9.96 94% c. Goods, Consultancy, Training and 7.05 7.05 5.54 79% Operating Costs d. Civil Works – physical infrastructure 3.50 3.50 4.42 126% Grand Total (1+2) 22.60 15.27 13.75 61% (b) Financing (in USD Million equivalent) Source of Funds Type of Co- Appraisal Revised at Actual/Latest %age of financing Estimates Restructuring in Estimate Appraisal Aug 2012 IDA 22.60 15.27 13.75 61% Total 26.10 15.27 13.75 Note: As per PAD (page 42), the Government of Sri Lanka was to contribute USD 3.50 mn by way of cost of land (USD 3.30 mn) and operating costs (USD 0.20 mn). The Government has provided land for the DCS and AGD buildings (as envisaged in the original project design) and in addition provided land for the Ratnapura training center building. Since the value of the land is not available, these have not been included in the table above. It can be concluded that the Government had fulfilled its commitment towards counterpart contribution. 25 Annex 2. Outputs by Components Component 1: Upgrading Statistical Capacity Activity Outputs at Project Close (Dec 31, 2011) Part A: Organizational Development: Improving the statistical system through strengthening the leadership capacity of DCS and improving coordination among all players in the statistical system. TA to design and  National Data Committee (NDC) formed on October 27, 2010 in order to help implement a data collection support and coordinate DCS activities with other agencies/ministries also and coordination strategy responsible for the production and/or use of statistics. The NDC is chaired by the by DCS with other data Deputy Secretary to the Treasury and includes representatives from various collection agencies to agencies involved with statistics, including a university lecturer. reduce duplication of work  A concept paper on the preparation of an Official Statistics Plan (OSP) was and transaction costs. developed and reviewed by the NDC but has not yet been implemented (awaiting MoFP approval)  MoU prepared between the Inland Revenue Department (IRD) and the DCS to facilitate transmission of tax data from IRD to DCS for the preparation of National Accounts; not yet signed.  A draft revised Statistical Ordinance was produced by the NDC and is awaiting approval by the MoFP.  The formation of 3 out of 7 planned Functional Coordination Committees (FCC) at the DCS was carried-out in order to improve inter-divisional coordination. The 3 committees include i) Sample Survey Coordination Committee; ii) Training and Research Coordination Committee; and iii) GIS Coordination Committee. Two other committees are related to the census and were postponed until after the census completion. A 4th FCC was later formed: Coordination Committee for Improving National Accounts Statistics. Skill development  A targeted multi-year, rolling Training Plan was carried out and benefitted DCS programs that follow staff as well as other agency staff. 26 training programs were implemented during international statistical the project time period. Training programs were conducted locally (80%) and standards, including abroad (20%), with India being the main destination for foreign training. The training, workshops, and majority of courses were related to Statistics (40%), followed by Language (30%), study tours. Economics (14%) and other courses.  Consultancy carried-out on “Upgrading the Training Division” of the DCS. As a “by-product” of this consultancy, a web-based learning management system was also prepared. Part B: Statistical and IT Infrastructure and Client Services: Upgrading IT to achieve greater efficiency in survey design, data collection, transmission, processing, management, and dissemination. Building and improving the statistical services to better serve data users. TA for IT-related needs  Consultancy carried-out by a World Bank-recommended expert on the assessment in terms of development of a methodology for Scanning Census Questionnaires – beneficiary survey design, data was Census Division of DCS. TA was also given on the technical basis for collection, transmission, drafting the tender of the “Pilot Census Scan Test”. processing, management and dissemination. Design, development and  An ICT Strategy was developed by Ernst & Young and has been partially implementation of the IT implemented. The partial implementation is due to the delay in the construction of and statistical systems the DCS building. required per the DCS needs assessment. Training and workshops on  Consultancy carried-out by Skills International Ltd on the Development of a Web- IT implementation issues. Based Library Catalog. Staff can now fully access the DCS library catalog on-line.  Consultancy carried-out by Access International Ltd for the Development of a Computerized Registry of Industries. The components include a Local Area Network (LAN) consisting of a server and five computers, and the staff of the 26 Activity Outputs at Project Close (Dec 31, 2011) Industries Division has been trained on how to keep the register updated. Hardware and software for  District offices have been provided with Internet connection and upgraded data development and hardware and software, in order to facilitate the decentralization of data collection information sharing. and processing. By end-2011, 40% of district offices were sending data electronically, and this figure is expected to reach 100% within a few years.  Where needed, district offices have also received necessary items such as computers, printers, and multimedia projectors.  Portable Data Assistants (PDA) have been procured for price collection, greatly increasing the processing time, and include on-line edit checks to minimize data entry errors. Consultant services to  Two user education workshops were conducted, as part of the Data User Education design and improve client- Program (9 workshops were initially planned, and will continue after the census). oriented data services and consultation, as well as a user education strategy to facilitate statistically accurate use of data among users. Training focused on user  A TA consultancy was carried out on the development of a data User Needs services and Survey. The staff of the Research and Special Studies Division in the DCS responsiveness to user participated in this training. needs. Web-based data  The DCS prepared a micro-data dissemination policy, based on feedback from the dissemination and clear user community. The policy and application form is available on the DCS website. articulation of policy to  A data User Needs Survey was conducted, following a TA consultancy on the improve data sharing. process.  18 methodological and procedural Manuals were produced by the DCS. Currently they are only available on the internal DCS website, however they are expected to eventually be translated into English and become available publicly. Part C: Data Development: Developing statistical series that are compatible with international standards and conform to the needs of an emerging middle-income country. TA to design and develop  DCS participated in the Accelerated Data Program (ADP) in order to develop a an integrated meta- and meta- and data catalog, which is fully operational. database on major economic and social statistics. Study visits to familiarize  Part of Training Program as described in Part A. Foreign study visits were made to technical staff with India, the Philippines, Thailand, the USA, and several other countries. international best practice. Statistical series training.  High-profile consultancy TA activities included “Improvement of National Accounts Statistics” and “Improvement of Agriculture Statistics”. Part D: Physical Infrastructure: Construction of an office building for DCS in order to house all 15 units of the DCS, which are located in 8 locations in and around Colombo. Construction of an office  A Civil Works Consultancy (Design, Construction Supervision, and Project building. Management) for the new office building was completed through the piling stage. The contract was awarded to Engineering Consultants Ltd. on October 1, 2009. Part E: Project Management Establish a Project Steering  Prior to the appointment of a Project Steering Committee, a Project Working Committee Committee was formed and was headed by the Director General of the DCS as the Chairperson. The Working Committee prepared the agenda for the Project Steering Committee and made recommendations on implementation of the project at department level.  Project Steering Committee was formed on December 2, 2008 with the Deputy Secretary to the Treasury as the Chairperson. The DG of the DCS, other DCS 27 Activity Outputs at Project Close (Dec 31, 2011) staff, and directors of the Dept. of the National Budget, Dept. of the National Planning, and Dept. of External Resources were committee members. Establish a Project  Project Management Unit was formed with Dr. A. Satharasinghe appointed as the Management Unit with Project Director. Other staff members were appointed gradually and included a staff Procurement Specialist, Finance Manager, Junior Project Manager, and support staff for a total of 13 filled positions by June 2009.  Consultancies were implemented for Procurement Specialist staffing and the appointment of a Finance Manager. Component 2: Improvement of Audit Standards Activity Outputs at Project Close (Dec 31, 2013) Part A: Audit Methodology: Enhancement of the capacity of AGD to expand audits to include investigative and performance audits over the medium term Consultancy services for  Separate audit manuals for FA’s, PA’s and IA’s (prepared by the consultants) field the preparation of manuals tested and approved by the AG and guidelines to conduct  PA unit established in 2010 under Assistant AG supported by 6 officers various types of audit  IA Division established under Deputy AG supported by a Superintendent of Audit and 6 officers Specialized training  Training curriculum finalized program on improved audit Performance Audit methodologies for  4 days training to core group of 40 Audit Superintendents and Audit Officers on investigative and draft PA manual conducted by consultants performance auditing  TOT conducted by consultants to 15 Officers on PA and training imparted to 60 officers by these trainers  Piloting of 7 PAs with 25 officers in 10 identified areas Investigative Audit  Core group Training to 40 audit officers on IA by consultants and selection of 15 trainers  Piloting of 7 IAs in 10 identified areas Part B: Human Resource Development: Development and implementation of a Human Resources Development strategy by developing individual skills to the highest professional standards by establishing training and development programs using best practices, and developing and implementing an objective staff valuation system. Consultancy services to  Computerized audit time management system for daily time recording (E-times) design HR development piloted in 4 branches Strategy and training  A new staff performance evaluation scheme linked to incentives developed and modules and develop an sent to Salaries and Cadre Commission for approval in 2010 (this included objective staff assessment proposal for recruiting specialists such as engineers, computer experts and and evaluation system lawyers) Development and  About 379 officers enrolled in a 6-month, 4 hr/week training course in English implementation of training language programs (local and  384 out of 400 vacancies filled by persons with relatively higher level of education foreign) in international and provided induction training best audit practice  77 officers pursued post graduate degree in local universities  A permanent Training Division under DAG set up for a more structured and systematic training Financial Audit  4 days core group training to 63 officers on new FA methodology completed by consultants and 20 officers selected as instructors  FA Training by consultants to 10 groups of 40 officers each total 400 by the core group trainers including electronic working papers  9 pilots in FA conducted between Jan to Sept 2011 covering 100 institutions using 28 Activity Outputs at Project Close (Dec 31, 2013) new methodology and TeamMate  2 days training to 20 FA division heads in new FA procedures manual and 60- member core group received 4 days training in preparing 10 pilot financial statements audit  300 staff trained in International Financial Reporting Standards in the new AGD building  New Training department provides financial audit training to 19 classes totaling 590 officers. Facilitation of Twinning  With SAI China for customized training in public works audit attended by 10 arrangements for public officers by AGD auditing staff Part C: Communications and External Relationships: Enhancement of the capacity of AGD to communicate with Parliament, auditees and the general public in order to build better understanding and relationships. Consultancy services to  Annual Reports of AGD for 2009 to 2012 tabled in Parliament respectively in Aug implement communications 2010, Oct 2011, Oct 2012 and Oct 2013 with enhanced look and content and infrastructure and tools for separate chapter on the PSCB Project till 2011 communicating with  MoU entered for development and maintenance of new website. Units established Parliament, auditees, and in AGD to support COPE and COPA in scrutinizing audit reports the general public  Hard cover coffee table book entitled "Auditor General's Department of Sri Lanka 1799-2012: Continuation of a Long Journey" produced for distribution at the opening of the building. The book describes the role of the AG and devotes 8 pages on PSCBP  International conference on “Supreme Audit Institutions: Meeting Higher Expectations” held in Colombo in June 2013 attended by 18 countries (activity added through restructuring)  Internal Control training provided to Health Ministry Funding for media access  Media access funding in selected areas. Acquisition of electronic  Fully automated digital mini press installed and other communication equipment to support improved media relations Part D: IT and Equipment/Fittings: IT is fundamental to running a successful and effective modem institution. IT supports the underlying management systems of the organization, including planning, HR, communications, and so on. IT can be used as a powerful audit tool to provide more efficient and effective use of audit resources. Hardware and software to  Two computer training labs with 60 desktop PCs established implement the IT operating  3 servers and 650 laptops procured 114 desktops systems  50 branches out of 220 linked with head office server and others are in progress  TeamMate – audit working paper software acquired and installed on 400 laptops  Establishment of LAN and installation of back-up generators  CAAT software acquired Training for staff on IT-  419 officers trained in Computer Driving License based audit over the  23 officers trained in CAAT by the consultants and 10 provided on the job training medium term in organizations where computerized systems available  Core group training to 40 officers in TeamMate by consultants; 3 officers providing further training as TeamMate trainers; 400 officers trained in TeamMate Furniture, fittings, Building completed in March 2012 with furniture & fittings, office equipment etc. communication, installed and networking completed networking, IT infrastructure for an independent AGD building and customization of offices through carrying out of works. 29 Activity Outputs at Project Close (Dec 31, 2013) Part E: Physical Infrastructure: Construction of an office building for AGD with connecting the head office with the regional offices. Subsequently, construction of a residential training center at Ratnapura was included. Construction of an office  Building construction commenced in May 2010 and was completed and building inaugurated in March 2012  Project cost savings utilized to construct additional floor for training department Construction of training  80% works completed, including material brought to site center at Ratnapura (additional activity from project cost savings) Part F: Project Management Establish a Project Steering  Established with AG, representatives from Parliament, Treasury and Presidential Committee Secretariat and two senior officers of AGD as members Establish a Project  Established with a retired Deputy AG as Project Manager. Procurement Specialist Management Unit with and Project Secretary engaged on contract basis. Project Accountant and two staff assistants placed on deputation on part time basis 30 Annex 3. ICRR Methodology Note: PSCB ‘Causal Chain’ and Indicator Interpretation Causal Chain In order to assess the extent to which the PSCB project achieved the highest-level PDO, the causal relationships between PSCB inputs, outputs and outcomes were scrutinized, as outlined in table 8 below. While it is difficult to attribute results with full certainty in complex institution-building projects, it was found that there were no other major initiatives, or donor-funded activities, that could have generated the results assessed in the ICRR. The ICRR considers the results that can be reasonably attributed to the project. * Table 8: PSCB ‘Causal Chain’ PDO Evidence of the improved effectiveness, efficiency, and productivity for each of the key public sector agencies (DCS Level and AGD). Evidence that improved public audit functions by Higher Evidence that DCS is relied upon as the main source of AGD have increased accountability and transparency Level national statistical data for the country, as reflected in the in the use of public funds, as reflected in the annual Outcome National Data Committee. public audit cycle submissions by AGD to Parliament. Improved Data access to data Percentage processing and Widened of technical Decreased of next disseminatio Timeliness public staff trained Number of processing HIES n of data of audit Inter- Percent of audit and using investigati times for survey outputs; reports mediate tech staff coverage recommend ve and completio conducted facilitating (Percent of outcome trained (Percent of practices performan n of key according to user- audits SOEs (including ce audits data series. HIES pilot education delayed) audited) IT-based data entry through audits) method workshops and website  New census methodologies.  New audit methodologies manuals.  Expert reports and action plans.  Expert reports and action plans. Key  Training programs completed.  Training programs completed. Outputs  Human resource development strategy.  Human resource development strategy.  ICT equipment.  ICT and office equipment.  Office premises  Office premises.  Technical services (e.g. consultancies)  Technical services (e.g. consultancies) Inputs  Physical infrastructure investments  Physical infrastructure investments  ICT equipment  ICT and office equipment *Note that this is not an exhaustive list of activities, but is indicative of the project causal chain assumptions. ICRR Interpretation of Indicators In order to carry out the ICRR assessment, a few clarifications were required as summarized here:  First, it is judged that the PSCB RF indicators are largely appropriate measures of progress towards the PDO, even if there some notable shortcomings as outlined below. In the ICRR, the achievement of the PDO is mainly based on an analysis of achievements against the following core target indicators: 2 higher-level outcome indicators, 8 IO indicators and 2 key output indicators. Where relevant, this is supplemented by a consideration of other results which, although not linked to a formal 31 indicator, reflect progress towards the highest-level PDO.47 Given that the PDO was restructured, results are judged against the original and revised PDO.  Second, the two output indicators that are assessed relate to the DCS and AGD buildings. They are explicitly considered in the ICRR even if they were not in the formally-agreed RF. Given that the buildings represented a significant portion of allocated project funds and given that they were a focus of ISRs, they are given explicit treatment here. However, as per IEG guidelines, their achievement is not weighted in line with disbursement.  Third, certain RF indicators lack clarity, so the ICRR has made its interpretation explicit in order to reach a judgment, as outlined in Table 9 below. Where necessary, the ICRR team has clarified the indicator and employed relevant sub-indicators based on project documentation. Note that these interpretations do not represent new indicators and, where necessary, the interpretation was validated in discussion with the most recent project Task Team Leader (TTL) and client. Where the target is partly beyond the project control, this is taken into account as far as is feasible in reaching a rating. The ICRR ratings also recognize that Component 2 had two additional years of implementation compared to Component 1. The judgment takes this into account and also notes that certain targets were revised upwards to reflect this extension.  Finally, in certain areas, the evidence is patchy with a lack of quantifiable data and there are some discrepancies between available client documentation of certain results compared to some ISRs’ documentation of results. For certain targets, this has made it difficult to reach robust conclusions. The ICRR thus makes clear where the evidence is patchy and reaches the rating accordingly, explaining any discrepancies in the endnotes. Furthermore, there are minor inconsistencies in the wording of some indicators in the project documentation, without accompanying justification. 48 Where there are discrepancies, the ICRR uses the PAD and legal agreements as the authoritative source. Table 9: ICRR Clarification of Key Target Indicators (when applicable) Target Interpretation/Clarification Issue ICRR Clarification PDO Level Strong evidence of Term ‘strong’ leaves some scope for ‘Strength’ assessed based on: (i) available improved judgment and interpretation. Also, difficult to documentation; (ii) achievement of key target efficiency, assess ‘strength’ in certain areas given limited indicators; and, (iii) other results under effectiveness and quantifiable baselines, targets and planned project activities. productivity of the measurement of results. DCS and AGD. Component 1 Quarterly/biannual A few points: (i) unclear if quarterly/biannual Target assessed based on: (i) establishment of NDC Meetings. meetings required for both years 2 and 3, or NDC and biannual NDC meetings taking can be averaged across project life; (ii) not place on average across Years 2 and 3 (i.e. 4 stated if biannual meetings are also adequate times); and, (ii) the sub-indicators agreed with (or quarterly preferred); and, (iii) regularity the client at MTR (although not formally of NDC meetings, along, may not give full integrated into RF). sense of progress. As such, the MTR added some sub-indicators related to the functions of the NDC: (a) data collection and co- ordination strategy drafted/in place; (b) MoUs signed with line agencies; (c) revised statistical legislation in place; (d) functional coordination committees are operational; and, (e) baseline assessments completed for agriculture, ICT, national accounts. National Census It is not immediately clear if this refers to: (i) Based on verification of project reduced to 24 completion of the entire census within 24 documentation, target is defined as data entry months months; or, (ii) completion of data entry processing time.49 process within 24 months. 32 Target Interpretation/Clarification Issue ICRR Clarification HIES survey A few issues: (i) does not specify the targeted Based on project documentation, target completed extent of the incorporation of the pilot data assessed based on: (i) adoption of according to a new entry methodology; (ii) does not specify decentralized data collection; and, (ii) what data collection HIES year of implementation; and, (iii) does can be judged a ‘reasonable’50 extent of scheme not fully elaborate on what new data scheme adoption given timing and sequencing collection scheme entails. of activities for the next HIES. Review of new data A few points: (i) unclear how to judge what Target assessed based on: (i) ‘review’ sharing policy end-target of policy ‘review’ is; (ii) how to considered to be an explicit (re)drafting of based on feedback judge what is ‘regular’ education workshops; policy; (ii) ‘regular’ considered as biannual; from users; regular and, (iii) how much/often feedback should be and, (iii) feedback to be attained via 1 major user education sought. MTR identified sub-indicators with survey. It also considers MTR sub-indicators workshops; the client to deepen the indicator: (a) but not ‘number of client visits to DCS feedback from implementation of user satisfaction survey; website’, as no baseline. users on data (b) user education workshops regularly access conducted; (c) number of client visits to the DCS website; and, (d) volume of information available on the DCS web-site. Component 2 AGD’s Additional indicators were not outlined for this Assessed based on the two relevant submissions to but relied on the combination of two IO targets intermediate indicators. Secondarily, project Parliament outlined below: (i) 100% audits within annual activities relating to improved accountability indicate that plan; and, (ii) 100% of SOEs audited (timely). and transparency are considered as per PAD maximum audit indicator related to this target. delay reduced to one year and 100% of SOEs covered by audit. 100% of audits The documentation does not explicitly define Based on documentation and discussion with within annual what is meant by ‘within annual plan’. client, ‘within annual plan’ understood as: (i) plan Moreover, the associated indicator at PDO clearing the backlog of audits; (ii) audit restructuring was reworded from ‘Improve reports on accounts submitted issued by AGD audit quality and timeliness of AGD’ to by September 30 (i.e. in time for inclusion in ‘Timeliness of Audit Reports’ without any submission to Parliament; and, (iii) tabling of change in baseline or target. The reason for audit reports in Parliament. this change was not mentioned in the restructuring paper. ‘Quality’ was not formally measured by a specific indicator in the project, although there were observed quality improvements which are mentioned in the ICRR. 100% of SOEs It is unclear how 100% coverage is defined/ Based on the findings of the CFAA and audited measured as AGD already had the mandate to discussions with the AGD, this target is audit 100% of SOEs at project outset (except assessed based on: (i) % of timely submission the 94 limited liability companies under the Sri of accounts by SOEs for audit (i.e. by Lanka Companies Act). The ISRs appear to February 2851); and, (ii) expanding mandate measure this indicator in terms of which SOEs of AGD SOE coverage to include the 94 were covered in audit mandate, but this status limited liability companies (via the Audit was unchanged. Act). The ICRR recognizes that this IO target is, at least in part, beyond AGD control. ‘Percent of The latter part of the indicator, ‘using Target assessed based on % of people trained. technical staff recommended practices (including IT-based While usage not directly measured in project trained and using audits)’ was not discussed specifically in documentation or in this ICRR, it is implicit recommended project documentation. based on observed changes in implementation practices of new audits and methodologies captured by (including IT- other indicators. based audits)’ 33 Annex 4. Technical Assistance Before or During PSCB Project name, ID and loan Start Date End Date Project description value (IDA only) Economic Reforms and March 10, 2003 June 30, 2008 The project aimed at strengthening the Technical Assistance (ERTA) government's capacity to implement its P077586 economic reform agenda by funding priority 15 Million USD technical assistance that provided the government with the executive capacity required to implement its reform agenda. The project served as the precursor to the development and strengthening of the statistical and auditing capacity of the country. Legal and Judicial Reforms July 1, 2000 June 30, 2005 The project aimed at improving upon the Project (LJRP) existing legal and judicial framework by P044809 making it more efficient, transparent and 18.19 million USD responsive to the needs of the public at large and of the private sector in particular. Strengthening judicial functions would be strengthened Improving the capacity of both DCS and AGD was an inherent part of the strengthening enabling institution that would promote good governance and the rule of law. Sri Lanka Central Bank August 24, 2001 July 31, 2005 The project aimed at supporting the Central Strengthening Project Bank of Sri Lanka in its efforts to effect a P071131 fundamental restructuring and reorganization 30.3 million USD program to create a lean, well-functioning, modernized, and efficient central bank capable of supporting strong economic growth over the medium to long term. 1 million USD was also provided by the Swedish International Development cooperation agency – 1) Sri Lanka Strengthening July 14, 2008 July 14, 2011 The objective of the grant was to strengthen Parliamentary Oversight (effectiveness) the effectiveness of the Committee on Public P111185 Accounts and the Committee on Public 0.49 million USD (cancelled Enterprises by i) enhancing the capacity of the 0.36 million USD; committee secretariat to maximize the disbursed 0.14 million efficiency and effectiveness of committee USD) meetings, ii) enhancing the capacity of committee members to attend and conduct more substantive meetings, and iii) enhancing public access to information about Parliamentary oversight committees. 34 Annex 5: Additional data for assessment of results for Component 2 Auditing Component (the data below used in the assessment of the indicators has been provided by the Sri Lanka AGD) Year-wise number of Performance Audits and Investigative Audits conducted Nature of Audit Year in which audit completed 2010 2011 2012 2013 Total Performance Audit (PA) 7 2 3 5 17 Investigative Audit (IA) -- 10 18 14 42 Timeliness of Audit Report in AGD Financial No. of No. of Total No. of Percent of audits Reports Tabled in Year reports reports reports issued completed prior to Sep Parliament / years of issued up issued after audited 30 to total audits submission to Sep 30 Sep 30 1 2 3 4 5=2/4 7 2009 372 472 844 44% 941 / 2010 to 2011 2010 509 355 864 59% 793 / 2011 to 2013 2011 574 273 847 68% 737 / 2012 to 2014 2012 569 264 833 68% 188 / 2013 to 2014 Widened public audit coverage Financial No. of accounts No. of accounts Total No. of accounts Percent of accounts rendered Year rendered to AGD rendered to AGD rendered to AGD for timely by Feb 28 after Feb 28 audit i. e. by Feb 28 1 2 3 4=2+3 5=2/4 2009 510 197 707 72% 2010 627 217 844 74% 2011 650 197 847 77% 2012 590 273 863 68% 35 Annex 6. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Name Title Unit Lending Minneh Mary Kane Lead Counsel LEGES (08137) Ismail Radwan Country Program Coordinator ECCU5 (07002) Olga Vadimovna Shabalina Senior Statistician DECDG (00566) Tatiana Nenova Adviser BPSVP (08446) Nobuo Yoshida Senior Economist PRMPR (07682) Asta Olesen Senior Social Development Specialist SASDS (08173) Asif Ali Senior Procurement Specialist SARPS (07080) Jiwanka B. Wickramasinghe Sr. Financial Management Specialist SARFM (07079) Samantha Prasada Wijesundera Water & Sanitation Specialist SASDU (08171) Luwisdura Lohitha Sanjee Karunasekera Private Sector Development Specialist SASFP (05621) Darshani De Silva Environmental Specialist SASDI (07206) Prabha Kumari Perera Team Assistant SARPS (07080) Farah Zahir Senior Economist SASGP (08243) Zeenath Marikar Program Assistant SASEP (07032) Vinod Beri Senior Executive Assistant WBIIN (08294) Pinaki Joddar Research Analyst SASGP (08243) Supervision Leslie Isao Kojima Sr. Financial Management Specialist SARFM (07079) Susan R. Razzaz Sr. Country Economist SASEP (07032) Miriam Witana Procurement Specialist EASR1 (08861) Monali Chowdhurie-Aziz Sr. Public Sector Spec. WBIOG (08392) Jyoti Sriram Senior Program Assistant SASDO (08174) Shashank Ojha Senior e-Government Specialist TWICT (01871) 8174 Haider Raza Senior Procurement Specialist SARPS (07080) Vidya Kamath Program Assistant SASGP (08243) Savita Dhingra Senior Program Assistant SASGP (08243) G. W. Anjali U. Perera Vitharanage Procurement Analyst SARPS (07080) Supul Chamikara Wijesinghe Financial Management Specialist SARFM (07079) Kaushik Sarkar Consultant SASGP (08243) Farah Zahir Senior Economist SASGP (08243) Zeenath Marikar Program Assistant SASEP (07032) Vinod Beri Senior Executive Assistant WBIIN (08294) Pinaki Joddar Research Analyst SASGP (08243) 36 (b) Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of Staff Weeks USD (incl. travel and consultant costs) Lending FY06 16.12 54810.7 FY07 32.5 100245 FY08 30.32 86451.65 Total Supervision FY09 22.41 77707.73 FY10 35.95 74766.8 FY11 31.43 64474.57 FY12 32.98 72384.95 FY13 24.62 51358.42 FY14 19.43 43154.31 Total 245.76 625354.1 37 Annex 7. Calculations on the Overall Outcome Rating Some of the calculations for determining the outcome rating are presented below, as per the IEG guidelines to be candid and transparent about the methodology employed and evidence reviewed. Some of these calculations gave the ICRR team a deeper understanding of the nature of the project results, however the hypothetical scenarios are not used to determine the ICRR rating, which is outlined and justified in the ICRR text. Table 10 illustrates the ICRR team workings in order to arrive at the original PDO rating. In following the IEG methodology, it was noted that the ICRR had split ratings (Row 2 entitled ‘PSCB ICRR Ratings’ shows the ratings applied in the ICRR). The ICRR based its final judgment on a rounding down of the split ratings (Row 3), given that no compelling reasons were identified to round upwards. Moreover, it was found that if the efficacy rating of the original PDO was rounded upwards to ‘substantial’, the overall outcome rating for the original PDO would be moderately satisfactory. Table 11 then shows what impact this would have had on the overall outcome rating. However, the team judged the original PDO rating to remain at MU, as explained in the ICRR. Table 10: PSCB Original PDO ICRR Split Ratings Relevance Efficacy Efficiency Objective Design/ Component Component Componen Componen Overal Implement. 1 PDO 2 PDO t1 t2 l Rating Individual ICRR High Substantial Modest Substantial Modest Substantial Ratings Aggregate Ratings (ratings Substantial Modest Modest MU rounded down as in ICRR) Aggregate Ratings (only Substantial Substantial Modest MS efficacy rating rounded up) Table 11: Project’s Overall Outcome Rating (August 2012 Restructuring) if Efficacy Rounded Up in the Original PDO Original PDO Revised Rating PDO Overall Outcome Rating 4 (MS) 5 (S) % Disbursed 97% 3% Weighted value (by % 3.88 0.15 4.03 = 4 disbursed) (MS) In order to assess the impact of the late restructuring on the overall outcome rating, the ICRR team ran another calculation. Table 12 shows that the rating would be moderately satisfactory if the PDO had been restructured when Component 1 closed in December 2011 (when 68% of the funds were disbursed, as opposed to 97% in August 2012). The late restructuring is the result of late processing on the part of the Bank. Table 13 also shows that, if the disbursement ratios were calculated based on actual expenditures (70% original PDO/30% revised PDO) rather than on advances to the client account (97/3 split), the rating would be moderately satisfactory. These are hypotheticals but give a fuller overview of the project results. 38 Table 12: Project’s Overall Outcome Rating (if restructured in December 2011) Original PDO Revised Rating PDO Overall Outcome Rating 3 (MU) 5 (S) % Disbursed 68% 32% Weighted value (by % 2.04 1.60 3.64 = 4 disbursed) (MS) Table 13 : Project’s Overall Outcome Rating (if based on actual expenditure at August 2012 restructuring) Original PDO Revised Rating PDO Overall Outcome Rating 3 (MU) 5 (S) % Disbursed 70% 30% Weighted value (by % 2.10 1.50 3.6 = 4 disbursed) (MS) 39 Annex 9. Beneficiary Survey Results Not conducted. 40 Annex 10. Stakeholder Workshop Report and Results No ICRR stakeholder workshop was held. 41 Annex 11. Borrower Comments on Draft ICRR 42 43 44 45 Annex 12. Comments of Co-financiers and Other Partners/Stakeholders Not applicable. 46 Annex 13. List of Supporting Documents and Information List of documents / reports referred to in the ICRR  SL PSCBP - Project Appraisal Document May 6, 2008  SL PSCBP - Aide Memoires of Implementation Support Missions – Jun 2009, Aug 2010, Apr 2011, Aug 2012, May 2013, Dec 2013  SL PSCBP - Implementation Status and Results Reports Nos. 1 to 11  SL PSCBP - Financing Agreement Jul 21, 2008 and Amendment Letters Oct 27, 2010, Aug 7, 2012  SL PSCBP - Restructuring Papers of Oct 18, 2010, Dec 13, 2011, Dec 17, 2012  Bank Mid-Term Review, February 2011  Auditor General’s Department of Sri Lanka – Project Status Report as of Dec 17, 2013  Auditor General’s Department of Sri Lanka, Annual Report - 2011 (tabled in Parliament in October 2012); 2010 (tabled in Parliament in October 2011); and 2010 (tabled in Parliament in XXX)  The World Bank, Sri Lanka Country Financial Accountability (CFAA), June 30, 2003  Sri Lanka Statistical Master Plan, version of July 2006  DCS Progress Report – January – June 2010  DCS Progress Report – January – March 2011  DCS Progress Report – April – June 2011  Minutes of NDC meeting - April 7, 2011  DCS Progress Report – September 21, 2011  DCS Project Completion Report, 2013  Updated Annex F from the DCS Project Completion Report, April 2014 List of individuals consulted for this ICRR Government of Sri Lanka Dr. A.J. Satharasinghe Additional Director General - DCS Mr. Bandulasena Director, ICT division - DCS Mrs. I.A.M Fernando Deputy Director, ICT division - DCS Dr. Swarnalatha Ukwatte Senior lecturer, Demography Department, University of Colombo Mr. Swarnajothi Former AG, AGD Mr. Chulantha Wikramaratne Deputy AGD, AGD L.K.I. Samantha Superintendent of Audit, AGD D. E.W. Gunasekara Chairman of COPE World Bank Francoise Clottes Country Director, Sri Lanka and the Maldives Jiwanka Wickremasinghe Senior Financial Management Specialist, SASFM Supul Wijesinghe Financial Management Specialist, SASFM Haider Raza Senior Procurement Specialist Doina Petrescu Senior Operations Officer, Sri Lanka and the Maldives Farah Zahir Senior Economist, SASGP Charles Undeland Senior Governance Specialist, SASGP Tatiana Nenova Adviser, BPSVP Leslie Isao Kojima Senior Financial Management Specialist, SARFM Development Partners and others A.L.M. Foumi Project engineer, Engineering consultants (pvt) LTD Mr. Chulantha DCS Building Contractor and Design Contractor 47 List of individuals on the ICRR Team Puneet Kapoor Financial and Audit Specialist Ingrid B. Ivins Statistician Hafiz Zainudeen Governance Specialist Savita Dhingra Senior Program Assistant 48 MAP 49 ICRR Supporting Information from Interviews and other Documentation 1 The RF from the PAD was not appended to the legal agreement. Hence, these indicators are taken from the PAD. Note that, there were some slight inconsistencies in the wording of the key indicators in the PAD. This table is based on the indicators in Table A 3.2 of the PAD. These indicators were used throughout the project albeit with some minor inconsistencies. 2 IDA would finance all component costs except the cost of land for the DCS building (US$l.9 million) to be provided by GOSL. 3 IDA would finance all component costs except the cost of land for the AGD building (US$1.4 million) to be provided by GOSL. 4 The Ratnapura building was first mentioned in AGD Annual Report of 2011 (Oct 2012) and then in the December 2012 restructuring paper. It was not mentioned in the amendment of the Financial Agreement (August 2012). 5 Implementation Completion and Results Report of the ERTA dated February 1, 2008 (Report No. ICR0000648). 6 These studies included (a) The World Bank, Sri Lanka Country Financial Accountability (CFAA), June 30, 2003; (b) The ROSC, May 21, 2004; (c) Peer Review by the National Court of Auditors, Kingdom of Netherlands; (d) Report on the Self Assessment of the Capacity of the Auditor General’s Department. 7 The IDP had a four point action plan agenda - (a) introducing international auditing standards; (d) having a professional qualified staff; (c) using up to date audit methodologies; and (d) improving external communications and external relations. (CFAA page 5). 8 The SMP was funded by a multi-donor Trust Fund for Statistical Capacity Building (TFSCB), which is administered by the Bank. TF054897, which closed in February 2007. 9 The SMP noted that a new, centralized building was essential for DCS to modernize. The SMP was funded by a multi- donor Trust Fund for Statistical Capacity Building (TFSCB), which is administered by the Bank. TF054897, which closed in February 2007. This TFSCB also included 4 separate components that targeted the establishment of a Statistical Research and Training Institute, workshops and the provision of technical expertise via consultants. 10 See Mid-Term Review (2010) and DCS Project Completion Report (2014). 11 The superstructure construction award was envisaged for July 2011, even if it was expected that DCS would not be able to occupy the building until mid-2013 at the earliest (Mid-term Review, 2010). The Bank team had concerns with this bidding process. The advertisement for Invitation for Bids (IFB) for construction of the building was published in leading newspapers of Sri Lanka on October 13, 2010 and subsequently 18 bidders submitted their bids to DCS. Three subsequent addendums were issued during November, 2010. While addenda 1 was issued with prior approval of the Bank, addenda 2 and 3 were not sent to the Bank for ‘no objection’ prior to issue to the bidders and they were issued electronically where some of the work items were removed from the Bill of Quantity (BoQ). The bid evaluation report for award of the contract for the construction of the office building was submitted on February 1, 2011. Due to non- compliance with the Bank guidelines and a lack of clarity whether the works items which were removed from the BoQ were evenly applied to all the Bids, clearance could not be issued for the award of the contract (Bank Various Email Correspondence, 2011). After discussions between the DCS management and the Bank team, the following options acceptable to the Bank were provided to DCS: (i) Re-inviting bids only from the 13 bidders who responded to the early invitation; (ii) Limit the bidding period to 14 days from the date of invitation; and, (iii) as provided in ITB 19.1, obtain a bid security declaration from the bidders instead of a bid security (MTR). 12 The USD $ 7.33 mn cancelled comprised of: (i) goods, consultancy ($ 4.55 mn); and, (ii) civil works ($ 2.78mn). 13 CPS, 2012 – 2016, Annex 13. 14 As indicated in draft SMP. Criteria for these standards refer to quality of data (using latest methodology) and timeliness of data, among other things. The GDDS adherence is for countries which want advice/TA to move to the more advanced SDDS adherence. 15 On this target, there were some discrepancies between results documented in the ISRs, MTR and client reporting (and current website). The ICRR team has reached a judgment based on a triangulation and cross-checking of all these different data sources. 16 As per last ISR (No 11, December 2013). Note that DCS commented that all districts had adopted the new data collection methodology but it is not clear if this was by December 2011. However, the ICRR team was unable to get documentation/verification of this and the DCS Project Completion Report notes that the new data collection was partially rolled out. In any case, the IO target rating would remain as substantial either way, thus having no bearing on the rating. 17 Source: ISR 11 of December 2013. 18 The micro-data access procedure is still cumbersome and lengthy, at least for access to “Licensed Files”, and gives the DCS the authority to decide if a research report based on the micro-data released can be published. For “Public Use 50 Files” (PUF; highly anonymized files) potential users can have direct access if they agree to follow specific criteria. However the ICRR team only found 1 survey (out of 348) on the Lanka Datta website that was a PUF (the 2011 LFS). 19 The functions of the NDC were to “…identify data needs of users and data producing institutions” and to promote “coordinating mechanism and data sharing” for the whole National Statistical System, not just the DCS. The NDC was also to be a “forum for promoting user -producer interaction”. (Source: Project PPT, Email from Former TTL , 3/6/2014) 20 Results compiled from DCS Project Completion Report (2014), email exchange with DCS (2014) and ISRs. 21 Annex F, DCS Project Completion Report (2014). 22 For instance, it has, in places, been difficult to quantify: (i) number of NDC meetings via available meeting minutes and reports; (ii) the activities that contributed to the reduction in census processing time; (iii) the precise intentions of the ‘new’ HIES data collection scheme and the PSCB contribution thereto; and, (iv) the review of the data sharing policy and the extent to which feedback was considered. 23 Accountant General 2009 Annual Report to Parliament. 24 The Annual Reports of the AG were submitted to Parliament by October 31 (i. e. in time) from close of the FY for 2009-2012. 25 Indeed, by December 2011, 77% accounts were rendered by the SOEs to the AGD for audit (based on FY2011) which was short of the end target of 100% but more than 25 basis points over the baseline and there was an increasing trend between 2009 and 2011. 26 As noted in Sections 2 and 4, it is recognized that AGD progress was also delayed by loss of staff and the requirement to undertake translation of reports. These factors, however, should have been taken into account in the design of the project targets and do not influence the rating here. 27 At PDO restructuring, the original target was reduced to 40% and end target set at 50%. 28 TeamMate is an audit management software system that includes risk based auditing, resource scheduling, electronic work paper management, audit issue tracking, and time and expense reporting with the objective of reducing time in documentation and reviewing, and providing more time for value added services such as risk analysis and audit planning. CAAT or Computer based Audit Tools are computer programs and data that the auditor uses as part of the audit procedures to process data of audit significance, contained in the entity’s computerized inform ation systems, with the objective of enhancing the effectiveness and efficiency of audit. 29 In its July 2013 report, the CoPE lauded the performance audits conducted by the AGD. 30 The AGD in its completion report assessed that ‘the new building has no doub t given worker friendly environment for our audit officers which will improve their efficiency’. 31 However, not all of the reports are available on the website (for example, Annual Reports for 2009 and 2012) 32 AGD, Project Completion Report, February 28, 2014. In addition, audit working papers are now maintained electronically, which has upgraded audit quality. The content and analysis of the Annual Reports also showed improvement over the years 2009 to 2012. 33 Note, also, that economic analysis of rates of return is not, at the time of writing, available. 34 One of the major recommendations made in the CFAA was to enact audit legislation to: specify the duties, powers and responsibilities of the AG; define more clearly the scope of work; and, provide for greater financial and administrative independence. The Act would also have provided unrestricted access to the AG to information, property and personnel associated with the management of significant public resources. The Act also proposed to bring the audit of limited liability companies within the ambit of the AGD, thus “widening the public audit coverage”. 35 As noted above, the institutional analysis of AGD (2003 CFAA) and DCS (2007 SMP) could have been explicitly updated and integrated into project design at the time of appraisal. 36 See, for instance, ISR 1 and 2 (of December 2008 and June 2009 respectively) and Sector Management Comments. One ISR also notes that: “Although the Financing Agreement of the project laid down implementation arrangeme nts and reporting requirements no record existed of whether both components have been adhering to these arrangements prior to ISR 4 of September 2010”. ISR 4 also notes that the slow progress on the project was linked to the inability of the Bank to understand and respond to the client needs on a timely basis. 37 ISR 4 of September 2010 38 ISR 11 of December 2013, page 2 paragraph 4 39 The letter from GoSL to not extend component 1 was received in December 2011. Given the need to cancel the IDA money and reallocate it to another area in the portfolio, a Level 2 restructuring was processed first. The Level 1 restructuring was initiated later. 40 Documented in ISR 3 for December 2009 and ISR 4 of September 2010 41 For example, the current CPS (p. 14) notes that the country has a high percentage of projects at risk (21% as of 2012), which is linked to the nature and complexity of implementation arrangements, weak implementation capacities of 51 implementing agencies and country financial management systems. The CPS also identifies scope for improvement in procurement (p. 16, 29) and that the relationship between the Government and World Bank has strengthened (p. 1). 41 However, as per IEG guidelines, the difficulties of the context are not for consideration in an ICRR but should be taken into account at design stage. 42 Four other Statcap operations which have closed and have completed ICRs were reviewed (Burkina Faso, Kenya, Tajikistan, and Ukraine). One key element which was consistent among the projects was the time period needed to complete the activities – a minimum of 5 years, regardless of whether the project components included a new building. This element supports the implementation period of 5 years, which is also found in most National Statistical Development Strategies (also referred to as “Statistical Master Plans”). 43 The DCS has therefore commented to the ICRR team that the job package for a procurement specialist could be improved to increase the prospects of retention. 44 Based on the AGD’s assessment in their Project Completion Report, February 28, 2014. This also proved more productive as compared to class-room based lectures as the officers gained practical experience in a real life situation. 45 This has been found, internationally, to be important especially as statistics offices in a number of countries may be comparatively weaker agencies. For example, the Statistical Master Plan in Lebanon is managed via the Economic department of the Prime Minister’s office, and therefore other partners in the Lebanese government are very responsive to requests, despite the fact that the statistics office itself, a semi-autonomous agency, is viewed as relatively weak. 46 For instance, one could develop a PFM reform strategy anchored around the findings and recommendations available from the ongoing Public Expenditure and Financial Accountability (PEFA) Assessment and ROSC in Sri Lanka. 47 The achievement of the PDO has been collectively assessed for the three attributes of ‘efficiency’, ‘effectiveness’ and ‘productivity’ as: (i) project activities transcended more than one of these three attributes; and, (ii) there is not any strong data that treats each area in isolation. 48 For example, in Table A3.1 (PAD), one indicator is: ‘Data processing of next HIES survey incorporates to a reasonable extent the data entry methodology as tested by the HIES pilot’. Aside from the fact that ‘to a reasonable extent’ is not explicitly defined for measurement purposes, Table A3.2 of the PAD expresses the same indicator without the phrase ‘to a reasonable extent’. Subsequently, the term ‘to a reasonable extent’ is used in some ISRs but not others. 49 ISR 6, 7, 8 and 9 (of May 2011, February 2012, September 2012 and December 2012 respectively) refer to “census processing time”. 50 The PAD does note that the ‘HIES survey incorporates to a reasonable extent the data entry methodology as tested by the HIES pilot’. However, this is not defined or included in the formal RF target. It is deemed necessary to retain the point here for the ICRR. 51 A 2002 Cabinet decision mandated submission of SOE reports within 2 months from close of the FY i.e. by Feb. 28. According to the CFAA, the baseline was that less than 50% of the SOEs rendered timely accounts for audit (and this baseline is noted in the PAD); and, the project sought to achieve an end target of 100%. 52