48032 v1 IMPROVING DEVELOPMENT RESULTS THROUGH EXCELLENCE IN EVALUATION Review of IDA Internal Controls An Evaluation of Management’s Assessment and the IAD Review Volume I Main Text and Overall Evaluation 2009 The World Bank This paper is available upon request from IEG-World Bank. Washington, D.C. ©2009 The Independent Evaluation Group, The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org E-mail: feedback@worldbank.org All rights reserved This volume, except for the elements contributed by group and institutions outside the Independent Evaluation Group, is a product of the staff of the Independent Evaluation Group of the World Bank Group. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. This volume does not support any general inferences beyond the scope of this evaluation, including any references about the World Bank Group’s past, current, or prospective overall performance. The World Bank Group does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank Group concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The Independent Evaluation Group encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request to the Independent Evaluation Group. ISBN 978-60244-110-1 Independent Evaluation Group Knowledge Programs and Evaluation Capacity Development (IEGKE) E-mail: eline@worldbank.org Telephone: 202-458-4497 Facsimile: 202-522-3125 Printed on recycled paper Acronyms and Abbreviations AAA Analytical and advisory activities GAO U.S. Government Accountability ARPP Annual Review of Portfolio Office Performance IAD Internal Audit Department BP Bank Procedure ICFR Internal Controls over Financial BPM Business Process Module Reporting CAS Country Assistance Strategy IDA International Development CFAA Country Financial Accountability Association Assessment IEG Independent Evaluation Group CGAC Country Governance and Anti- IL Investment lending Corruption INT Department of Institutional Integrity CODE Committee on Development IRMF Integrated Risk Management Effectiveness Framework COSO Committee of Sponsoring ISR Implementation Status (and Results) Organizations (established by the Report Treadway Commission) IT Information technology CPAR Country Procurement Assessment KPI Key Performance Indicator Report LEG Legal Department CSR Controller, Strategy, and Resource N/A Not applicable Management OP Operational Policy DPL Development Policy Loan OPCS Operations Policy and Country DIR Detailed Implementation Review Services ECDM Enterprise Content and Document PR Procurement processes Management PRIMA Portfolio and Risk Management ELCQ Entity-Level Controls Questionnaire System EPR Evaluated pass rate RAPMAN Risk and Portfolio Management ESW Economic and sector work System F&C Fraud and corruption QAG Quality Assurance Group FM Financial management QSA Quality of Supervision Assessment FR Fiduciary Review SPC Strategy and Performance Contract GAC Governance and Anti-Corruption SPR Simple pass rate Council TTL Task Team Leader ii Contents: Volumes 1–5 Evaluation Managers  Vinod Thomas Director-General, Evaluation  Cheryl Gray Director, Independent Volume I: Main Text and Attachments Evaluation Group-World Bank  Nils Fostvedt Preface ....................................................................................................................... xi Task Manager About this Report.................................................................................................... xiii Key Technical Terms ............................................................................................... xv IEG Evaluation Summary ...................................................................................... xvii Chairmen’s Summary: Committee on Development Effectiveness (CODE) and the Audit Committee of the Board of Executive Directors ................................ xxiv Summary of Management’s Response ............................................................. xxviii Statement from the Advisory Panel ................................................................... xxxii 1. Origins of the Review, Status after Completion of Parts I and II ............. 1 Purpose of the Report .................................................................................... 1 Background and Recapitulation...................................................................... 1 Phasing of the Review ...................................................................................................... 2 Status of the Review at the End of Part I ......................................................... 3 Part II: Assessing Entity-Level Controls.......................................................... 4 The Definition of Entity-Level Controls ............................................................................. 4 The Internal Controls Framework under COSO ............................................................... 6 The Review Becomes Comprehensive ............................................................ 7 2. THE IEG EVALUATION ......................................................................... 9 IEG’s Approach and Tools .............................................................................. 9 Evaluating IDA’s Internal Controls Framework .............................................. 10 Approach and Method .................................................................................................... 10 Effectiveness of Entity-Level Controls: Calculated ELCQ Pass Rates........................... 10 The Composite Evaluation ............................................................................................. 13 Material Weaknesses and Significant Deficiencies ........................................ 14 Material Weakness ......................................................................................................... 14 Significant Deficiencies ................................................................................................... 19 Evaluation of Controls over Efficiency and Effectiveness .............................................. 20 Concluding Observations ............................................................................................... 22 v CONTENTS Future Evaluations of IDA Internal Controls .................................................. 23 3. Summary of Management’s Assessment and the IAD Review .............. 25 Management’s Assessment .......................................................................... 25 I: Issues Relating to the Current Policy and Procedural Framework for Investment Lending ...................................................................................... 26 II: Issues Relating to Fraud and Corruption ................................................... 29 III: Issues Relating to Procurement (PR) and Financial Management (FM) ..... 31 IV: Issues Relating to Risk Aggregation ........................................................ 33 V: Other Significant Deficiencies .................................................................. 34 Resolving Issues Outstanding at the End of Part I ......................................................... 35 Monitoring Arrangements ............................................................................................... 36 IEG Observations on the Overall Assessment Report ................................................... 36 The IAD Review and Opinion ........................................................................ 37 Overall Objective ............................................................................................................ 37 Scope, Approach, and Method ....................................................................................... 37 IAD’s Approach and Method........................................................................................... 38 Summary of IAD Results and Findings .......................................................................... 39 4. Summary of Key Findings, Lessons, and Recommendations of IEG ... 45 Overall Summary of Findings ....................................................................... 45 Lessons Arising from the Review ................................................................. 46 Recommendations ................................................................................................. 47 Annex: Management Response ...................................................................................... 51 Boxes Box 1. Phases, Content and Timing of the Review ........................................ 2 Box 2. IEG Recommendations at the Conclusion of Part I ............................ 4 Box 3. The COSO Framework ....................................................................... 6 Box 4. Standards and Evidence Which Led to a Finding of a Material Weakness .................................................................................................... 15 Figures Figure 1. Simple Pass Rates by COSO Component ................................... 11 Figure 2. IEG’s Evaluation of the Effectiveness of IDA’s Internal Controls Framework ................................................................................................... 13 Figure 3. Remedies to Combat a Material Weakness in Controls over Fraud and Corruption ............................................................................................. 19 vi CONTENTS Volume II: Completing Part II and Integrating Parts I and II Annexes and Statistical Appendix Section I: The Entity Levels Controls Review (Part II) Annex A: Analysis and Evaluation of Management’s Approach and Method in Part II Annex B: Analysis and Evaluation of ELCQ Results Section II: The Integrated Internal Controls Framework (Combining Parts I and II) Annex C: Integrating Parts I and II: Scope Limitations and Controls Deficiencies Annex D: Factors Combining to Form a Material Weakness Annex E: IEG’S Composite Rating of the Internal Controls Framework Annex F: Statistical Appendix Volume III: Attachments 1. The Management Assessment Report 2. The IAD Review and Opinion Volume IV: Report on the Completion of Part I – Incorporating Compliance Testing of Key Controls (Part IB) 1. Background and Status After Completion of Part IA 2. Management’s Assessment 3. The IAD Review 4. The IEG Evaluation 5. Conclusions and Recommendations vii CONTENTS Annex A. Recapitulation of Main Findings and Summary of Conclusions from Part IA Annex B. Summary of the IEG Analysis of Results from Management’s Compliance Testing of Key Controls in Part IB Annex C. Summary Account of the Disposition of All Reported Internal Control Issues Uncovered During Part I Annex D. A Description of the Quality Rating Process Used by IEG in Evaluating the Approach and Methods in Management’s Assessment and the IAD Review of Part I Annex E. Statistical Appendix Attachment 1. Management Report on its Review of Internal Controls Attachment 2. IAD’s Review of Management’s Assessment Attachment 3. Statement of the External Advisory Panel Volume V: Report on the Completion of Part IA – Process Mapping and Effectiveness of Control Design 1. Background and Description of Approaches 2. Management’s Assessment 3. The IAD Review and Report 4. Conclusions and Recommendations Annex A. The COSO Framework Annex B. Standards Agreed by Management, IAD and IEG to Be Used in Assessing Deficiencies, Significant Deficiencies, and Material Weaknesses Annex C. Illustration of Potential Internal Control Design Weaknesses Annex D. A Typical BPM: Descriptive Material Annex E. Does the Cluster of BPMs Represent the Universe of IDA Controls? Annex F. Method and Results in Applying the Business Process Template viii CONTENTS Annex G. Statistical Appendix Attachment 1. Management Report on Its Review of IDA Controls—Part IA Attachment 2. IAD Review of Management's Assessment ix CONTENTS x Preface Bank management, in its IDA14 Replenishment Report, committed to “carrying out an independent comprehensive assessment of IDA’s control framework, including internal controls over IDA operations and compliance with its charter and policies.” Each part of this review was to be done in three phases: the first phase would be a self- assessment by management, the second a review by the Internal Audit Department (IAD) and report on management’s self- assessment, and the third an Independent Evaluation Group (IEG) independent evaluation of both management and IAD work. Part IA of the review was completed in late 2006 (IEG report dated October 18, 2006) and Part IB was completed in mid-2007 (IEG report dated June 30, 2007). In the present report, IEG evaluates management’s assessment and IAD’s review, which were provided at the completion of Part II. The report incorporates the results of the Part II entity-level assessment with the earlier results from Parts IA and IB, and thus covers the full COSO (Committee of Sponsoring Organizations) framework. The basis for IEG’s current evaluation consisted of management’s report on its Part II assessment (attachment 1); all the underlying materials that management generated in its entity-level questionnaires, follow-up discussions, and other analysis; and the IAD report (attachment 2). This report was prepared by Ian Hume, under the task management of Nils Fostvedt and the overall guidance of Cheryl Gray and Vinod Thomas, with the assistance of a core consultant team that included Dexter Peach (strategic advisor, formerly Assistant Comptroller General for planning and reporting, U.S. Government Accountability Office), James Campbell and Rosemary Jellish (consultants, both former Assistant Directors of the Government Accountability Office). Jed Shilling, Hiran Herat, and Domenico Lombardi assisted the core team on selected topics. An international Advisory Panel reviewed the final drafts of IEG’s reports at the end of both Part I and Part II. The Panel members were all former Auditors General, Patrick Barrett (Australia), Vijay Shunglu (India), and Bjarne Mork-Eidem (Norway). The panel statement at the end of Part II is shown in volume I. xi CONTENTS xii About this Report The organization of this report has had to accommodate a number of requirements: To evaluate the considerable amount of work carried out under the current Part II by both management and IAD; to summarize and comment on the key findings and conclusions presented by both management and IAD in their work; and to present IEG’s integrated evaluation of IDA’s controls framework, reflecting both the completion of Part II (the entity levels control review) and the earlier findings from Parts IB and A (transactions level review) shown in volumes 4 and 5, respectively. This integrated evaluation thus draws on materials from three separate parties (management, IAD and IEG itself), and from a total of three reports. To this end, the review also involved sifting through a very substantial volume of material and background information, the use of standardized evaluative templates created for this work, and the generation of a very substantial statistical data set. In order to structure the presentation of all this material, the report has been presented in five volumes, as follows:  Volume I presents a synthesis of IEG’s overall evaluation of the integrated internal controls framework, drawing on both Parts I and II, together with a summary of the conclusions of the independent Advisory Panel to this evaluation (with their report attached in full in Volume I). It also contains the key findings and conclusions of both management’s Assessment and the IAD Opinion and Review; their full reports are also attached in Volume III. Finally, Volume I presents IEG’s overall conclusions and recommendations from this evaluation.  Volume II contains, in addition to a final Statistical Annex, five annexes that place on record the more detailed analyses that were conducted. They are essentially reference sources, containing data and analysis which provide the underpinning for the findings and conclusions reflected in Volume I. The annexes are divided into two sections:  Section I (Annexes A and B) deals with the review of the entity level controls (i.e., the subject of Part II of the overall review). Annex A addresses approach and method, while Annex B provides the detailed analysis of the results from xiii ABOUT THIS REPORT management’s questionnaire, the Entity Level Controls Questionnaire (ELCQ).  Section II (Annexes C, D and E) deals with the integration of findings from Part I and Part II, to form the basis for the overall evaluation of the integrated framework of controls that was discussed in Volume I. Annex C deals with content that was not completed during Part I (scope limitations and resolving identified deficiencies); Annex D provides the detailed basis for the judgment that a material weakness exists in controls over F&C; and Annex E gives the analytical basis for IEG’s composite evaluation of the integrated controls framework.  Volume III contains two attachments: Attachment I is management’s report on the overall exercise and on the current Part II, and Attachment II is IAD’s report on Part II.  Volume IV contains the earlier report on Part IB of the review: on the completion of Part I incorporating compliance testing of key controls.  Volume V contains the first report on Part IA of the review: on process mapping and effectiveness of control design. xiv Key Technical Terms Audit Standards Criteria established by recognized accounting and audit bodies (in this case COSO and the Accounting Standards 2 (AS2)) for conducting audits and reviews of internal controls that offer a basis for providing assurance that controls are well designed and working as intended, and for identifying deficiencies, significant deficiencies, and material weaknesses. Bottom-up Approach The approach adopted by management in its assessment did not begin with a top-down, entity-level review, but focused first on business processes at the transactions or operating level. Hence, it has been described as a bottom-up approach. Business Process Modules Management chose to conduct this review of internal controls by (BPMs) identifying the main business processes in which IDA is engaged on a daily basis in the course of its operations. There were 35 procedures in all, covering IDA allocation; the Country Assistance Strategy (CAS) process; the main lending products (Specific Investment Loans, or SILs, and Development Policy Loans, or DPLs); and the fiduciary, contractual, safeguards, and quality assurance processes that support lending. Each process was mapped and described as separate business process modules, each containing the key internal controls that are the subject of the review. Business Process Template A standardized assessment questionnaire and rating system used by IEG to provide quality ratings of management’s method and approach in identifying, describing, and mapping the business processes, and of its method in assessing the effectiveness of control design and of control operation. COSO Integrated Framework A framework of management principles (COSO components) in an organization that, when collectively operating as intended, will (“Internal Controls-Integrated provide reasonable assurance as to the attainment of three key Framework”) organizational goals (COSO objectives): reliable financial reporting, operational effectiveness and efficiency, and compliance with laws and regulations (in IDA’s case, with its charter and internal policies and procedures). The COSO components are: Control Environment, Risk Assessment, Control Activities, Monitoring and Learning, Information and Communications.1 Deficiencies, Significant Design flaws, omissions, or noncompliant operation of controls, Deficiencies, Material discovered in the course of a controls review, denoting an ascending Weaknesses order of seriousness. The precise criteria by which the three categories of materiality are distinguished are explained in Annex B 1. See World Bank Web site for COSO Framework. xv of the Part IA Report. However, in the case of operational as against financial reporting, there are no such clear yardsticks by which to measure the materiality of a given weakness or set of weaknesses. Some judgment is required. The criteria to be used as a guide in making the needed judgments are those outlined in Annex B of the Part IA report. Entity-Level Controls Entity level controls refer to those internal controls applicable to the entity as a whole (i.e.”high level” controls). As such, appropriate entity level controls established and supported by management are a critical ingredient in creating an effective control environment. Examples of entity level controls include creating effective systems and processes for performance management (performance measurement and results), human resource management (hiring, performance evaluation, and training), and ethics (code of conduct and ethics regulation). Examples also include the creation of control units with responsibilities that cut across the organization and exist for the purpose of monitoring the effective achievement of objectives and/or implementation of internal controls such as IEG, IAD, QAG, INT and others. Entity-Level Controls A questionnaire designed by management to be answered by Questionnaire (ELCQ) managers throughout the operating units in the Bank, with questions aimed at soliciting opinions from managers about the effectiveness of controls. Where questions received “yes” responses the presumption is that the control in question was seen to be working, and where “no” or qualified responses were given, there was presumed to be a weakness in the control. Entity-Level Template A standardized questionnaire and rating system used by IEG to evaluate and give quality ratings to both management’s approach and method in its assessment of the entity-level controls framework, and to evaluate the strengths and weaknesses of the framework, as viewed across the five COSO components. Evaluated Pass Rate (EPR) An ELCQ question about a given control could be answered “yes” by some managers while being answered “no” by some others. The EPR is the number of questions deemed to have been answered “yes” on balance, taking into account also the number, type and reasons for the “no” responses given for the same question, as a percentage of the total number of questions. Since IEG and management used different criteria for making these judgments, the EPRs calculated by each party were different. Evaluation Panels In applying its Business Process Template, IEG assembled panels of 3-4 people, including controls specialists, and with experts in the particular discipline covered by the given BPM. The panels arrived at consensus judgments on the ratings that should be applied to each section of the module, according to their evaluation of the materials presented by management. Exceptions Non-compliances deemed to be of a less serious or material nature than deficiencies. xvi Exceptions/Deficiency Rates The number of exceptions/deficiencies found during the Part IB testing of key controls, divided by the number of control steps in the sample. Internal Controls Controls, individually or collectively, are structured means within an organization to enable it to achieve its business objectives while addressing risk. Control instruments include the control framework (in IDA’s case, the COSO framework), organizational checks and balances, published policies, and required procedures, among others. Integrated Internal Controls The combined system of key controls contained in the transactions- Framework level business processes and the entity level controls that provide for governance of the organization as a whole Key Control A gateway and decision point, involving key units and IDA staff, in a given business process module, through which a business transaction being processed must pass. It is the effectiveness in design of these controls and the subsequent testing of the effectiveness of their operation that is at the center of this review. Non-compliances Controls or control steps found during testing to be not operating in conformity with the design of the control. The concept of non- compliance includes both exceptions and deficiencies. Process Map The flow chart that graphically depicts all steps in a business process module. Review The term used to refer to the entire process of this study. Management conducted an assessment, the Internal Audit Department (IAD) conducted a review and opinion, and IEG conducted an evaluation. When referring to all three processes as an entity, the term used is “review.” Risk Focal Points In the adaptation of the COSO framework by the Bank and IDA to meet their own needs, management has defined and added to the framework four key points of risk that face the mission of the Bank Group and are especially relevant to IDA. These are: Strategy Effectiveness, Operational Efficiency, Financial Soundness, and Stakeholder Support. Simple Pass Rates (SPR) The number of “yes” responses received for each category of responses in the Entity-Level Controls Questionnaire, divided by the total number of responses. The SPR therefore gives a direct measure of what the ELCQ results show regarding the effectiveness of controls; hence it directly reflects the perceptions of the managers responding to the questionnaire. xvii Walkthrough An interactive interview and review of process documentation conducted by management with relevant teams of IDA staff knowledgeable in a particular business process and its associated controls, with a view to verifying that controls are designed in the way described and operate as intended. xviii IEG Evaluation Summary IDA stakeholders want to be assured that IDA complies with its Articles and policies, and that the funds they provide for development purposes are used as intended and have measurable results. It is a key purpose of IDA’s control system to provide such assurance. Hence, the Board of Executive Directors requested a full assessment of the system by the Independent Evaluation Group (IEG), through an assessment by IDA management and a review by the Internal Audit Department. The assessment is the first of its kind not only for the Bank but also for all international financial organizations. In this concluding step in the exercise, IEG finds that, with some important qualifications, IDA’s internal controls framework operates to a high standard overall, giving reasonable assurance that the controls operate effectively. The weaknesses are concentrated mainly in the areas of fiduciary controls and the related lack of a specific focus on controls at the transactions level against fraud and corruption in operations supported by IDA. With regard to the management assessment, IEG finds its approach and method as transparent, well documented, and comprehensive. The analysis indicates several remedial actions. First, controls over possible fraud and corruption in IDA operations should be addressed on a broad front, starting with risk management processes and country assistance strategies, and including the development and deployment of specific additional instruments directed at fraud and corruption issues at the level of programs and projects. Second, the implementation of remedies for the other control deficiencies should be closely monitored. Management has recognized the need for such remedies, and many are contained in the Governance and Anti-corruption (GAC) program currently being implemented (including some still under preparation). These remedies appear to address the key issues but they are not yet sufficiently operative to be tested and, if effective, thereby lessen the materiality of the controls weaknesses identified. IEG thus believes it would be premature to conclude that F&C risks have been successfully resolved under the current IDA controls framework. Approach and Method During the IDA14 Replenishment process, in response to shareholder concerns, World Bank management committed to have carried out (by IEG) “an independent comprehensive assessment of IDA’s internal control framework, including internal controls over IDA operations and compliance with its charter and policies.” In the process agreed with the Board, management would assess the controls, the Internal xviii Audit Department (IAD) would then review the assessment, and IEG would conduct an independent evaluation of both the management and IAD reports. Management used the COSO (Committee of Sponsoring Organizations of the Treadway Commission) framework as the basis for its assessment. It divided its study into two parts, Part I dealing with compliance issues within business transactions, and Part II dealing with efficiency and effectiveness issues within IDA entity-level controls. Part IA of the review was completed in late 2006 and Part IB in mid 2007. This report presents IEG’s evaluation of the effectiveness of IDA’s integrated internal controls framework. The evaluation covers both methods and findings, taking Parts I and II together. IEG has evaluated management’s approach and method as transparent, well documented, and comprehensive, though it would have been preferable to have examined the entity-level controls before the transactions level controls—in other words, for Part II to have preceded Part I, because this would have enabled a more prioritized, risk-based focus to the transactions level assessment. Key Findings of Part II Evidence presented by management for both the entity- and transactions-level controls gives reasonable assurance—except for weaknesses identified in certain parts of the overall framework—that controls operate effectively. With these exceptions, the controls framework provides Senior Management and the Board with reasonable assurance that the three COSO objectives are being achieved: Reliable financial reporting, compliance with policies and procedures, and the efficiency and effectiveness of operations. Evidence of controls effectiveness at the entity level (based on questionnaire results) includes pass rates ranging from 92 percent to 95 percent, depending on the method. The earlier evidence at the transactions level includes a pass rate of 93 percent (document-based testing of key controls). Management, IAD, and IEG all found that, while the overall framework is robust, there are weaknesses that are concentrated in a few key areas. The three parties generally agree on the nature of the deficiencies uncovered, but there are somewhat different judgments as to their materiality: IEG found one material weakness and six significant deficiencies. Management found significant deficiencies in five areas but no material weakness. IAD found that a material weakness will arise if a combination of significant deficiencies in fiduciary controls, entity-level controls, controls over fraud and corruption, and information technology (IT) controls are not remedied in timely manner. Evaluating Controls Under COSO. IEG evaluated the overall effectiveness of the entity-level controls framework under COSO, and compared the relative strengths of controls within each of the five COSO components, using the audit standards agreed for the review. The overall rating is satisfactory with qualifications, and this rating was given equally to controls within all five components. xix Material Weakness. Evidence emerged during the review that suggested that there are significant risks of fraud and corruption (F&C) impinging on IDA’s lending operations, not fully matched by appropriate controls. There has been progress in building the Bank‘s global anti-F&C agenda. However, both the specific tools to address F&C issues at the transactions level in IL and heightened efforts to support the building of client country systems that can protect IDA funds from F&C in DPL/PRSC-type lending (i.e., budget support) have been put in place only recently, and their effectiveness cannot yet be tested. These weaknesses are reinforced by significant deficiencies found in other related controls: in risk management, project financial management, and procurement. Since the risk of fraud and corruption by local beneficiaries, contractors, and other stakeholders can result in diversion of funds that, in the worst case, can impair IDA’s mission, IEG considers this weakness to be a material weakness. EG stresses that this finding is based on the risk of F&C rather than any clear measure of the extent to which F&C may have actually occurred in operations supported by IDA financing. It should also be kept in mind that weak governance is a widespread problem and a fundamental dimension of the development challenge, and the risk of misuse of funds exists not only for IDA but also for its development partners. The challenge, which IDA is now addressing, is to bring it more into the open and match it with risk management controls. Significant Deficiencies. At the conclusion of this final part of the evaluation, IEG found six significant deficiencies: (i) a need to maintain the currency of the Bank’s Operational Policies and Bank Procedures (OP/BPs); (ii) a need for improved systems of document retention and accessibility; (iii) generic weaknesses in controls over financial management and procurement processes (Part I); (iv) a need for improved management oversight of project processing and supervision, coupled with improved staff incentive structures and performance accountability; (v) a need to improve risk management, including inserting specific F&C risk factors into the Risk Scan, and in integrating risk treatment from the entity level to the activity level; (vi) a need for greater IT security in some areas. Other Deficiencies. During the two parts of the review a total of over 160 deficiencies of various kinds were identified. These are numerous but relatively minor weaknesses, which neither individually nor collectively rise above the level of minor deficiencies. Most of these have now been remedied, or their remedies are in progress. IAD Review and Opinion. IAD noted the significant deficiencies and other issues uncovered by the assessment in Parts I and II, and based on its review, expressed the opinion that management’s assessment and qualified conclusions as to the effectiveness of IDA’s internal controls review were fairly stated. However, it pointed to the identified significant deficiencies relating to fiduciary controls, entity-level controls, IT controls, and fraud and corruption controls, which in combination could create vulnerabilities which, if not remediated in a timely manner, could lead to a material weakness. IEG is unclear about the meaning of this, since if any weakness or deficiency has been identified, it should be considered to exist until mitigating measures have been introduced and proven to be effective. xx Accomplishments of the review: This was the first review of this kind for any multilateral financial institution. It has thus broken new ground both in creating methodologies (controls mapping and testing, the ELCQ, the IEG templates) and in building strong factual knowledge about the Bank’s internal controls framework. The corpus of materials emanating from the review provides a solid basis for mounting similar reviews in the future and for other analytical exercises. The review has also led to an acceleration in the developing of new controls for good governance within its client countries, and specifically within IDA operations. Advisory Panel IEG was assisted by a senior international Advisory Panel. It concluded that the evaluation of IDA’s controls has been a comprehensive, timely and responsible initiative, and that the approaches and specific tools have been consistent with what the Panel would expect from an independent evaluation. The Panel agreed with the IEG finding of one material weakness and endorsed the reasoning underlying the finding, as it did for the six significant deficiencies. The Panel expressed the view that for an organization as significant and complex as the Bank, such findings would not be uncommon for a first review, and it concluded that the outcome of the overall Review reflects a high level of effectiveness compared to results in other organizations of similar size and complexity but with less international involvement. Recommendations Based on its evaluation, IEG makes the following recommendations to IDA management: (a) Address on a broad front the controls needed to ensure that F&C practices in IDA client countries and among participating stakeholders do not impinge on IDA’s mission. Actions could include:  Accelerate implementation of the ongoing Governance and Anti-Corruption (GAC) program, and devote additional attention and resources to building an organizational culture and incentive structure in which the risks of F&C are explicitly and cost-effectively addressed in the management of IDA’s operations. While Management has correctly observed that such awareness has been spreading, including through the follow-up to the Volcker report, the systematic integration of this awareness into daily operations still has some way to go and needs to be given sustained emphasis going forward.  Develop and deploy specific F&C related instruments into the Bank’s Risk Scan processes, CASs, lending and project designs, and ISRs. Remedies have already been initiated as part of the GAC initiative and the Volcker Report, and INT has recently become involved in helping to design toolkits to address F&C at various levels of the lending cycle, although it is too early to judge the impact of these initiatives. It is also important to link country-based risk assessments through the Risk Scan to specific tools to address lending risks in both IL and DPL/PRSC type lending. xxi  Continue the ongoing reforms of FM and PR processes (launched in response to the findings of this review) and link them closely to the F&C agenda. These are key elements in the Bank’s fiduciary and governance systems but evidence from the review suggests that new toolkits (such as those being developed under the “GAC in Projects” program) need to be deployed, made operative and later tested for effectiveness.  Intensify IDA support to strengthen clients’ fiduciary and governance systems, recognizing that this is a principal means to guard against F&C and to ensure the effective use of IDA resources (and the only means to do so in the case of budget support operations such as PRSCs). In the case of DPL/PRSC operations, special emphasis needs to be given to developing tools that could attach, for example, to the Letter of Development Policy and to CFAA requirements, to raise the attention to systemic F&C issues at the country level.  Make arrangements for testing the operating effectiveness of these and other new controls at some appropriate time in the future, since the material weakness and other identified deficiencies will be deemed to persist until this has been done. (b) Closely monitor the implementation of remedies for control deficiencies, including:  The measures currently in progress to update the OP/BPs. These also need to be extended to key areas (AAA, F&C) not yet covered or where new policies are being developed.  A mechanism to ensure the future currency of OP/BPs. There has been progress in bringing the body of OP/BPs into conformity with overall Bank and IDA policies and strategic goals, and IEG has therefore downgraded the weakness uncovered in this area during Part I from a potential material weakness to a significant deficiency.  Improved documentation retention and accessibility and a user-friendly documentation management system. In its Part IB report IEG had already downgraded the materiality of this issue from a potential material weakness to a significant deficiency. However, the needed IT systems are not yet in place and the Enterprise Content and Document Management (ECDM) system of which they will be a part should be developed as a matter of priority.  Mechanisms to correct and monitor the several IT systems deficiencies identified. These included password management, business continuity and change management, and need for tighter control over IT access privileges for staff who rotate into new positions.  Measures to address the about 100 identified other as yet unresolved deficiencies. Remedies for many of these are already in progress, but specific monitoring is needed given the wide front and many areas in which remedial actions are needed. xxii xxiii Chairmen’s Summary: Committee on Development Effectiveness (CODE) and the Audit Committee of the Board of Executive Directors Review of International Development Association (IDA) Internal Controls: An Evaluation of Management’s Assessment and the Internal Auditing Department (IAD) Review Report on the Completion of Part II and Draft Management Response 1. The Audit and Development Effectiveness (CODE) Committees met on January 14, 2009 to jointly consider an Independent Evaluation Group (IEG) paper, Review of International Development Association (IDA) Internal Controls: An Evaluation of Management’s Assessment and the Internal Auditing Department (IAD) Review: Report on the Completion of Part II and draft Management Response. 2. Background. In its IDA14 Replenishment Report, Management committed to an independent comprehensive assessment by IEG of IDA’s control framework, including the internal controls over IDA operations and compliance with its charter and policies. An earlier Audit Committee and CODE discussion of the review had concluded that consistent with the undertaking made under IDA 14, it should be limited to IDA and not be extended to other Bank Group entities. The scope of work and the findings of earlier parts of the review were discussed during prior Committee meetings. The approach agreed with the Committees entailed a Management self-assessment, IAD review, and an independent evaluation of both by IEG. Part I (the transaction-level controls) identified specific issues with outdated OP/BPs, document retention and accessibility, and some fiduciary controls. In response, Management initiated corrective measures, enabling some of the issues to be remedied before the completion of Part II of the review. Members and Management also agreed that the findings could help accelerate efforts to modernize and simplify the control framework. 3. Overall conclusions. Members expressed satisfaction with the work of IEG, IAD, and Management on this first review of its kind among multilateral institutions, which provided a comprehensive assessment of IDA’s internal controls utilizing the COSO internal controls framework. IEG’s evaluation concluded that IDA’s internal controls framework operates to a high standard overall, with some important qualifications. IEG, IAD and Management emphasized their consensus on the issues xxiv identified and the direction of the corrective actions needed to address them. Members discussed findings and recommendations of the review and underscored the importance of a comprehensive plan of corrective actions, including: (i) timely implementation of management’s plan of corrective actions (described in Management’s Report and summarized in Management’s Response); (ii) strengthening specific fiduciary controls and enhancing controls for managing the risk of fraud and corruption in IDA-supported operations as a matter of priority; (iii) providing periodic updates to the Board on progress made in implementing the corrective action plan; and (iv) consistency with other institutional initiatives. The Main Issues Discussed Included 4. Findings of the Review. IEG, IAD and Management stated their broad agreement on the findings of the review and the issues identified, with some differences as to characterization of these issues. Members discussed the concerns identified by IEG and IAD related to controls for managing the risk of fraud and corruption in IDA- supported operations, which were assessed as a “material weakness” by IEG, and a “significant deficiency” by IAD and Management. IAD explained the reasons for its conclusion of significant deficiencies which if not addressed in a timely manner and monitored on an ongoing basis could in totality represent a material weakness. In discussing this difference, Management indicated that, in contrast to controls relating to financial reporting, in the context of operational controls there is not as clear a standard by which to measure the materiality of a given weakness, it being a matter of judgment. Management also expressed the view that IEG’s findings in this area reflect the situation that prevailed when IEG did its fact-finding a year ago and do not fully reflect recent corrective actions. IEG noted that many recent corrective actions are at an early stage, and results cannot yet be evaluated. IEG indicated that its assessment of material weakness relates to risks of fraud and corruption, rather than their extent, in operations supported by IDA. 5. Enhancing controls for managing the risk of fraud and corruption in IDA- supported operations. Members emphasized the importance of strong proactive actions by Management to enhance controls for managing the risk of fraud and corruption in IDA-supported operations. Management confirmed its commitment to mainstreaming governance and anticorruption efforts into its development work by tackling the anti- corruption agenda at all levels and implementing broad–based actions to strengthen the Bank’s controls over the fraud and corruption risks in operations, as outlined in the five- point plan of corrective actions included in its Management Response. Management also confirmed that its action plan and follow-up actions would be embedded in and fully consistent with the Governance and Anti-Corruption framework and the work of the Institutional Integrity Vice Presidency. 6. Remedying specific fiduciary controls. Members underscored the urgency of remedying the specific fiduciary controls that did not pass compliance testing during Part IB as a matter of priority. Management agreed to give priority to remedying the specific fiduciary controls in question and involving IAD in verifying progress made. 7. Follow-up actions. Members urged timely implementation and monitoring of the management action plan and sought clarification of the accountability for the xxv retesting of controls. Management confirmed its commitment to correct the issues identified through timely implementation of effective remedial actions outlined in its action plan. Management also emphasized its agreement with IEG and IAD on the importance of effective monitoring and reporting on the progress achieved, with specific responsibilities for monitoring and retesting of the control framework to be worked out going forward and with the first progress report to the Board prior to the IDA15 Mid- Term Review. Management noted that it is creating an Implementation Oversight Panel (IOP) to oversee implementation of the corrective actions. Abdulrahman Almofadhi, Chairman, Audit Committee Giovanni Majnoni, Chairman, CODE xxvi xxvii Summary of Management’s Response Background 1. Pursuant to an undertaking made under IDA 14, management, IAD and IEG carried out a comprehensive assessment of internal controls over IDA’s operations. The assessment has been conducted in the context of the internal control framework developed by the Commission of Sponsoring Organizations of the Treadway Commission (COSO), adapted to fit the unique nature and operations of IDA. It has been carried out in three tiers: Management self-assessment; Internal Auditing Department (IAD) review; and an independent evaluation of both by the Independent Evaluation Group (IEG). Management appreciates the close cooperation with IAD and IEG throughout this important exercise and welcomes the valuable IAD review and IEG evaluation of management’s assessment. The results of this work present an opportunity to accelerate measures to enhance IDA’s internal controls and maximize IDA’s efficiency and effectiveness. 2. This was the first of its kind review, not only for the Bank, but for all international organizations. As noted by IEG, it has broken new ground in creating new methodologies, allowed the Bank and IDA to take an important lead in assessment of its internal controls, and helped accelerate the development of new and improved controls to enhance Bank’s and IDA’s operations. Results 3. While as noted by IEG the results of this review demonstrate a high level of effectiveness compared to results in other organizations of similar size and complexity, IDA is always looking to improve its performance and management sees this as a real opportunity to maximize IDA’s efficiency and effectiveness generally and in combating fraud and corruption in particular. Management therefore has moved swiftly in formulating and beginning the implementation of a robust program of corrective measures to address issues identified and strengthen IDA’s controls, with most actions in process and many expected to be completed by June 2009. These actions will strengthen and refocus IDA’s internal controls to better address governance and anti- corruption issues, enhance risk identification and management at transaction and entity- levels, and improve effectiveness and efficiency of investment lending, the Bank’s primary lending instrument. 4. Management is pleased to note that at the end of this intensive effort, IEG, IAD and management agree on overall conclusions, findings and the appropriateness of the remedial actions proposed by management. As noted by IEG, with some noteworthy qualifications, IDA’s internal control framework operates to a high standard overall and provides Senior Management and the Board with reasonable assurance that the three COSO objectives are being achieved: Reliable financial reporting, compliance with policies and procedures, and the efficiency and effectiveness of operations. As IEG has pointed out, according to the testing results, evidence of controls effectiveness ranges xxviii from 92 percent to 95 percent at the entity level assessed during Part II, and 93 percent at the transaction-level (reflecting document-based testing of key controls) assessed during Part I. IAD also found reasonable assurance that IDA’s financial statements are being prepared reliably, and that IDA complies with the relevant provisions of its Articles of Agreement and operational policies and procedures, taking into account the exceptions identified as significant deficiencies by management. 5. IEG, IAD and management are also in broad agreement, with some differences as to degree, on the issues uncovered. As summarized in table C.2 included in Volume II of IEG’s final Report, IEG, IAD and management agree on the four significant deficiencies relating to: (i) policies and procedures governing investment lending (IL); (ii) risk management and accountability at project and entity levels; (iii) financial management and procurement oversight in projects; (iv) Information Technology (IT) and Analytic and Advisory Activities (AAA) controls. In addition, IEG, IAD and management identified an issue with design and integration of controls for management of fraud and corruption (F&C) risk in operations, with management and IAD classifying the issues identified as a “significant deficiency”, and IEG classifying them as a “material weakness.” In addressing these issues, management has taken due note of the recommendations set out in paragraph 4.10 of the IEG report, and has incorporated these recommendations in the management action program discussed in this Management Response. 6. The Bank is firmly committed to mainstreaming governance and anticorruption efforts into its development work. To this end, management is actively tackling the anti-corruption agenda at all levels, as evidenced by: (i) the swift and decisive actions it has taken over the past 6-12 months in response to the Volcker Panel Report, the India DIR and in implementing the GAC strategy; (ii) the remedies invoked to address F&C issues in operations; and (iii) the detailed and candid coverage of fraud and corruption risks in Bank documents (including CASs and PADs). Key among improvements already in place are the actions associated with the Bank’s implementation of 16 of 18 recommendations made by the Volcker Panel. These include: the creation of an Independent Advisory Board to protect the independence and strengthen the accountability of INT; enhancement of INTs’ advisory services and support to the regions in guarding against F&C risks through INT’s Preventive Services Unit; the increase in INT staffing; greater focus on review and management of high-risk cases; and stepping- up of staff training on managing F&C risks. Management acknowledges that more needs to be done to strengthen the Bank’s efforts in this area, and is committed to implementing broad–based actions to strengthen the Bank’s controls over the F&C risks in operations. Given the actions already in place, however (including the measures added over the past 6-12 months), management believes that the remaining issues are at a level of a “significant deficiency” rather than a “material weakness.” 7. In this regard, IEG’s finding of a “material weakness” needs to be viewed in the context of the following factors. First, as IEG pointed out in paragraph 2.19 of its final report, a finding of one “material weakness (together with some significant deficiencies) should overall be considered a quite respectable outcome for the first (and very detailed) exercise of its kind for IDA.” Second, as explained by IEG, “in the case of xxix operational as against financial reporting there are no… clear yardsticks by which to measure the materiality of a given weakness or set of weaknesses.” Third, as IEG explained, “weak governance and F&C risks are part of the challenge of development in many IDA countries and F&C risks affect the work of all donors, not just IDA.” Fourth, IEG’s findings in this area are a snapshot of a review period that ended a year ago, and therefore they do not capture/evaluate the improvements introduced in the F&C area in the past 6-12 months. Finally, it is important to note that IEG found management’s assessment to be “transparent, well documented and comprehensive”. (Emphasis added throughout this paragraph.) Follow-up Action 8. Management is in full agreement that timely and effective remedial action is needed to address all of the issues identified. It also agrees with IEG that controls over the risk of possible fraud and corruption in IDA-supported operations should be addressed on a “broad front” and that implementation of all the remedial actions should be closely monitored. To this end, management has adopted and begun implementation of a detailed and comprehensive 5 point action plan, with many actions to be completed by June 2009. These actions will:  Improve investment lending by rationalizing IL policies, processes and controls, strengthening supervision, and focusing resources on high risk projects;  Enhance the Bank’s risk management tools, incentives, and accountability to ensure better management and timely reporting of risks at project and entity level;  Integrate enhanced management of the F&C risk into operations through implementation of the GAC strategy at country and project levels, continued integration of INT work, enhanced training, and “smart project design”;  Tighten financial management and procurement controls to incorporate risk management and fraud and corruption issues and remedy as soon as possible the 10 (out of 50 tested) fiduciary controls that did not pass compliance testing during Part I;  Strengthen role of IT in risk management and improve processes and controls for AAA. 9. To ensure effective monitoring and reporting on the progress achieved, management is creating an Implementation Oversight Panel (IOP) to monitor, oversee and report to the President and the Board on implementation of these corrective actions. xxx xxxi Statement from the Advisory Panel 1. Background The External Advisory Panel (the Panel), involving the same membership, conducted a review in Washington from 5-9 March 2007 of the IEG Report on the Completion of Part 1A (dated October 16, 2006) and Preliminary Draft IEG Report on the Completion of Part 1B of the above Review. As well, the Panel provided comments on IEG’s thoughts on Part 2 of the Review. The Panel members subsequently received a copy of IEG’s Report on the Completion of Part 1: Incorporating Compliance Testing of Key Controls (Part 1B), including its Conclusions and Recommendations (Chapter 5). The Panel’s Statement was included as Attachment 3. The Panel was asked to reconvene in Washington from 21-25 April 2008 to provide input on key elements of analysis and conclusions that will form the basis for IEG’s final report to the Bank Board on the overall Review. As an indication to the Panel of matters to be covered, the IEG provided the following list of Principal Discussion Points for the technical interchange between the IEG Team and the Panel:  The Questionnaire Approach  Aggregating the Questionnaire Data  Use of Non-Questionnaire sources  Integrating Transactions and Entity Level Reviews  Extent of Revealed Weaknesses  Material Weakness  Future Reviews The Panel was also provided with a Draft of IEG’s Report on the “Review of IDA Internal Controls: An Evaluation of Management’s Assessment and the IAD Review.” As well, the Panel was given a copy of the Management Scoping Paper and was advised that IEG had not seen Management’s final report nor had the Team received anything from IAD. However, the Panel received a copy of IEG's Final Report (Volumes 1 and 2), Management’s Assessment Report, and the IAD Report on its Review and Opinion in mid October last. The Panel's final report also reflects its consideration of those reports. 2. Approach Taken by the Panel In preparing for the April discussions, the Panel familiarized themselves again with the Part 1 reports, the COSO (Committee of Sponsoring Organizations of the Treadway Commission) Integrated Framework, and the draft IEG final report. We structured our participation in discussions with the IEG Team, Senior Management, the Management xxxii Project Management teams (PMT), Auditor General and the Internal Audit Department (IAD) PMT, and the Chair of the Audit Committee to broadly cover:  The Approaches taken to Part II of the Review  Indications of likely Conclusions by the Participants on Parts I and II of the Review and their Integration  Identified issues from IEG Draft Report for discussion and any Observations  IEG Draft Report Conclusions and Recommendations  Panel Suggestions for Consideration  Panel's Overall Preliminary Observations and Conclusions The Panel’s final assessment was undertaken through e-mail contact with the IEG team and with each other based on documentation provided, as noted earlier. 3. Discussion of Issues First, we would like to reiterate an observation we made in our 2007 Statement. The strength of the approach in Part II of the Review is the top level strategic focus reflecting decisions made about the application of the overall integrated COSO risk management framework and associated entity-level controls within the governance arrangements that reflect both “tone at the top” and the authority and accountability that is assigned to the Review and any agreed outcomes. These issues are important to recognize in evaluating and concluding on the Integrated Controls Framework arising from both Parts I and II of the Review. In relation to controls, the Panel stresses the importance of a sound governing framework supported by a robust enterprise risk management (ERM) approach. This goes further than the earlier model framework adopted by IDA on which the entity level Questionnaire used in the assessment activities was based, but is consistent with it. As the COSO documentation indicates, ERM is geared to achieving an entity’s objectives in four categories:  Strategic: high-level goals, aligned with and supporting its mission;  Operations: Effective and efficient use of its resources;  Reporting: Reliability of reporting;  Compliance: Compliance with applicable laws and regulations; Internal control is therefore seen as an integral part of enterprise risk management. The process is a means to an end, not an end in itself. It is carried out by people. It can be expected to provide reasonable, not absolute, assurance to an entity’s management and Board. The focus of ERM, in particular, and on internal control, is an integration of inter- related components, setting of strategic and operational priorities, and achieving appropriate balance between competing imperatives, such as control and performance. These requirements are central to good governance and management of an entity with the associated accountability for performance (results) and an increased emphasis on implementation (that is, making sure it happens). As such, an enhanced COSO framework would help build on the learning associated with the IDA Internal Controls Review. xxxiii 3.1 PANEL ASSESSMENT OF THE REVIEW PROCESSES The Panel was again impressed by the professionalism, competence, understanding, and commitment of the IEG people involved in the evaluation. The approaches taken were consistent with what the Panel would expect from an independent evaluator. Not surprisingly, the evaluation approaches and the specific tools IEG used in its evaluation were based on the type of analysis that management conducted during its assessments. For Part II, the Entity Level Template used to rate management’s questionnaire-based approach, its assessment of implementation within IDA of each of the five COSO components1, and the assessment of controls relating to efficiency and effectiveness was assessed as reasonable. The Panel thought that reflecting IEG’s Template-based ratings in a statistical “diamond” format (Chart A.2) is a simple, useful way of illustrating the relative effectiveness of controls within each of the five COSO components. While questionnaire- based surveys of the kind used by Management have limitations that IEG recognizes, the Panel was satisfied that the latter were largely ameliorated by several non- questionnaire sources of information obtained by Management (including participation by IEG) and by the IEG itself from interviews with selected Bank units and other studies commissioned by IEG and by the Bank, for example the Volcker Report on the Department of Institutional Integrity (INT). The Panel accepted that the perceptions of managers, who completed Management’s questionnaire as to the effectiveness of controls, were adjusted by IEG judgments based on all sources of evidence (see Box A.3). Those judgments did not seem unreasonable to the Panel (see Chart A.3). The Panel cannot “second guess” the actual judgments made but is supportive of the approach to “standardize” the responses to reasonable confidence levels. On the issue of aggregating questionnaire data, the Panel recognizes the practicality of modifying some responses in the four-point scale used. Once judgments have been made about the allocation of responses to the scale, the Panel accepts their aggregation is a reasonable basis on which to make indicative assessments. This also recognizes the manner and level at which the Unit managers responded and the fact that the responses are their perceptions. Interpretation of the aggregate responses, including attaching some different weighting to the perceived importance/knowledge, is an added complication. The simple point is that if there were reservations about the capacity of participating Units to respond, they should not have been included in the exercise, or at least made aware they were not expected to answer questions unrelated to their responsibilities or experiences. The Panel also notes the relatively “conservative” nature of IEG’s use of the data and judgments made, as noted above. It supports the IEG comment that warns the results should not be “over-interpreted” and that they are “indicative.” The Panel considered that the IEG Evaluation Approach and methodology were well articulated and succinctly explained. This comment particularly applies to Annexes A 1. Control Environment, Risk Assessment, Control Activities, Monitoring and Information and Communication. xxxiv and B which provide the analytical basis for IEG's evaluation of IDA's entity-level controls. Reference has already been made to the COSO “Diamond” Figure 2, which will also be useful as a reference point for future reviews. We also considered Figure B3 in Annex B showing the distribution of “No” responses by Organizational Group has important implications for the Review. The Panel refers again to Figure B2 showing the ranking of COSO components by effectiveness of entity level controls, using a composite IEG rating index, as a very useful outcome of the Review. This Chart also reflects the significant interrelationship between Parts 1 and 2 of the Review and the value of presenting an integrated view of findings from both Parts. The Panel also considered the linkages between Activity/Unit and Entity level findings for each of the five COSO components were well illustrated and should assist both greater integration and understanding at all levels of the organization. 3.2 PANEL ASSESSMENT OF MAIN ISSUES Support for risk management and related controls has to be seen to come from the very top of any organization. This usually is part of the governance framework that provides strategic direction and decides on the appropriate balance between sometimes- conflicting objectives of control and performance (results). Importantly, it also conveys the requirement for accountability in these respects. The Panel did not see any evidence of a coordinated approach to identifying and prioritizing identified risks nor of an organization-wide view of the major risks that needed to be dealt with. It was suggested that this might have been due to the structured “silo-type” approach to dealing with identified risks. The presentation of an integrated approach as part of the final report on the Review may encourage consideration of an enterprise-wide risk management framework to facilitate governance and management. The Internal Audit Department indicated that it already used this framework (eight components)2 in planning for its audit program each year and for examination of Trust Funds. In the Panel’s experience, for an organization as significant and complex as the World Bank, it would be common to find one material weakness and six significant operational management deficiencies in Internal Controls. Admittedly, they would more likely be in the information and communications systems areas. As well, there is generally little criticism from stakeholders when such findings are accompanied by remedial action and explanation. Effective implementation as part of clear associated accountability is essential. It is also relevant that this is the first major review of controls in the history of the Bank. However, while the Panel was cognizant of, for example, the need to examine many of the deficiencies identified in Part I of the review in the context of Part II, it remains of the view that timely interim, or final action, was highly desirable to resolve as many of such deficiencies as possible by the time the Review is completed. The panel was quite impressed by IEG’s treatment of the material weakness in the internal controls to prevent fraud and corruption (F&C) in Annex D to its report. This is clearly a subject of much sensitivity to the Bank. Besides having an impact on the effectiveness of achieving its objectives, F&C issues involve a considerable reputation 2. Internal Environment, Objective Setting, Event identification, Risk Assessment, Risk Response, Control Activities, Information and Communication and Monitoring. xxxv risk, involving at least a potential loss of confidence by its various stakeholders, internal and external. While it is not possible, nor indeed cost effective, to provide absolute assurance in this area, there are considerable “public interest” concerns in this area that need to be kept in mind. In the public sector, these would be an integral part of any audit going well beyond financial statement issues. There would be no question about the responsibility to be open and transparent in reporting the weakness, its ramifications, and corrective action being taken. The Panel understands the need to put this finding in perspective without understating its seriousness but, at the same time, not risking unwarranted apprehension about the Bank’s operations and any over-reaction by the media or other parties that would be prejudicial to the interests of all concerned. It could be argued that, even if all the criteria agreed by Management, IAD, and IEG for the Review3 to determine material weakness were in evidence, as concluded by IEG, action taken since discovery would now have changed that situation. The Panel agrees that, unless there is clear evidence that the remedies set out in Figure D.1 of Annex D in IEG's final report have been put in place and are working effectively, it would not be prudent to draw such a conclusion. The Panel notes the IAD observation in its final report that “any conclusion on the adequacy and effectiveness of key fiduciary controls to ensure use of funds for intended purposes would be premature until remediation plans have been implemented and verified as effective.” Therefore, the Panel agrees with the IEG finding of one material weakness. The Panel also notes the IAD opinion that “unless recommended corrective actions are implemented in a timely manner and effectively monitored on an ongoing basis, the identified significant deficiencies relating to fiduciary controls, entity level controls, IT controls and fraud and corruption controls, in combination, could represent a material weakness.” The Panel also was satisfied that there is sufficient evidence to justify the grading of the six significant deficiencies identified by IEG. (See Annex E; setting out its composite evaluation of the Internal Controls Framework). The Panel draws particular attention to Annex C of the IEG's final report. . The Panel earlier expressed general reservations about operational management oversight and risk management implementation. However, the Panel draws attention to some particular concerns about the non- observance of well-designed controls, deficiencies in procurement mechanisms and the need for improved IT security. Such deficiencies can markedly affect basic business processes and on stakeholder confidence. The Panel supports the development of the “Deficiency Tracker " as it is clearly necessary to provide assurance to all concerned that the identified deficiencies are being dealt with effectively and in a timely manner. The Panel also supports the need for clear accountability, for example by sign-off, by the responsible managers that there has been effective implementation. Deficiencies identified may be indicative of broader issues to be addressed such as changing culture; document management; systems access; improving management 3. Published in Annex B to the Report on Part 1A. See also Annex 3 (page 35) reproduced in Box D1 (Annex A) in IEG’s final report. xxxvi oversight in project preparation; understanding risk management and the role of controls, including the relationship between transaction and entity type controls; effectively using sound controls that are already in place; developing user-friendly risk management tools; ensuring values and ethics are embedded in business processes and outputs and in performance management, including results being achieved; accountability for regular reviews; and facilitation of learning, personal awareness and support. As mentioned earlier, the last mentioned is a recognition that people effect Internal Control. Financial Management, Purchasing, Business Continuity Planning, and Disaster Recovery systems and related controls are of fundamental business importance and need ongoing attention. This, in turn, requires specific attention to people, and their performance, management. 4. Panel Suggestions for Consideration The foregoing paragraph started to indicate some suggestions for consideration as part of addressing identified deficiencies. As well, the Panel stressed earlier the importance of a sound governance framework supported by a robust enterprise risk management approach consistent with the latest COSO developments. There are now a number of Better Practice Guides available that give practical advice on the development and maintenance of different kinds of governance frameworks, on the basis that it is not a “one size fits all” consideration. The same comment applies to the latest developments in risk management and compliance. There would be benefit at least in reviewing the application of standards, particularly world standards, to the Bank’s/IDA’s environment such as the ISO31000 standard “Risk management - Principles and Guidelines on implementation.” The Panel notes that questionnaire results indicate that the Bank has well articulated risk assessment procedures. We agree that it is a critical link component in the COSO framework and further note, for example, that the Bank’s risk scan omitted the Fraud and Corruption risk. Therefore, it is important risk identification and assessment are done well. In the Panel’s experience, organizations often struggle with adequately assessing their risks at both the transaction and entity levels, particularly when there is lack of guidance and of suitable tools. There is also often a lack of appreciation that risk assessment is an ongoing task, not simply because the business environment is constantly changing. The concern is not just about process but in ensuring that those responsible have the capacity and support to properly assess risk and remain vigilant. The Panel was asked whether it has a view as to the frequency and possible form of future reviews. We agree that the entire Review is likely to have made a substantial contribution to the Bank’s knowledge of its internal controls system and the important role of risk management. As the IEG noted, an annual review with the scope of the present Review would seem to be impractical even with the learning that has occurred. While supportive of the IEG recommendation for follow-up, more selective reviews to be undertaken periodically, perhaps every two or three years, consideration could also be given to a planned set of rolling reviews, say, over a three year period, aimed at a similar coverage with an overall evaluation at the end of the period. We would envisage that any such reviews would be complemented by work undertaken by IAD in its xxxvii annual audit program. The IEG view that all such reviews should, as a matter of course, cover both IBRD and IDA would seem logical. Where the questionnaire approach is used in future reviews, the Panel agrees with IEG that it is essential for those involved to know whether the questionnaire is asking for perceptions about the design or operation of the controls when testing the controls in the future, both at the entity and the transactions levels. It is also important for the questionnaire approach to be supported by “walkthroughs” of the kind conducted in Part I of the Review and to involve IEG if it is to perform a similar role in the future. Finally, the Panel raises the question of adherence to timetables, perhaps as more of an issue of planning and commitment, including timely information and involvement of all participants. While there are many planning methodologies to choose from, the Panel commends the use of at least the principles of critical path analysis aimed at keeping the review on track and focused on the outcomes to be achieved while minimizing the frustration, cost, and impact on the capacities and effectiveness of the various participants. 5. Panel Observations on IEG’s Overall Conclusions/ Recommendations on Parts I and II of the Review The Panel draws attention to its earlier observations and suggestions relating to the IEG specific conclusions and recommendations. The Panel agrees with the IEG overall finding that its evaluation provides a solid and reliable basis on which to draw the conclusion that there is reasonable assurance that the compliance aspect of Internal Controls for the business processes are working as intended, with some notable exceptions, described as a material weakness and a number of significant deficiencies. The latter need to be put into perspective given the coverage and the first-time nature of the overall Review. The Panel has made some relatively minor observations on the IEG recommendations to Management that they regarded as being of the highest priority. However, it supports those recommendations as being balanced and appropriate, based on a thorough and comprehensive evaluation. The Panel also agrees, based on its experience, that the outcome of the overall Review reflects a high level of effectiveness compared to results in other organizations of similar size and complexity but with less international involvement. This perception would be reinforced by a clear and timely commitment including reporting back to stakeholders, to overcome the identified deficiencies and improve the effectiveness of internal controls within both the COSO and governance frameworks. 6. Panel’s Observations and Conclusions on the Overall Review In the Panel’s view, the Review of IDA Controls is a comprehensive, timely, and responsible initiative. It has been undertaken at a time when governance arrangements and public concern about accounting and accountability controls and corporate performance has been at its highest for many years. The result should be of great xxxviii satisfaction and source of confidence going forward for the Board, Management and other stakeholders. While the Panel had most exposure to the IEG team and its professional competence and commitment, we were also similarly impressed with other senior personnel in Management and IAD. In particular, we were very grateful for their openness and cooperation. These observations are made because they explain a lot about the quality of the overall review. As always is the case, the challenge is now the successful implementation of the Review’s Findings and Recommendations, as accepted by the Board. A similar professional commitment will be required. Patrick Barrett Bjarne Mork-Eidem Vijay Shunglu xxxix Evaluation Essentials  This report focuses on IDA’s internal controls under the COSO framework 1. Origins of the Review, Status  IEG evaluated after Completion of Parts I and II Management and IAD outputs from Part II (entity level), supplemented with findings from Part I Purpose of the Report (transactions level) 1.1 This report contains IEG’s evaluation of IDA’s internal con-  For the first time in the trols framework. The report reflects the results of IEG’s evaluation of review, final overall entity-level controls, conducted as Part II of the overall review, but it conclusions of the effectiveness of IDA also integrates all findings from Part I. Hence, it is a comprehensive controls are possible evaluation of the whole framework. The report is about IDA, but since IDA is managed by the Bank and governed by Bank controls, the report may refer in some places to the Bank in a generic sense. 1.2 Controls are tools to manage and mitigate risks. The focus of this report is therefore on internal controls, including the identifica- tion of weaknesses where they may exist. However, it is not within the scope of this report to identify the detailed remedial mechanisms, nor to address any underlying institutional issues. Background and Recapitulation 1.3 IEG has conducted its evaluation in accordance with the planned approach that was endorsed by Audit Committee and the Committee on Development Effectiveness in 2006. The overall purpose of IEG’s evaluation has been to offer an independent conclusion to the Board as to the effectiveness of IDA’s controls over compliance with its charter and relevant policies and procedures. This did not encompass an evaluation of the quality or development effectiveness of such policies. 1.4 The Board agreed that the review would involve three parties: management would undertake a self-assessment, the Internal Audit Department (IAD) would provide a review and opinion on manage- ment’s assessment, and IEG would undertake an independent evalua- tion of both management’s and IAD reports. 1.5 Management and IEG agreed that the review would be con- ducted using the framework of the Committee of Sponsoring Organi- zations of the Treadway Commission (COSO).1 The Bank and IDA adopted the COSO framework as guiding principles for corporate go- vernance in 1995. Since 1997, management has issued an annual re- 1 CHAPTER 1 ORIGINS OF THE REVIEW, STATUS AFTER COMPLETION OF PARTS I AND II port (shared with the Board since 2000) reflecting the extent to which the Bank has adapted to these operating principles. Until now the COSO framework has been applied and adapted in the Bank mainly for financial reporting, where the Bank has attained a standard that permitted external auditors to give an attestation not only on the ex- ternal financial reporting but also on the internal assessment that un- derlay the external financial reports. 1.6 Management discussed its most recent assessment of financial reporting controls with the Audit Committee of the Board on October 31, 2007. At that time, the external auditors also presented their Man- agement Letter on the controls, and their attestation over manage- ment’s assessment of the internal controls over external financial re- porting. This report incorporates the results of these assessments where appropriate, and IEG has verified that the external auditor pre- sented both the Management Letter and its attestation. PHASING OF THE REVIEW The Review was 1.7 Given the annual assessment of internal controls over financial conducted in reporting, management and IEG agreed that this review would not two parts: Part I need to include this area within its scope (apart from IEG’s verifi- dealt with cations, mentioned above). Management also decided and informed compliance; the Board that the review would be conducted in two parts: Part I Part II with would deal with controls over compliance with IDA’s charter and its in- efficiency and ternal policies and procedures, the transaction-level controls; Part II effectiveness would focus on controls over the efficiency and effectiveness of IDA oper- Box 1. Phases, Content and Timing of the Review Ongoing—Reliability of Financial Reporting: Man- Part II—Controls over Efficiency and Effective- agement and the Bank’s external auditors annual- ness of Operations (entity-level controls): Man- ly report on the effectiveness of internal controls agement assessed whether the existing internal over financial reporting. control framework, including corporate gover- nance and entity-level controls, provides reason- Part I—Compliance with IDA’s Articles and Pol- able assurance that IDA’s operations are carried icies (transactions-level controls): This part was out efficiently and effectively, focusing also on split into two steps: the processes and controls identified in Part I. A. Identified and mapped the Business The content of Part II is the substance of the Process Modules (BPMs) and key controls present report. in each process. Management reviewed the design effectiveness of the business processes and recommended remedial ac- tions (IEG report completed October 2006.) B. Management tested a sample of products and transactions for weaknesses in the op- erating effectiveness of the key controls and recommended remedies (IEG report completed June 2007.) 2 CHAPTER 1 ORIGINS OF THE REVIEW, STATUS AFTER COMPLETION OF PARTS I AND II ations, the entity-level controls under the COSO framework. 1.8 To better manage the reporting of its progress to the Board, In Part I management decided to divide Part I into two stages (Part IA and management Part IB), each dealing with distinct components of the assessment of found the internal controls. How the overall review has been divided, and compliance what topics are covered in each part, including Part II—the subject of pass rates at the the present report—is described in Box 1. 93rd percentile; IAD said that management Status of the Review at the End of Part I assessment was fairly stated 1.9 IEG completed two reports under Part I. The first (October 2006) subject to Part II covered Part IA.2 In May 2007, management delivered to IEG its Part IB confirmation; and overall Part I assessment, and IAD delivered its review. IEG com- IEG found two pleted its evaluation of both management’s assessment and the IAD re- potential view in its report on Part I sent to the Audit Committee in late June material 2007.3 Some key findings of the Part IB report included the following: weaknesses,  Despite starting with a transactions-level assessment, the BPM later (Part IB) approach was well focused, transparent, detailed and empiri- reduced to one. cally based, with measurable outcomes;  Controls testing in Part IB was found satisfactory, robust, and credible;  Controls at the transactions level complied with required poli- cies and procedures at a rate exceeding the 90th percentile;  Results provide a reasonable level of assurance that the com- pliance aspect of internal controls for BPMs is working as in- tended, with some exceptions;  The principal exceptions included a potential material weakness relating to OP/BPs which were not current with changes in the Bank, together with a significant deficiency in fiduciary controls. 1.10 Following the conclusion of its Part I evaluation IEG made a number of recommendations in its Part IB report, which served as a basis to ensure continuity and comprehensiveness between the two components of the review. These are summarized in Box 2. 1.11 Advisory Panel: As part of its evaluation, IEG was assisted by The Advisory an international Advisory Panel, which visited Washington in early Panel statement March 2007. The panel prepared a statement that was supportive of the supported IEG approach, method, and conclusions reached to that point in the review. approach and A copy of the statement is appended to the report of June 22, 2007. findings 3 CHAPTER 1 ORIGINS OF THE REVIEW, STATUS AFTER COMPLETION OF PARTS I AND II Box 2. IEG Recommendations at the Conclusion of Part I Completion of the Entity-Level Assessment (Part II): The challenge for Part II would be to redress the scope deficiencies in Part I and to draw linkages between actual findings from the transactions-level assessment and the COSO framework elements at the entity level. IEG recommended that the following topics be the subject of specific focus in Part II:  Controls and Project Processing: Design  Managing the Risk Framework and flaws were less important than non- Extending COSO: As IEG observance of controls in Part I, which recommended on Part IA, management suggests there is a need for greater should consider extending the COSO management oversight (i.e. improvement framework by adding a fourth objective in the control environment) (strategy—high-level goals) and three new components (objective setting, event  Potential Material Weaknesses: identification, and risk response). This o Documentation Retention and suggestion was also made by the Accessibility: This significant Advisory Panel. deficiency suggests the need to  Efficiency and Effectiveness: As the draw links with both the Control overall review would move from the Environment and the Information and transactions level to the entity level, and Communications component at the from compliance to effectiveness and entity level, where improved efficiency, a challenge in Part II would be information technology (IT) systems to build on the results from Part I, would be part of the solution. linking these to IDA’s Monitoring and o Dated OP/BPs: This potential Learning activities (including the Quality material weakness was an essential Assurance Group and IEG), in order to element of the Control Activities provide the element of effectiveness and component, which it would be well efficiency testing that was lacking in Part to accelerate and complete (as I, and was needed before conclusions much as possible) in time for the could be drawn regarding the overall completion of Part II. effectiveness of IDA’s internal controls. Part II: Assessing Entity-Level Controls THE DEFINITION OF ENTITY-LEVEL CONTROLS 1.12 Entity level controls refer to those internal controls applicable to the entity as a whole (i.e.”high level” controls). These controls in- clude to the structure, practices, and culture of the organization (so-called “high level” controls) which, supported by management, establish an ef- fective Control Environment. The entity-level controls refer to those ele- ments of the five COSO components of internal control that have an overarching or pervasive effect on an agency. 1.13 Examples of entity-level controls include:  systems and processes for performance management (perfor- mance measurement and results); 4 CHAPTER 1 ORIGINS OF THE REVIEW, STATUS AFTER COMPLETION OF PARTS I AND II  human resource management (hiring, job descriptions, per- formance evaluation, and training);  ethics (code of conduct and ethics regulation);  control units (including both regional and sector operating line The Entity-level units as well as cross-cutting central control and monitoring controls are units) with responsibilities that cut across the organization and structure (key exist for the purpose of monitoring the effective achievement control units), of objectives and/or implementation of internal controls (for practices, and IDA, such as IEG, IAD, the Quality Assurance Group, and De- culture, as partment of Institutional Integrity). exercised through the 1.14 In addressing its assessment of entity-level controls in Part II, COSO management adopted a two-pronged approach that focused on key framework control units as the basis, and used a questionnaire to corroborate re- sults. 1.15 Key Responding Units (Control Units): As it had done in Part I Management by mapping business processes to represent IDA operations, in Part II identified 31 management mapped the key units in the Bank most related to its op- responding erating processes and hence part of its entity level controls framework. units These were the units which were sent the ELCQ (para. 1.16) and re- sponded to it. In all there were 31 units, including Managing Directors and the Chief Financial Officer (CFO), the operating Regions and Net- works, nine central control units, and nine other service units. The units are listed in Box A.1 in Annex A, Volume II. Where relevant, the units were also linked to the respective components in the COSO framework. Management mapped the mandate and main responsibilities and role of each unit in the internal control framework; reviewed the primary products and outputs from each unit in the recent past against their re- spective mission goals; reviewed the processes in place for follow up on recommendations for action by management; and identified some gaps and overlaps in the overall framework. 1.16 The Entity-Level Questionnaire (ELCQ): Management de- ELCQ design signed a questionnaire (based on source information from the public was based on accounting profession and from the U.S Government Accountability GAO and other Office, or GAO, adapted to the IDA review), to be used to give addi- accounting tional evidence in its analysis of key control units. The ELCQ was sent industry to managers in all the units identified in the framework. The ques- principles; it tionnaire was similar to, and in some cases used the same questions as was partly the questionnaire used annually in the assessment of Internal Con- linked to the trols over Financial Reporting (ICFR), but the ELCQ focused mainly annual ICFR and on operational issues and controls over efficiency and effectiveness, was sent to all issues not routinely covered in the ICFR. 31 controls units 1.17 The ELCQ was organized around the five COSO components, and had a sixth section on anti-fraud programs and controls. Each COSO component contains a number of sub-topics which define key 5 CHAPTER 1 ORIGINS OF THE REVIEW, STATUS AFTER COMPLETION OF PARTS I AND II aspects of the controls framework which are intended to govern man- agement and staff practices in that particular area. For example, the Control Environment component is divided into management commit- ment to excellence (“Tone at the Top”), emphasis on ethical behavior, organizational structure, HR policies, and procedures. Using the GAO Manual as a guide management devised a set of questions for each sub- component designed to solicit responses from managers as to how they saw the controls operating in practice in each area. 1.18 IAD and IEG were shown a draft of the ELCQ and were given the chance to comment on it before it was distributed to managers. Management conducted detailed exchanges with a number of res- ponding units to verify and clarify the responses. THE INTERNAL CONTROLS FRAMEWORK UNDER COSO Questions in the 1.19 The approach taken in the review has treated the transactions- ELCQ were laid level controls and the entity-level controls separately, the former be- out by COSO ing addressed in Part I and the latter in Part II. In reality the internal components, controls framework is integrated and consists of both these levels of framed to controls, and it is further integrated with the overlay of management capture controls principles that constitutes the COSO framework. Management’s as- issues in each sessment was organized around the COSO framework, most particu- component larly in dealing with the entity-level controls. The COSO framework, as used by IDA, was described in the two previous IEG reports, but a summary is repeated for easy reference in Box 3. Box 3. The COSO Framework Elements in sound governance: For all public companies and other operating entities COSO postulates the key objectives of sound governance and the means to ensure they are attained. The COSO Objectives: Sound governance requires the attainment of:  Reliable financial To minimize risk of misstatements in external reporting reporting  Compliance with laws In IDA’s case, compliance with its charter and and regulations internal policies and procedures  Effectiveness and Being aware of the extent that a company or efficiency of operations organization is achieving objectives of efficient and effective operations The COSO Components Aspects of corporate management to ensure attainment of corporate objectives: Continued 6 CHAPTER 1 ORIGINS OF THE REVIEW, STATUS AFTER COMPLETION OF PARTS I AND II Box 3 (continued)  Control Environment Integrity and ethical values, Board of Directors, management philosophy and operating style, organizational structure, authority and responsibility, human resources  Risk Assessment Identifying operational, financial, and fraud risk, internal and external to the organization  Control Activities Matching identified risks with specific, cost- and effectiveness-weighted Control Activities; policies and procedures; secure information technology systems  Information and Internal: Capturing financial, operational, and Communication internal control information, using it at all levels in the organization, and distributing it in timely fashion to achieve reporting objectives External: Effectively communicating matters affecting financial and operational reporting objectives to outside parties  Monitoring and Learning Enabling management to determine whether the other components of internal control continue to function effectively over time through ongoing monitoring and separate evaluations The Review Becomes Comprehensive 1.20 The completion of Part II brings the overall review to a stage where, with both transactions- and entity-level controls assessed and tested, it is now possible to reach conclusions regarding the effectiveness of the overall controls framework under COSO. Volume II of this report gives an account of the elements that went into completing Parts I and II. The review has now become comprehensive in the following respects:  The entire framework of controls has now been assessed—at both the transactions and entity levels—which means that con- clusions can be drawn regarding how transactions controls and framework controls interact, and whether any conclusions drawn during Part I might need to be reconsidered in light of their assessment within the whole controls framework.  Effectiveness of controls under COSO requires that the as- sessment examine the degree of attainment of all three COSO objectives. IEG confirms that the assessment of controls over fi- nancial reporting was conducted by the external auditors. The 7 CHAPTER 1 ORIGINS OF THE REVIEW, STATUS AFTER COMPLETION OF PARTS I AND II nancial reporting was conducted by the external auditors. The present part of the review has addressed the remaining two COSO objectives. With the completion of Part II this means that all three objectives have been assessed. With the  Completion of the entity-level review in Part II focused on all completion of five COSO components, whereas Part I focused mainly on only Parts I and II, two (Risk Assessment and Control Activities). The COSO frame- the Review has work suggests that, for reasonable assurance that controls are now covered the operating effectively, all five components should be present in entire controls the assessment. framework; all  In Part I there were scope limitations (omission of some busi- COSO ness process modules; postponement of the assessment of in- objectives, all formation technology [IT] controls, fraud and corruption, and COSO the decentralized field offices), which implied that the controls components, framework being assessed was somewhat incomplete, thus and all scope preventing clear conclusions regarding overall effectiveness. limitations have These remaining components have now all been assessed. been addressed;  In the same vein, the discovery during Part I of a number of is- definitive sues, exceptions, and deficiencies that arose during the trans- conclusions are actions-level assessment meant that these needed to be further now possible assessed for the review to be complete and comprehensive. Some were addressed during Part I, several remained to be addressed during Part II. This has now been done (although the resolution of a number of these issues is still in progress). 1.21 In short, management’s decision to conduct the review in stages always implied that final conclusions would have to wait until the full framework and entity-level controls had also been assessed. That has now been accomplished. The balance of this report lays out how IEG has evaluated the controls framework as whole, also taking into account the reports from management and IAD. In doing so it gives a brief account of IEG’s method and approach in making its evaluation, which is then followed by a summary of the main findings and conclusions. 1. Committee of Sponsoring Organizations of the Treadway Commission, NOTES which published a report in 1992: Internal Control—Integrated Framework. 2. See “Review of IDA Internal Controls: An Evaluation of Management’s Assessment and the IAD Review: Report on the Completion of Part IA: Process Mapping and Effectiveness of Control Design” AC 2006-0099 October 18, 2006. 3. See “Review of IDA Internal Controls: An Evaluation of Management’s Assessment and the IAD Review: Report on the Completion of Part I: Incor- porating Compliance Testing of Key Controls (Part IB)”. AC2007-0044 June 22, 2007. 8 Evaluation Essentials  IEG uses multiple tools in its evaluation  Management’s and IAD’s 2. The IEG Evaluation approach and method were found to be sound  IEG rates effectiveness of controls framework as IEG’s Approach and Tools satisfactory with 2.1 IEG has performed the role of an independent evaluator, con- qualifications, reflecting sistent with the initial agreements reached with the Board and man- high pass rates for both agement. To this end, IEG has made its own analysis of the effective- entity-level and transactions-level ness of IDA’s internal controls, while also using management’s controls, but with one assessment reports and the IAD reviews as primary references. The material weakness and elements that went into the evaluation are the following: six significant deficiencies  Independent analysis of basic materials supplied by manage-  The review has been very ment and IAD, including the mapping and survey of key con- useful and should be trol units, the results from the ELCQ, certain background pa- repeated in the future pers commissioned by management (and others by IEG itself), external articles from the auditing profession, and Bank re- ports on related topics;  Extensive interviews with many of the key control units com- IEG’s evaluation prising the entity-level controls framework; is based on  Critical review of the final reports from management and IAD; independent  Application of IEG-designed templates to evaluate the design analysis, and effectiveness of controls at both transactions and entity le- interviews with vels, as the basis for quantified rating of controls quality; key units,  The expertise of IEG’s audit-experienced core consulting team, background which has kept IEG in close contact with developments in the reports, and use internal controls profession, regarding COSO itself, and with of its own relevant sources, such as the requirements of the Sarbanes- template for Oxley legislation, accounting standards (AS2 and 5) and other rating implementing standards that are recognized as benchmarks effectiveness of for such reviews. controls A summary of the approaches IEG used in its evaluation is in Annex E. 2.2 For its evaluation of internal controls over financial reporting, IEG relied upon reports issued by management and the external audi- tors.1 IEG examined the results of these reviews for FY 2006 and 2007, including management’s report and the external auditors’ Manage- ment Letter for both years, and conducted follow-up work on defi- ciencies noted and actions taken as a result of the reviews. 9 CHAPTER 2 THE IEG EVALUATION Evaluating IDA’s Internal Controls Framework APPROACH AND METHOD 2.3 Since management undertook the self-assessment, it also de- termined the approach and method for the review, in consultation with IAD and IEG. IEG has conducted a thorough evaluation of man- agement’s approach and method in both Parts I and II. These were al- so given detailed ratings using the IEG templates. With a few caveats mentioned below, IEG has found the approach for the review metho- dologically sound. IEG found 2.4 IEG had commented in its earlier reports on certain shortcom- management’s ings it observed in the approach adopted, mainly relating to scope li- approach and mitations and the fact that management started with the transactions method sound, rather than the entity-level assessment. Following completion of Part scope II this sequencing issue has become largely moot because the entity- limitations were level controls have now been assessed, and the earlier scope limita- adequately tions have also been satisfactorily addressed. addressed, and the ELCQ 2.5 Management’s use of the questionnaire approach in Part II was appeared to give appropriate; this is a common method for such reviews. However, a robust results, question with this method is whether it may introduce some biases into though some the results. There is some evidence that operating managers were less improvements likely to see fault with internal controls than managers in the central could be control and monitoring units (see paragraph 11 and Figures B.3 (a) and suggested (b) in Annex B), with 75 percent of all negative ELCQ responses (indi- cating controls were failing) coming from managers in central control units. Given their responsibilities and perspective on the institution, it should not be surprising that the central control units are more aware of possible control weaknesses. The remaining 25 percent of “no” res- ponses from operating managers is not a trivial number, however, and suggests that substantial candor was present in this segment of the ELCQ. 2.6 In Annex A (Box A.2) IEG raises some more technical observa- tions on the questionnaire approach, mainly to do with the focus of responses between individual units and the Bank as a whole. Howev- er, IEG concludes that none of these observations have caused any significant distortions in the overall results of the review. Also, the ELCQ was but one among a number of other sources of evidence con- sulted by IDA management, including detailed interviews. Its ap- proach was thus essentially two-pronged, with these other sources of evidence being used to corroborate the findings from the ELCQ. EFFECTIVENESS OF ENTITY-LEVEL CONTROLS: CALCULATED ELCQ PASS RATES 2.7 As one measure of the effectiveness of entity level controls IEG used the responses from the ELCQ to construct “pass rates” i.e. the fre- 10 CHAPTER 2 THE IEG EVALUATION quency with which managers gave positive responses to questions in A first analytical the ELCQ, implying that they thought the controls related to the ques- step was to use tions were operating effectively. IEG defined the aggregate number of ELCQ data to positive responses divided by the total number of individual responses construct a as a simple pass rate (SPR) which showed the average extent to which simple pass managers gave positive responses to questions about the effectiveness rate, showing of the entity level controls. The ELCQ response data showed that, de- that controls fined in this way, the simple pass rate was in the 95th percentile.2 This is appear to similar to the 93 percent pass rate for key transactions-level controls operate with more objectively observed (based on documented evidence, not ques- effectiveness at tionnaire responses) during Part I. Annex B gives a more detailed ac- the 95th count of these results. percentile; with some 2.8 The question was also addressed whether controls in some differences parts of the COSO framework might have been systematically weaker between COSO than in other parts. The simple pass rates (see Figure 1) showed that components there were indeed some differences between COSO components but these were not particularly significant. The pass rates for controls ef- fectiveness were above the 90th percentile in the case of each compo- nent, though responses relating to both the Control Environment and to Information and Communication appeared to receive somewhat fewer positive responses than the average. Figure 1. Simple Pass Rates by COSO Component 100 99.6 98 97 96 95.6 94 93.8 92 92 0 90 Control Risk assessment Control activities Monitoring Information and environment communications 2.9 However, the simple pass rate concept provides rather too simple a picture, because within each individual question there were both pos- itive and negative responses, and managers responded differently to given questions. Since the questions were designed to test the effec- 11 CHAPTER 2 THE IEG EVALUATION tiveness of the controls, a more accurate aggregate count or pass rate would be given by showing how many individual questions out of the total 157 could be deemed to have been responded to positively on bal- ance (i.e. constructing a net balance between positive and negative res- ponses to each question). This approach was defined by IEG as an eva- luated pass rate (EPR) because it requires a judgment for each question to balance out the positive and negative responses. 2.10 Judgments to determine the EPR were based both on the num- ber of individual responses of each type (positive and negative), and also on the origin and nature of the responses: i.e. which Bank units made negative responses; how close were they to the subject matter of the given control being addressed; and what evidence or reasoning did they offer to explain their responses. As is described in more detail in Annex B (see paras. 6-7) management and IEG used different criteria to make these judgments and arrived at slightly different results. Man- agement found an EPR of around 96% while IEG, using slightly stricter criteria, found an EPR of 92 percent. Both findings, therefore, were in the 90th percentile and suggested, again, that perceptions from the ELCQ results supported a broad conclusion that the controls frame- work operated effectively overall. 2.11 Another statistic from the ELCQ results supported the finding that managers saw controls as being generally effective: A majority of the individual questions received no negative responses at all. Out of a total of 157 questions only 70 (43%) had any negative responses and of these more than two-thirds had only one or two negative responses. IEG po- sited the standard that where three or more negative responses were given to a question it was a signal of possible deficiency in the control being addressed. Out of the total of 157 questions only 18 contained three or more negative responses, which shows how, by this measure, the controls weaknesses appeared to be concentrated in a relatively few areas. 2.12 The analysis was then conducted on these 18 questions, being the areas where most negative responses had been given in the ELCQ, while also consulting evidence from sources other than the ELCQ. From this process emerged the identification of major weaknesses and other deficiencies in the controls framework, ending finally in a compo- site evaluation of the overall framework. An account of the several building blocks that were used to compile the composite evaluation is given in Volume II in the following annexes: Annex A (Analysis and Evaluation of Management’s Approach and Method in Part II); Annex B (Analysis and Evaluation of ELCQ Results); Annex C (The Deficiency Tracker Consolidating Weaknesses Found in Parts I and II); Annex D (Factors Combining to Form a Material Weakness); Annex E (IEG’s Composite Rating of the Internal Controls Framework). 12 CHAPTER 2 THE IEG EVALUATION THE COMPOSITE EVALUATION 2.13 In making its overall evaluation IEG used these different ana- IEG rated lytical building blocks: it consulted all sources of evidence available; it effectiveness of closely reviewed management’s assessment and the IAD Review; and the controls it applied its own template rating system to give a composite rating of framework— the effectiveness of the overall controls framework. Taking all factors using all forms into account (pass rates, other evidence, identified weaknesses) and of evidence—as using the language of its template rating system (see Annex E in Vo- satisfactory with lume II), IEG rates IDA’s controls framework as satisfactory with quali- qualifications, fications, a finding which is consistent with management’s qualified reflecting assurance and also with IAD’s conclusion that management’s assess- identified ment is fairly stated, but qualified by a number of deficiencies de- weaknesses in scribed in its review. However, as described below, IEG concluded certain areas that there was one material weakness and six significant deficiencies. This was close but not identical to the findings of management and IAD, because of somewhat different judgments regarding the mate- riality of some of the findings. 2.14 IEG’s composite rating also included a rating of controls effec- tiveness within each COSO component, since it is not excluded that controls weaknesses may occur to different degrees in the five com- ponents. IEG chose to present the results of this composite, compo- nent-based evaluation in the form of a statistical “diamond” shown in Figure 2 below Figure 2. IEG’s Evaluation of the Effectiveness of IDA’s Internal Controls Framework Control envrionment 14 3 2 Information and 2 3 Risk assessment communications 4 1 Monitoring Control activities 2.15 The figure shows a graph with one arm for each COSO com- ponent. The effectiveness of controls within each component was 13 CHAPTER 2 THE IEG EVALUATION Controls rated on a scale of 1 to 4.3 Had all five COSO components been eva- weaknesses luated as fully satisfactory (rating 1), each arm of the polygon would were found to have been positioned at the outer perimeter of the diamond. As it similar degree in happens the evidence shows that the controls within each of the five all COSO components were equally rated (at “2”) which therefore equates to sa- components tisfactory with qualifications, which equates to the composite rating of the framework as a whole. Controls 2.16 While this depiction accurately expresses IEG’s overall finding, weaknesses as a statistical aggregate it also masks some important features of the were found in assessment. One is that, as shown in the analysis in Annexes A and B, only a few the more serious weaknesses in controls, even though cutting across areas, the COSO components, were concentrated in a relatively few areas in the effectiveness of controls framework. Generally, the framework operates effectively, ex- most controls cept for these weaknesses. In the same way, IEG’s ratings of 2 for the was rated fully controls within each component, while valid overall, hide the fact that satisfactory many individual control items were rated 1 or fully satisfactory. Out of a total of 74 individual items rated, 43 (58 percent) received a rating of 1. IEG finds one 2.17 The source of the qualifications are the one material weakness material and six significant deficiencies that were uncovered during the re- weakness and view. These findings result from IEG’s own analysis but also take ac- six significant count of the similar (though not identical) findings arrived at by both deficiencies management and IAD. The basis for the findings is contained in Vo- lume II (Annexes B, C, and D). The management and IAD findings are summarized in Chapter 3. What follows is a description of the materi- al weakness and significant deficiencies that IEG has identified. Material Weaknesses and Significant Deficiencies MATERIAL WEAKNESS A Material 2.18 Fraud and corruption (F&C) risks are widely assumed to exist weakness has in many countries, and this evaluation has shown that the Bank‘s tra- been found ditional internal controls to ensure that F&C does not impinge on IDA because risks of projects have not always been effective. Based on the evidence F&C are not yet brought to light in the present review, and the agreed criteria under- covered by lying the review (see Box 4 below) IEG concludes that the weakness in controls which the existing framework of controls addressing F&C issues are such as are fully to rise to the level of a material weakness. While commitment to inte- operative and grity has always been and remains a central feature in the Bank, there effective are also aspects of the culture that have resisted dealing openly with the potential for F&C at the local level in Bank and IDA operations. These facts have been well recognized by the Bank and more specific F&C related controls are now being introduced as part of manage- ment’s GAC Implementation Program and in other ways, e.g. in im- plementing the recommendations of the Volcker Report. However, 14 CHAPTER 2 THE IEG EVALUATION until these new measures are fully operative and effective it would be premature (from the view of both process and substance) to conclude that the F&C issues have been successfully resolved under the current IDA controls framework. Box 4. Standards and Evidence Which Led to a Finding of a Material Weakness At the start of this review management, IAD and IEG agreed on the standards that were to be fol- lowed in determining the materiality of any controls weaknesses that were uncovered during the review. These were presented in Annex B to IEG’s the report on Part IA. Identified deficiencies could be significant deficiencies or material weaknesses depending on whether one or all of the materiality criteria cited below are present. MATERIALITY SPECIFIC EVIDENCE FOUND CRITERIA  Evidence from INT DIRs and FRs in five countries show F&C indicators which suggest funds may have been diverted from the purposes intended; evidence of actual F&C has also  Impair the been found in some of these cases achievement of  INT DIRs/FRs show that F&C indicators appeared in all six IDA’s objectives studied countries, suggesting actual F&C may not be rare but may occur in many countries  IDA’s Articles require that IDA operations be governed by its  Violate require- policies and procedures ments of IDA’s  INT DIR evidence (India, Kenya) shows that F&C indicators charters or other have been found even when Bank procedures have been cor- contractual rectly followed agreements  In IL operations evidence from Part I showed significant defi-  Significantly wea- ciencies in IL fiduciary controls and absence of F&C con- ken safeguards trols against waste, loss,  In DPL/PRSC operations procurement controls do not apply, or unauthorized so F&C has to be addressed through IDA FM and Loan Ad- use of funds, prop- ministration controls and through country systems which are erty, or assets often weak or non-existent  Involve conflicts  Bank culture, management priorities, staff incentives and HR of interest, involve practices have not given priority to safeguarding against F&C systemic problems in Bank/IDA operations in country assis-  Evidence from Part I (fiduciary weaknesses) and Part II (enti- tance, partner- ty level incentives) shows a gap between progress in build- ships and project ing a global Bank agenda against F&C and establishing lending, or require tools to make the agenda operational; more is needed from the attention of Senior Management in linking the global perspective to daily Senior Manage- operations and to internal systems in client countries ment, the Board as well as the aware- ness of external stakeholders 15 CHAPTER 2 THE IEG EVALUATION F&C is part of 2.19 It is important to emphasize key aspects of the context for this development finding: First, the finding is based not on evidence of the occurrence context; the of actual F&C (though there is evidence that some F&C has actually material occurred) 4 but rather on the fact that there is evident risk of F&C oc- weakness is curring in a significant number of IDA countries. Second, weak go- based on risk of vernance and F&C risks are part of the challenge of development in F&C, not actual many IDA countries and F&C risks affect the work of all donors, not occurrence of just IDA. Third, this is the first evaluation of its kind for any IFI, F&C which places the Bank out front within the development community. Fourth, the finding of a single material weakness (together with some significant deficiencies) should overall be considered a quite respecta- ble outcome from the first (and very detailed) exercise of its kind for IDA.5 Addressing F&C 2.20 A full accounting of the factors that supports the IEG judg- requires country ment of a material weakness is presented in Annex D. As explained in systems to be the Annex , the evaluation revealed that substantial progress has been stronger made by the Bank over the past decade in building a global strategy to combat F&C, and a number of important steps have been taken to build entity level mechanisms to pursue an anti-corruption agenda in client countries. However, the specific tools needed to translate global goals into transactions level controls have been missing or have only recently been put in place (and so in many cases are not yet fully operative), while some are currently being developed. It should also be emphasized that addressing governance and F&C issues requires country systems to be improved as much as it needs improved internal controls within the Bank. This is so in investment lending, where the Bank is moving towards increased reliance on these local systems, but the Bank also provides general budget support through PRSCs (which are controlled entirely by local systems). Dealing with weak gover- nance, including F&C in operations supported by IDA, is a funda- mental dimension of the development challenge. 2.21 To elaborate further on the evidence summarized in Box 4, the key findings underlying the material weakness included the following:  Inadequate Tools to Address F&C: The Bank has until recently had few if any specific tools to supplement its existing controls systems to address F&C at all stages in the lending cycle. Specif- ically, tools have been lacking to diagnose F&C risks systemati- cally in the Country Assistance Strategy (CAS) process and to address it systematically in project design and appraisal docu- ments. The Implementation Status Report (ISR) lacks F&C indi- cators, and F&C issues are not treated systematically in Imple- mentation Completion Reports. The Bank’s generic fiduciary and other controls have historically been assumed to be ade- quate protection against F&C in projects, an assumption that 16 CHAPTER 2 THE IEG EVALUATION has been called into question by recent evidence. Furthermore, the question of how best to support clients in building country systems to address F&C – which is critical in ensuring appro- priate use of general budget support – has not been fully ad- dressed (although efforts have increased substantially in recent years).  Weaknesses in Generic Fiduciary Controls: Part I of this re- view showed that the noncompliance rates among some fidu- ciary processes were higher than the average. Management, IAD, and IEG agreed that this should itself be considered a significant deficiency (see Annex D, para. 19).  Absence of F&C in Risk Management: The Risk Scan does Several factors not include routine assessment of country fraud and corrup- combine to tion risk, even though it is widely known that IDA deals with influence the countries where those problems are endemic. material weakness; the  Control Environment Factors: The ELCQ results suggested remedies must that HR policies and incentives, job descriptions, and man- also be multi- agement behavior regarding the treatment of ethical issues in faceted IDA operations are insufficiently geared to emphasizing the need to explicitly address F&C issues in routine IDA opera- tional work. 2.22 Taken overall and measured against the specific criteria by which to judge materiality, as shown in Box 4, the evidence that there is significant risk of actual F&C occurring (whether or not it has ac- tually been widespread) suggests the need for offsetting controls. However, specific tools are lacking in the Bank’s armory of controls to address F&C in Bank/IDA strategy and risk management, as well as in its operations cycle (both DPL and IL lending). These weaknesses are reinforced by deficiencies in some of the key fiduciary elements of the existing controls system, and in cultural and incentive structures which all combine to convince IEG that a material weakness exists. 2.23 Identifying Remedies: Given the composite nature of the ma- Tackling F&C terial weakness IEG proposes that the remedies would also need to be requires a multifaceted. This fact has been very well recognized by management partnership (and by IAD) and is reflected in the wide range of GAC program between Bank components currently being implemented or prepared. In the sche- controls and matic shown in Figure 3 below, IEG displays the key elements it re- country systems gards as contributing to the weakness and shows the types of reme- dies that could be applied at the entity level and in the business processes across the project cycle. IEG highlights the following ele- ments: 17 CHAPTER 2 THE IEG EVALUATION  Remedy generic deficiencies in existing controls and install appropriate tools to combat F&C within Bank/IDA opera- tions. The objective should not be to attempt to eradicate all risk of F&C with stifling controls, but to have a risk-based, prioritized approach to explicitly address F&C issues in IDA operations, with the aim of understanding, signaling a focus on and reducing the risk of F&C. The costs/benefits and fea- sibility of such new controls need to be kept in mind, but what is being suggested here relates closely to the work already be- ing done to address identified weaknesses in controls to ad- dress F&C under the GAC initiative (and the response to the Volcker Report). IEG believes what it is suggesting in this re- port could probably be accomplished with only minor addi- tions to what is already underway. Also, the new risk man- agement framework adopted by COSO should provide additional guiding principles in managing F&C risks and se- lecting risk priorities to be pursued going forward (see also para 2.28, bullet five below).  Address country systems and harmonize with other donors, to be sure that those remedies internal to the Bank and IDA are matched by efforts to enhance the scope and capacity for country fiduciary and governance systems within the client countries themselves, working also with other donors and par- ticipating partners. This would be the appropriate approach to address F&C in budget support operations and to achieve longer-term development objectives more broadly. The reme- dies being proposed here should not be taken to suggest that IEG is advocating more “ring fencing” of IDA projects (apart from the already agreed need to strengthen fiduciary controls). On the contrary, as more emphasis and reliance is being placed on country systems (particularly for PRSC type lend- ing), the remedies should rather be seen as simply ensuring that country systems themselves also focus on F&C issues, in parallel with the Bank’s efforts in this area. 18 CHAPTER 2 THE IEG EVALUATION Figure 3. Remedies to Combat a Material Weakness in Controls over Fraud and Corruption Control Environment •Clear management signals •HR policies, staff incentives •Results-based management Country Strategy •CAS process to reflect F&C country risk •F&C safeguards in DPL/PRSC Risk Assessment Project Design •F&C in Risk Scan •F&C section in PAD from CAS country risk Project Supervision •Implementation Status Report (ISR): F&C toolkit •Adequate budgeting New Specific F&C Tools •INT assist design •Results monitoring •GAC program •OPCS VPUs Financial Management/Procurement Controls •Remedy deficiencies •Add F&C tools 2.24 These IEG findings and suggested remedies, therefore, echo and make explicit what the Bank itself has already recognized and is acting to correct. 2.25 For the material weakness to be resolved, such new controls would have also to be tested as being effective, which leads to the question how such effectiveness could be measured. In the first stage it would be progress enough (as is already underway) if explicit con- trols until recently absent were to be put in place in the key areas of the operations cycle outlined in Figure 3 below, and, as a second step, to demonstrate their effectiveness. It is in not the scope of this study to go beyond that, but it could be envisaged that the combined efforts to apply explicit tools aimed at focusing on F&C will themselves lead to an accumulating data base which will become a part of the Bank’s routine operations monitoring. SIGNIFICANT DEFICIENCIES 2.26 The six significant deficiencies that IEG has identified (includ- ing those identified during Part I) are as follows:  Lack of currency of OP/BPs:6 In light of progress made in up- dating OP/BPs that were in need of reform, IEG judges this weakness to rise to no more than a significant deficiency.  Timely accessibility to operational documents:7 In light of the findings from the testing completed in Part IB, IEG concluded that the availability of documentation was substantially im- 19 CHAPTER 2 THE IEG EVALUATION proved compared to the previous round of testing and that this issue no longer rose to the level of a material weakness.  Weaknesses in FM and PR processes: Testing of key controls during Part I showed that these fiduciary modules were IEG finds six among those with the highest rates of noncompliance, in part significant because of regional variations in process. This is a significant deficiencies, deficiency, which has also contributed to the material weak- some of which ness described above. Management is in process of mounting have an action program to remedy these deficiencies. contributed to  Need for improved management oversight: Evidence from the overall various sources (Part I findings, the ELCQ results, IAD country material audits, the INT India DIR) suggests a lack of adequate man- weakness of agement oversight of project processing, and most particularly, controls over project supervision, which has been a significant factor contri- F&C buting to the weakness of controls in IDA’s operations.  Need for improved risk management:8 The need to extend the COSO framework to introduce two more risk-oriented com- ponents was identified in Part I. In line with current COSO developments, several additions were suggested: a fourth COSO objective, Strategy— High-level Goals Aligning with Supporting Mission; and three new components: Objective Setting, Event Identification, and Risk Response. Part II notes the failure to include an F&C risk element in the Risk Scan, in the CAS, and in project design and supervision processes, which all contribute to the identified material weakness con- cerning F&C.  Need for improved IT security: The Bank’s current routines concerning managers’ SAP access privileges, and their amendment when staff rotate, may leave open the possibility Management of lengthy periods when segregation of duties is breached, states that when staff reassign but carry with them access privileges from processes exist their previous positions. This could raise the risk of fraud. Re- to ensure that lated findings have been reported by IAD and the external au- IDA operations ditor (during the ICFR review). There are also issues relating are conducted to password sharing and improved IT systems for decentrali- with due regard zation, and ensuring business continuity in light of natural or for efficiency other disasters, as was brought to light in both the ICFR and and the ELCQ results. effectiveness and that senior EVALUATION OF CONTROLS OVER EFFICIENCY AND EFFECTIVENESS management is 2.27 Management concluded that its assessment “gave reasonable kept well assurance to Senior Management and the Executive Directors that informed they are made aware, in a timely fashion, of the extent to which IDA is moving toward effective and efficient use of its resources in meet- ing its operational objectives” (Management’s Assessment report pa- ragraph 15). The basis for this statement was as follows: 20 CHAPTER 2 THE IEG EVALUATION  Findings in Part I that showed reasonable assurance that IDA operations were conducted in compliance with its policies and procedures, that IDA resources were used for the purposes in- tended, and that scope was identified for certain streamlining of controls over IL operations; and  Descriptions during Part II of the various budgetary processes (which contain efficiency indicators and incentives) and moni- toring mechanisms by the Bank’s central control units, which ensure that Senior Management and the Board are kept in- formed of trends in efficiency and effectiveness. 2.28 IEG concurs, as management asserts, that Bank Senior Man- agement is kept informed of the progress and performance in the an- nual budget cycle. Key tools include the annual budget process (and quarterly QBR processes), the Quality Assurance Group (QAG) An- nual Review of Portfolio Performance (ARPP), and the IEG Annual Review of Development Effectiveness and Annual Report on Opera- tions Evaluation (now a combined document). It is also clear, as man- agement says, that the Board is kept current on efficiency and effec- tiveness outcomes. 2.29 However, in IEG’s view, management did not go far enough Management in its treatment of controls over efficiency and effectiveness in its Part could have gone II assessment, in two respects: further in referring to  A focus on the controls over efficiency and effectiveness is a actual efficiency necessary but insufficient condition for sound judgment on and this question. While IEG agrees with management that there effectiveness are extensive controls in the Bank’s budgeting and monitoring outcomes, but systems to ensure that Senior Management is informed, the ef- the scope of this fectiveness of those controls cannot be judged without some review did not reference to outcomes. For its part, IEG consulted a number of include a review efficiency indicators (FY04-FY08; see Table SA.5 in the Statis- of outcomes tical Appendix). All four lending-related efficiency indicators trended positively in each year (except in FY08), but comple- tion costs of economic and sector work (ESW) and technical assistance (TA) have risen in recent years. While trends may not offer complete clarity, IEG is satisfied that Bank staff pur- sue the operational objectives of IDA with due regard to effi- ciency and effectiveness requirements.  Management assembled a significant body of information dur- ing its Part II assessment that described ways in which the ef- fectiveness of IDA’s operations is measured and priorities are selected and presented to Senior Management and the Board each year. This information could have been used for a more substantive statement. 21 CHAPTER 2 THE IEG EVALUATION CONCLUDING OBSERVATIONS The review has 2.30 This exercise has been a major undertaking for IDA, and IEG made major acknowledges the commitment that management made to completing contributions to both parts of the review. The exercise has contributed substantially to the Bank’s the Bank’s knowledge of its internal controls systems and has heigh- knowledge tened controls consciousness among managers and the staff, and cor- of its controls rective actions already underway will strengthen controls and better systems, at both ensure that IDA’s business objectives are being attained. The combi- transactions nation of mapping the 35 transactions-level business process modules and entity (and their rigorous testing), together with the COSO-framed ques- levels; it tionnaire in Part II, has provided the Bank with powerful new know- provides a basis ledge about its internal controls system and how the Bank has for streamlining adapted to the COSO principles. There were other benefits as well: the and future mapping has provided a strong basis for further work to streamline testing; and has operations for greater efficiency, and the framework now established highlighted the offers a benchmark against which future reviews can be conducted. need to accelerate 2.31 IEG acknowledges the thorough reviews by IAD and the in- development of sights that were offered from its auditor’s perspective on how con- new controls to trols should be designed and operated. A good deal of useful evi- address F&C dence for the review was also obtained from IAD’s ongoing work program, not done as part of this review but whose findings have been usefully applied. 2.32 Summary: IEG makes a number of observations and draws several conclusions from all of the foregoing analysis, as follows: The controls  The controls framework as a whole operates at a high level of framework effectiveness, but there are controls weaknesses in some areas. operates  At both entity and transactions levels more serious weak- effectively nesses (noncompliance with controls) were not widespread, overall, with but were found to be concentrated around a relatively few some areas in the framework. exceptions; at  Weaknesses were found in both the design of controls and in the entity level, the nonobservance of well-designed controls. At the entity- control design level, control design was the dominant source of weakness. At is an issue, at the transactions level, most noncompliance arose from nonob- the transactions servance of existing controls, not from control design, though level, non- some need for the latter was also shown. observance is  Observing the incidence of controls weaknesses at the entity more frequent level within the COSO components, IEG found that there were than flaws in differences within each component, but that overall weak- control design nesses were also cross-cutting and interrelated, so effective- ness ratings were the same for all components and rated satis- factory with qualifications.  With regard to specific controls weaknesses, IEG found one material weakness in the controls over fraud and corruption in 22 continued CHAPTER 2 THE IEG EVALUATION operations supported by IDA funding, being a composite of several factors. It also found six significant deficiencies, three of which were identified in Part I and three which were added after evaluating entity-level controls in Part II. Future Evaluations of IDA Internal Controls 2.33 At the direction of the Board, management conducted this as- Similar reviews sessment as a one-time exercise. An outside review organization has should be earlier recommended that internal control assessments be conducted systematically annually.9 Future assessments would be able to build on the substan- conducted in tial body of knowledge and analysis that is now available, but an an- the future nual review with the scope of the present review would be impractic- al. However, IEG recommends that more selective reviews be undertaken periodically, perhaps every two to three years. All such reviews should cover both IBRD and IDA. 2.34 IEG recommended in its Part IB report that IDA should con- sider the value of adopting a policy requiring: (1) ongoing monitoring and reporting on internal controls in the course of operations for all three COSO objectives; and (2) developing a policy on separate evalu- ations and reporting on special topics.10 Such a policy could delineate criteria for determining when separate monitoring would be initiated and when IDA would rely upon its ongoing monitoring efforts (IEG, IAD, Department of Institutional Integrity [INT], Quality Assurance Group [QAG]) for assurance as to the effectiveness of internal con- trols. The COSO framework provides potentially useful guidance for determining when the types of monitoring are appropriate, such as when there has been significant organizational change. 1. As provided in AS5, which was used as a source of guiding principles for NOTES this review, IEG relied on the results of the work completed by management and external auditors for evidence supporting the assessment of control over financial reporting. In making this decision, IEG considered the combination of management’s work and that of the independent public accounting firm to offer a high degree of competence and objectivity for evidence on the effec- tiveness of internal controls over financial reporting, enabling IEG to make full use of that work for its evaluation. 2. The ELCQ contained 157 individual questions. The questionnaire was sent to 31 units in the Bank, so in aggregate there were potentially 4,867 individu- al responses to the questions (31 x 157). Since some questions were not ans- wered by all units, and some were not applicable to all units, the actual number of effectively possible responses was 4,149. Of this total 3,673 were positive, 177 were negative( and a number were answered ambiguously), giving a simple pass rate of slightly over 95%. See Annex B, para. 5. 23 CHAPTER 2 THE IEG EVALUATION 3. The ratings were based on a combination of all forms of evidence available to IEG. This included all management materials, including the ELCQ, inter- views by IEG with Bank units, all background papers and other external in- formation, and the final reports from management and IAD. Hence, the rat- ings are a composite of all materials and reflect IEG’s judgment on the overall effectiveness of controls within the framework. 4. INT has conducted several Detailed Implementation Reviews and Fidu- ciary Reviews (DIRs and FRs, which examine the presence of F&C Indicators in IDA lending operations, including India (07). Some of the reviews were undertaken at INT’s initiative, while the others were initiated by the Regions. The IEG team examined all the reviews, which all showed evidence of F&C indicators, and some of actual F&C. The most recent DIR (on the health sec- tor in India) became public in December 2007, and the key findings were presented to the Board in January 2008. See Annex D para. 14 (Volume II page 40) for types of findings that emerged from these reviews. The Volcker report on INT also contained relevant evidence. 5. Thus the US Treasury was found to have eight material weaknesses in 2005, reduced to four by 2008 (see Statement of Assurance, included in its 2008 Performance and Accountability Report); and USAID was found to have four material weaknesses in 2007 (see USAID FY08 Performance and Accountability Report (PAR). Visit Treasury web site: http://www.treas.gov/offices/management/dcfo/accountability- reports/2008-par.shtml and USAID:http://www.usaid.gov/policy/afr08/ 6. Downgraded from potential material weakness in Part I. 7. This was judged by IEG to be a potential material weakness in Part IB, and is now downgraded to a significant deficiency. 8. Treated as part of material weakness in controls over F&C. 9. See GAO Report,”World Bank Group: Important Steps Taken on Internal Control but Additional Assessments Should Be Made” (GAO-03-366), June 2003. 10. See IEG Part I report, para. 4.7, final bullet, p. 42. 24 Evaluation Essentials  Management states that the controls give adequate assurance that 3. Summary of Management’s IDA’s resources are used efficiently and effectively Assessment and the IAD Review and in compliance with its policies, but finds significant deficiencies in five areas Management’s Assessment  IAD states that 3.1 Management’s final assessment report is in Attachment 1. The management’s qualified main objective for management’s review was to assess IDA’s internal assessment is fairly controls framework and to ascertain whether the framework gives stated; IAD finds a group Senior Management and the Board reasonable assurance that IDA’s of significant deficiencies, operations are in compliance with its policies and procedures and that including in controls over its resources are used with due regard for efficiency and effectiveness. F&C, which could become a material 3.2 Management’s Overall Assessment:1 “Under the COSO inter- weakness if not soon nal controls framework, Management is responsible for establishing corrected and maintaining an effective internal control system. IDA’s manage- ment has assessed internal control over IDA’s operations, and based on this assessment, and reflecting the work undertaken by manage- ment and attested to by the external auditor during the FY07 ICFR Management’s process, management is of the view that the internal controls system assessment over IDA’s operations is adequate to provide reasonable assurance to concludes that, Senior Management and the Executive Directors, that: with some  IDA’s published financial statements are being prepared exceptions, reliably; IDA’s internal controls provide  IDA carries out its operations in compliance with the relevant adequate provisions of its Articles of Agreement and operational assurance that policies and procedures, including provisions relating to the the three COSO use of funds for intended purposes; and objectives are  They are made aware, in a timely manner, of the extent to being attained which IDA is moving toward effective and efficient use of its resources in meeting its operational objectives.”2 3.3 Management goes on to say that, integrating its findings from both Part I and Part II,3 it has identified a number of significant defi- ciencies that require remedial actions—and effective monitoring of their implementation—in order to strengthen the internal control sys- Management tem over IDA operations. These significant deficiencies fall into five found categories, for each of which management also recommended moni- significant torable remedial actions, to be reported on in a progress report to be deficiencies in issued in time for the IDA 15 mid-term review. These are all paraph- five major areas rased in the paragraphs which follow. 25 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW IEG finds 3.4 IEG Observation: IEG takes the above statements by man- management’s agement as assurance to IDA stakeholders that, except for certain assurance weaknesses identified in parts of the internal controls framework, the justified, but IEG framework provides reasonable assurance that IDA’s operations are finds one in compliance with its policies and procedures and are conducted material with due regard for efficiency and effectiveness. Following its own weakness in evaluation of the effectiveness of the controls framework, which in- addition to six cluded confirmation that management’s approach and method were significant sound, and taking account also of IAD’s conclusion that manage- deficiencies ment’s assurance was well stated, IEG believes that management’s as- surance, with the stated significant deficiencies, is justified. However, IEG goes further than both management and IAD regarding the weaknesses of controls over prevention of fraud and corruption in IDA operations, judging it to be a material weakness. I: Issues Relating to the Current Policy and Procedural Framework for Investment Lending Process inefficiencies, including a lack of focus on key risks and con- trols during preparation/design and supervision stages of investment lending projects Investment 3.5 Management points to the findings arising out of the transac- lending tions-level assessment in Part I, which revealed a number of ineffi- processes are ciencies in the processes governing investment loans (ILs), which are cumbersome, duplicative, rigid, “one size fits all,” and are not risk-based or respon- not risk-based; sive to different country situations. management proposes 3.6 Recommended Remedies: Management has prepared a streamlining statement containing a new risk-based investment lending (IL) policy and enhancing framework, which will be shared with the Board (by FY09-Q3) and risk-based which contains the following principles: controls  Rationalize and prioritize key controls governing IL.  Facilitate appropriate differentiation of processes and re- quirements based on risk.  Set appropriate additional controls, resources, and reviews for higher risk operations/environments; maximize efficiency and effectiveness of each control through rationalization and work- flow automation.  Strengthen risk identification and monitoring during supervision by, among other things, clarifying roles, responsibility and accountability between IDA and borrowers within the institution. 3.7 IEG Observations: IEG agrees with these findings and consid- ers the remedies appropriate. 26 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW Reforming the outdated and complex policy framework for invest- ment lending 3.8 During Part I of the review management found that the con- Simplifying the trols framework governing IL operations is complex—multiple policy OP/BPs applied to ILs, the framework is over-regulated, and a num- framework for ber of OP/BPs have not been updated to reflect changes in the Bank investment and its client situations. During Part II, management found further lending can be that “the current status of OP/BPs does not have a material impact on achieved in a compliance and operations objectives of IDA” because there have new controls been mitigating measures taken that “fill in” the gaps with interim framework, decrees and other measures. Management attributes the failure to based on “core keep current with OP/BPs mainly to their “prescriptive and micro principles and style approach,” which will be improved once the new controls key controls” framework, based more on “core principles and key controls” 4 (de- scribed below) is in place. 3.9 Recommended Remedies: Management has launched a two- phase program to reform IL processes: Phase I to deal with immediate streamlining issues (completion by FY09Q4); Phase II to deal with a complete realignment of the IL framework, described below (comple- tion by FY10Q2). A concept note on IL reform, which reflects the find- ings of this review, was discussed with the OVPs on October 20 and with the Board in November-December 2008. The new framework will:  Reflect the main principles governing IL;  Replace the current “one-size-fits-all” requirements with a risk-based approach to IL due diligence, processing, and de- sign options; and  Replace the rigid “ring-fenced” project model with a flexible menu of design and funds flow options to better meet devel- IEG stresses opment and funding needs of IDA’s varied clients. that 3.10 IEG Observations: IEG agrees with management’s findings management regarding the status of OP/BPs. However, based on management’s oversight is also observation in Part I that noncompliance is frequently traced to non- important and observance of OP/BPs, IEG stresses that ensuring observance and warns against compliance with policies (that is, management oversight) may be as losing important as maintaining the currency of OP/BP content. Regarding standardized the proposed remedies, IEG considers them appropriate but warns approach to that in seeking more flexibility and moving away from “one-size-fits- PADs all” project preparation approaches the Bank should not lose at least a minimum degree of standardization in the PAD. Risk Risk identification and reporting, information sharing management 3.11 Management found weakness in the timeliness and scope in needs reporting of risks generally, and in the context of project supervision improvement 27 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW in particular. As evidence for this, management cited the results from questionnaires (both ICFR and ELCQ) and from the Risk Opportunity Workshops (ROW), as well as data from QAG, IAD, and IEG reviews. More realistic 3.12 Recommended Remedies: Management calls for an im- measures of risk provement in staff incentives and accountability, as part of an effort to and project improve the use and application of the ISR tool and process. It points outcomes are to management action already underway, which includes: needed in the ISRs and  Issuance of a “Quick Reference Guide on Results and Results management Terminology” to help task teams preparing ISRs in preparing has an action results-focused ISRs against which to monitor project progress; program  Modification of the project realism index in April 2007, ben- underway chmarked against IEG evaluations of project outcomes at exit; and  ISR reviews by every Region, to ensure adequate baselines, and to review quality and realism. 3.13 IEG Observations: IEG agrees with these findings and consid- ers the remedies appropriate. Need to address diffused accountability and compliance IEG again 3.14 Management finds, mainly from its ICFR and ELCQ results, emphasizes that the links between performance and accountability are not as management strong as they could be: accountability, compliance, and internal con- oversight and trol responsibilities are not adequately reflected in performance man- response: agement tools such as OPE or promotion criteria. Accountability is- project sues were also identified as part of the Volcker report and the India performance DIR. and staff accountability 3.15 Recommended Remedies: Management is to conduct a corpo- need to be rate accountability review that will focus on assessing: linked and management  Existing oversight arrangements to identify shortfalls in quali- needs to focus ty, including the roles and methods of central control units on monitoring (QAG, IEG, INT, IAD, and Regional Quality Teams); units  Existing accountability arrangements for operations, including mechanisms to detect operational shortfalls, launching an e- module for ISRs (FY08Q4) and ensuring regular reviews of ISRs by regions, with guidance from OPCRX, and several measures to enhance performance management, including the HRS initiated Performance Management Working Group and the HR Insight web site to improve transparency of HRS data. 3.16 IEG Observations: IEG agrees with these findings, including the diffusion of management accountability and weak reflection of ac- countability and compliance in performance management tools. Evi- 28 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW dence from different sources (the ELCQ; IAD country audits; QAG assessments; and INT DIR studies) all point to the fact accountabilities seem to be diffused by the matrix management system, and this is likely to contribute its own element of weakness to other controls de- ficiencies. Also, in evaluating the proposed remedies, IEG would re- mind management that the ELCQ results suggest that a significant is- sue with the Bank’s Monitoring and Learning component is that operational management often does not take timely actions to re- spond to recommendations from the central monitoring units (IAD, QAG, IEG, and INT). More fundamentally, these issues may reflect deeper weaknesses in the Bank’s internal governance structures. Timely accessibility of operational documents primarily relating to investment lending operations 3.17 Difficulties with accessing key operational documents was a Document major finding that emerged from the testing processes in Parts IA and accessibility IB, to such an extent that IEG classified this issues as a potential materi- issue is al weakness. Even though improvements were evident by the time of downgraded testing in Part IB (there was a 93 percent success rate in accessing doc- from a potential uments), it was evident that significant issues remained. The entity- material level assessment in Part II showed that aspects of both the Control Envi- weakness to a ronment and Information and Communication (principally the IT solutions significant required to support IDA’s operations) needed attention to resolve these deficiency issues (as mentioned also by the IEG in its Part IB report). 3.18 Recommended Remedies: In FY07 the Bank established a task force, including ISG, to build a new system of automated key controls for IDA’s primary operational tools (CAS, Development Policy Lend- ing [DPL], investment lending). The initial work has focused on IL as the main instrument, will be closely coordinated with the general re- view of IL controls (to be in effect by FY10Q2), and will be part of the Enterprise Content Data Management (ECDM) effort. 3.19 IEG Observations: IEG agrees with management’s findings and its formulation of the issues, which coincide with IEG’s own ear- lier observations. IEG also considers the remedies appropriate. As de- scribed in Chapter 2 (para. 2.24), IEG has downgraded this issue from a potential material weakness to a significant deficiency. II: Issues Relating to Fraud and Corruption Need for better integration of fraud and corruption (F&C) issues (including lessons learned from precedents and INT work) into daily operations 29 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW Progress has 3.20 Management points to the leadership role the Bank has played been made in in addressing the fraud and corruption agenda (acknowledged also in building an anti- the Volcker Report), and distinguishes three phases, as follows: corruption agenda, but  Phase 1: Setting out the intellectual case that good governance more needs to and an attack on corruption must be key parts of efforts to sus- be done to tain economic growth and reduce poverty. introduce  Phase 2: Formulating an appropriate anti-corruption strategy specific tools to and its adoption by Senior Management and the Board. combat F&C in  Phase 3: Moving into the implementation of the strategy, the IDA operations phase management explains the Bank and IDA have now en- tered. Using lessons learned, including from the work of INT, much remains to be done to integrate specific F&C tools into daily operations, particularly in countries where corruption is known to be endemic: in risk management, program and project design, and project supervision and evaluation. 3.21 Recommended Remedies: Management cites the response it gave to the Volcker report and recommendations as the main thrust of its action program to address F&C issues. Recognizing the need to mainstream good practices across the entire Bank, management also makes a number of specific recommendations for actions: Management  Clarify responsibilities and accountabilities in addressing has extensive, fraud and corruption, including adoption of a new INT strate- time-bound gy by end 2008. action program  Introduce improved tools and processes (by November 2008) to address in order to create a more effective risk management frame- weaknesses in work to help prevent, deter, detect, and address fraud and combating F&C corruption through “smart” project design and more effective and more appropriately resourced project supervision (progress to be reported on as part of the overall GAC Imple- mentation Report, discussed at the Board October 21, 2008).  Expand staff skills and broadening behavioral change in order to mainstream good practices across all of the Bank Group’s work (progress was reported on as part of the overall GAC Implementation Report).  Review staff incentives (performance reviews, promotions, rewards, and visibility) to ensure that they are aligned with the anti-corruption agenda (an MD-chaired GAC Governance Council has been created and this issue is being discussed at that level).  Mainstream lessons learned from precedents and INT work through, among other things, forming a consulting unit within INT for non-investigative purposes. The INT consulting unit has been created and core materials are expected from them on this topic by end September 2008. 30 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW  Prepare and monitor (with the support of Operations Policy and Country Services [OPCS]) specific action plans for follow- ing up on INT reports (a small team for the follow-up on post- INT report action plans is proposed to be established in OPCS with effect from FY 2009). 3.22 IEG Observations: IEG agrees with management’s formula- IEG regards tion of its role in the area of combating F&C, and it agrees also with weakness in the broad findings that weaknesses in the Bank’s specific tools in this controls over area suggest the urgent need to mainstream anti-F&C practices F&C as a throughout the Bank. IEG also considers appropriate the remedial material steps management is taking in this area, including those it committed weakness to in response the Volcker Report. However, as described in Chapter 2 because IDA’s (para. 2.18-2.27 and in more detail in Annex D), IEG has concluded mission is at that the weaknesses in this area rise to the level of a material weakness risk of being in IDA’s internal control framework, rather than the significant defi- impaired, and it ciency found by management. This judgment is based inter alia on the would be current absence of specific anti-F&C tools in IDA’s controls frame- premature to work coupled with the risk of IDA’s mission impairment that this im- conclude F&C plies. IEG fully recognizes the urgent and comprehensive set of re- issues have medial actions management has launched. However, where a been material weakness (or any other deficiency) has been identified it successfully must stand until all remedies are completed and found to be in opera- resolved tion and effective, which is not yet the case. III: Issues Relating to Procurement (PR) and Financial Management (FM) Strengthening quality and accountability arrangements for procure- ment 3.23 During Part I, management found that quality assurance Generic processes for PR are in place and that the regional variances that ex- deficiencies in isted were in line with the decentralized Bank, but it also identified two FM and PR principal areas where controls need to be strengthened, and which, include the need when combined with findings uncovered during Part I, it regards as for better rising to the level of a significant deficiency: integration of PR staff with  To ensure consistent follow-up on PR issues by the task teams, project teams, including the need for better integration of PR staff in task and better post- teams and clarification of accountabilities for procurement is- reviews sues and decisions;  To ensure consistency in implementation of post-reviews; 3.24 Recommended Remedies: OPCS will set up a working group, by November 2008, that will be responsible for developing the following: 31 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW  Mechanisms for early and full integration of PR staff in the project teams and of PR tasks during the project cycle,  Clarifications regarding the sharing of responsibility for key PR decisions at preparation and implementation stages be- tween TTL and PR staff and Sector Manager and RPM, includ- ing mechanisms to resolve disagreements and raise awareness of TTLs/sector managers;  a Procurement Certification system for this that would in- crease awareness and importance given to procurement work  Guidance and criteria for assigning PR ratings for the ISR, and mandatory recording of revisions to such rating by sector staff  Updating of OP/BP 11.0 Procurement (by end October 2008), and revising the PAD Procurement Annex (by December 2008) to include specific fraud and corruption provisions in addition to mainstreaming the risk-based PR assessment tool developed by SAR in 2004-5 (to be piloted by early 2009).  Reviewing the roles of the Procurement Sector Board (PSDB) and OPCPR, to expand its ability to guide the regions and standardize regional practices and to provide guidance on risk assessment, including F&C risk, and improved post review monitoring (by December 2008). There is a matrix  Developing a protocol with INT to systematically address management fraud red flags. dimension to the PR and FM 3.25 IEG Observations: IEG agrees with management’s findings issues, and and formulation of the issues regarding PR and other fiduciary weak- remedies will nesses and, as it did in Part I, has found in its evaluation that these need to put in weaknesses remain a significant deficiency. IEG takes particular note place to clarify of the accountability uncertainties that derive from the Bank’s matrix responsibilities management system and which were featured in the India DIR as an important factor relating to F&C (and which IEG regards as one factor contributing to the material weakness in this whole area). IEG consid- IEG also sees ers management’s suggested remedies appropriate. need for Improved use Issues relating to financial management and 3.26 For FM, management finds that quality assurance (QA) ar- management of rangements have been put in place, but that the regional QA ar- the PR rangements and documentation are “inconsistent and do not fully complaints comply with the FM Practices manual (FMPM).”5 While some differ- database ences may be justified, there is a need for consistency across the Bank. Management cites three sets of issues:  In some Regions, it was not possible to verify regular financial management specialist (FMS) review of audit reports be- cause Audit Report Compliance Systems (ARCS) data entry was considerably out of date. 32 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW  There are inconsistent quality arrangements for the documen- Some regional tation of FM supervision work, including supervision plan- FM quality ning, supervision reporting, and follow-up on FM action assurance items. Quality arrangements to support the filing of FM doc- measures are uments are also inconsistent, making it difficult to validate inconsistent that FM work has been done. with the FMPM  There are inconsistencies between regions in timing of FM quality interventions on risky projects and gaps in oversight practices. 3.27 Recommended Remedies: Actions are already underway un- Regions will der the FM Sector Board to strengthen quality arrangements, includ- consolidate and ing a Joint Evaluation by the Controller (CSR) and Operations Policy update quality and Country Services (OPCS) of Network Quality Arrangements arrangements, (FY09), implementation of Portfolio and Risk Management (FY09), update the and a review and update of the FMPM (by August 2008). Manage- ARCS, pending ment also proposes that: development of the ECDM  Regions will consolidate and update their quality arrange- ments (by August 2008).  Regions will update the ARCS for earlier years, and the FSB will monitor to be sure this has been done and that in future the data are entered in timely fashion. The Sector Board will develop and communicate good FM practices, pending the development of the ECDM system (by September 2008). 3.28 IEG Observations: IEG agrees with management’s identified findings and considers the proposed remedies reasonable and appropriate. In both PR and FM, however, IEG would have suggested giving more recognition to the question of how to support the development of country systems, which is key to both F&C control and IDA effectiveness over the medium term. IV: Issues Relating to Risk Aggregation Potential gaps in risk coverage 3.29 Despite important review and evaluation work carried out by There is a need the Bank’s central control units, there are gaps in risk coverage and for better IDA lacks a system for aggregating and prioritizing risk. aggregating and prioritizing risk 3.30 Recommended Remedies: Management intends to move to- ward an annual Integrated Risk Report (to be in place by FY10) which is intended to (a) describe overall risks facing the institution, (b) set out actions to address them, (c) assess potential gaps and overlaps, (d) develop a dashboard of risk findings from the various assessment ac- tivities, and (e) over time assess the adequacy of the processes in 33 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW place. Management also intends to conduct more specific reviews of central control units roles in the integration of risks and in providing checks and balances, and is suggesting a broad review of QAG by January 2009. 3.31 IEG Observations: IEG agrees with management’s findings and points to the observation IEG made in its Part IB report to the ef- fect that the identified risks facing IDA had not been aggregated, pri- oritized, and differentiated. IEG considers the proposed remedies ap- propriate in general, but more information is needed on the scope and purpose of the QAG review. V: Other Significant Deficiencies Processes and controls for analytical and advisory activities AAA processes 3.32 Following the request made by IEG in its Part IB report, sup- were also ported by a similar request from the Audit Committee, management mapped and during Part II mapped processes and controls for analytical and advi- tested, with a 75 sory activities (AAA) and tested these for a selection of sample tasks, percent pass in the same way they had done for other BPMs in Part I. The actual rate. tasks were all for ESW, which represents 80 percent of AAA expendi- tures. The tests revealed a 75 percent pass rate. However, many of the non-compliances were trivial—almost half were due to simple differ- ences in dates between actual meeting dates and those recorded in the SAP, so the substantive pass rate is certainly higher than 75 percent. Management has launched a 3.33 Recommended Remedies: Management has already launched major review of a broad review of AAA processes, controls, and monitoring to ad- AAA dress both these deficiencies and some other issues revealed by recent QAG reviews (mainly concerned with the monitoring of AAA). Man- agement expects to complete this review and present the findings to the Board by late FY09. IEG regards these 3.34 IEG Observations: IEG notes the successful conclusion of the weaknesses as AAA process mapping and testing, and it evaluates both manage- part of the ment’s approach and its findings as being relevant and credible, and overall considers the proposed remedies appropriate, while noting that a re- significant cent IEG evaluation had found that the Bank needed to take the re- deficiency in sults tracking framework for ESW (and TA) much more seriously.6 OP/BPs However, IEG sees the main controls weakness here as relating to un- stated or unclear policies (i.e. OP/BPs needing amendment) so it sees these as part of the overall weakness in OP/BPs, and does not regard this weakness as rising to the level of a significant deficiency in itself. 34 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW Information technology controls relating to password sharing and Infrastructure Change Management 3.35 Management states that the FY07 ICFR process identified three Password significant deficiencies: sharing of passwords for access into the SAP; sharing and issues (scope of access and monitoring) relating to access to privileged other issues of accounts; and the need to ensure consistency in application of controls IT security need over Infrastructure Change Management. remedies 3.36 Recommended Remedies: Management has recommended actions to:  Address the password sharing issue.  Strengthen controls around information security to limit privi- leged access to system applications and to monitor changes made by staff using such privileged IT systems access.  Strengthen controls around Infrastructure Change Management. 3.37 IEG Observations: IEG notes management’s findings from the Password ICFR and notes also its recommended remedial actions. IEG has not access to identified these three significant deficiencies in its evaluation since it certain did not review the ICFR questionnaire in detail. However, the results databases of the ELCQ also showed that access privilege issues were arising needs to be from the fact that the Bank system for managing privileged access for removed more staff who rotate to other positions is not sufficiently fine tuned. This quickly from could make fraud easier and was therefore found by IEG, along with staff when they other IT issues, to be a significant deficiency.7 rotate to new assignments RESOLVING ISSUES OUTSTANDING AT THE END OF PART I 3.38 At the completion of Part I, a number of issues remained to be A number of addressed and resolved during Part II. Among these were some scope issues left limitations: three BPMs—ESW, Debt Sustainability Analysis (DSA), unresolved at and Safeguards (Corporate Risk List)—which hade not been mapped end of Part I and tested; there were other items on which further testing was have been needed—the large number of N/As, one further DPL, and CASs from addressed, and other regions; and there were items that had been postponed to Part some resolved II—IT, field offices, and fraud and corruption; finally, a large number of miscellaneous deficiencies remained unresolved. 3.39 These various outstanding items have now been assessed, re- viewed by IAD, and evaluated by IEG. For the sake of review com- pleteness, a summary account of the results is given in Annex C. MONITORING ARRANGEMENTS 3.40 Management intends to implement the above remedial actions over the next 18-24 months and report to Senior Management and the 35 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW Executive Directors on the progress of their implementation and re- sults achieved. OPCS and CSR will take the primary responsibility for such periodic monitoring and reporting. The first progress report to be prepared and discussed with the Executive Directors in the early fall of 2009, in time for the IDA 15 mid-term review. IEG Observation: These arrangements are welcome but management must ensure that the remedies program covers all deficiencies identified (over 100 in total) whose remedies may be in progress but are not yet operative. IEG OBSERVATIONS ON THE OVERALL ASSESSMENT REPORT IEG rates 3.41 Management’s assessment report was subjected to an exten- management’s sive IEG review, including evaluation ratings for approach and me- approach and thod, findings, and conclusions. The approach and method, as de- method as fully scribed in Chapter 2, was rated fully satisfactory, but with some satisfactory suggestions for improvements. IEG rates the overall assessment as sa- (with some tisfactory with qualifications. The assessment was made by COSO com- suggestions for ponents, and IEG rates the assessment of the Control Environment and improvement); Control Activities as fully satisfactory with the assessment of Risk As- IEG rates the sessment, Monitoring and Learning, and Information and Communications overall rated as satisfactory with qualifications. The qualifications stemmed assessment as mainly from the fact that, by conducting the entity-level review after satisfactory with rather than before the transactions-level review, management missed qualifications an opportunity to prioritize some areas (particularly regarding risk assessment relating to fraud and corruption), where controls needed to be strengthened. IEG did not 3.42 In viewing the entire package of management’s findings IEG can identify any confirm that IEG identified no significant deficiencies that management significant did not also identify. This commonality of findings is significant because deficiencies that IEG did not base its findings on management’s final report. Rather, IEG’s management did findings were made on the basis of a review and analysis of the raw re- not also identify, sults of the ELCQ responses, and the other materials provided to IEG. but there were IEG’s analysis is therefore independent of management’s report. IEG also differences in applied a statistical analysis of the ELCQ responses to support its find- organizing the ings, something management chose not to do. findings and one difference 3.43 Notwithstanding the broad commonality of findings, there are as to materiality also some differences between management and IEG findings, most particularly in two areas:  Material Weakness: IEG found that the deficiencies in controls over detection and prevention of F&C rose to the level of a material weakness, which affects its evaluation of the state of the overall controls framework. However, this does not imply that IDA’s financial reporting has been misstated, because this material weakness is over operational not financial reporting. This issue is further elaborated in Chapter 2 and Annex D. 36 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW  Organizing the Findings: Management grouped its findings of deficiencies around key areas, and grouped many of them around processes tied to IL business processes, F&C, FM and PR, and others; IEG tended to stay with the framework of the COSO components, but it also used a conglomerate grouping when describing the material weakness; while grouped diffe- rently, the commonality of coverage of the findings remained. The IAD Review and Opinion 3.44 IAD’s final report “IAD Review of Management’s Assessment of IDA’s Internal Controls: Part II,” is in Attachment 2. OVERALL OBJECTIVE 3.45 IAD’s overall objective in the review was to: “review the basis IAD set out to of management's assessment, and express an opinion on whether it is review if fairly stated based on the criteria established in Internal Control— management’s Integrated Framework issued by COSO.” assessment, based on COSO SCOPE, APPROACH, AND METHOD principles, was 3.46 Part I: Assessment of Business Process Controls: In its Part IB fairly stated report8 IEG gave a full account of the scope, approach, and method adopted by IAD in its Part I review and opinion on management’s as- sessment of the transactions-level key controls, as well as of the key results. The highlights of this approach were as follows:  Review of high-level process flow charts and accuracy of man- IAD adopted a agement’s mapping of key processes; multifaceted  Attendance at workshops/review sessions to verify effective- approach to its ness of design of key controls; review: in Part I  Application of its deficiency tracker to identified weaknesses reviewing in controls design; details of  Review of sampling and methods of testing for operation of approach and key controls; critical  Review and critical validation of management test results (it is- validation of test sued 32 transmittals containing observations to management); results; in Part II  Used related results of IAD work conducted for other purpos- examining es, where relevant to the controls review. management’s work in five 3.47 Part II: The Entity-Level Controls Review: With regard to the main areas scope of its review and opinion in Part II, IAD described this as cover- ing five areas:  “overarching control framework for IDA, including all aspects of corporate governance and entity-level controls;  efficiency and effectiveness of IDA’s operations; 37 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW  information technology (IT) controls;  fraud and corruption controls; and  other outstanding items carried over from Part I, including (i) quality assurance arrangements for procurement and financial management in IDA’s operations (ii) three additional processes relating to Safeguards Corporate Risk, IDA’s Grant Allocation and Debt Sustainability, and Economic and Sector Work (iii) QAG’s Quality of Supervision Assessment, and (iv) additional testing of CAS and DPL modules.” IAD’S APPROACH AND METHOD 3.48 IAD’s approach to its Part II review consisted of four segments of work: (a) review of basic instruments; (b) review and verification of management’s testing results; (c) use of other ongoing IAD audits and other reports; and (d) review of management’s assessment report. IAD reviewed 3.49 Review of Basic Instruments: As part of the process of re- the scoping viewing and commenting on management’s approach before the start memorandum of the assessment, IAD reviewed both the Part II Scoping Memoran- and ELCQ dum and the draft pre-issue ELCQ, it commented on each and fol- design lowed up on the comments. IAD thoroughly 3.50 Review and Verification of Results: IAD conducted a tho- reviewed all rough review of the output from the instruments used in manage- management ment’s assessment, covering: outputs, attended  Workshops/Review Sessions: IAD attended walkthrough ses- workshops, and sions to validate process documentation for key controls in issued four business modules not covered during Part I (ESW/AAA, transmittals Safeguards and Corporate Risk, Grant Allocation, and Debt Sustainability).  Revised Process Documentation: IAD reviewed the revised process documentation that arose out of the workshop/walk- through sessions.  Management Test Results: IAD reviewed the test results for Part II, including the documentary evidence supporting man- agement’s conclusions.  Transmittals for Communicating Entity-Level Review Re- sults: IAD issued seven transmittals communicating prelimi- nary results from the responses to management’s ELCQ. Ongoing IAD 3.51 Use of Other, Ongoing IAD Audits: In its review report, IAD audits also were mentions 20 audit reports from which it drew evidence for the IDA used as controls review. These included 12 country program or other audits of evidence Bank or IDA activities in a range of countries; an audit of Bank Identi- ty and Access Management; audit of key Bank IT and wireless tele- phone systems; an audit of the Bank’s COSO process; and audits of 38 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW the Integrated Risk Management Framework and the Quality Assur- ance Group. IEG Observation: With regard to the country program audits (most of which were also reviewed by IEG) the Bank-wide learning would have been more pronounced had the main findings been organized and analyzed by IAD into a synthesis report where common themes and their materiality could have been identified. Such a report was prepared in September 2008. 3.52 Draft Management Assessment: IAD reviewed the draft re- IAD reviewed port and supporting annexes and provided preliminary comments management’s with regard to the candor of management’s conclusions and consis- draft tency with the objectives and results of the assessment. Management assessment reformulated its overall conclusions to address some of IAD’s prelim- report inary comments, and additional comments were also made in July 2008. SUMMARY OF IAD RESULTS AND FINDINGS 3.53 IAD presented its findings in the form of a general observa- tion, several specific observations, and a concluding statement. 3.54 General Observation: The IAD general observation is as fol- The IDA lows: “As noted in IAD’s Part IA and Part IB reports, this IDA as- controls review sessment is the first comprehensive internal exercise undertaken by is unique among management to review its operational/compliance internal control international framework. Furthermore, it appears to be unique in the multilateral financial development banking environment, and to our knowledge, in the institutions broader international financial institution community. While the ef- fort underlying the commitment was clearly underestimated at the outset, substantial commensurate benefits are anticipated: its results will provide a compelling baseline to identify opportunities for streamlining IDA’s (and concurrently IBRD’s) operations and internal controls while significantly improving consistency and efficiency.” 3.55 IEG Observation: IEG agrees with this general statement and has itself commented on the fact that this is a first-in-history review of internal controls, whose accumulated body of knowledge and mapped processes will be of great value going forward. 3.56 Specific Observations: The principal content of IAD’s find- ings is contained in specific observations it makes in five areas: 1. Adequacy and Effectiveness of Key Fiduciary Controls: IAD Controls over concurs with management that the findings of Parts I and II fiduciary regarding controls over procurement and financial manage- processes ment processes collectively constitute a significant deficiency. constitute a However, IAD disagrees with management’s broader conclu- significant sion that, despite this significant deficiency, key controls in deficiency 39 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW IDA’s fiduciary processes still give assurance that IDA’s re- sources are used for the purposes intended, saying that man- agement’s assertion is inadequately supported, for three rea- sons: management did not sufficiently examine the underlying quality (that is, outcomes) of the fiduciary controls; there were inappropriate regional variances in FM and PR processes; and some transactions key controls were found to need redesign. IEG Observation: IEG agrees with the finding of a significant deficiency in this area, it also takes note of IAD’s reasons for this finding and notes management’s remedial actions that are underway. IAD agrees with 2. Entity-Level Controls: IAD concurs with management that management weaknesses in five areas of the entity-level controls which regarding each individually rise to the level of a significant deficiency: significant outdated policy framework for IL, need to integrate F&C into deficiencies in daily operations, inadequate mechanisms for risk aggregation, five areas issues relating to AAA, and need for controls over information systems. IAD suggests that there are also other aspects that should be added to this list, including governance and accoun- tability for Integrated Risk Management; oversight of opera- tional risk; and QAG assessments. IEG Observation: IEG notes that IAD concurs with management’s assessment as to the significant deficiency at the entity level, and that IAD sug- gests other issues should also be considered, but notes also that IAD implies that the individual materiality of these issues remains at the level of a significant deficiency. 3. Controls Over Efficiency and Effectiveness of Operations: IAD concurs with management’s conclusion relating to effi- ciency and effectiveness of operations, but states that in its opinion “several opportunities exist for enhancing effective- ness and efficiency of IDA’s operations, including:”9 the need for more reliable and candid ISR reporting; IEG outcome rat- ings in Africa in recent years have shown steady declines; and opportunities identified for streamlining could significantly improve the efficiency of IDA operations; and HR practices (skill mix and performance management) need improvement. IEG Observations: IEG agrees with the areas highlighted by IAD where improved efficiencies and effectiveness may be achieved. IEG believed that examining “controls over E&E” was the necessary condition of the review, but to be fully suf- ficient at least some reference also to E&E outcomes would be Weaknesses in helpful. At the same time, IEG does not consider the IDA con- certain IT trols review the appropriate occasion for a full examination of controls E&E outcomes, which would be a major undertaking on its constitute a own. significant 4. Information Technology Controls: IAD refers to manage- deficiency ment’s FY07 ICFR, which found significant deficiencies in 40 CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW password management, information security rationalization, and change management controls. IAD also mentions certain significant deficiencies that it found in IAD audits covering business continuity, IT governance and strategy, and wireless security controls. IAD concurs with management that weak- nesses in the controls in these areas constitute a significant de- ficiency. IEG Observation: These findings are noted and IEG concurs with their materiality. In addition, from its own anal- ysis of the ELCQ results IEG finds that a lack of effective con- trols over data access privileges when staff rotate to other po- sitions is part of a significant deficiency in IT controls, along with other deficiencies found. 5. Fraud and Corruption Controls: IAD states that issues relat- IAD identifies ing to F&C collectively constitute a significant deficiency, significant while pointing out that such deficiencies in these controls deficiencies in create vulnerabilities to F&C in countries where systemic cor- controls over ruption is not adequately addressed during program and F&C and points project design. IAD also states that the identified significant to remedies deficiencies relating to fiduciary controls, entity-level controls, being IT controls, and F&C controls, in combination could represent implemented a material weakness unless remedied in a timely manner and effectively monitored on an ongoing basis. IEG Observation: IEG is of the view, based on the audit criteria agreed for this review, that the deficiencies in this group of controls rise to the level of a material weakness, and that this weakness must be seen to remain until remedies are not only in place but have been tested as being effective. IEG is of the view, therefore, that the IAD conclusion implies a material weakness but does not, as IEG suggests it should do, explicitly state this. 3.57 Status of IAD’s Part I Recommendations and Issues: IAD re- Thirteen issues fers to a total of 13 issues that were deferred or otherwise left open at left over from the end of Part I (including from Part IA), and explains that, with one Part I are being exception (relating to treatment of trust funds), remedies are in remedied, with process of being implemented. IEG Observations: IEG considers this one exception is an accurate accounting of the outstanding issues, and does not see relating to trust grounds for finding any significant weaknesses in these areas of de- funds ferred treatment. Regarding trust funds, management had noted at the end of Part I that these had been dealt with separately, including as part of the ICFR process. IAD concludes 3.58 Overall Conclusion: IAD concludes that, subject to certain that, subject to stated exceptions, management’s assessment is fairly stated that the certain exceptions, internal control system over IDA’s operations is adequate to provide management’s reasonable assurance to Senior Management and the Executive Direc- assessment is tors that IDA’s financial statements are reliable; that IDA carries out fairly stated its operations in compliance with its Articles, policies, and proce- 41 continued CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW dures; and that management is made aware in a timely manner that IDA’s operations are carried out effectively and efficiently. The significant 3.59 Regarding the mentioned exceptions (significant deficiencies deficiencies in in fiduciary controls, entity-level controls, IT controls, and F&C con- several areas trols) IAD indicates that these could constitute a “ material weakness could become a unless remedied in a timely manner” because there is a “reasonable material possibility that internal control failures may not be prevented.” weakness if not remedied in a 3.60 IEG Observation: IEG notes the finding that IAD regards timely manner management’s assessment as fairly stated, but subject to certain ex- ceptions. IEG arrives at a similar assessment. However, IEG comes to different findings in relation to the precise treatment of the excep- tions, in two respects:  Material Weakness: IEG has concluded that a material weak- ness already exists in the controls over F&C (contributed to by weaknesses in other associated controls); IAD has found a sim- ilar nexus of associated controls (all of which IEG has also identified as deficiencies or significant deficiencies) but has concluded that this creates vulnerabilities to F&C and other is- sues which collectively could lead to a material weakness if not remedied in a timely manner.  Efficiency and Effectiveness: IEG observes that process con- trols at the entity level are designed not only to inform Senior Management (and the Board) of the extent of efficiency and ef- fectiveness of IDA operations, but they also contain specific incentive mechanisms to encourage operational managers to conduct their operations with due regard for efficiency and ef- fectiveness, so IEG regards the spirit of the Scoping memoran- dum on this topic to have been achieved in management’s Part II assessment. NOTES 1. Management Assessment OPCS and CSR May 2008, para. 17, page 10 (At- tachment 1). 2. In its final report, management distinguishes between the financial report- ing and compliance objectives, which are largely under IDA’s internal con- trol, and the operational objective of efficiency and effectiveness, which can- not be entirely under IDA’s control, so the degree of assurance that can be provided by the internal controls is less absolute than with the former two objectives. 3. As well as from the FY07 ICFR where relevant. 4. All excerpts from Management’s Assessment, para. 24 (Attachment 1). 5. Management’s Assessment, para. 41. 42 continued CHAPTER 3 SUMMARY OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW 6. Using Knowledge to Improve Development Effectiveness: An Evaluation of World Bank Economic and Sector Work and Technical Assistance 2000-2006. (Report 46803, Washington, DC: World Bank, 2008). 7. The issue is occasioned mainly by staff rotations. If a staff member has access (that is, a password) to a classified database tied to his/her function in one position, this access should end when the member rotates to another posi- tion because the segregation of duties principle may be breached if the access continues. At present HR reviews rotations of this kind only periodically, so access may continue for some time in a member’s new position, allowing con- tinued entry to a database that should not be authorized. There needs to be a system that ends such access on the same day as rotation occurs. 8. See “Review of IDA Internal Controls: An Evaluation of Management’s Assessment and the IAD Review: Report on the Completion of Part I: Incor- porating Compliance Testing of Key Controls (Part IB).” AC2007-0044 June 22, 2007. 9. See Volume III, Attachment 2, “The IAD Review and Opinion, “ para. 3, p. 6. 43 4. Summary of Key Findings, Lessons, and Recommendations Overall Summary of Findings 4.1 In its final assessment report, management provides an assur- Management ance, which states that the internal controls framework governing provides an IDA operations has been shown to have been designed and to operate assurance that in a way that gives adequate assurance that it is in compliance with controls provide IDA’s charter and internal policies and procedures, and that it ensures reasonable the efficiency and effectiveness of IDA’s operations. assurance that IDA’s operations 4.2 Management also states that it found significant deficiencies are conducted in five areas of the controls framework governing IDA operations, as in accordance well as a number of other deficiencies, all of which are well elabo- with COSO rated. objectives, but cite significant 4.3 In its final review report, IAD provides an opinion that, fol- deficiencies lowing its extensive and detailed review of management’s assess- ment, including its general approach, methods, results, and conclu- sions, management’s assurance has been fairly stated, but subject to certain exceptions. IAD broadly concurs with management’s findings regarding a number of significant deficiencies, but, unlike manage- ment, regards the combination of the deficiencies of controls over fi- IAD finds duciary, entity-level, IT, and fraud and corruption issues will result in management’s a material weakness unless effectively addressed in a timely fashion. assurance fairly 4.4 IAD also makes a number of specific observations about stated, but also weaknesses over fiduciary controls and entity level controls (need for finds that policies and procedures reform, integration of F&C into operations, significant improved risk aggregation and treatment of AAA, among others). deficiencies in IAD also questions whether management’s treatment of efficiency several areas, and effectiveness was adequate (citing declining IEG performance rat- including over F&C, which ings for the portfolio in Africa) and gives a reminder of certain signifi- could become a cant deficiencies that were uncovered during the ICFR on IT controls. material 4.5 IEG’s Evaluation: In concluding its own evaluation (based on weakness if not the extensive analysis described in the annexes contained in Volume soon addressed II) IEG arrived at the following conclusions: 45 CHAPTER 4 IEG’S CONCLUDING STATEMENT IEG concludes  Approach and Method: IEG rated the approach and methods that used by management fully satisfactory but with some observa- management’s tions relating to certain improvements that could have been approach and made to the questionnaire design. However, these would not method is fully have significantly affected the results. IEG also finds IAD’s satisfactory, but approach and method satisfactory. To this extent, IEG finds improvements both management’s and the IAD’s approach and outcomes can be credible and transparent. suggested  Results and Conclusions: IEG concludes that IDA’s internal controls framework operates to a high standard overall, except, as management and IAD also find, it contains some weak- nesses. IEG’s evaluation is therefore qualified in a way broadly IEG evaluates similar to management’s assurance, except that IEG believes the the materiality of one set of weaknesses – in fraud and corruption -- effectiveness of rises to the level of a material weakness. It also finds six addi- IDA’s controls tional significant deficiencies. Consistent with these findings, framework as IEG rated the overall quality (strength and effectiveness) of the satisfactory with controls framework as satisfactory with qualifications. qualifications 4.6 The detailed analysis that underlies these IEG findings is de- but finds one scribed in the annexes in Volume II. The detailed justification for the material finding of a material weakness in the area of prevention of fraud and weakness and corruption is given in Annex D. There is a high degree of commonality six significant in the findings of management, IAD, and IEG as to individual weak- deficiencies nesses. However, these weaknesses have been presented and grouped somewhat differently in each case, as was described in Chapters 2 and 3. IEG believes that deficiencies can be grouped, but this should be done only where they are related, where they reinforce each other, and/or where needed remedies are common to all deficiencies in a group. IEG has applied this principle to its analysis and in its finding of a material weakness in controls over F&C, where multiple deficiencies are cited but where their linkages are clearly described. Lessons Arising from the Review The entire 4.7 Benefits of the Review: IEG regards this entire review as a review has been substantial contribution to IDA’s knowledge of its internal controls beneficial, system. Whereas IDA has, since adopting the COSO framework a yielding new decade ago, achieved conformity with COSO standards with regard knowledge and to its financial reporting—including obtaining an attestation from the redressing external auditor on the effectiveness of its internal controls over ex- imbalance in ternal financial reporting—it has lagged behind in achieving these IDA reporting standards with regard to the compliance and efficiency and effectiveness standards under objectives of COSO. Completion of this review has redressed this im- COSO balance in IDA’s reporting standards, subject to the weaknesses that have been identified and whose remedies are part of the recommen- dations of this review. 46 CHAPTER 4 IEG’S CONCLUDING STATEMENT 4.8 It is important to put these findings in a broader perspective. For an agency as complex as IDA and the Bank—with global reach over multi-faceted program content and diverse clients—and for a first-in-history controls review, the outcome of IEG’s review, with one material weakness and five significant deficiencies, is by most standards quite reasonable. The statement of the external panel expresses similar sentiments. 4.9 Implications for IEG: IEG has undertaken this major evalua- tion at the request of the Board. For the exercise, there has been clear additionality and mutual learning from the involvement of manage- ment, IAD, and IEG. Going forward IEG would assume that follow- up work would best be undertaken by management itself, reporting directly to the Board through the Audit Committee. IEG will reflect carefully on the main findings of this evaluation, and in particular the issues related to fraud and corruption in IDA operations, and how these issues would be appropriately addressed in IEG's evaluation program. An evaluation of the GAC currently planned for FY11 should report, inter alia, on the implementation through that vehicle of the current recommendations. In addition, IEG will review how best to incorporate F&C issues in its project evaluations, taking into account the division of labor with INT. Recommendations 4.10 Based on its evaluation, IEG recommends the following to IDA management: (a) Address on a broad front the controls needed to ensure that F&C practices in IDA client countries and among participating stakeholders do not impinge on IDA’s mission through possible Fraud and Corruption in IDA operations. Actions could include:  Accelerate implementation of the ongoing Governance and Anti-Corruption (GAC) program, and devote addi- tional attention and resources to building an organization- al culture and incentive structure in which the risks of F&C are explicitly and cost-effectively addressed in the management of IDA’s operations. While Management has correctly observed that such awareness has been spreading, including through the follow-up to the Volcker report, the systematic integration of this awareness into daily opera- tions still has some way to go and needs to be given sus- tained emphasis going forward.  Develop and deploy specific F&C related instruments into the Bank’s Risk Scan processes, CASs, lending and project designs, and ISRs. Remedies have already been initiated as part of the GAC initiative and the Volcker Report, and 47 CHAPTER 4 IEG’S CONCLUDING STATEMENT INT has recently become involved in helping to design tool- kits to address F&C at various levels of the lending cycle, al- though it is too early to judge the impact of these initiatives. It is also important to link country-based risk assessments through the Risk Scan to specific tools to address lending risks in both IL and DPL/PRSC type lending.  Continue the ongoing reforms of FM and PR processes (launched in response to the findings of this review) and link them closely to the F&C agenda. These are key ele- ments in the Bank’s fiduciary and governance systems but evidence from the review that new toolkits such as those be- ing developed under the “GAC In Projects” program need to be deployed, made operative and later tested for effec- tiveness.  Intensify IDA support to strengthen clients’ fiduciary and governance systems, recognizing that this is a principal means to guard against F&C and to ensure the effective use of IDA resources (and the only means to do so in the case of budget support operations such as PRSCs). In the case of DPL/PRSC operations special emphasis needs to be given to developing tools that could attach, for example, to the Letter of Development Policy and to CFAA require- ments, to raise the attention to systemic F&C issues at the country level.  Make arrangements for testing the operating effectiveness of these new controls at some appropriate time in the fu- ture, since the material weakness and other identified defi- ciencies will be deemed to persist until this has been done. (b) Closely monitor the implementation of remedies for control deficiencies, including:  The measures currently in progress to update the OP/BPs. These also need to be extended to key areas (AAA, F&C) not yet covered or where new policies are being developed.  A mechanism to ensure the future currency of OP/BPs. There has been progress in bringing the body of OP/BPs in- to conformity with overall Bank and IDA policies and stra- tegic goals, and IEG has therefore downgraded the weak- ness uncovered in this area during Part I from a potential material weakness to a significant deficiency.  Improved documentation retention and accessibility and a user-friendly documentation management system. In its Part IB report IEG had already downgraded the materiality of this issue from a potential material weakness to a signifi- cant deficiency. However, the needed IT systems are not yet in place and the Enterprise Content and Document Man- 48 CHAPTER 4 IEG’S CONCLUDING STATEMENT agement (ECDM) system of which they will be a part should be developed as a matter of priority.  Mechanisms to correct and monitor the several IT systems deficiencies identified. These included password man- agement, business continuity and change management, and need for tighter control over IT access privileges for staff who rotate into new positions.  Measures to address the about 100 identified other as yet unresolved deficiencies. Remedies for many of these are already in progress, but specific monitoring is needed given the wide front and many areas in which remedial actions are needed. 49 Annex: Management Response REVIEW OF IDA INTERNAL CONTROLS: AN IEG EVALUATION OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW: REPORT ON THE COMPLETION OF PART II MANAGEMENT RESPONSE AND UPDATED SUMMARY OF MANAGEMENT’S OVERALL ASSESSMENT OPCS AND CSR February 24, 2009 ABBREVIATIONS AND ACRONYMS AAA Analytic and Advisory Activities IDA International Development Association ARCS Audit Report Compliance System IDF Institutional Development Fund AS2 PCAOB’s Auditing Standard No. 2, An IEG Independent Evaluation Group Audit of Internal Control Over Financial IFI International Financial Institution Reporting Performed in Conjunction With IL Investment Lending an Audit of Financial Statements IMF International Monetary Fund AS5 PCAOB’s Auditing Standard No. 5, An INT Department of Institutional Integrity Audit of Internal Control Over Financial IOP Implementation Oversight Panel Reporting That Is Integrated With an ISG Information Solutions Group Audit of Financial Statements ISR Implementation Status and Results Report BP Bank Procedure IT Information Technology CAS Country Assistance Strategy LEG The Legal Vice Presidency CGAC Country Governance and Anti-Corruption MD Managing Director CFAA Country Financial Accountability MIC Middle Income Country Assessment OKSP Operations and Knowledge Management CFO Chief Financial Officer Systems Program CFP Concessional Finance and Global OP Operational Policy Partnerships OPCFM Financial Management Anchor OPCS CODE Committee on Development Effectiveness OPCPR Procurement Policy and Services Group, COSO Commission of Sponsoring Organizations OPCS of the Treadway Commission OPCRX Results Secretariat Unit, OPCS CPAR Country Procurement Assessment Report/ OPCS Operations Policy and Country Systems Review OPE Overall Performance Evaluation CSR Controller's, Strategy, & Resource OVP Operational Vice-President Management PAD Project Appraisal Document DIR Detailed Implementation Review PCAOB Public Company Accounting Oversight DPL Development Policy Lending Board EAP East Asia and Pacific Region PEFA Public Expenditure and Financial ECA Europe and Central Asia Region Accountability ECDM Enterprise Content and Document PR Procurement Management PREM Poverty Reduction and Economic ELC Entity Level Control Management Network ESW Economic and Sector Work PRIMA Portfolio and Risk Management EU European Union PRSC Poverty Assessment Support Credit / EXT External Affairs Poverty Reduction Support Credit F&C Fraud and Corruption PSB Procurement Sector Board FM Financial Management PSU Prevention Service Unit with INT FMPM Financial Management Practices Manual QA Quality Assurance FMS Financial Management System/ Financial QAG Quality Assurance Group Management Specialist RAPMAN The Financial Management Risk and FY Fiscal Year Portfolio Management Model GAAP Governance and Accountability Action ROW Risk and Opportunity Workshop Plans RMS Results Measurement System GAC Governance and Anti-Corruption RPM Regional Procurement Manager GSD General Service Department SAP Systems, Applications, and Products HD Human Development SAR South Asia Region HR Human Resource SRI Salary Review Increase HRS Human Resource Services TTL Task Team Leader IAD Internal Auditing Department US United States IBRD International Bank for Reconstruction and USAID United States Agency for International Development Development ICFR Internal Controls over Financial Reporting VP Vice-President iv REVIEW OF IDA INTERNAL CONTROLS: AN IEG EVALUATION OF MANAGEMENT’S ASSESSMENT AND THE IAD REVIEW: REPORT ON THE COMPLETION OF PART II I. Overview and Follow-Up Management Action 1. Background. As reflected in the IDA 14 Replenishment Report1 management committed to carry out an independent comprehensive assessment of its control framework including internal controls over IDA operations and compliance with its charter and policies, and making such assessment available to the public after its disclosure has been approved by IDA’s Executive Directors. The assessment has been conducted in the context of the internal control framework developed by the Commission of Sponsoring Organizations of the Treadway Commission (COSO),2 adapted to fit the unique nature and operations of IDA. It has been carried out in three tiers: Management self-assessment; Internal Auditing Department (IAD) review; and an independent evaluation of both by the Independent Evaluation Group (IEG). The assessment has been divided into two parts: Part I, which focused on design and operating effectiveness of controls over IDA operations at the transaction level; and Part II, which focused on (a) entity-level controls, (b) outstanding transaction-level controls deferred from Part I, and (c) the operations objective of COSO. 2. Benefits of this exercise. In management’s view, this has been a very important and highly useful exercise. It enabled management to take a comprehensive and holistic look at internal control system over IDA’s operations, focusing on management and business processes conducted at transaction and entity levels across IDA. It also allowed management to assess mechanisms in place to monitor and adjust these processes to respond to identified risks and support IDA’s strategy and priorities as they evolve in response to client needs. As noted by IEG, by undertaking this “first of its kind” assessment of internal controls not only for IDA but also for all international financial organizations, “the Bank and IDA have taken an important lead in assessment of internal control” (emphasis added). Being the first of its kind, this unprecedented assessment was much more time and resource consuming than originally anticipated at all levels and for all assessment participants (management, IAD and IEG). As noted by IEG, however, the time and effort spent by management has resulted in development and application of a “transparent, well documented, and comprehensive” approach to this 1 See, Report from the Executive Directors of the International Development Association to the Board of Governors, Additions to IDA Resources: Fourteenth Replenishment, Working Together to Achieve the Millennium Development Goals, (approved by the Executive Directors of IDA on March 10, 2005), paragraph 39, under the Disclosure bullet. 2 COSO published a report in 1992, “Internal Controls—Integrated Framework,” which is widely referred to by leading financial institutions in the United States and is also seen as a model in many other parts of the world. IDA adopted the COSO framework as its controls methodology in 1995. The framework is an all encompassing process which covers all aspects of internal control of an organization’s operation. It considers not only the evaluation of formal controls, but also informal controls, such as ethics, trust, communication, organization behavior and leadership, and incorporates “top-down” as well as “bottom-up” analysis. 5 exercise, thus breaking “new ground both in creating methodologies … and in building strong factual knowledge about the Bank’s internal controls framework.” 3. Management’s conclusion and findings. Under the COSO internal controls framework,3 management is responsible for establishing and maintaining an effective internal control system. Management has assessed internal control over IDA’s operations, and based on this assessment, and reflecting the work undertaken by management, and attested to by the external auditor, during the FY07 ICFR process, management is of the view that the internal control system over IDA’s operations is adequate to provide reasonable assurance to Senior Management and the Executive Directors that:  IDA’s published financial statements are being prepared reliably;  IDA carries out its operations in compliance with the relevant provisions of its Articles of Agreement and operational policies and procedures including provisions relating to the use of funds for intended purposes; and  they are made aware, in a timely manner, of the extent to which IDA is moving toward effective and efficient use of its resources in meeting its operational objectives. As part of its assessment, however, management identified a number of significant deficiencies which require remedial actions—and effective monitoring of their implementation—in order to strengthen the internal control system over IDA’s operations. These significant deficiencies, described in more detail in Part III below, fall into five main categories: (i) the framework governing investment lending (IL); (ii) risk management and accountability at the entity and project levels; (iii) integration of controls for managing fraud and corruption risk in operations; (iv)) oversight of Procurement (PR) and Financial Management (FM) work related to IDA-financed projects; and (v) the role of IT in risk management and issues relating to processes and controls over AAA. 4. Agreement on overall conclusion and follow-up to this review. Management is pleased to note that at the end of this intensive effort, IEG and IAD have concurred with management’s overall conclusion and findings. As stated by IEG: “evidence presented by management for both the entity- and transactions-level controls gives reasonable assurance—except for weaknesses identified in certain parts of the overall framework—that controls operate effectively… [and subject to identified exceptions] provide Senior Management and the Board with reasonable assurance that the three COSO objectives are being achieved: Reliable financial reporting, compliance with policies and procedures, and the efficiency and effectiveness of operations.” As a result, IEG concluded that, “with some important qualifications, IDA’s internal controls framework operates to a high standard overall, giving reasonable assurance that the controls operate effectively” (emphasis added). Management is also pleased that the senior international Advisory Panel assembled by IEG in connection with this review has opined that the results of this review demonstrate a high level of effectiveness compared to results in other organizations 3 See the COSO report: “Internal Controls—Integrated Framework,” July 1994, page 21. 6 of similar size and complexity but with less international involvement4 (emphasis added). 5. Management, IAD and IEG also agreed on the nature of the issues uncovered and on remedial actions needed to address them, including IEG’s recommendation that controls over the risk of possible fraud and corruption in IDA-supported operations should be addressed on a “broad front” and that implementation of the remedial actions should be closely monitored. Management formulated and began implementation of a comprehensive program of such remedial actions, as described in management’s report and Part III of this Management Response. The key areas being addressed by this program are summarized below and in the five-point action plan set out in Annex 1 to this Management Response:  IL reform, expected to: (a) improve IL instrument efficiency and effectiveness; (b) rationalize and strengthen control framework for IL by adopting a risk-based model to appraisal and supervision of IL operations; and (c) eliminate current backlog of outdated OPs and BPs by creating a principles-based umbrella policy for IL operations.  Strengthen risk management capacity, incentives, and accountability at the project and entity levels by: (a) ensuring appropriate delineation and exercise of responsibilities and accountabilities at management and staff levels, including consequences for failure to report serious issues; (b) aligning incentives and management support and oversight to ensure more timely risk identification, reporting and information-sharing within the task team and between management and staff; (c) launching an annual Integrated Risk Report that would identify overall risks facing the institution, units responsible for managing said risks, including potential gaps and overlaps, and areas needing strengthening; and (d) using the review of QAG to inform a broader review of gaps and overlaps among the existing control units.  Better design and integration of controls for managing fraud and corruption risk into preparation and supervision/monitoring of IDA-supported operations, expected to strengthen the Bank’s ability to detect, deter and address, in partnership with country counterparts, issues of fraud and corruption and form part of the GAC Implementation Plan, discussed with the Board in September 20075 and follow-up actions to the Volcker Panel Report, issued in September 2007,6 and the INT’s Report on the Detailed Implementation Review of the India Health Sector (the “India DIR Report”), issued in December 2007. 7 4 See the IEG Review of IDA Internal Controls: An Evaluation of Management’s Assessment and the IAD Review Report on the Completion of Part II: Final Report on the Effectiveness of IDA Internal Controls for Assuring Reliable Financial Reporting, Compliance with IDA’s Charter and Policies, and Operating Efficiency and Effectiveness, AC2008-0147/CODE2008-0098, December 29, 2008, (the IEG Report), Volume III, Attachment 3, Statement of the Advisory Panel –Section 5. 5 Implementation Plan for Strengthening the World Bank Group Engagement on Governance and Anti-Corruption, OPCS September 28, 2007 (the “GAC Implementation Plan”). 6 Implementing the Recommendations of the Independent Panel Review of the World Bank Group Department of Institutional Integrity, Report of the Working Group, January 23, 2008 (Management’s 7  Tighten fiduciary controls and strengthen quality arrangements in the PR and FM areas, by: (a) enhancing regional and overall monitoring of quality arrangements in the PR and FM areas; (b) raising awareness and achieving better integration of fiduciary staff in task teams, and re-clarifying accountabilities within the task team for PR and fiduciary issues generally and oversight of contract management in particular, including more consistent implementation of PR post-reviews; (c) updating relevant policy and guidelines and mainstreaming risk-based assessment tools, and (d) improving IT systems and data bases function and usage to support FM and PR work.  IT strengthening and improvement of AAA processes and controls, expected to: (a) address password and access issues; (b) improve compliance monitoring through automation of operational processes starting with IL; (c) address current issues with filing and accessibility of operational documents, and (d) simplify and rationalize processes and systems for the wide scope of AAA activities and strengthen compliance with this new improved framework. 6. Management believes that implementation of the agreed remedial actions will make a substantive and critical contribution toward enhancing internal controls over IDA’s and Bank’s operations by strengthening and refocusing internal controls to better address governance and anti- corruption issues, enhance risk identification and management at transaction and entity-levels, and improve effectiveness and efficiency of investment lending, the Bank’s primary lending instrument. Management is committed to implementing the remedial actions swiftly and effectively. 7. Monitoring and reporting on implementation progress. Management also shares IAD’s and IEG’s views on the importance of timely implementation and effective and focused monitoring of these actions. To aid with this effort, the President will be establishing in March, 2009 an Implementation Oversight Panel (IOP) to monitor and periodically report to the President and the Board on the implementation progress achieved.8 Management expects that the first implementation progress report of the IOP would be discussed with the Board in early fall of 2009 prior to the IDA 15 mid-term review. II. Methodology and Differences with IAD and IEG 8. Throughout this assessment management followed the COSO definition of internal control, namely, a process affected by an entity’s board of directors, management and other Response), and Independent Panel Review of the World Bank Group Department of Institutional Integrity, September 13, 2007 (the “Volcker Panel Report”), paragraph 140. 7 Detailed Implementation Review India Health Sector 2006-2007 issued December 19, 2007, Department of Institutional Integrity. (the “India DIR”); and Implementing the Recommendations of the Independent Panel Review of the World Bank Group Department of Institutional Integrity, September 13, 2007, Report of the Working Group January 23, 2008 (Management’s Response to India DIR). 8 The IOP would be co-chaired by an outside party, the CFO, and VP OPCS, with VP CSR, VP EXT, VP LEG, VP CFP, and VP ISG as members, and central control groups (IAD, IEG, and INT) as observers. 8 personnel, designed to provide reasonable assurance regarding the achievement of objectives in three COSO categories: financial reporting, compliance and operations. Management (as did IAD and IEG) also applied the definitions and terminology developed under COSO to identify the seriousness of the issues uncovered during its review, ranging in ascending order from “deficiencies, to significant deficiencies, to material weaknesses.” This has proved challenging when assessing the compliance and operations objectives since at present no precise definitions of these terms exist beyond the area of financial reporting, for which they were developed.9 Therefore, as explained by IEG, when applying these terms “in the case of operational as against financial reporting, there are no such clear yardsticks by which to measure the materiality of a given weakness or set of weaknesses,” requiring each reviewer to exercise “some judgment” (emphasis added). 9. In applying these definitions to the issues uncovered by management, while all three parties agreed on the nature, scope, and the ultimate characterization of most of the issues identified, there was some variation with respect to certain issues: management classified each of the issues identified as a “significant deficiency;” IAD agreed, but noted that these deficiencies, in combination, “could represent a material weakness … unless remediated in a timely manner and effectively monitored” (emphasis added); and IEG classified issues relating to integration of fraud and corruption controls as a “material weakness”10 (emphasis added). Regardless of the ultimate characterization of the issues uncovered, management, IAD and IEG are in agreement that a timely and effective implementation of the remedial actions identified by management will address these issues and strengthen the efficiency and effectiveness of internal controls over IDA’s operations. Management began implementation of these actions and is fully committed to a swift and effective completion of the entire program, including dedication of the resources needed to support it, and to monitoring and reporting on the results achieved through the Implementation Oversight Panel. 10. In assessing the differences between management, IAD, and IEG in characterizing the issues identified, it is important to view these differences in the context of (i) a very limited experience with reviews of this nature, (ii) the rather nascent and somewhat subjective standards 9 Under the relevant financial reporting standards (AS2), a material weakness is defined as “a significant deficiency, or a combination of significant deficiencies, that results in more than a remote likelihood that a material misstatement of the annual or interim financial statements will not be prevented or detected.” A significant deficiency, on the other hand, is defined as “a control deficiency, or a combination of control deficiencies, that adversely affects the company's ability to initiate, authorize, record, process, or report external financial data reliably in accordance with generally accepted accounting principles such that there is more than a remote likelihood that a misstatement of the company's annual or interim financial statements that is more than inconsequential will not be prevented or detected.” 10 As the IEG Report pointed out in paragraph 2.19: “It is important to emphasize key aspects of the context for this finding: First, the finding is based not on evidence of the occurrence of actual F&C (though there is evidence that some F&C has actually occurred) but rather on the fact that there is evident risk of F&C occurring in a significant number of IDA countries. Second, weak governance and F&C risks are part of the challenge of development in many IDA countries and F&C risks affect the work of all donors, not just IDA. Third, this is the first evaluation of its kind for any IFI, which places the Bank out front within the development community. Fourth, the finding of a single material weakness (together with some significant deficiencies) should overall be considered a quite respectable outcome from the first (and very detailed) exercise of its kind for IDA.” (emphasis added) 9 when expanding the COSO methodology beyond financial reporting, including the definitions used and the timeframe on which the final conclusion is based, and (iii) the focus placed by IAD and IEG on results relating to fiduciary controls. In terms of comparative experience, as noted by the senior international Advisory Panel assembled by IEG, the results of this review demonstrate a high level of effectiveness of IDA’s internal controls compared to results in other organizations of similar size and complexity but with less international involvement (with a finding of one material weakness and several significant deficiencies being quite common for the first review of this nature).11 On the applicable standards beyond the area of financial reporting and the time-frame used for the review, it is important to note that IEG’s finding of a material weakness relating to controls over F&C risk in operations reflects a snapshot of the controls in place as of 12 months ago, when most of the diagnostic work was completed, and does not take into account the substantive changes implemented by management in response to the Volcker Panel Report and the India DIR, which have been factored into management’s opinion. 11. During Part I of this exercise, management identified and tested controls in fiduciary areas (including FM, PR and disbursements) and found that 10 out of 50 controls did not pass compliance testing.12 To address this issue, management is undertaking a concerted effort to improve the specific fiduciary controls in question in order to enable a remapping and retesting of their operating effectiveness as soon as possible. Management also plans at the same time to achieve substantial progress regarding the strengthening of specific controls for managing F&C risks in operations. It is discussing with IAD and IEG a detailed program for assessing these controls, including a methodology for retesting and reporting the results and any related evaluation findings to the IOP, Senior Management and the Board. III. Management’s Assessment, Findings and Recommendations 12. This section sets out management’s overall assessment regarding the effectiveness of the current internal control framework over IDA’s operations, followed by more detailed findings of significant deficiencies identified by management and recommended actions to address them.13 11 For example, similar comprehensive reviews conducted in the US using the COSO (Committee of Sponsoring Organizations of the Treadway Commission) framework have identified several material weaknesses in the internal controls of the agencies reviewed, e.g. 8 material weaknesses in the internal controls of US Treasury in 2004, reduced to 4 by 2008, and 4 material weaknesses in USAID in 2007 (2 financial and 2 operational). See US Treasury FY 2008 Performance and Accountability Report; and USAID FY08 Performance and Accountability Report (PAR). Visit Treasury web site: http://www.treas.gov/offices/management/dcfo/accountability-reports/2008-par.shtml and USAID:http://www.usaid.gov/policy/afr08/. 12 During Part I of this exercise management identified and tested 50 controls in fiduciary area (including FM, PR and disbursements). Ten out of the 50 controls tested (or 20%) did not pass compliance testing. A list and description of these 10 controls is attached hereto in Annex 2 and is included in Annex 1, Attachment 2 to the Management Report on Its Review of Internal Controls, Part IB, see Attachment I, Review of IDA Internal Controls: An Evaluation of Management’s Assessment and the IAD Review: Report on the Completion of Part I: Incorporating Compliance Testing of Key Controls (Part IB), AC2007-0044, June 22, 2007. 13 This Section III of this Response substantively reflects the text included in Section III of, and the Attachment to the Management Overall Assessment, set out in Attachment I, Volume III of the IEG 10 A. MANAGEMENT’S OVERALL ASSESSMENT 13. Under the COSO internal controls framework,14 management is responsible for establishing and maintaining an effective internal control system. IDA’s management has assessed internal control over IDA’s operations, and based on this assessment, and reflecting the work undertaken by management, and attested to by the external auditor, during the FY07 ICFR process, management is of the view that the internal control system over IDA’s operations is adequate to provide reasonable assurance to Senior Management and the Executive Directors that:  IDA’s published financial statements are being prepared reliably;  IDA carries out its operations in compliance with the relevant provisions of its Articles of Agreement and operational policies and procedures, including the provisions relating to the use of funds for intended purposes; and  they are made aware, in a timely manner, of the extent to which IDA is moving toward effective and efficient use of its resources in meeting its operational objectives. 14. As part of its assessment, however, management has identified a number of significant deficiencies which require remedial actions—and effective monitoring of their implementation— in order to strengthen the internal control system over IDA’s operations. These significant deficiencies fall into five categories as follows: (a) Issues relating to the current policy and procedural framework for IL: o Process inefficiencies, including a lack of focus on key risks and controls during preparation/design and supervision stages of IL projects resulting from a non- rationalized “one-size-fits all” requirements irrespective of risk. o Over-focus on project preparation at the expense of project implementation and supervision. o Outdated and complex policy framework for IL as reflected in multiple OPs and BPs that apply to IL operations. (b) Risk management and accountability at the entity and project levels: o Inadequate emphasis on risk identification, reporting and information-sharing within the task teams and between staff and management during project supervision, including diffused accountability in this area and for operational quality more generally. Report, revised as needed to ensure consistency between the detailed description of the deficiencies and recommended actions and their summary in the action plan matrix included in Annex 1 to this Response. 14 See the COSO report: “Internal Controls—Integrated Framework,” July 1994, page 21. 11 o Inadequate mechanisms for risk aggregation and timeliness and consistency in monitoring, identifying (and formulating an appropriate response to) systemic risks. (c) Issues relating to integration of controls for managing fraud and corruption risk in IDA-supported operations: o Need for better design and integration of controls for managing fraud and corruption risks (reflecting, inter alia, lessons learned from precedents and INT work) into design/preparation and supervision/monitoring of IDA-supported operations. (d) Issues relating to PR and FM: o Need to strengthen controls in place to ensure consistent follow-up on PR issues by the task teams, including the need for better integration of PR staff in task teams and clarification of accountabilities for procurement issues and decisions. o Inconsistent implementation of PR post-reviews. o Inconsistency in quality arrangements for the documentation of FM supervision and some inconsistency in quality arrangements for oversight and monitoring of FM supervision. (e) Controls Over IT and AAA: o Need to strengthen Information Technology (IT) controls relating to password sharing in SAP, privileged access, and Infrastructure Change Management. o Need to improve timely accessibility of operational documents primarily relating to IL operations. o Need to review and rationalize processes and controls relating to AAA work. 15. A summary of each of the above significant deficiencies identified by management together with recommended remedial actions are set out below. B. FINDINGS AND RECOMMENDATIONS 1. IL Reform to Improve IL Efficiency, Effectiveness and Controls a. Rationalizing IL processes and strengthening controls 16. Transaction-level work conducted by management in Part I (including identification, mapping and testing of IL processes and controls), has identified significant inefficiencies in the current processes and controls that apply to IL, especially during preparation stages (from concept to approval). As reflected in management’s Part I report, IL is currently hampered by inefficient and often duplicative processes and reviews, applied in a “one-size-fits-all” manner, irrespective of the level of risk, borrower capacity, track record, and similar factors. The absence of a risk-based approach to IL processes and controls undermines efficiency and effectiveness of these controls by over-regulating low risk projects while diverting resources and management attention from addressing higher risk situations. Attention to proper risk identification and results monitoring during project supervision is diverted by excessive focus on preparation/appraisal stages. Such 12 focus also contributes to increased project preparation/appraisal costs and delays in bringing projects to the Board for its consideration and approval. 17. Recommended action. In response to these and related findings of this assessment, management has launched a comprehensive effort to reform the investment lending instrument. The proposed reform will also build on and reflect other developments, including the Middle Income Countries (MIC) and Fragile States strategies, the GAC strategy, and the findings of the Volcker Panel Report and the India DIR.15 A concept note on IL reform, which reflects the findings of this review, was discussed with the OVPs in November 2008 and with the Board in February 2009. Integration of a risk-based model into the control framework governing IL is a key aspect of the rationalization of IL processes and controls. The principles of the new IL policy and the new risk-based control framework for IL are expected to:  rationalize and prioritize key controls governing IL;  facilitate appropriate differentiation of processes and requirements based on risk;  set appropriate additional controls, resources, and reviews for higher risk operations/environments;  maximize efficiency and effectiveness of each control through rationalization and work-flow automation; and  strengthen risk identification and monitoring during supervision by, among other things, clarifying roles, responsibility and accountability between IDA and borrowers and within the institution. 18. Management expects to complete the first phase of IL reform, which will address the process and control issues during all stages (including more timely risk identification and reporting during supervision, clarification of roles, responsibilities and accountability within the task team, as well as better integration of fraud and corruption issues in project appraisal and supervision) in June 2009. b. Reforming IL policies 19. During Part I of the assessment, management identified that various OPs/BPs have not been keeping pace with the changes needed and/or introduced on the ground. As the main fiduciary policies (including OP/BP 12.00 related to disbursements and 10.02 related to FM) were issued in March 2007 and major instrument policies (including OP/BP 8.60 related to DPLs and OP/BP 8.00, related to emergency lending) are in place, this issue primarily affects OPs and BPs that apply to appraisal and supervision of IL. Consistent with the scoping memo for Part II, management has assessed (a) how the proposed reform and consolidation of IL policies will help address this issue; and (b) how the current processes for policy revisions could be made more efficient so as to facilitate more timely updates of OPs and BPs. 15 See footnote 5, 6 and 7 for references to the GAC Implementation Plan, the Volcker Panel Report and India DIR. 13 20. Based on these assessments, management is of the view that the proposed reform of IL policies will address most of the outstanding policy updates issues. While this work is important from the Control Activities and Information and Communication points of view, the entity-level review conducted during Part II, including information from the ELC Questionnaires, confirmed that the current status of OPs and BPs does not have a material impact on compliance and operations objectives of IDA as a result of compensatory measures adopted to “fill in” the gaps created by the outdated OPs and BPs. These mitigating measures include issuance of Operational Memoranda and central and corporate guidance to staff. Based on management’s identification and mapping of the current process for policy revisions, management believes that the current backlog in OP/BP updates is due to the prescriptive and micro style and approach to OP/BP drafting and content rather than the process for introducing policy changes that require OP/BP revisions. The principles-based approach to preparation of new OPs and BPs reflected in the more recent policy reforms,16 which focuses the OP and BP statements on core principles and key controls rather than detailed rules and procedures characteristic of the older statements, should help minimize any future backlogs in OP/BP updates. 21. Recommended action. During Part I, IL was identified as the most over-regulated Bank instrument and is subject to approximately 30 of the Bank’s operational policies and procedures, many of which are misaligned with the current practice and evolution of the IL instrument. To address this shortcoming, which is the source of much inefficiency surrounding the current IL instrument, the IL reform effort will focus on consolidating and rationalizing the policies and procedures governing IL by creating a single principles-based “umbrella” policy for IL that would govern IL projects from preparation through completion. Reflecting lessons learned, including findings of this internal control review, the new IL policy would:  replace the rigid “ring-fenced” IL project model with a flexible menu of design, funds flow and financing options to better meet development and funding needs of IDA’s varied clients;  replace the current “one-size-fits-all” requirements with a risk-based approach to selection of IL design options and associated due diligence, processing, and monitoring requirements; and  reflect main principles governing redesigned IL instrument in a new principles- based umbrella policy governing IL from “identification to exit.” 22. Management expects to make a proposal for policy consolidation during the second phase of IL reform, following the completion of (a) Phase I that would address preparation/appraisal process inefficiencies and significant deficiencies in supervision as a matter of priority, and (b) broad consultations inside and outside the Bank to ensure that the new IL model meets and reflects the needs and expectations of the Bank’s clients, partners and shareholders. This second and final stage of IL reform is expected to be completed in June 2010. 16 These include: OP/BP 6.00 related to expenditure eligibility, OP/BP 10.02 related to financial management, OP/BP 12.00 related to disbursement, OP/BP 13.20 related to additional financing and OP/BP 8.00 related to rapid response to crises and emergencies. 14 2. Strengthen Risk Management Capacity, Incentives and Accountability at Project and Institutional Levels a. Risk identification and reporting at the project-level 23. Management’s assessment of entity-level controls has found an issue with timeliness and scope in reporting of risks generally, and in the context of project supervision in particular. Sources for this finding include management’s assessment of responses to the ICFR and ELC Questionnaires, a review of the Risk and Opportunity Workshop (ROW) results, and findings reflected in recent outputs/reports of several central control units, including IAD, QAG and IEG. Given that timely and accurate reporting on risks and project implementation issues is essential to ensure that appropriate and timely decisions and actions are taken to address risks and improve outcomes, this issue merits prompt management attention to formulate and implement an effective remedial action. 24. Recommended action. This finding has pointed to the need to strengthen incentives, including: (a) staff incentives and accountability; (b) management tools and accountability; and, (c) resource reallocation mechanisms. These will promote open and timely identification and reporting of risks. Some management actions are already underway to improve candor and realism in implementation supervision reporting in response, at least in part, to IAD’s, QAG’s and IEG’s findings. These include:  completion of a cross-regional review of ISRs by OPCRX and its input into the Focus on Results—“IDA 14 Results Measurement System and Directions for IDA 15”;  preparation and dissemination of important reference documents,17 including the launching of an e-learning module in FY08 Q4 to assist teams in preparing and updating ISRs;  regular reviews of ISRs by the regions, with guidance from OPCRX, including a guidance on Maintaining a Results Focus in ISRs; and  preparation by each Region of specific guidance for ISR reviews to ensure adequate baselines, and to review quality and realism. 25. In addition, as part of the first phase of IL reform management also expects to introduce in June 2009 measures that would improve timeliness and accuracy of risk identification and information sharing within the task team and between management and staff, including such measures as (i) mechanisms for reallocation of resources to address newly-identified risks; (ii) incentives and management actions to encourage accurate and timely reporting of risks and issues in operations, including project performance; and (iii) a more explicit focus on risk monitoring and reporting in the revised ISR template and requirements. 17 For example, the OPCS Results Agenda Website includes a Quick Reference Guide on Results in Operations; the IDA RMS Fact Sheet; Maintaining a Results Focus in ISRs; Baselines in ISRs; and Results and Harmonization in Bank Operations (including supervision). In addition, the results terminology is periodically updated and posted on the Website. 15 b. Accountability and compliance incentives and instruments 26. Management’s assessment of entity-level controls has identified that accountability, compliance and internal control responsibilities are not adequately reflected in performance management tools such as OPEs or promotion criteria. Nor does there seem to be a strong link between performance evaluations and individual accountability: for example, job descriptions and performance evaluations do not usually contain specific references to internal control related duties, responsibilities, and accountability. Concerns about the adequacy of current incentives to promote accountability, compliance and quality were also reflected in several units’ responses to the ELC and ICFR questionnaires. Several responders expressed a concern that individual staff and units are primarily rewarded for specific results such as project delivery to the Board and project disbursements and completion, with less attention being paid to the method or quality of these achievements. Potential issues with accountability arrangements were also reflected in the Volcker Panel Report (referenced in footnote 6) and the India DIR (referenced in footnote 7). 27. Some of these concerns are being addressed by ongoing activities in the accountability and ethics areas, such as emphasis on ethics and controls in the orientation program provided for all new Bank hires and implementation of appropriate disciplinary action in response to departures from approved policies and procedures or violations of the code of conduct pursuant to Staff Rule 8.01. In June 2008 the Bank has issued a strengthened, dedicated whistle-blowing policy and amended the Principles of Staff Employment to enhance its current handling of whistle-blowing reports and claims of whistle-blowing retaliation. The new policy, principle and staff rule puts the Bank Group at the leading edge of whistle-blowing protections amongst international organizations, consistent with the highest standards of good governance and the on- going efforts of the Bank to safeguard its integrity and effectiveness. 28. Management has long-recognized the challenge of managing performance in the Bank— which is key for promoting individual and professional accountability. This has been emphasized most recently in the Report of the Performance Management Working Group published in July 2007. Prompted by the Report, HRS launched HR Insight, a new website dedicated to sharing HR-related data, information and research with Bank Group staff. Efforts have also been made to exercise more differentiation in the recent SRI awards, to ensure that high performers are rewarded. 29. Recommended action. Given the importance of individual and professional accountability to the effectiveness and efficiency of an internal control system, Senior Management is undertaking a corporate review of responsibilities, accountabilities and quality oversight. This review will include several components, including (i) existing oversight and quality assurance arrangements to identify shortfalls in quality, including the roles and methodologies of the institutional mechanisms such as QAG, IEG, Regional Quality teams, (ii) the existing quality and accountability arrangements for operations, including roles and responsibilities within the task team and processes and controls in place to ensure proper oversight, (iii) development by HRS of a comprehensive roadmap/strategy for enhancing performance based on the diagnosis of the root causes of dissatisfaction with current performance management, developed by the Performance Management Working Group, and (iv) continuation of HRS efforts to enhance the HR Insight website to increase further the transparency of HR-related data. Components of this work will be reflected in: the QAG review, expected to be completed in March 2009, the first phase of IL, expected to be completed in June 16 2009, including in particular the integration and clarification of responsibilities within the task team and strengthening of the supervision model. c. Risk aggregation, monitoring and reporting at entity-level 30. Management has identified that while important review and evaluation work is being carried out throughout the institution by the central control units (including IAD, IEG, INT, QAG, CSR, and the Inspection Panel), risk and control activities remain fragmented, resulting in duplication of effort and potential gaps in risk coverage. IDA also currently lacks a consistent mechanism for aggregating and prioritizing the results of this work. This finding is based on management’s review of the terms of reference, key outputs (including recommendations) of the central control units, assessment of responses to the ELC Questionnaire, and relevant observations set out in the Volcker Panel Report (referenced in footnote 6) and the India DIR (referenced in footnote 7). 31. Recommended action. This finding merits strengthening of the existing management tools for integrated risk management. To this end, management (through CSR) intends to move toward an annual Integrated Risk Report intended to: (a) describe overall risks facing the institution; (b) identify units responsible for management and oversight of risks identified; (c) assess potential gaps and overlaps; (d) develop a dashboard of risk findings from the various assessment activities; and (e) over time, assess the quality and consistency of the processes in place. 32. In addition, a more in-depth review of the central control units may be merited, including complementarities in their respective methodologies, terms of reference, outputs and current mechanisms for information-sharing and consolidation of their respective findings and recommendations. In management’s view, such review should:  identify ways to rationalize and improve effectiveness and efficiency of checks and balances provided by the central control units;  explore various mechanisms for better risk aggregation and assessment/review rationalization, including mechanisms in place to validate and prioritize key risks for monitoring and testing as well as defining opportunities for improvements to controls and management activities; and  assess mechanisms for linking systemic risks identified from time to time with objectives, initiatives, and business processes, opportunities for alignment and coordination across the institution. 33. The first Integrated Risk Report is expected to be completed in October 2009 for FY 09. Management is also finalizing a review of QAG, which could inform a broader review of central control units described above. 17 3. Strengthening Controls for Managing F&C Risk at Country and Project Levels a. Managing F&C risk: a phased approach 34. IDA’s anti-fraud and corruption strategy can be divided into three phases: Phase I, which focused on setting out the intellectual case that good governance and an attack on corruption must be key parts of efforts to sustain economic growth and alleviate poverty,18 Phase II, which focused on formulating an appropriate anti-corruption strategy and its adoption by Senior Management and the Board; and Phase III, which focuses on the implementation of the anti- corruption strategy through its integration into the Bank Group’s (including IDA’s) daily operations. Having been the leader in making an intellectual case against corruption (as noted in the Volcker Panel Report discussed below), and having demonstrated the depth and breadth of the management and Board commitment to this agenda through the adoption of the GAC strategy,19 IDA has actively moved into Phase III of this work. 35. Phase III builds on and reflects: (i) the firm commitment and actions of IDA’s Senior Management, starting with the President, to make its fight against fraud and corruption a core element of IDA’s operations; (ii) diagnostic and assessment work performed or commissioned by management to identify areas where the controls for managing F&C risks need to be strengthened; (iii) ongoing efforts and experience with strengthening country client institutions and systems that are essential for effective and efficient management and utilization of public resources, including safeguarding these resources from fraud and corruption; and (iv) roll-out and implementation of specific actions to strengthen controls and improve effectiveness of IDA’s efforts in managing F&C risks that reflect the work and recommendations flowing from the work under items (i)-(iii). As summarized below, much progress has been achieved in this area particularly over the past 6-12 months, putting the Bank well ahead of other institutions dealing with these complex and challenging issues. Management is committed to deepen and broaden these efforts to: (i) continue building and strengthening mechanisms to monitor and manage F&C risks in IDA-supported operations, including expanded Bank-wide staff (and particularly TTL) training, and (ii) refine and improve F&C controls as more knowledge is gained about the weaknesses that still exist (which needs to be a continuous effort to reflect on lessons learned and a growing body of experience). b. Recent actions to strengthen controls over F&C risk 36. High-level commitment and actions. As the President stated in his recent address to staff during the Integrity Day on December 3, 2008, the Bank’s commitment to integrity is at the core of all it does. The Bank, at its highest levels, is not just making the case but is actively implementing the anti-corruption agenda, as evidenced by: (i) the swift actions it has taken and specific measures it has put in place over the past 11 months in response to the Volcker Panel Report (discussed below); (ii) Senior Management hands-on approach and involvement in decisions relating to delays or suspension of operations affected by fraud and corruption issues; and (iii) detailed and candid coverage of fraud and corruption risks in Bank documents, including CASs, project documents, and periodic updates by management to the Board on specific actions and progress achieved in this area. 18 See the Volcker Panel Report, paragraph 140, footnote 6 above. 19 See the GAC Implementation Plan, footnote 5 above. 18 37. Diagnostic work. The Volcker Panel, India DIR, and this IDA internal controls exercise have all served as key diagnostic tools to assess the status of controls over management of F&C risks in operations and recommend areas for improvement. To maximize effectiveness of its implementation efforts, including identification of areas for improvement, in February 2007,20 the World Bank President, in consultation with the Board, requested an independent panel of experts (the Volcker Panel) to review the operations of INT, the Bank’s central control unit responsible for, among other things, investigating allegations of fraud and corruption in operations financed by IDA and other members of the World Bank Group. The findings and recommendations of the Volcker Panel, as set out in the Volcker Panel Report issued in September 2007, identified that much remained to be done in mainstreaming fraud and corruption issues into IDA’s daily operations and has included 18 specific recommendations to help with this effort. Additional concerns about the effectiveness and robustness of controls over fraud and corruption in Bank-supported operations also were raised in the India DIR, issued by INT in December 2007 (referenced in footnote 7). The issues identified in these reports were consistent with management’s findings resulting from its assessment of entity and transaction level controls. Key among these were findings relating to the need to: (i) better integrate INT and the results of its work into IDA’s operations in order to improve management of F&C risks that may arise in the context of IDA-funded projects; (ii) specifically embed fraud and corruption risk assessment in many of the existing risk assessment processes; (iii) undertake assessment of F&C risks in a specific and systematic way throughout the institution; (iv) address more consistently as part of program and project design at a portfolio and project level, the vulnerabilities to fraud and corruption in countries with systemic corruption; (v) focus on incentives that would lead to consistent monitoring, reporting, and timely response to, implementation risks generally, and fraud and corruption issues in particular during project supervision; and (vi) develop new and/or improve existing tools and methodologies in all areas (project design, supervision and evaluation) using lessons learned, including the work of INT, in order to ensure appropriate flagging and treatment of fraud and corruption issues. 38. Experience and tools for country-level work. IDA has much experience with conducting country-level ESW work that helps assess and identify systemic issues that could undermine the country’s development efforts due to weaknesses in the country’s systems, institutions, and internal controls frameworks responsible for management and utilization of public funds. Such work is reflected in fiduciary ESW (in particular CPARs and/or joint fiduciary assessments i.e. combined CPAR/CFAAs which have become the quasi norm in the last 4/5 years), which provides the basis for assessing country systems and defining mitigation measures in DPLs and PRSCs as well as assess fiduciary risks at the country level that is an important input into CASs. In addition, many projects that address public sector management have included PR and FM components, several grants (e.g. IDF) have focused on fiduciary strengthening, and close coordination with other donors as part of the harmonization efforts has helped address the capacity-building issues especially in fiduciary areas. This is monitored by the number of completed CPARs and the various Bank initiatives. Much work is being done to strengthen country systems under the country system pilot in the procurement area, which focuses on assessing, strengthening and building on country’s procurement systems. Similar efforts are also underway in the FM area, as reflected in increasing number of projects relying on a country’s own FM systems and a growing number of DPLs (including PRSCs) that reflect assessments and strengthening measures in the area of public financial management. In this regard, it is also important to note the ongoing PEFA Program (with Phase III agreed with the EU, IMF and other bi- laterals in June 2008), which enhances IDA’s ability to strengthen country systems through the use of a 20 See the Volcker Panel Report, footnote 6 above. 19 common results measurement framework and joint fiduciary diagnostic work (including with PREM and Procurement). Finally, strengthening and building country systems in these key areas has been, and increasingly is, an important focus of the Bank’s institution-building, governance and fiduciary support work in its client countries. Indeed, over 50% of DPLs in FY07 (up from an average of 24% in 1990-94) included conditionality on public sector governance, including in the areas of procurement, financial management, and budget transparency. While it is not a new area of emphasis, the importance of this work under the GAC strategy, combined with the work underway to implement the recently approved country systems pilot in procurement area, provide an added emphasis to the Bank’s efforts to strengthen the effectiveness, efficiency, reliability and transparency of control systems and institutions in client countries with the help of all of the Bank’s existing tools and instruments. 39. Management actions to strengthen controls for managing F&C risks at country and project level. Over the past 12 months, management has undertaken assertive and concrete actions in response to findings and recommendations made in the context of the primary diagnostic tools, and particularly the Volcker Panel Report, India DIR and this IDA internal controls assessment. The progress and results achieved to date with implementation of the GAC strategy are set out in the recent progress report on GAC implementation, discussed with the Board in October 2008.21 Moreover, the overlap between the issues raised in the context of the India DIR, management’s actions designed in response to the Volcker Panel Report, and the findings of this IDA internal controls assessment, has greatly contributed to management’s ability to better define and begin implementation of a concrete plan of systemic Bank-wide actions for heightening the Bank’s effectiveness in the governance and anti- corruption area. These actions have included development and roll-out over the past 6-12 months of the following specific tools for better management of the F&C risk at both country and project levels: 1. Dissemination of lessons learned, including:  distillation of lessons learned from INT work by its Preventive Services Unit (PSU), designed as a tool which could be used for didactic purposes throughout the institution;  delivery of Bank-wide learning events by OPCS, SAR, and INT to disseminate lessons learned from India DIR;  training seminars conducted by the PSU Bank-wide; and  PSU collaboration with specific project teams to help define governance, accountability and anti-corruption plans at the project level. 2. Strengthening project supervision:  compiling and disseminating good practices for better management of F&C risks, including 3rd party supervision and “smart” disseminating through the work of the “GAC in Projects “network. 21 Strengthening the World Bank Group Engagement on Governance and Anti-Corruption, October 21, 2008 (the GAC Progress Report). 20 3. Specific measures focused on F&C risk management in procurement area:  development and roll-out of PR Risk Model/Risk Management Tool that specifically focuses on F&C issues, scheduled for piloting in the spring of 2009, which draws on the F&C red flags identified by INT and the regions;  deepening of the cooperation between the Procurement Anchor in OPCS and INT, including (i) signing in July 2008 of an INT/OPCPR Memorandum of Understanding on the Prevention of Fraud and Corruption in Procurement, which sets out a framework for facilitating a joint approach to preventing F&C in operations; and (ii) agreement on a joint INT/OPCPR work program relating to prevention of F&C in operations; and  improvement of the procurement complaints data base (in terms of completeness, operations and usage) which is an important source for identifying potential F&C issues. 4. Specific tools for addressing fraud and corruption at project level:  development of GAC in project guidelines by “GAC in Projects” team under the GAC Implementation Plan (these have been discussed with teams across the Bank and are being finalized for issuance in March 2009); and  development, posting, and ongoing updating of lessons learned and best practices on GAC in projects on the “GAC in Projects” website, supplemented by peer learning and other training events in this area across the Bank. 5. Strengthening financial audits:  development of standard terms of reference to widen financial audits to cover performance issues and procurement. 6. Enhancing tools for monitoring procurement in the health area, including:  specific assessment and Bank-wide discussion of the Bank procurement procedures for pharmaceuticals; and  enhancing training on managing F&C risks in the health sector through such modules as the session developed and delivered to HD staff during the HD week in the fall of 2008 on “Governance and Accountability: Issues, Diagnostics, and Implementation Tools for Health.” 7. Strengthening accountability:  clarifying and re-enforcing the accountability of managers on all sides of the matrix with respect to their respective areas of responsibility. 40. Specific actions in response to the Volcker Panel Report. Management also took swift and proactive actions in response to the recommendations in the Volcker Panel Report, with 21 actions already implemented on 16 of the 18 recommendations (with the remaining two well underway). These actions have gone a long way to transforming INT from a segregated and rather insular function to a core part of the Bank in its fight against corruption. INT has rapidly moved toward the core of operations, without, however, losing its independence as one of the central control units that forms an essential part of the Bank’s internal control system. Key among these transformative actions are:  elevation of the INT head to a VP level;  the establishment of a PSU within INT, which has as its major role the dissemination of the results of INT’s investigative work, and collaboration with operations to build these lessons into project design;  revision of INT’s disclosure policy relating to its work, thus making its findings and lessons that can be drawn from it much more accessible;  the establishment of a very close relationship and partnership between INT and the operational complex through, in part, its close partnership with OPCS and the regions;  inclusion of INT in Bank-wide Senior Management dialogue and reviews through INT participation in Operations Committee reviews and OVP meetings;  INT assistance to task teams that seek its advice, and increased training and learning on fraud and corruption issues carried out across the Bank in close partnership between INT, OPCS, PREM, and the regions; and  the establishment of the Independent Advisory Board which is an important tool for confirming and building confidence relating to fairness and effectiveness of INT work. 41. The new INT strategy, which was discussed with the Audit Committee in February 2009, also emphasizes this transformation and charts a road ahead to maintaining and deepening the role of INT as a core part of the Bank while maintaining its independence. 42. Actions and examples of country-level and project work. As management also reported in its GAC Implementation Progress Report, discussed with the Board in October 2008 (referenced in footnote 5), much work has been done to mainstream the GAC agenda at the country level. Specifically, 27 countries are now participating in the CGAC program, which allocated specific additional resources of $100,000 per participating country to identify, deepen, systematize and mainstream engagement on governance and anti-corruption at the country level. The CGAC initiative has resulted in development and refinement of systemic diagnostics of governance challenges and reflection of these issues in CAS for several countries. As indicated in Table 1 of the GAC Progress Report, GAC issues were adequately integrated in 64% of CASs discussed with the Board in FY 2007/08, with 32% (a jump from 19% compared to CASs discussed in FY 2006) of the CASs rated “high” in terms of providing a diagnosis of governance conditions and corruption risks. Indeed, engagement on governance and anti-corruption issues has increasingly become a part of the process for CAS design and preparation and its content. In countries where corruption issues have 22 surfaced in the context of Bank projects, there have been serious and concerted responses at the CAS level. Examples include the CAS update in Kenya, following the DIR for health and education, and the recent Indonesia CAS which has GAC as its centerpiece. Recent examples of CASs that specifically address GAC issues also include the India CAS and the Zambia CAS, which are particularly striking in the candor and depth in how these key issues are addressed. In many other countries which have not yet gone through the CGAC process, GAC issues also feature prominently in the CAS. In addition, issues of governance and fraud and corruption are increasingly permeating project design and are reflected frankly and comprehensively in project documents, including in specific Governance and Accountability Action Plans (GAAP) Annexes (see for example the Water and Sanitation Service Improvement Project in Kenya (P096367); Federal Roads Development Project in Nigeria (P090135); National Vector Born Disease Control and Polio Eradication Support Project in India (P094360); ID-BOS KITA Project in Indonesia (P107661); Second Health Sector Support Program in Cambodia (P102284). c. Recommended further actions to strengthen F&C controls 43. While the efforts described above have strengthened IDA’s controls for identifying and effectively managing F&C risks in operations, more needs to be done to strengthen the controls for managing F&C risk on a “broad front” in order to “expand staff skills and broaden behavioral change in order to deepen, systematize and mainstream good practices across all of the Bank Group’s work.”22 To this end, management is committed to implementing additional corrective actions outlined below: 1. Clarifying responsibilities and accountabilities for addressing F&C through:  adopting of the new INT strategy (discussed with the Audit Committee in February 2009);  establishing appropriate protocols of cooperation between INT and the Regions on handling allegations of F&C;  reforming processing and supervision of IL operations, including specific focus on addressing F&C risk during project appraisal and supervision, as part of Phase I of IL reform, expected to be completed by FY09-Q4; and  reviewing staff incentives (performance reviews, promotions, rewards, and visibility) to ensure that they are aligned with the anti-corruption agenda through discussions at an MD-chaired GAC Governance Council (ongoing). 2. Deepening, systematizing and mainstreaming tools for better management of F&C risk in operations through the work of the GAC in Projects Working Group, supported by OPCS, including:  development and launch by OPCFM of the GAC Audit and Assurance Toolkit, designed to help task teams enhance the effectiveness of financial audit and provide guidance on other types of audit and assurance engagements that focus on fraud and corruption risks, such as technical and contract audit; 22 The Volcker Panel Report, page 2, footnote 6 above. 23  preparation of specific guidance on managing fraud and corruption risk for inclusion in the current update of the FM Practices Manual;.  preparation of guidance for FMSs by OPCFM on better identification and management of F&C risk through smart project design (to be supported by web–based knowledge sharing tools);  preparation for issuance in FY09-Q3 of GAC good practices at the project level, entitled “Dealing with Governance and Corruption Risks in Project Lending: Emerging Good Practices” (draft completed and circulated for comment)”, which would: (a) provide task teams with a common conceptual framework for understanding GAC issues; (b) highlight some of the key lessons learned over the past several years and provides examples of emerging good practices at the sector and project level; and (c) indicate areas where further work is required in order to fully support task teams;  development of a comprehensive training program for task teams that would cover 4 major components of the GAC agenda: (a) the CGAC initiative at the country level; (b) assessing governance and corruption risks at the sector and project level; (c) mitigating project risks through “smart project design” and the development of project-specific Anti-Corruption Action Plans; and (d) the supervision of “high corruption risk” projects;  completion of an initial stock-taking of AAA and investment lending operations with significant GAC components and its conversion to a searchable database that can be accessed by Task Teams to provide examples of innovative approaches to risk assessment and risk mitigation;  preparation of "case studies and good practice notes” that illustrate and elaborate on the tools and approaches being developed to improve governance and reduce corruption at the sector and project level (to be delivered through training sessions and disseminated as a publications series and on the new GAC in Projects web portal);  establishment of a GAC in Projects Peer Learning Network , with a rapidly growing “community of practitioners” that will be supported by the interactive web portal under preparation;  identification of Practice Leaders at the regional and sector levels and establishment of full-time focal points (EAP and SAR) or “on demand” Advisory Units (ECA and INF); and  building on the progress achieved in developing and testing “smart” project design and more effective and more appropriately resourced project supervision, which reflect lessons learned, including systemic issues drawn from INT investigations and DIRs and creating a more effective risk management framework to help prevent, deter, detect and address fraud and corruption, which are to be reflected in the new approaches to project design/appraisal and supervision to be developed as part of IL reform. 24 3. Other initiatives to strengthen controls for managing F&C risk:  preparation and monitoring (with OPCS support) specific action plans for following up on INT reports (a small team for the follow-up on post-INT report action plans has been established in OPCS with effect from FY2009);  at project-level, inclusion of the F&C risk among categories of risks to be assessed during project appraisal (and reported in the PAD) and monitored and reported on during project supervision (as reflected in the revised ISR template);  at the entity-level, inclusion of the F&C risk among a list of specific risks facing the institution in the new annual Integrated Risk Report, mentioned in paragraphs 31 and 33 above, which will replace the current Risk Scan, with the first such Report expected in October 2009; and  implementation of actions in the procurement area focused on addressing the F&C risk in operations (reflected in paragraph 45 below). 4. Strengthening Procurement and Financial Management 44. Procurement. During Part II management assessed quality arrangements over PR work relating to project appraisal and supervision and has identified the need to strengthen quality arrangements, business processes and adequacy of documentation relating to PR during project preparation/appraisal and supervision stages. This review found that (a) quality assurance arrangements for procurement are in place and are generally sound and (b) the regional variances identified are in line with the high degree of decentralization and broad mandate of the Regional Procurement Managers (RPMs). However, management’s review also identified two issues that need to be addressed: (a) adequacy of controls in place to ensure consistent follow-up on PR issues by the task teams, including the need for better integration of PR staff in task teams and clarification of accountabilities for procurement issues and decisions; and (b) consistency in implementation of post-reviews. In management’s view, these issues, combined with the findings of Part I of this exercise, rise to a level of a significant deficiency in PR area, which requires implementation of monitorable corrective actions. 45. Recommended action. To address the issues identified, management recommended a plan of corrective actions, some of which have been implemented already, while others are underway. Actions in place: 1. Improving controls and quality of complaints data base:  as of FY07, the procurement complaints database has been made a key control and management tool, and enhancements to the database were introduced, including (i) better controls: automatic alerts and reports for pending complaint cases, and required RPM clearance to close cases: (ii) improved and more detailed information: updated pick lists with new attributes (e.g. “nature of complaint” and “resolution of complaint”) and tracking complaints related to late 25 payments; and (iii) improved database management: strengthening of OPCPR monitoring of complaint follow-up. 2. Strengthening PR/INT cooperation on management of F&C risks in operations:  an INT/OPCPR Memorandum of Understanding on the Prevention of Fraud and Corruption in Procurement was signed in July 2008; and  a joint INT/OPCPR guide/pamphlet on identification and handling of red flags during the project cycle was issued in December 2008. 3. Strengthening of procurement post reviews:  OPCPR in collaboration with the RPMs has developed a single PPR system for (i) centrally filing PPRs uploaded by RPM offices and (ii) rating the findings in terms of procurement systems, procurement procedures, and contract administration against the four risk levels (low, moderate, substantial, high). This system is accessible through the Operations Portal and already being used in several projects. The mandatory roll out is planned after the announcement of the updated PPR/IPR guidance note which is pending the approval by the PSB. Actions underway: 1. Full Integration of PR staff and tasks in project teams (by December 09):  a mechanism for early and full integration of PR staff in the project teams and of PR tasks during the project cycle is being developed;  new instructions and guidance are under preparation to ensure full understanding by all staff of the appropriate sharing of responsibility for key PR decisions at preparation and implementation stages between TTL and PR staff and between Sector Manager and RPM;  Procurement Certification system is being enhanced with a view to increase awareness and importance given to procurement work; and  guidance is being prepared to clarify criteria for assigning PR ratings for the ISR, including a mandatory process to be followed for making any revisions to such rating by sector staff. 2. Updating procurement policy and procedures (by March 2009):  update of OP/BP 11.00, Procurement, is being finalized to incorporate, inter alia, risk management, handling of fraud and corruption and the already enhanced complaints handling, as well as to revise the matrix of responsibilities and the various clearance thresholds (expected to be issued in March 2009). 3. Bank-wide roll out of a Procurement Post Reviews module:  The module is now being extended to all Regions and will be mandated in FY10. In addition, OPCPR requested the RPMs to identify the outstanding post review actions of the FY07/08 reports that require follow-up. RPMs will be held 26 accountable to follow-up on unresolved post review findings. RPMs will also provide OPCPR a summary of what they have done to meet this requirement by March 2009. 4. Bank-wide roll out of PR risk assessment tool and revised templates (FY09 Q3/Q4):  a PR Risk Model/Risk Management Tool that goes beyond the traditional capacity assessment template and aims at defining and tracking risk mitigation measures based on an enhanced risk assessment including inter alia F&C issues has been developed. The tool is being finalized and is scheduled to be piloted in the spring of 2009, in particular under the procurement Use of Country Systems piloting program; it is expected to be made mandatory for Bank-wide use later in 2009; and  the PAD Procurement Annex template is being revised as part of the first phase of IL reform to reflect new initiatives, including risk-based procurement assessments that include inter alia fraud and corruption, and to reflect the work on risk-based approach to processing of IL operations. 5. Strengthening OPCPR/PSB Roles:  Review, by June 2009, the roles of the Procurement Sector Board (PSB) and OPCPR with a view of: (a) expanding them to identify areas that may merit harmonization of regional practices; (b) strengthening the advisory role of the PSB to respond to the Regions’ needs; and (c) monitoring regional fiduciary compliance and quality. 6. Further improvements to controls and quality of complaints data base:  The Procurement Sector Board has approved a few additional features that will be developed during FY09, including (a) the centralization of the reopening of closed cases; and (b) automated case reporting to INT and follow-up handling. 7. Strengthening PR/INT cooperation on management of F&C risks in operations:  A joint protocol to provide guidance as to the roles of, and the interactions between, operational staff, regional management and INT regarding the reporting of allegations of fraud and corruption and handling of requests for no-objections and post-investigation will be issued after vetting by the OS panel in March 2009. 46. Financial Management. In FM, management found that while Quality Assurance (QA) arrangements have been put in place to oversee FM arrangements for the use of IDA financing, the quality and documentation of regional QA arrangements is inconsistent and does not fully comply with the FM Practices Manual (FMPM), particularly during project implementation. Variations in Regional QA exist, and while many of these make sense, there is a need to ensure consistency in the quality of FM work, as defined in the FMPM. Management’s assessment identified, however, issues in three specific areas meriting remediation: The first issue relates to documentation of review of audited financial statements by an FM specialist and management oversight of Audit Report Compliance Systems (ARCS) data given that in some Regions, it was 27 not possible to verify regular FMS review of audit reports because ARCS data entry was considerably out of date. The second issues relates to documentation of FM supervision, with some inconstancy in quality arrangements for the documentation of FM supervision work, including supervision planning, supervision reporting, and follow-up on FM action items. Some inconsistencies also exist in quality arrangements to support the filing of FM documents, making it difficult to validate that FM work has been undertaken. The third issues relates to the timing of quality interventions during supervision and identification and monitoring of risky projects, with some inconsistency and gaps in quality arrangements for Regional FM managers’ oversight and monitoring of FM work during project supervision. For example, some Regions do not have adequate systematic monitoring of interim financial reports, which could result in untimely identification of FM issues and/or delayed implementation of corrective actions. 47. Recommended action. The FM Sector Board formulated and began implementation of actions needed to strengthen quality arrangements in FM work, which include:  completion of Phase I of the Joint CSR/OPCS Evaluation process (Phase II has been launched in January 2009);  adoption of the RAPMAN/PRIMA system by all regions;  centralized monitoring of the ARCS audit tracking system by OPCFM and a rapid reduction in the number of outstanding audit reports;  actions to strengthen specific F&C controls in the FM area: o issuance in June 2007, of an FM Approach Paper to GAC;23 o establishment of an FM GAC Working Group to support the development of good practices, guidance and training for FM staff; o issuance of an Audit and Assurance Toolkit; o preparation and circulation to the FM Sector Board of FM Guidance on dealing with fraud and corruption in project design; o development of guidance on enhanced project supervision and FM "red flags", supported by web-based tools and guidance, including a database of projects featuring FM anti-corruption features; and o delivering training on GAC to the FM community, including the training provided during the 2008 Fiduciary Forum;  additional actions completed or underway to address deficiencies identified in the FM area including: o review and update of the FM Practices Manual, which serves as the main operational guide for FM staff, with the revised FMPM expected to be issued in 23 Financial Management Sector: Approach to Governance and Anticorruption, Financial Management Sector Board, June 8, 2007. 28 June 2009 (findings of this review will be incorporated into the final methodology for the Joint Evaluation); o consolidation and updated by the regions of the regional QA arrangements (including QA-related information on regional websites), to be followed by introduction of further updates as necessary; o aligning the QA arrangements with the issuance of an updated FMPM (exposure draft) in June 2009; o progress in updating the ARCS by the regions for all actions related to audits that were due in FY05 through FY07 and in clearing backlogs relating to earlier years; o entry of baseline data by the regions on interim financial reports into PRIMA (completed in October 31, 2008); and o integration by December 2009 of IT systems tracking project performance in FM with other Bank systems to ensure FM is fully reflected in all assessments of project performance. 5. Deficiencies in IT and AAA Areas a. IT issues identified as part of the ICFR exercise 48. During the ICFR review three significant deficiencies were identified in connection with IT-related issues. The first related to password policy breaches in SAP, as identified by IAD in its FY07 “Identify and Access Management” audit. The audit identified that SAP passwords are widely shared by Bank staff which may have resulted in some unauthorized expenses in the financial statements. The second related to the scope of privileged access and monitoring of activities in privileged accounts which were deemed to need rationalization and strengthening to limit risks of misuse. The third related to change management controls associated with Infrastructure Change Management and the need to ensure consistency of application of these controls. 49. Recommended action. In response to these findings, management has recommended and as part of the ICFR program is carrying out corrective actions to (a) address the password sharing issue, (b) strengthen controls around information security to rationalize and further limit privileged access to system applications and monitor changes made by IT staff using such privileged accounts; and (c) strengthen controls around Infrastructure Change Management to ensure that change management controls are applied consistently and exceptions are reviewed and authorized by appropriate authority. b. Timely accessibility of operational documents 50. During “compliance testing” conducted by management in Part I, management identified a problem with accessibility of operational documents. The documents requested by management to support the processes and controls identified were not easily accessible. Although after an extensive effort management was able to obtain 93% of the documents requested, this exercise identified the need to strengthen document retention practices and improve accessibility of operational documents. During Part II management confirmed that this 29 issue is indeed linked to both, the Control Environment and Information and Communication components of COSO at the entity level, as was suggested by IEG in its Part IB report. Consistent with IEG’s preliminary recommendation, the review conducted by management also concluded that improved IT systems to support IDA’s operations would have to be an integral part of the solution to these issues. 51. Recommended action. Management has already begun to address the document retention and accessibility issue by setting up a Task Force in FY07 to look at retention, filing and accessibility of operational documents and come up with recommendations for improvement. Working closely with ISG, the Task Force has made a proposal for addressing the issues identified through automation of key controls for IDA’s primary operational tools (CAS, DPL, IL) into the Bank’s documentation system. Under the proposal, this work would commence with automation and integration of IL processes and controls, where most issues with documents accessibility occurred. The work on the automation system will proceed in close coordination with the first phase of IL reform and will be part of the Operations and Knowledge Management Systems Program (OKSP, previously known as the Enterprise Content Data Management (ECDM)). The new system for IL will be piloted in FY09 and is expected to be fully in place by the end of FY10, in close coordination, and forming a part of, the roll-out of IL reform. c. AAA processes and compliance 52. As mentioned in the Scoping Memo for Part II, and following the request of IEG and the Audit Committee to include AAA within the scope of this exercise, during Part II, management (a) identified and mapped current processes that apply to the main AAA product line, Economic and Sector Work (ESW) reports, (b) conducted a walkthrough of the mapped processes, and (c) tested compliance with such controls using a sample of randomly selected ESW reports completed and delivered to clients during FY07, in a process similar to that performed for IL, DPL, and CAS product lines during Part I. The focus on ESW (and for sampling purposes, on ESW reports) was in large part due to the fact that ESW funding has traditionally absorbed the major share of AAA funds and ESW reports account for over 80% of total ESW spending.24 The process mapping exercise will inform management’s larger-scale review of AAA covering both IBRD and IDA, currently underway. 53. In ESW Reports randomly selected for compliance testing, non-compliance with controls was identified in 25% of control steps involved in the ESW process. It should be noted, however, that roughly half of the instances of non-compliance were due to mismatches between dates recorded in SAP and the actual dates of application of such controls (e.g. the dates of the Concept Review meetings of some of the transactions differed from the dates for such meetings recorded in SAP). 54. Recommended action. Management is undertaking a broad review of the processes and controls, including systems and monitoring, that apply to AAA in order to simplify and strengthen them where needed, and ensure they are updated to take into account the wide variety of AAA currently carried out by the Bank. This review will also address the compliance 24 The sample was drawn from the universe of ESW Reports completed as of September 16, 2007, and whose cost exceeded $100,000. These types of ESW Reports accounted for 90% of ESW Reports completed. ESW as a whole has consumed approximately 67% of the AAA budget. 30 weaknesses observed, along with other issues that have been raised by IEG and QAG in recent related reviews of AAA. Management expects to complete this review and discuss the recommended changes with CODE in FY10. 31 32 Annex 1. Management 5-Point Action Plan Summary of Corrective action (with concordance to more detailed recommendations in Management’s Problem identified Response) Timeline On point Inefficiencies and gaps in control I. Improve efficiency, effectiveness and controls for IL framework governing investment (paras 17, 18, 21 and 22 of Management Response) lending (IL), in particular (i) non-rationalized “one-size-fits- 1. Match the demands of the process to the level of risk and focus June 2009 OPCS all” requirements irrespective resources on higher risk projects. of risks, 2. Strengthen IL supervision by increasing resources, support and June 2009 OPCS (ii) over-focus on project management oversight of project implementation. preparation at the expense of 3. Tailor design and financing options under the IL instrument more June 2010 OPCS implementation, and closely to the needs, capacity and risk profile of clients. (iii) outdated and complex policy 4. Consolidate multiple rules into clear key principles to inform June 2010 OPCS framework design and processing. II. Strengthen risk management capacity, incentives and accountability at the project and institutional levels (paras 24, 25, 27-29, 31, 32, 33 of Management Response) Diffused management and staff accountability and responsibilities At the project level for operational quality. 5. Review lines of accountability at the management and staff level Launch January MDs, RVPs, Inadequate mechanisms for (including management oversight) to ensure appropriate 2009 to be OPCSs institutional risk identification, delineation and exercise of responsibilities and accountability and completed by June monitoring and management. consequences for failure to report serious issues. 2009 6. Introduce incentives and greater management support and June 2009 (as part OPCS, RVPs oversight and communicate to staff expectations to ensure accurate of IL reform) and timely reporting of risks. At the institutional level 7. Prepare an annual Integrated Risk Report to: a) describe overall First report to be CSR risks facing the institution, (b) identify units responsible for prepared for FY09 management of risks identified, (c) assess potential gaps and (October 2009) 33 Summary of Corrective action (with concordance to more detailed recommendations in Management’s Problem identified Response) Timeline On point overlaps, (d) develop a dashboard of risk findings from assessment activities, and (e) over time, assess the quality and consistency of processes in place 8. Review QAG, to inform a broader assessment of gaps and overlaps March 2009 OPCS, MDs among the existing central control units (IEG, IAD, INT, QAG and Inspection Panel). Inadequate integration of fraud III. Better integrate fraud and corruption prevention into operations and corruption issues (including (paras 36- 42 of Management Response set out actions already in lessons learned from INT work) place; para 43 sets out recommended further actions summarized into daily operations. below) 9. Establish clear responsibilities and accountabilities for addressing June 2009 INT/RVPs fraud and corruption in Bank operations (per new INT strategy). 10. Establish appropriate protocols of cooperation between INT and ongoing INT/RVPs the regions on handling allegations of fraud or corruption. 11. Promote good practices across the Bank Group’s work by:  Intensifying staff training; ongoing INT/OPCS  Increasing management signals on the importance of this issue ongoing MDs/OVPs INT  Ensuring staff incentives (OPEs, promotions, rewards and ongoing HR visibility) are aligned with anti-corruption strategies.  Propagating lessons learned, including through preventive has begun issuing INT services unit within INT. core materials  Issuing guide/pamphlet on identifying and handling red flags done in December INT/OPCS relating to fraud and corruption during the project cycle. 2008 12. Improve tools such as smart project design (drawing on lessons in progress MDs/OPCS learned), and more effective and more appropriately resourced (per GAC /RVPs 34 Summary of Corrective action (with concordance to more detailed recommendations in Management’s Problem identified Response) Timeline On point supervision to help prevent, deter, detect and address fraud and progress report) corruption. 13. Prepare and monitor specific action plans for following up on INT ongoing (small OPCS/RVPs reports. team created in OPCS) 14. Issue OPCS Guidance on addressing GAC issues in projects. March 2009 OPCS IV. Tighten fiduciary controls (paras 45 (for PR) and 47 (for FM) of Management Response) Issues relating to fiduciary controls in the areas of financial In financial management (FM): management and procurement, particularly during project 15. Institute corporate monitoring of quality of FM work in operations. Initiated in FY09 CSR/OPCS implementation. 16. Integrate IT systems tracking project performance in FM with other December 2009 OPCS/ISG Bank systems to ensure FM is fully reflected in all assessments of project performance. 17. Ensure that all records relating to quality arrangements for FM ongoing RVPs/OPCS work, periodic project audits and financial reports submitted by country clients, are maintained and up to date. In procurement (PR): 18. Ensure more consistent follow-up through, e.g., earlier and fuller June 2009 (as part OPCS/RVPs integration of procurement staff in project teams, review of of IL reform) procurement certification system, issuance of guidance for assigning procurement ratings, and establishment of clear mechanisms to resolve disagreements between procurement staff and task team leaders and sector managers; 19. Update procurement policy to: (i) incorporate risk management March 2009 OPCS 35 Summary of Corrective action (with concordance to more detailed recommendations in Management’s Problem identified Response) Timeline On point and fraud and corruption issues, (ii) document the already enhanced complaints handling; and (iii) mainstream a risk-based procurement assessment tool. V. Deficiencies in the IT and AAA areas (paras 49 (IT at entity-level), 51(IT an project level, and 54 (AAA) of Management response) IT system vulnerability. 20. Prevent password sharing; and strengthen controls to further limit Ongoing ISG/GSD privileged access to system applications and to monitor changes to privileged accounts. Difficulties with timely 21. Improve accessibility of operational documents through June 2010 (as part OPCS/ISG accessibility to operational automation (commencing with IL) and new electronic filing of IL reform) documents system. Mismatch between existing AAA 22. Rationalize processes and controls governing AAA to better reflect Complete review OPCS processes and controls and the the wide range of AAA work, address compliance issues identified and discuss with wide range of AAA work by IEG and QAG, and improve system support and monitoring. CODE by June 2010 36 Annex 2. Control Failures by Process Module (BASED ON COMPLIANCE TESTING CONDUCTED DURING PART IB) Operating No. Process Name Tested Effectively Failed Description of Failed Control 1 IDA Allocation 4 4 – 4 CAS Products 3 3 – 5 SIL: Specific Investment 9 9 – Loan 6 Project Changes 1 1 – 7 DPL: Development Policy 7 7 – Lending 8 Corporate Review 1 1 – (ROC/OC) 9 Contractual Remedies 3 3 – 10 Legal – IL 10 10 – 11 Legal – DPL 8 8 – 12 Financial Management – 4 2 2 During FY06, the FM Sector Board issued new IL guidelines for FM practices in Bank-financed investment operations. We could not verify because of lack of documented evidence, and change in Regional practices, that the review and approval of the FM Assessment and appraisal stage PADs and Financing Agreements by the RMFM or appointed delegee occurred. The sample testing, based on date prepared, identified that the majority of FMSRs prepared after November 2005 were not in accordance with the suggested requirements in the FM Guidelines as Regions were transitioning to preparing the FMSRs in accordance with the guidelines. Approx. 40% of the projects reviewed had no documentation evidencing that the risk rating identified by the FM specialist was sent to the TTL for inclusion in the ISR. In one instance we noted the ISR had a different rating from the FMSR - and no explanation was attached. 13 Financial Management – 2 2 – DPL 37 Operating No. Process Name Tested Effectively Failed Description of Failed Control 14 Procurement – IL 8 6 2 Issues in preparation of the Form 384 included: (i) a few months delay in preparing the Form 384 after the contract received date; (ii) the 384 not showing the LAS disbursement categories; (iii) the 384 not corresponding to the no objection letter; and (iv) the contract amount recorded in the 384 was lower than that of the bid documents. Unable to verify that the procurement post reviews were carried out in accordance with the timing requirements in the most recent procurement supervision plan or PAD in approx. 40% of our sample, due to lack of documentation provided. In one case we also noted a lack of audit evidence to support the post procurement review and the results from it. 15 Procurement – 2 1 1 Audit evidence on file was missing to Complaints indicate internal review and communication with the borrower in some cases. 16 Procurement – Non- 2 2 – Compliance 17 LOA – IL 5 4 1 Loan master data created at the time of credit set-up in LAS, was not consistent with the financing agreement and/or disbursement letter. The majority of issues related to the set-up of prior review and/or SOE thresholds. 18 LOA – DPL 6 6 – 19 LOA – Application 5 5 – Review 20 LOA – SC or Application 1 1 – Problem 21 LOA – 2 2 – Amendment/Extension 22 LOA – Refund Process 1 1 – 23 LOA – Cancellation 2 2 – Process 24 LOA – Suspensions 6 4 2 Controls surrounding FO approvals of notices related to threat of suspension, suspension, and lifting of suspension were not testable in many cases due to the lack of documentary evidence. Verifiable historical audit trail relating to imposing or lifting of suspensions is not readily available in LAS. 25 LOA – Closing - Standard 2 1 1 Lack of evidence of Finance Officer clearances – documentation not made available/provided for testing. 38 Operating No. Process Name Tested Effectively Failed Description of Failed Control 26 LOA – Closing - Special 2 1 1 Housekeeping of the Credit information in LAS is not always performed in a timely manner. 27 QAG – Quality at Entry & 6 6 – Supervision 28 Safeguards – IL 3 3 – 30 Debt Reporting 3 3 – 31 CPIA 4 4 – 32 PCPI 3 3 – 115 105 10 Less items not included in testing: Safeguards - Corporate 29 Risk (QACU) – – – Controls included in testing 115 105 10 39