75681 UKRAINE PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT UKRAINE PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT 2012 © 2013 The International Bank for Reconstruction and Development / The World Bank Website: http://www.worldbank.org.ua All rights reserved. The �ndings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the World Bank Office in Ukraine. Funded by the Swiss Confederation About the Swiss Cooperation in Ukraine In 1997 the Swiss Confederation and Ukraine signed a bilateral agreement for technical cooperation, in the framework of which the Swiss Cooperation Office was opened in Kyiv to coordinate the cooperation programs in Ukraine. The Swiss support to Ukraine focuses on Financial and Economic Sustainability, Sustainable Energy Management, Local Governance and Public Services as well as Reproductive Health. The yearly budget of the Swiss Cooperation Program in Ukraine amounts up to 14 million Swiss Francs. For more information please visit http://www.swiss-cooperation.admin.ch/ukraine/ and http://www.seco-cooperation.admin.ch . 3 Currency and Equivalent Units (Exchange Rate Effective as of December 13, 2012) Currency Unit = Hryvnia UAH 7.99 = US$1 US$ 0.13 = UAH1 Government Fiscal Year January 1 – December 31 Abbreviations and Acronyms BCR Bene�t Cost Ratio CBA Cost Bene�t Analysis CPAR Country Procurement Assessment Review CSO Civil Society Organisation EIRR Economic Internal Rate of Return IFI International Financial Institution GFS Government Finance Statistics IPSAS International Public Sector Accounting Standards MEDT Ministry of Economic Development and Trade MTFF Medium Term Fiscal Framework NGO Non-Government Organisation O&M Operations and Maintenance PEFA Public Expenditure and Financial Accountability PER Public Expenditure Review PFM Public Financial Management PI Performance Indicator PIM Public Investment Management PIP Public Investment Project PIM-PR PIM Performance Report PPP Public Private Partnership QFA Quasi Fiscal Activity SAI Supreme Audit Institution SFI State Financial Inspection of Ukraine SIP State Investment Program SNG Sub-National Government SOE State-Owned Enterprise VFM Value For Money Vice President: Philippe H. Le Houérou Country Director: Qimiao Fan Sector Director: Yvonne M. Tsikata Sector Manager: William L. Dorotinsky Task Team Leader: Oleksiy Balabushko 4 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Table of Contents Overview of the Indicator Set ........................................................................................................ 5 Summary Assessment ..................................................................................................................... 6 1. Introduction ............................................................................................................................... 8 Country Context ....................................................................................................................................................................... 8 Assessment Background and Objectives.................................................................................................................. 9 Scope and Methodology .................................................................................................................................................... 9 2. Overview of Public Investment and Public Investment Management.......................... 11 Public Investment Composition and Trends .......................................................................................................11 Institutional Mapping ........................................................................................................................................................12 Political Economy of the PIM Reforms.....................................................................................................................14 The Cost of Corruption......................................................................................................................................................16 3. PIM Assessment ....................................................................................................................... 17 Budget Credibility and Transparency as a Basis for Sound PIM.................................................................17 Guidance and Preliminary Screening ........................................................................................................................18 Formal Project Appraisal....................................................................................................................................................19 Independent Review of Appraisal ...............................................................................................................................20 Project Selection and Budgeting .................................................................................................................................21 Project Implementation.....................................................................................................................................................22 Adjustment for Changes in Project Circumstances ..........................................................................................23 Facility Operation ...................................................................................................................................................................24 Ex-post Evaluation .................................................................................................................................................................24 PIM Indicators Report ..........................................................................................................................................................25 4. Outline Reform Strategy ........................................................................................................ 38 In the Short Term....................................................................................................................................................................39 In the Medium-Term ............................................................................................................................................................40 Capacity Building / System Improvements ...........................................................................................................41 Annex A. PIM Assessment ........................................................................................................................................................42 Annex B. Summary of PIM Performance Indicator Set with Links to PEFA and PIM 8 ‘Must- Have’ Features ....................................................................................................................................................................................48 List of Tables Table 1: Capital Expenditures by Source of Funding, 2009-2011 UAH mn ..................................11 Table 2: Institutional Mapping of PIM Activities in Ukraine – Central Government .............................14 Table 3: Construction Costs vs Perceived Corruption .............................................................................................16 List of Figures Figure 1: Planned and Actual Capital Expenditures as a Share of GDP, 2009-2011 ..............................11 Figure 2: Actual Capital Investments and Transfers as a Share of GDP, 2009-2011. ..............................12 Figure 3: State Budget Capital Expenditure Execution Rates, 2009-2010. .................................................12 Figure 4: Chart Showing Preparation and Selection Practices in Ukraine ..................................................13 Overview of the Indicator Set 5 Overview of the Indicator Set Reference Area/Indicator Score PIM 1 Aggregate capital expenditure out-turn compared to original budget D PIM 2 Composition of capital expenditure out-turn compared to original budget D PIM 3 Aggregate revenue out-turn compared to original budget A PIM 4 Stock and monitoring of capital expenditure payment arrears A PIM 5 Classi�cation of the budget C+ PIM 6 Comprehensiveness of information included in budget documentation C PIM 7 Extent of unreported capital spending C+ PIM 8 PIM-related inter-governmental �scal relations C+ PIM 9 Management of �scal risks from capital spending outside central government C+ PIM 10 Public access to key information on capital spending A PIM 11 Investment guidance, project development, and preliminary screening D+ PIM 12 Formal project appraisal D+ PIM 13 Independent review of appraisal D PIM 14 Orderliness and participation in the annual budget process B PIM 15 Multi-year perspective in �scal policy and management D+ PIM 16 Project selection and budgeting D+ PIM 17 Project Implementation C+ PIM 18 Predictability in the availability of funds for commitment of expenditures B PIM 19 Value for money in procurement B PIM 20 Effectiveness of internal controls and internal audit of capital project expenditure C+ PIM 21 Project Adjustment D+ PIM 22 Facility Operation C+ PIM 23 Basic Completion review and evaluation D+ PIM 24 Quality and timeliness of in-year budget reports A PIM 25 Quality and timeliness of annual �nancial statements B+ PIM 26 Scope, nature and follow-up of external audit D+ PIM 27 Legislative scrutiny of capital spending in the annual budget law B PIM 28 Legislative scrutiny of external audit reports on capital spending D+ Overall Assessment Score: C 6 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Summary Assessment Ukraine has extensive public infrastructure inherited from the Soviet times but much of it has fallen into disrepair over the past decades and needs major rehabilitation or replacement so that growth may continue. Creating �scal space for investing more is one of the critical tasks that facing the country, but a constrained �scal space together with the use of investments as a stimulus for growth call for more efficiency in public investment management practices. While many countries in the region facing similar conditions have attempted public investment management (PIM) reforms, reforms in PIM have been more difficult to achieve than in other areas of PFM because responsibilities in selection and monitoring are often fragmented across the public administration, and on many occasions the process of planning public investments is inherently politi- cal.. Some of the same institutional weaknesses that affect the public administration as a whole have an impact on what can be achieved and how quickly in PIM as well, for example, human resource capacity and political economy constraints contribute to the continued weakness in project appraisal and selection, despite extensive training programs. This assessment aims at a comprehensive analysis of the PIM cycle in Ukraine, benchmarking against good practices, and identi�cation of key constraints to efficient functioning of the PIM system in Ukraine. There are a number of fundamental issues that need to be addressed if Ukraine is to make prog- ress in its reform ambitions for Public Investment Management. The most signi�cant are: (1) Most projects avoid scrutiny due to loopholes in classi�cation (lack of de�nition of a public in- vestment project); (2) There is no effective economic appraisal and appraisal review procedures in place due to limited HR capacity, and no common technical standards; (3) The PIM system does not seem to block new projects from entering the budget but allows min- istries to delay ongoing ones and squeeze in new ones; and (4) Lack of strategic guidance with which to prioritize complicates project selection. At the heart of the problem lies a lack of de�nition of what constitutes a Public Investment Project and an understanding of the fundamental reasons why governments engage in them. At the same time there are multiple de�nitions of investment related activities such as capital expenditures, investment projects, national projects, state investment programs, and construction projects. Amongst these deep seated issues appears to be a growing recognition that reforms are needed in order to meet the huge infrastructure needs of the country. The reform process has so far been slow to move and has been piecemeal in nature. The evidence of strong political will to enact strong reforms in this policy area remains elusive, despite encouraging statements from the government. In order to move forward, reform should begin with a single clear strategic framework of what a well-functioning PIM system should look like together with a list of reforms that the government intends to implement. There is a non-de�ned difference between so called Construction Projects and State Investment Programs (SIPs) a difference that appears to be regularly exploited for convenience and expediency. The reason is that SIPs are required to undergo an economic appraisal and Construction Projects are not, therefore officials and politicians in a hurry do not want to classify their project as a SIP for that reason. This situation leads to a strong built-in disincentive to appraise projects, creating a situation where over 90% of projects avoid any form of economic justi�cation or assessment as they are not Summary Assessment 7 considered to be investment projects. If that was truly the case, the logical question might be: ‘if it is not an investment project then why is it being built?’ One of the fundamental building blocks of a sound PIM system is a clear, legal de�nition of what counts as a public investment project and what does not. This de�nition is missing, allowing arbitrary and subjective judgements to be made, leading to a situation where capital allocations and project selection can be made without consideration of the social and economic bene�ts to the nation and the �nancial costs of doing so. The elements of a system that appear to function well are those associated with engineering and technical assessments but that pre-supposes that certain projects are necessary at all and con- ceived in the most economically efficient form in the �rst place. Links to speci�c national policy docu- ments and infrastructure plans are often identi�able but links between objectives and options ap- praisal (other than technical options) are almost always completely missing. Projects are almost always conceived on the basis of input based thinking instead of output/ performance targets linked to the objectives and the need for the investment. Because demand for economic appraisal has been so low for the reasons explained above, the supply of capacity to prepare projects and undertake appraisals is minimal. If a new de�nition of what comprises an investment project is created it will need to be clear about the level of appraisal required and will need to be careful in setting thresholds that match available capacity to do project prepara- tion (in sponsoring entities) and project appraisal (in MEDT) in order to avoid overwhelming the sys- tem. Indeed one option might be to raise the existing threshold in the short term in order to reduce numbers but improve quality. The current threshold for appraisal is 100m UAH ($12.5m) as de�ned in Ukrainian legislation. More investment will need to be made into the preparation of projects – a clear analogy being that if you aim to build a good house you need to invest in good quality foundation. Improved project appraisal and selection are two interventions that are likely to increase efficiency and quality of invest- ment projects the most. Better project preparation will lead to better project outcomes. Whilst there have been improvements in the overall procurement framework in recent times, there remain some doubts about its overall trustworthiness. There are concerns also that construction out-turn costs in Ukraine are estimated at 23% higher than in Germany and 22% higher than the entire EU average for an equivalent category of building, without any perception of extra value being ob- tained for this extra cost. The difference between Ukraine and neighbouring Poland is approximately 20% higher. It should be pointed out that this already high discrepancy is only a comparative mea- sure of input values. Developing projects that are output/performance driven should yield even greater efficiencies. This for the time being may at least serve to draw attention to the inefficiencies in the current system and the costs of delaying reforms to develop a well-functioning Public Investment Manage- ment system in Ukraine. The functioning PIM system will bene�t Ukraine nationally through better targeted investments that could be directly linked with bene�ts for the nation. Proper medium term investment planning would also be possible, allowing budget officials to take a strategic view of what the nation can re- alistically afford. In doing this, if the process is transparent enough, private companies that may wish to bene�t from such public investment in infrastructure can design their own business development plans accordingly. 8 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT 1. Introduction Country Context 1.1. Between 2000 and 2008, Ukraine was an average growth performer in a fast grow- ing region, with GDP growth averaging 7 per cent. Growth helped to signi�cantly reduce poverty in the country. The poverty and vulnerability headcount index fell steadily from 46.9 percent in 2002 to just 12.3 percent in 2007 (measured by the USD 5 in Purchasing Power Parities poverty line). Growth over these years was primarily driven by external and temporary factors as opposed to the structural changes in the economy. 1.2. As the global �nancial crisis hit the Ukrainian economy it contracted by 15 per cent in 2009, exposing its underlying macroeconomic and structural vulnerabilities. These included: (i) a weak maturity structure of the fast growing private sector external debt; (ii) banking sector vulner- abilities associated with rapid loan growth supported by predominantly external funding and weak regulatory and supervision controls; (iii) volatile terms of trade and lack of diversi�cation in external demand (mainly related to the steel and heavy industry sectors on the export side and the gas sector on the import side), (iv) expansionary �scal policies in the context of problematic expenditure and revenue structures; (v) weak competition and ability to diversify and generate higher value added products; and (vi) an overall burdensome regulatory environment and a large government footprint that hampers private sector development. 1.3. As a result of the insufficient structural transformation and impact of the econom- ic crisis, Ukraine now faces substantial �scal pressures that threaten economic stability and growth. These include large and growing infrastructure investment needs; growing public and pub- licly guaranteed debt; and a high average tax burden and high marginal direct tax rates coupled with an onerous revenue administration system. 1.4. Addressing these issues in Ukraine in recent years was complicated by political in- stability and frequent changes in the Government. Between 2005 and 2010 there were four dif- ferent governments. And even within one government, ministers and senior officials have changed frequently. Following the last Presidential election, the situation stabilized in a way that Ukraine had one Government for over a year. 1.5. The Government of Ukraine recognized the need for a modern PFM system and put considerable emphasis on several aspects of PFM reforms. Important progress has been made, including the establishment of a Treasury system, the abolishment of central bank direct �- nancing of the Government, the adoption of a new Budget Code and budget classi�cation system, the creation of an internal audit function and improvement of the availability of �scal information and adoption of the new Tax Code. The Procurement Legislation addressing what used to be one of the most corrupt practices between 2005 and 2009 is �nally in place since June 2010 and is broadly in line with international standards although some concerns remain as identi�ed and dis- cussed later in this assessment. 1. Introduction 9 Assessment Background and Objectives 1.6. Capital budgeting has been identi�ed as one of the weakest aspects of the PFM system by PEFA assessments in 2007 and 2011 and several Public Finance Reviews. Ukraine has vast investment needs – the Public Finance Review 2011 estimated the public investment required for sustained economic growth at a level of over USD 100 billion over the next 10 years. 1.7. At the same time, Ukraine’s capital budgeting system has a number of drawbacks resulting in high under execution rates for public investments. Based, on that the Dutch TF was requested to focus on improvements to project selection and evaluation as well as public asset man- agement practices. The Dutch TF included technical assistance on project selection and evaluation, public asset management, and establishment of the PPP framework in Ukraine. In addition, the TF �nanced capacity building activities such as a workshop on capital budgeting for high level govern- ment officials, and a four week training course on cost bene�t analysis. 1.8. From WB and other donors’ support, Ukraine has initiated some improvements in the area of capital budgeting such as (i) the establishment of multi-year budgeting for invest- ments �nanced from the budget; (ii) the Law on Investment Activity which established the necessity to conduct an appraisal procedure for State Investment Projects; and, (iii) Government Resolutions establishing procedures for preparation and evaluation of PPPs which approved by the Government in the spring of 2011. At the same time, the implementation of these measures requires further work on preparation of sub-legal acts, institutional changes and building capacity in government agencies dealing with investments. 1.9. Based on the understanding of high investment needs and a limited �scal enve- lope, capital budgeting reform is highlighted as one of the key areas in the presidential Eco- nomic Reform Program for 2010-2015. PIM is high on the agenda of policy-makers. 1.10. This assessment is intended to serve both as a baseline for continuing and future assessment and monitoring of the capital budgeting system in Ukraine and the basis for con- tinuing policy dialogue and development. Scope and Methodology 1.11. The assessment team included Oleksiy Balabushko (ECSP4), Martin Darcy (consul- tant) and Irina Shcherbyna (consultant). The team carried out the assessment between July- November 2012. 1.12. The assessment examined the stages of the process that a typical public invest- ment project goes through in Ukraine. The methodology is based on Rajaram et al. (2010), Jacobs (2008) and the PEFA PIM drill down tool developed by PRMPS. The assessment was conducted on the basis of existing analytics including Public Finance Reviews I and II, PEFA assessments and other reports prepared by the Bank. The assessment started by identifying in detail the path that a typical investment project follows in Ukraine. It then investigated the sub processes in line with the indicators of the PEFA PIM tool. Finally, it examined a number of systemic or cross-cutting issues that affected the performance of the entire process. 1.13. The assessment was also conducted as a pilot for the PEFA PIM drill down tool de- veloped by PRMPS and it adopted its methodology. Some minor adjustments were adopted to reflect speci�c issues in Ukraine and some due to proposed improvements to the methodology. 10 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT 1.14. The assessment is focused more on PIM speci�c parts of the general budget, and budget reporting. External scrutiny components of the system were recently evaluated under the 2011 repeat PEFA Assessment (World Bank, 2011). Some of the indicators are not assessed separately but a reference to the PEFA report is provided. 1.15. The baseline year for the assessment is 2011 although evidence was also gathered about current practices undertaken during the time of the review in mid-2012. In the rare ex- ception where there are notable differences between the two, suitable notation is made. With the resources available it was not possible to assess practices in SNGs or in local authorities and the as- sessment therefore covered only a selection of central entities that regularly developed and managed capital projects. The assessment team was heavily reliant on the willingness of these entities to co- operate and volunteer their time and share their experiences. It was not always easy to �nd suitable and willing candidates to participate. A number of changes to the initial methodology were necessary to reflect this situation. 1.16. The activities were comprised of: (a) An initial high level workshop to re-invigorate reform and policy dialogue on public investment management with the MoE, MoF, Presidential Administration, State Financial Inspection. The workshop was used to launch the assessment and to hold a high level discussion on what the most critical constraints to efficient PIM are, and build consensus around what needs to be done. (b) Conducting a review of capital budgeting processes as well as related processes that affect pub- lic investment project performance countrywide as well as in speci�c agencies. This work is comprised of an analysis of the process that an investment project has to follow through its lifecycle from the identi�cation to completion and benchmarking this process against good practice and, by doing this, establishing a baseline assessment for Ukraine 1.17. The PIM Assessment commenced with a two week mission during May 2012. The �nal preparations for Euro 2012 and the event itself, followed by the usual summer slowdown pre- vented any further work on the assessment until late August when a second mission concluded the assessment. Agreement had initially been reached in planning to make a detailed assessment of one classical infrastructure sector and one social infrastructure sector. However, as a result of implementa- tion and inability to �nd detailed case study, it was decided instead to replace the detailed case stud- ies with a wider range of discussions with multiple other governmental entities associated with the subject of PIM. 1.18. The assessment was based on statistical data from the State Statistics Service, State Treasury of Ukraine, Ministry of Finance, and Ministry of Economic Development and Trade as well as a number of interviews with private sector representatives and government agencies (Ministry of Economic Development and Trade, Ministry of Finance, Ministry of Emergencies, Ministry of Regional Development, Housing and Communal Services, State Treasury Service, State Statistics Service, State Tax Service), state owed enterprise Ukrhydroenergo. The assessment also bene�ted from cooperation with the Coordination Center for Economic Reforms under the President of Ukraine. 2. Overview of Public Investment and Public Investment Management 11 Overview of Public Investment and 2. Public Investment Management Public Investment Composition and Trends 2.1. The public investments in Ukraine are carried out by either central or local govern- ment or by SOEs. The central government invest through the state budget, local government �nance their investments from local budgets and also some centralized programs from the state budget. State owned enterprises are in charge of a considerable portion of public investments in Ukraine. Many SOEs �nance core infrastructure such as railways, sea ports, etc. The structure of public investment by investing agency is shown in table 1 below. There is a disproportionate growth in state budget capital expenditures during this time, which is mainly attribute to the Euro 2012 championship. Table 1: Capital Expenditures by Source of Funding, 2009-2011 UAH mn Year State Budget Local Budgets SOE Investment 2009 10,365.6 11,428.6 6,681.7 2010 21,057.1 13,256.8 7,038.3 2011 31.281.4 17,396.9 6,802.3 Source: State Statistics Service, State Treasury. 2.2. As the state budget is a major source of public investments and this assessment is mainly focused on central government PIM practices, we should go into more detail on state budget public investment composition and trends. Following a deep reduction in the capital expenditures following the crisis, they have been picking up in recent years. At the same time, the execution levels remain low. Figure 1: Planned and Actual Capital Expenditures as a Share of GDP, 2009-2011 0.04% 0.03% 0.02% 0.01% 0.00% 2009 2010 2011 Planned Actual Source: State Trveasury. 2.3. Capital expenditures in Ukraine consist of capital investments and capital trans- fers. It is important to distinguish between these two categories as capital investment means creation of assets, while transfers include a considerable share of subsidies to enterprises and special transfers to spending agencies (such as the Ministry of Fuel and Energy, Ministry of Agriculture) to subsidize certain sectors. Transfers occupy a considerable share of capital expen- 12 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT ditures, suggesting that scarce �scal space for improving basic infrastructure is diverted towards sub- sidies. The breakdown of state budget capital expenditures according to these two categories is shown below. Figure 2: Actual Capital Investments and Transfers as a Share of GDP, 2009-2011 0.04% 0.03% 0.02% 0.01% 0.00% 2009 2010 2011 Capital investments Capital transfers Source: State Trveasury. 2.4. Capital expenditure execution rates have been improving recently but still are low. The capital expenditure execution rates in Ukraine have dropped below 60% following the crisis of 2008, and while there were improvements to over 70% in 2010-2011, the rates remain low indicating poor planning process and weak implementation. Figure 3: State Budget Capital Expenditure Execution Rates, 2009-2011 80% 70% 60% 50% 40% 30% 20% 10% 0% 2009 2010 2011 Source: State Treasury, Bank Staff Calculations. Institutional Mapping 2.5. Institutional responsibility for proposing and implementing projects rests with bud- get entities in central government, with full autonomy for SOEs and SNGs. Budget entities �rst pre- pare projects on the basis of the required inputs (rather than outputs). In the case of construction projects, the inputs are estimated using well known cost and input norms and benchmarks. According to the State Statistical Bureau, these data and benchmarks are regularly updated and widely used by public entities in the preparation of their construction estimates. Such feasibility studies that are conducted are pre- occupied with technical feasibility rather than economic feasibility. 2.6. In the case of Agencies and public entities that report to Ministries, project propos- als are �rst sent to the relevant ministry for basic checking. Again the checking consists mainly of ensuring that the correct administrative procedures have been observed as there is no appraisal ca- 2. Overview of Public Investment and Public Investment Management 13 pacity in the ministries. The key divergence in institutional routing follows on from this point as shown in the Chart below. Figure 4: Chart Showing Preparation and Selection Practices in Ukraine Line Ministries Project identi�cation Line Agencies Preparation Pre-feasibility Studies State Investment Programs The By-pass Route! (8.9% by value of executed projects) Other Captival Projects not considered as State Investment Programs (91.1% by value of executed projects) Ministry of Economy Document Compliance Registry Basic Screening of Ministry of Finance Project Selection Project Appraisal scores Projects Government Budget Scrutiny Flow of Approved projects back to the Implementing Verkhovna Rada Final Approval Agencies for Procurement / Implementation 2.7. Proposing entities appear to have a legitimate choice about how to classify their project proposals. If they choose to classify their projects as State Investment Programs then they must send the project proposal for an economic appraisal to the MEDT. MEDT is not only responsible for conducting appraisal work but also for veri�cation and checking which might imply an inherent conflict of interest. If proposing entities choose to classify their project as a Construction Project they are able to go directly to the Ministry of Finance for direct negotiation on funding. 2.8. When projects go through the MEDT route as SIPs, the Investment Division of MEDT has the roles of checking compliance with administrative procedures and basic screening of any preparatory work. Although it is possible for projects to be rejected and returned to the propos- ing entity, at the moment this appears unlikely to happen for any reason not connected to a proce- dural mistake. There is understood to be frequent pressure applied by proposers on MEDT to approve projects quickly, regardless of the degree of complexity. 2.9. Once projects are checked and approved by MEDT they are included in a register of projects that is still in its infancy and not yet fully developed. All other projects that might oth- erwise been de�ned as PIPs are not included on any projects database. Therefore there is no way for anyone to have a complete overview of the scale and scope of public investment activities in Ukraine even at the level of central government. Much less so when considering the activities of SNG and SOEs. 2.10. All projects whether checked and approved by MEDT or not, ultimately arrive at the Ministry of Finance where they compete for what ultimately is an inadequate amount of �nancial resource relative to the demand. The strength of political pressure appears to be the main selection method. Along with other forms of proposed expenditure, all programs and projects that are included in the draft budget are then subject to scrutiny by the government where it is possible that amendments might be made. It is also possible at this late stage to include politically motivated projects. 14 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT 2.11. Once the draft budget has been agreed by the government, it is sent to the parlia- ment where subject to any further debate, it is approved. Budget approvals then flow back to the relevant budget entities which are then responsible for the implementation of approved projects. 2.12. Project adjustments are usually initiated by the implementing agency, however budget virement rules are rather stringent and reallocation between budget programs re- quires Ministry of Finance and Parliament consent. 2.13. Facility operation and maintenance is performed by the facility owner. In some sec- tors, the facility network is well structured and organized (e.g., roads where the national road network is maintained by the Road Agency and local roads are responsibility of local governments), while in other sectors the network is cumbersome (e.g., in healthcare where hospitals belong to local govern- ment, Ministry of Healthcare, National Medical Academy and multiple sector agencies and ministries). 2.14. In the base assessment year of 2011, there was no functioning internal audit sys- tem. This has been corrected from 1st January 2012 but it is too early to say how effective this function will be. Oversight is provided by the State Financial Inspection1 in �nancial and wider compliance with procedural and administrative matters; and the Anti-Monopoly Committee in the case of public pro- curement. The Accounting Chamber of Ukraine is the Supreme Audit Institution and has been at- tempting in recent times to make a start on assessing performance of public investment projects but their scale and scope is very limited. Thus, full scale ex-post evaluation is not in place. Table 2: Institutional Mapping of PIM Activities in Ukraine – Central Government Institution Role / Major Tasks Comments Line Ministries (16), • To conceive and prepare projects that directly affect their central Excludes SOEs and SNG Central Government functions and implement those that receive funding. Bodies (53) • To receive and administer capital requests from agencies and other entities under their control. • Design, implement centrally �nanced projects including budgeting, contracting Ministry of Economic • Policy development regarding PIM Some (but limited in capacity) Development and • Prepare and review economic appraisals for State Investment capability in project appraisal Trade (MEDT) Programs • Development and maintenance of a projects database for SIP • procurement regulatory function Ministry of Finance • Direct negotiations with many public entities regarding project funding • Project Selection • Budget Allocations Accounting Chamber • The supreme audit institution responsible to Parliament Attempt to do some limited performance based audit Anti-Monopoly • To receive and administer complaints about public procurement Commission procedures State Financial • To check compliance with rules, controls / �nancial and Under the Ministry of Finance Inspection administrative norms Political Economy of the PIM Reforms 2.15. The appetite for substantive reform of PIM norms and practices in Ukraine remains hard to de�ne. The President and the Government have offered encouragement in public speeches about the importance of infrastructure projects in particular but subsequent actions to deliver reform 1 Formerly known as KRU. 2. Overview of Public Investment and Public Investment Management 15 appear to be lackluster. A number of slow but encouraging, piecemeal reforms have taken place in the last two to three years and the Bank has been a close partner in those initial efforts. 2.16. Despite this, practices and procedures that normally comprise an effective PIM sys- tem are often carried out in isolation of each other, meaning that a cohesive and holistic sys- tem does not exist in Ukraine. 2.17. Those officials working in discreet elements of PIM are rarely aware of the wider pic- ture and therefore a deep understanding of what a Public Investment Project (PIP) represents and its role in the wider economy appears to be missing. The �nancial and economic bene�ts of a functioning PIM system and the costs and risks attached to the current system are not understood by most people involved. 2.18. Differing classi�cations of what essentially are PIPs adds to the confusion. Evi- dence was found of two different hospitals being built, one of which was considered a State Investment Program and the other a Construction Project. The former would be subject to an appraisal procedure, the latter would not. 2.19. These arbitrary classi�cations underlie a culture of bringing forward projects di- rectly to the Ministry of Finance in a direct plea for funding in the next budget round. Whether for expediency or convenience, successful pleas are often allocated a separate budget line devoid of any justi�cation or economic appraisal. This indicates a heavily politicized selection procedure. Like many other countries, politicians clamor to announce new projects that might offer them political kudos. This behavior is often at the expense of funds to complete other projects under implementa- tion. Furthermore, budget requests are often done on a market style bargaining basis where officials know that if they ask for the actual amount they need to implement a project, they will receive only a fraction. This can lead to arbitrarily inflated budget requests that are expected to be negotiated down to close to the real �gure. 2.20. The entity of the government with the most signi�cant central responsibility for PIM is the Ministry of Economic Development and Trade (MEDT) and it has been the prime mover in the reform agenda for PIM. It has successfully argued that some projects (SIPs) should be subject to appraisal and therefore review by them. In the eyes of some proposers of projects, this places an un-wanted and un-necessary administrative burden on them. They would much prefer a direct negotiation over budget allocations with the Ministry of Finance in order to win funding for their project. Fortunately for them but unfortunately for PIM reformers, this ‘by-pass’ route is fully per- missible and commonly used. The full institutional map illustrating these parallel systems, with a fuller narrative, is given in the previous section. 2.21. Any country wishing to reform a particular area of public policy needs ‘champions’, namely a group of high level officials and politicians that are closely associated with the sub- ject matter and consistently promote the bene�ts of reform to that policy area. In the case of PIM in Ukraine, there is a consistent champion in MEDT and positive reform steps so far undertaken have been as a result of this dedication. For further improvements to be made, and for an increase in the pace of reform, further political support will be necessary, not only from MEDT but also from the Ministry of Finance and wider government. 2.22. From the perspective of the MEDT, the prospects of being the architects and imple- menters of a sound and economically productive PIM system will gain them much credibility both at home and internationally. Managing a fully developed system of project appraisal and scru- tiny will place them at the lynchpin of the system. MEDT would sign off projects that ful�l the neces- sary criteria for socially and economically productive public investment projects and programs. They 16 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT would then hand them to the Ministry of Finance for �nal selection according to the available scope for investment within any particular budget year. 2.23. Clearly from the perspective of the Ministry of Finance, having a stream of quality checked project proposals (sometimes known as ‘shovel ready’ projects) reduces the risk in them giving �nal approval and speeds up the time between approval and implementation. This would give them a valuable management tool during the budgeting process. 2.24. Nationally, Ukraine stands to gain from projects that are more targeted towards meeting speci�c social and economic objectives, are delivered more efficiently, both in terms of terms of time and cost and are more likely to deliver the required project outputs more ef- fectively. This must over a period of time ultimately result in better infrastructure and public services which are more widely available and at a cost that is �scally sustainable. The Cost of Corruption 2.25. Construction contracts offer major opportunities for corrupt practices in all coun- tries. Better performing countries will deal more effectively with the threat than poorer per- forming countries. Any assessment of the efficiency and effectiveness of capital investment projects or overlying systems that govern them should attempt to measure the perceived cost of corruption. 2.26. By drawing enough attention to this subject in a manner that could be monitored, over time Civil Society Organisations (CSOs) might be able to bring enough pressure to bear to improve the situation. The corrupters and the corruptible may at least moderate their behaviour with the resultant improvement in outcome efficiency for projects. A reasonably measurable indica- tor can provide one of the more effective tools in reducing corruption in PIM. It might also prove to be one of the more effective interventions towards improving the efficiency of public investment projects. The data from table 3 suggests that construction costs in Ukraine are between 25-30% higher than in Germany, despite lower labour costs. This does not necessarily point to issues with procure- ment processes but rather abuse of the systems involved. 2.27. Below is an illustration of the perceived relationship between the premium that is paid for built projects and the Transparency International Corruption Perception Index. Table 3: Construction Costs vs Perceived Corruption CCI Score2 % of EU Average TI CPI3 TI CPI Ranking5 Score4 Germany 112 -1 7.9 15 EU-27 Average 114 6.30 33.81 Czech Republic 114 0 4.6 53 Lithuania 115 +1 5.0 46 Estonia 115 +1 6.5 26 Poland 116 +2 5.3 41 Ukraine 139 +22 2.4 134 Bulgaria 141 +24 3.6 73 Romania 147 +29 3.7 69 Georgia 3.8 68 Turkey 4.4 56 2 Construction Cost Index: Eurostat 2010 (based on residential construction projects). 3 Transparency International Corruption Perception Index 2010. 4 Score: 10 = Excellent, 0= Poor. 5 Out of 182 Countries - #1 = Least corrupt, #182 = Most corrupt. 3. PIM Assessment 17 3. PIM Assessment 3.1. This section applies the PEFA PIM Drill Down Tool methodology (World Bank, 2012) in order to assess where Ukraine stands compared to good practice in different dimensions of the PIM system. The write up describes PIM system features organized by eight broad areas and pointing to the most critical weaknesses. At the end of the section is a table that summarizes an as- sessment of 28 indicators using letter grades with A indicating best practice, while D indicating non- existent element of the PIM system. Budget Credibility and Transparency as a Basis for Sound PIM 3.2. The Ukrainian budget system has some sound elements in place such as a stable and transparent budget process, high execution rates on both the expenditure and revenue side and low arrears in capital expenditures. At the same time, the credibility of the budget expenditures in terms of structure and comprehensiveness of information requires more efforts to be satisfactory. The Budget Code of 2011 envisaged important changes to promote medium term and program based budgeting, however implementation of these changes has not been completed. As a result, capital budgeting practices during the period of assessment (2011) are still weak and have a number of weaknesses: a) A low level of capital expenditures indicates a weak public investment management system and shows both inadequate public capacity to properly plan revenues and capital expenditures and a “delay� in capital expenditures till the end of the budget year; b) Existence of social insurance funds that are not part of the budget as well as investments carried out by the SOEs; c) Absence of an efficient system that would link objectives and investments in the investment part of the intergovernmental �scal relations. 3.3. The recent crisis negatively affected the execution of capital expenditures of the budget. The execution of capital expenditures in 2006-2008 stood at 89.3 %, 82.7 % and 79.7 % re- spectively. The execution rate dropped to 56.9% in 2009 and came back to 71% in 2010-2011. The criti- cally low execution rate of 2009 could be attributed to underexecution of revenues (actual revenues collected were 82% of the budgeted �gure). The execution rate of current expenditures also was lower than usual at 93.1%. At the same time, low execution rates in 2010-11 occurred with high revenue execution rates. Thus, a tight �scal situation can explain low execution rates only partially. Persistent weak capital budgeting is another major explanation – namely weak appraisal and selection, resulting high number of the new projects that are frequently favored over ongoing ones. 3.4. The comprehensiveness of the budget is satisfactory. Budget expenditure classi�ca- tion includes economic, functional and program classi�cations that are compliant with GFS. At the same time information related to public investment projects is not sufficient for planning, monitoring and evaluation. 18 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT 3.5. The transparency of the budget is negatively affected by existence of four social funds and SOE investments, which are not included into the budget. The share of SOE in public investments has almost halved over the last three years but still was at a considerable 18% of overall public investment. However, in nominal terms, SOE investments have been stable and a decrease in share occurred as a result of growing capital expenditures from the state budget, mainly due to the Euro 2012 program. Social funds are outside the budget; however the new Budget Code envisages that all four funds will be reporting according to the national public sector accounting standards com- pliant with IPSAS starting 2013. When that happens, it will be a considerable step towards consolida- tion of all capital investments carried out by the state and will eventually lead to improved efficiency of capital budgeting. 3.6. Intergovernmental �scal relations are regulated according to the equalization for- mula that de�nes transfers from the state budget to local budgets. At the same time, the capital transfers to the regions are not subject to clear rules and criteria for allocation across regions or project selection. In addition the timeline for capital transfers is not optimal for planning by local govern- ments. The share of capital transfers is allocated as part of the annual budget law and the other share is allocated based on government resolutions following approval of the annual budget law. Guidance and Preliminary Screening 3.7. There are numerous strategy and policy documents of the government relating to general economic and policy objectives. Sector strategies underpin these documents which out- line speci�c physical, economic and social objectives. Line ministries and other implementing entities of government are adept at linking their project proposals to these sector strategies and clearly recog- nise the importance of doing so. 3.8. There appears to be no guidance on prioritisation which might be helpful against a background of what many officials complain of as ‘constantly changing priorities of the government’. Political and economic instability has been a cause of these changes in priorities in recent years but it is hoped that a more stable environment might allow for better planning in the immediate and medium term future. 3.9. Projects are developed either by line ministries themselves for central functions but in many more cases through their subsidiary or devolved entities. Proposing entities below the level of line ministries are required to send their project proposals to the responsible ministry for basic screening. However the requirements for the contents of project proposals are de�ned by each ministry rather than through a harmonised template or format developed centrally. It might be con- sidered that the contents of a project proposal, although generally proportionate to the scale of the project, are generally good on input data and physical technical assessments but super�cial on per- formance based objectives and target setting. The result is that economic appraisal is in short supply. 3.10. Critically there appears to be little in the way of policy based elucidation of the need for a project, screening on the question of whether a proposed project might be the best option for achieving the need or whether there is even a need in the �rst place. 3.11. Preliminary screening done in line ministries of projects being proposed by second tier entities appears to be super�cial. The screening process consists mainly of checking the com- pleteness of the paperwork and ensuring that administrative procedures have been ful�lled in the correct manner. 3. PIM Assessment 19 Formal Project Appraisal 3.12. There is no central guidance on appraisal to officials in proposing entities so offi- cials are largely left to interpret speci�c requirements themselves. At present in Ukraine the only formal requirement to conduct project appraisal comes from the 2011 revision to the Law on Invest- ment Activities (the Law) which for the �rst time requires State Investment Programs to be subject to a form of economic appraisal. Whilst the Law clearly de�nes the contents of appraisal documentation, it does so only in list form and there are no supporting regulations to help officials in proposing enti- ties to make a meaningful attempt to ful�ll the requirements. Instead the responsibility for preparing economic appraisals is left to the MEDT using inputs provided by the proposing entities. Without formal appraisal guidance, it becomes relatively easy for officials to propose and accept form instead of substance. Where the Law is found particularly wanting is in the lack of clarity in de�nition of the subject matter. The Law uses the rather unclear de�nition from the original law of 1996 and suggests that further de�nition might be clari�ed by the Council of Ministers. At the time of this assessment, that clarity was still awaited. Most unhelpfully of all, the de�nition states that ‘…construction projects could be classi�ed…’ essentially making it a discretionary choice for officials instead of a mandatory requirement. 3.13. The result of this is that the majority of capital expenditure can be classi�ed as con- struction and therefore avoid, for reasons of convenience or expediency, any form of appraisal. Estimates drawn from budget documentation point to actual expenditures under the budget pro- grams, which have the identi�er of state capital investments (5th digit in the program code is “8�) total UAH 2.8bn in 2011, whilst budget programs which could be categorized as state capital investments total UAH 32.5bn. Therefore only 8.6% of the total is captured through existing practices practices of appraisal and selection. Even current thinking on the subject centers on whether to exclude projects in the social sectors from appraisal regardless of size, signi�cance or impact. At the same time all capi- tal investment bene�ts from budget prepration, approval, execution (i.e., internal controls, procure- ment) processes. 3.14. Without a clear and comprehensive de�nition of a Pubic Investment Project / Pro- gram, other reform initiatives may be completely undermined or rendered ineffective. 3.15. One of the most important elements in building a PIM system is to encourage the widespread consideration of high quality objective setting with strong and clear links to pub- lished policies of the government. The objectives should be measurable and based on outputs or in other words should allow consideration of their effectiveness in meeting the policy requirements. Based on those objectives, the possible options for their realization should be considered with the thought that building something physical might not always be the most economically effective way to achieve those objectives. 3.16. Strategic investment planning of this nature appears to be almost absent from PIM activities in Ukraine and the value of doing so seemed lost to many officials. Almost all project planning relates to ensuring that inputs are as well de�ned as possible rather than outputs. 3.17. The assessment found no evidence that recurrent costs are considered on a system- atic basis in the development of project proposals. Whole life cost of ownership of the resulting assets cannot be formally calculated without these inputs. The accrual of bene�ts from a proposed project similarly is not assessed over a multi-year period (or assessed at all in most cases). Together 20 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT these issues will hinder the prospects for assessing future projects based on comparative indicators of economic value such as EIRR or BCR. 3.18. No evidence was found of mechanisms for rejecting proposals on grounds other than the necessary documentation is missing / not complete. Essentially any project that follows the correct procedures and administrative steps can be put forward for funding from the budget. 3.19. The appraisal system, such as it is allows the weakest projects to have an equal chance of receiving budgetary funds as those projects that might offer the best economic re- turns for the nation’s investment. Independent Review of Appraisal 3.20. MEDT’s Department of Investment and Innovation Activity (DIIA) is the main entity responsible for providing limited oversight of project proposals. At the moment because rules permit projects that are not classed as SIPs to avoid the need for appraisal, less than 10% of projects that might be classi�ed as PIPs are appraised and reviewed. Since DIAA also prepares economic ap- praisals for proposing entities based on supplied data, the potential for conflicts of interests needs to be managed carefully. 3.21. The 2011 revision to the Law on Investment Activities now includes a de�nition on what represents an investment project but this is subject to further clari�cation and as yet does not appear to capture those projects that are still considered ‘construction projects’. It also mentions the need for a Registry of Investment Projects, which is currently maintained by MEDT) and that an economic appraisal for each proposal is required before it can enter on to the Registry. These new requirements have increased the work load of DIIA. 3.22. There are 5 positions available for appraisal staff in DIIA. In 2011 the department handled approximately 1200 projects. This means that each official needs to assess 240 proj- ects each per year in the event that all 5 positions are permanently �lled and the staff is engaged full-time only in those duties. The quality of the appraisal and review that is possible under these conditions appears to frustrate officials and raises the question of what can be achieved in so little time. Another capacity constraint is that all of these 1200 projects don’t arrive in a consistent manner throughout the year. Instead they arrive in greater numbers in the run-up to the budget deadline mak- ing it impossible to provide any meaningful assessment. 3.23. Projects coming in to the department can be as low in value as 100,000 UAH ($12,500) up to very large infrastructure projects. There appears to be no formal minimum thresh- old for projects within the MEDT. 3.24. The main sources of projects by sector to DIIA for assessment and review are High- ways, Water/Wastewater, Mining, Social and culture (including tourism). 3.25. Such guidelines that exist to date appear to be a checklist of documents to be sub- mitted e.g., Environmental statement, Energy efficiency etc. All projects that submit the correct documentation theoretically go onto the Registry after the DIIA has done its checking. Therefore it is perfectly possible for bad projects (but nonetheless compliant), scoring just 1 point to be on the same Registry, as good projects with 100 points. Good projects and bad projects go on the Registry together. 3.26. The Registry is still a work in progress. It is intended that basic information will be shown on the Ministry of Economy’s website. 3. PIM Assessment 21 3.27. It is intended that once on the Registry, projects can remain there for a maximum of 3 years without challenge despite that fact that many of the assumptions on input values and the output bene�ts may have changed considerably during that time. 3.28. Appraisal and review capacity is one of the greatest challenges to short and me- dium-term improvements to the system, yet there is no strategy for rectifying this problem. The dynamic nature of the civil service presents additional obstacles with people being moved often frequently from one position to another. It was notable to learn that of the officials who attended the Bank sponsored CBA training in 2010; only 20% are still in the same job. Project Selection and Budgeting 3.29. Given the weaknesses described above in the appraisal and screening of project proposals, it is no surprise that project selection and budgeting presents a number of chal- lenges and weaknesses. Due to the lack of proper screening and the absence of a mechanism for rejecting weak projects, too many projects arrive at the Ministry of Finance all of which jostle for atten- tion and funding. This situation also means that it is possible for weak projects to attain the same level of attention as stronger ones. 3.30. Worse still, public investment projects can arrive at the Ministry of Finance in a number of different guises. They can present themselves as SIPs with some basic form of Appraisal or as Construction projects with none at all. They can also be packaged together within transfers to SOEs or SNGs with seemingly little visibility. All of this makes it impossible to have a national overview of the full scale and impact of public investment. It makes a rational selection strategy impossible. Given all of that, the scale of the challenge and task of the Ministry of Finance is an unenviable one. 3.31. The Ministry of Finances workload might be expected to be reduced if clear guide- lines and transparent selection criteria were published. Despite this, there are none. The ab- sence of selection criteria also opens the way for large numbers of politically inspired projects to enter the funding arena often without any previous analysis, pre-feasibility or appraisal and on the most basic form of justi�cation. 3.32. The large volume of new projects competing for attention and funding puts intense pressure on the Ministry of Finance to provide funding so that politicians can ‘announce’ new projects on a regular basis. The demand to fund new projects inevitably constrains the capacity to fund on-going projects at the required level. This can and has led to insufficient funds being allocated to existing projects with the consequent delays leading to inflationary pressures, resulting in cost over- runs. The increase in input costs and the delays in realizing the bene�ts of a project caused by this practice only worsen the economic performance of PIPs. 3.33. Whereas contracting authorities are allowed to sign multi-year contracts to realize investment projects, they are not allowed to commit funds beyond the annual budget appro- priations. For contractors this is a business risk but nevertheless a risk they will factor into their pricing – a price that the government will end up funding through the cost of contracts. 3.34. The full impact of the whole life cost of ownership of a building is not always con- sidered during the selection process. Information supplied in project proposals usually relates to manpower costs and some consideration of utility costs. Future maintenance costs, sufficient to maintain the asset in a basic condition are not considered at this stage. 22 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Project Implementation 3.35. Other than for technical regulations concerning construction matters and procure- ment aspects of the implementation sphere, there are no speci�c guidelines issued centrally for the implementation of PIPs. Some implementing agencies on the other hand have developed their own guidance on the subject. 3.36. One of the better improvements to the PIM framework in recent times is the en- actment of the new Public Procurement Law, drafted in consultation with the World Bank and EU which was passed by the Parliament and came into effect in June 2010. This Law is broadly compliant with international best practice and EU Directives, but requires further amendments which are pres- ently being deliberated by the Parliament. 3.37. Public procurement is regulated by the Public Procurement Law which has prece- dence over all other acts that may mention the subject matter. The implementation arrange- ments are set by Government resolutions while methodology is regulated by Ministry of Economy orders. It applies to spending units which use budget funds to �nance contracts (in full or in part) with a total cost above US$12,000 for goods and services and US $37,500 for civil works. 3.38. According to the Law, open competition is the default method of procurement. However the PPL lacks clarity about which contracts may be considered exempt from the require- ments of the Law and this offers scope for abuse. The most apparent opportunity for abuse is through wording that allows single source tendering in ‘urgent cases’. Procuring entities wanting to avoid com- petition only need to delay the process and create an arti�cial ‘urgent case’. 3.39. This anomaly is still being addressed by the government. As a �rst step, from Octo- ber 2011 each procuring entity was required to justify in writing its reasons for selecting single source tendering and its justi�cation. 3.40. The law requires all information related to public procurement is made publicly available. Bidding opportunities, contract awards, and data on resolution of procurement complaints are publicly available on the web site of the authorized agency accessible upon free registration and in a printed edition of Official Procurement Bulletin. It seems clear that this is one of the most improved aspects of the PIM system in Ukraine. 3.41. Participation rates for all tenders appear good and this is a reasonable indicator of con�dence in the system. Projects involving construction regularly attract in excess of 3-4 bid, which is enough to promote competitive pressure. In projects that involve ICT, participation rates are said to be even higher. 3.42. Discussions with officials suggested that there were very few complaints regarding public procurement processes. However at the same time, according to the Anti-Monopoly Com- mittee, complaints about violations of procedures for publication of information represent 29% of the total number of complaints in Public Procurement. However complaints are not always an accurate indicator of con�dence in the system. In many countries, they are merely an indicator of the style of business culture. 3.43. One problem often highlighted was the inflexibility of rules regarding tenders that were above the initial estimate. This rule applied even when an open competition demonstrated the best price/value. The rule means that in the event of a bid being higher than the original estimate, 3. PIM Assessment 23 the procurement process would be cancelled and a new procedure would have to be started. This causes inevitable delays and some upward pressure on prices as a result of the delay. 3.44. There is little or no tracking of physical progress against costs incurred during the implementation of a project. Due to the strong discipline of the budget and the inability of imple- menting authorities to spend more than their delegated limit, costs are rigorously tracked. The same cannot be said of the tracking of physical progress and it is difficult to determine during the course of implementation whether, for example, when 40% of the costs have been disbursed whether this translates in to 40% completion. 3.45. Possibly the most signi�cant problem in implementation in multi-year projects is that signi�cant in-year amendments / adjustments to the budget are frequent to the extent that budget entities might usually expect to suffer a negative adjustment. This creates uncer- tainty about completion targets for implementing entities and uncertainty about payment amongst contractors, the risk of which will inevitably be factored in to tender prices. 3.46. Handover of �nished assets is relatively well formulated and is certi�cated rather than declarative. Assets that are considered SIPs are registered on to the State Property Register, the rest are not. This makes it impossible to build a picture of what assets the state owns and runs on behalf of its citizens. Developing better information about the assets in its ownership remains a key challenge for Ukraine. Understanding the availability and condition of assets is a key informer of what new assets may be required and what surplus assets may be disposed. Adjustment for Changes in Project Circumstances 3.47. One of the major tell-tale signs of inadequacies in project preparation and review are to be found in the number and frequency of changes to projects during the implementa- tion. Tellingly, many projects in Ukraine are regularly adjusted during implementation. Adjustments can be of a signi�cant nature and many could have been prevented through better preparation work or have been identi�ed through adequate scrutiny. 3.48. Changes can materially affect the performance and objectives of investments but in the case of Ukraine, since objectives are not set and performance is not measured, the true economic and social costs of this behavior remain hidden from the government and the citizens. 3.49. When problems occur that result in the slowing down of project implementation, virement is possible between different projects in the same program (within the same budget entity) and can be authorized either by the Ministry of Finance or the Council of Ministers depend- ing on the level of authorization. Virement between different budget entities is also possible but only through an amendment to the Budget Law. This is not an uncommon occurrence. 3.50. There is no known mechanism that would trigger a review of whether a struggling project would continue to be justi�ed, other than it being abandoned due to lack of budget sup- port to �nish it. Given the situation described above, this is a pity since there are likely to be a number of cases where the project could or should be halted in conditions where signi�cant changes ren- der the project to be of little or no economic value. Without this mechanism it is possible that ‘good money’ is given to �nishing ‘bad projects’. 24 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Facility Operation 3.51. The State Property Register registers �xed assets starting 2012 but the score re- flects the situation in 2011 – the assessment year. Even at the time of the assessment, the adequacy of the information inputted on to the register is yet to be determined. It is hoped and expected that the score in this indicator will improve in the next assessment. 3.52. Although completed projects are formally handed over in a certi�cated procedure, the assessment team discovered that there are a quite a number of examples of newly handed- over assets that require further expenditure in order for them to be �t-for-purpose. This might indicate poor monitoring during implementation, contractors cutting corners on quality in order to deal with budget pressures (with or without the collusion of the contracting authority) or poor hando- ver procedures. 3.53. Generally, sufficient funding is made available to operate new assets in the short term. Budget funds to do so are only allocated annually and can easily be reduced. The word ‘suffi- cient’ is interpreted to mean ‘enough to maintain the basic level of service required of the asset. It is not certain in any way however whether sufficient funds are made available for maintaining assets. On the physical evidence of the asset condition of other state properties and on the basis of discussions with officials, it would seem that funds for maintenance are far from sufficient, in fact it might be con- sidered that maintenance is the �rst expenditure item to be cut when funds are limited as they most often are. Funding for the maintenance of new assets in the short term should be minimal. Budget funds to pay for maintenance are only allocated annually and can easily be excluded or reduced to an unsatisfactory level. 3.54. Some service delivery agents collect information on service recipients depending on the degree of visibility of the service. These surveys appear to be orientated towards customer service satisfaction rather than asset condition surveys. Ex-post Evaluation 3.55. There is no policy or guidance regarding post project reviews. In recent times the Ac- counting Chamber of Ukraine has attempted to undertake some basic form of performance auditing, but more generally there appears to be no institutional understanding of the potential value of doing such reviews. When the subject matter was raised in the assessment, it was said that results should be ‘obvious’. Actually undertaking performance based ex-post evaluations is somewhat undermined by the fact that there is little or no current basis for setting project objectives based on performance indicators. Until this element of a PIM system is improved to provide at least some basic indicators of success in project formulation, it would hardly be meaningful to attempt to assess the value of a proj- ect ex-post. 3.56. There is no impact or investment evaluation and little comprehension of the value in doing so. Although it was claimed that many projects were subject to a form of basic post project review but further questioning revealed that this relates only to physical input elements of the project rather than its subsequent performance or capacity to meet its objectives. It should be pointed out that some evidence of external audit takes place through the Regulator in the case of Ukrenergo. 3. PIM Assessment 25 PIM Indicators Report Indicator/Dimension Score 6 Brief Explanation Notes PIM 1 While overall expenditure outturn compared to original Multiyear budgeting with Aggregate capital D budget Ukraine scores well both in PEFA 2007 and in credible ceilings could expenditure out-turn PEFA 2011, the situation with capital expenditures is improve situation considerably. compared to original considerably worse and while some improvements As the new Budget Code budget occurred starting 2011, under-execution remains a already established legislative problem for now. grounds for that, it is a matter of implementation over the medium run. 1.1. Capital expenditure D The execution rate for capital expenditures was 56.9% - out-turn compared to 2009, 70.8% - 2010, 71.4% - 2011. Thus, the actual capital original budget expenditures deviated from the planned once by more than 25% leading to a core of D for the sub-dimension. 1.2. Donor project D The execution rates for donor-�nanced projects was expenditure outturn 57,3 % in 2009, 56,7 % in 2010, and 95,7 % in 2011.While compared to original 2011 has shown considerable improvement, low rates in estimates 2009-2010 lead to a score of D. PIM 2 In 2008 the Stabilization Fund was established that Composition of capital D existed in 2009-2010. The Fund was created to deal with expenditure out-turn the crisis and bank re-capitalization; however other compared to original capital expenditures were �nanced through this fund. budget The Government resolutions have de�ned objects and sectors for investment and were frequently changed within a year. The size of the fund was at 7% of the total expenditures and was �nanced from privatization and internal borrowing. As a result of crisis and poor planning, out of 25 main spending units 15 had capital expenditures execution deviating from plans by more than 25%. In 2010 only 8 out 25 spending units were deviating by more than 25% from the plan. In 2011, only 3 out of 25 main spending units deviated by more than 25%, and other 3 spending units deviated by more than 20%. PIM 3 Aggregate revenue A out-turn compared to original budget 3.1. Revenue out-turn A The revenue outturn compared to original budget was compared to original 82% in 2009, 97% in 2010 and 100% in 2011 justifying the budget score of A. 3.2. Quarterly donor A The sub-dimension was not projects disbursements as assessed. compared to plan PIM 4 Stock and monitoring A of capital expenditure payment arrears 6 Indicator scores are in bold capital letters: A, B, C, or D. Dimension scores are in small letters (See Annex – Conversion Table for Indicator Scoring). 26 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Indicator/Dimension Score Brief Explanation Notes 4.1. Stock of capital A The payment arrears on capital expenditures were at a low The payment arrears are not expenditures payment level in course of 2009-2011. In 2009 the arrears stood at evenly distributed across arrears as a share in total only 2.3% of capital expenditures, arrears were reduced to different types of capital capital expenditures 0.3% in 2010 and then increased in 2011 to 1.1%. expenditures. The payments arrears in capital transfers were only 0.5% in 2011, 0.2% in 2010 and 1.2% in 2009. The highest level of payments arrears was observed in capital reconstruction and renovation – 10.8% in 2011; 0.9% in 2010 and 5.6% in 2009. 4.2. Availability and A The Budget Code envisages that the Treasury servicing The consolidated information on comprehensiveness includes commitment control and payment control. accounts payable and receivable of the information The Ministry of Finance order sets the procedure for is available on the Treasury on payment arrears registration and accounting for budget commitments in website on a monthly basis (including time pro�le) the Treasury (Order of March 2, .2012 № 3097). Annual budget execution According to this order, spending units submit registry reports are also available on the of commitments to the Treasury within 7 days of taking Treasury website and contain the commitment. The information is submitted with a data on accounts payable and break down by economic classi�cation with reference to receivable with time pro�le by program classi�cation budget program, and include payment arrears data. The budget programs allow to identify investments and thus information on payments arrears related to capital expenditures is available .on an annual basis PIM 5 Classi�cation of the C+ budget 5.1. The classi�cation A Ukraine uses economic and functional classi�cations in system used for line with GFS/COFOG standards, which is now in transition formulation, execution from GFS 1986 to GFS 2001. As part of this transition the and reporting of the IPSAS compliant modi�ed accrual accounting standards central government’s would be introduced starting 2013. Ukraine also uses the budget program classi�cation of the budget since 2001. 5.2. The de�nition of a C Ukraine uses program classi�cation of the budget project. according to the Order of the Ministry of Finance of January 14, 2011 #11 “On Budget Classi�cation�. The program budget classi�cation contains a identi�er for budget programs that �nance investments (5th digit of the code should be 8). At the same time application of classi�cation is not universal. In 2011 the expenditures on reconstruction of Medical University were classi�ed as investments while construction of clinical hospital was not. These cases are not infrequent. At the same time it is usually possible to understand from the name of the program whether it is investment program or not. Given the considerable discrepancies in the process and discrepancies in the de�nitions of investment program, project., etc. the score for this sub-indicator is C 7 http://zakon2.rada.gov.ua/laws/show/z0419-12/ed20120406 3. PIM Assessment 27 Indicator/Dimension Score Brief Explanation Notes PIM 6 The annual budget law ful�lls only 4 out of 9 criteria as The criteria (8) is addressed Comprehensiveness of C described below: in the 2011 Budget Code information included in (1) While the information on budget programs including that envisages the Cabinet of budget documentation investment programs is provided as part of the budget Ministers de�ning methodology law. At the same time the information on the current year for assessing the �scal impact investment program and anticipated level of execution is of quasi-�scal operations and not submitted including it into the annual (2) The amount appropriated by MDA for the budget budget submission but only year for each major central government budget-�nanced starting with Budget 2014. capital spending program is included into the budget law including the Road Fund (where Ukravtodor is the main spending unit) (3) The total approved multi-year cost of each major investment programs is included as a forecasted amount for the next two years starting budget of 2012. At the same time, these amounts are not binding and it is difficult to say whether they would be adhered to going forward (4) The donor projects are usually included into the budget as budget programs with the name indicating the title of the project. Additional detailed information is submitted in supplementary budget documents. (5) As part of 2012 budget process, all main spending units submitted information on objectives, and target results for 2012-2014, however this practice did not exist in 2011. (6) Details of capital grants or transfers, and of lending (including on-lending) by central government to State Owned Enterprises and/or Sub-national Governments that is predominantly intended to �nance capital expenditure is submitted according to program classi�cation however the information the detailed information is not submitted as part of annual budget law (7) The detailed information on investment programs requiring sovereign guarantees is submitted as part of the annual budget law (8) The nature, rational and �scal impact of quasi-�scal operations in not included into the budget document (9)PPP are not currently covered by the annual budget law PIM 7 Extent of unreported C+ capital spending 7.1. The percentage D The SOE and four social insurance funds are major of unreported capital omissions in reported capital spending. The level of SOE spending (by source) in investments in 2009-2010 exceeded 20% of total public total capital spending capital spending. (excluding any donor funded projects) 7.2. Income/expenditure A The donor projects such as World Bank investment information on donor- projects, EU budget support programs, and other similar funded projects which is projects are included into the budget execution reports. included in �scal reports PIM 8 PIM-related inter- C+ governmental �scal relations 28 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Indicator/Dimension Score Brief Explanation Notes 8.1. Policy coherence C The strategic planning is done under the framework of for PIM across levels of the Law of Ukraine “On State Forecasting and Social and government Economic Development Programs� #1602, which envisages a program of social and economic development programs for short and medium term as well as regional and local strategies of different levels. At the same time the medium term forecast of social and economic development was not approved lately except for 2010 when the Parliament approved such program by the Law #2278, however this program did not address issues of PIM. Thus, there is no clear hierarchy of strategic documents and insufficient coordination that would ensure consistency of PIM across levels of government. The Cabinet of Ministers usually allocates investment subsidies to the regions following budget approval. 8.2. Transparency C The allocation of capital transfers is regulated by the and objectivity in the procedures set by the Cabinet of Ministers. There are two horizontal allocation of types of capital transfers that are managed by the Ministry capital transfers of Finance and Ministry of Regional Development. While some rules do exist the system is subject to considerable discretion and vulnerabilities. In 2011 the list of capital transfers were approved by the Government resolution #98 of February 9, 2011. However the resolution was amended 25 times in the course of the year. 8.3. Timeliness of reliable B According to the Article 75 of the Budget Code of Ukraine, budget information to SN the Government ensures providing all the budget governments information necessary for detailed budget process to local government following approval of the annual state budget. While the budget calendar envisages approval by November 20, in 2011 the budget was only approved during 2011, which delayed provision of information to SN government by a month. Also, until 2011 (budget 2012), there was a considerable share of unallocated capital transfers to local governments that were then allocated by government. However starting with 2012 budget, over 97% of capital transfers were allocated in the budget law. 8.4. Extent of A Article 7 of the Budget Code set that all the revenue and consolidation of expenditure information from the state budget, AR Crimea �scal data for general budget and local budgets should be consolidated. Thus. government according to All local budget capital spending information is collected sector categories and consolidated as part of the budget execution reporting including by functional classi�cation. PIM 9 Management of �scal C+ risks from capital spending outside central government 9.1. The extent of central D There appears to be little effective monitoring of SOEs government monitoring on-lending or government guaranteed lending by MoF. of SOEs that are recipients Instead oversight is delegated to line ministries. There is no of on-lending and/or uni�ed methodology evident for recording the aggregate government guaranteed of �scal risks from state guarantees. lending No oversight of SOE activity by the Accounting Chamber. 3. PIM Assessment 29 Indicator/Dimension Score Brief Explanation Notes 9.2. The extent of central A SNGs operate through the Treasury and cannot spend government monitoring more than their budget. Only local authorities with of SNGs that are populations of >300k can issue debt but only then within recipients of on-lending rules established in the Budget Code. and/or government Annual Financial Reports consolidate national and sub- guaranteed lending national accounts. 9.3. The estimated size of QFA is impossible to measure but due to the SOE sector capital spending related alone comprising 20% of GDP, a reasonable estimate might QFAs be in the range 1-5%. Prospects of additional un-managed �scal risks through future PPP projects raise the possibility of a worsening situation. Infrastructure projects negotiated directly with foreign governments that require sovereign guarantees also need to be monitored. PIM 10 Availability at a strategic level through the Treasury It might be considered that Public access to key A website. the information is presented information on capital Year-end �nancial statements are made available to the in a highly technical form that spending public even before Audit. frustrates detailed scrutiny by Information about contract awards is published through media and CSOs. the Public Procurement Herald within 7 days. PIM 11 Investment guidance, D+ project development, and preliminary screening 11.1. Strategic guidance C There are sector strategies that outline plans for physical and social developments but provide no guidance for prioritization. A constant complaint is that priorities are constantly changing. 11.2. Project pro�les D Project pro�les are prepared for those projects that go through the MEDT but are inconsistent in scope and quality. 11.3. Preliminary Should be carried out by the responsible LM/Agency. No screening C evidence that any projects are screened out for reasons other than incorrect paperwork. 30 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Indicator/Dimension Score Brief Explanation Notes PIM 12 D+ Formal project appraisal 12.1. Clarity of roles in Despite the fact that there appears to be a reasonable planning process B understanding of the roles that each institution plays, the roles of each institution are not well co-ordinated. Nevertheless the arbitrary use of different terminology allows this situation to be abused. 12.2 De�nition of steps in D There is no central guidance on the basic steps that project development might be expected in a well-functioning PIM system. A systemic flaw. 12.3. Technical guidance C Technical appraisal guidelines do not exist but there are formal procedures related to the physical nature of the proposal. They are input based and administrative in nature. 12.4. Extent of project D Less than 10% of projects are subject to any appraisal by appraisal law. The capacity to appraise even this small amount of projects effectively is compromised by totally inadequate capacity, severely affecting quality. 12.5. Quality of project D Key elements of appraisal missing. Particular weaknesses appraisal are objective setting, option appraisal and bene�t assessment. 12.6. Proportionality of Appraisal is based on the arbitrary judgment of proposing appraisal D officials as to whether the project is considered a SIP or not. Two very similar projects in the same sector may in one case be subject to appraisal if it is considered a SIP or no appraisal if it is not. Currently plans to widen the scope of appraisal still intend to exclude social projects regardless of size or signi�cance. PIM 13 D In only SIPs is there some form of review. This amounts Independent review of to less than 10% of the total of capital that might be appraisal considered to be Public Investment Projects. In most cases there is no review. PIM 14 Orderliness and B participation in the annual budget process 14.1. Existence of and The budget calendar is clearly set out in the Budget Code. Score would be B for current adherence to �xed C However projects can be proposed at any time of the year. expenditure but is C for capital project planning and Funding for unplanned projects can be found through because budget entities don’t budget calendar new lines in the budget. There is little independent review know what amount might be of projects. available until very late in the Capital budget ceilings are not part of the budget circular. process. 14.2. Guidance on Procedural and administrative guidance exists but it is not preparation of capital C of an economically substantive nature nor is it speci�cally spending proposals addressed to capital spending but to expenditure in general. 3. PIM Assessment 31 Indicator/Dimension Score Brief Explanation Notes 14.3. Time for central Resources for analyzing project proposals are highly The score A is consistent with agency analysis and A inadequate. Most project proposals are not analyzed at all. the scoring methodology advice Those that are subject to analysis are put under external because it is a time-bound pressure to approve them. Politically motivated projects dimension only. are able to �nd a place in the budget without any scrutiny or analysis. However in the event that analysis was desired, there is sufficient time to do this (8 weeks). 14.4. Timeliness of B Parliament approves the budget in a timely manner before The 2009 anomaly prevents budget approval by the the start of the �scal year with the exception of 2009. this score from being A as the legislature The 2011 budget was approved within the prescribed scoring relates to the previous timetable. 3 years. PIM 15 Multi-year perspective D+ in �scal policy and management 15.1. Multi year forecasts The 2010 Budget Code introduced a medium term The recent PEFA 2011 provides and functional allocations C perspective. The government also has to submit to additional details on the multi- the legislature estimates for the following two years year budgeting. Currently, for each budget program that requires a multi-year multi-year budgeting still implementation. permits more and more projects to be �t in and that gives project promoters no certainty of future �nancing. The main reason for that is that the multi- year budget ceilings are not respected and at least up until 2012 the annual budgeting process is not strongly linked to the MTEF. 15.2. Existence of costed Sector Strategies are estimated on the basis of individual sector strategies D projects. They often lack substantiated capital estimates and rarely if ever, include a consideration of the associated recurrent costs. 15.3. Multi-year Project There is no database of proposed and approved projects, Databases C the exception being State Investment Programs which represent less than 10% of the total. 15.4. Quality of a PIP Not applicable. (where relevant) PIM 16 D+ Project selection and budgeting 16.1. Transparent criteria D Criteria are not made clear for the selection of projects in for project selection to the budget. This makes it easier for politically motivated projects to enter the system without analysis or scrutiny. 16.2. On-going projects C There have been problems in this area however the State receive sufficient funding Financial Inspection and the Accounting Chamber have pointed out the problem with a view to it being addressed. Whether it will be addressed remains to be seen. 32 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Indicator/Dimension Score Brief Explanation Notes 16.3. Multi-year budget C Multi-year contracting is permitted and is common but the authority risk of non-availability of budget funding in subsequent years remains a possibility. There have been many instances where this has been a problem. 16.4. Capital and current It remains quite rare for capital projects to have their spending fully integrated C associated recurrent costs incorporated into budgets except in cases where capital is used to renovate existing assets and the operational costs are already being borne. New or green�eld projects still do not take account of operational recurrent expenses in the project planning process putting at risk the availability of budgetary funds. PIM 17 C+ Project Implementation 17.1. Guidelines on Guidelines exist in line ministries and are related to project implementation C technical norms and regulations concerning physical components of the project. There appears not to be any centrally based support or advice on the methodology of project management. There appears to be no culture of project management practices. 17.2. Clear accountability Implementation does not appear to be a planned activity and implementation C with few project management practices or disciplines in plans place. Accountability is blurred. 17.3. Total project cost There appears to be no capacity to track total project management system D costs over time against the approved budget. De�nition of investment project doesn’t exist so it becomes difficult to track something that doesn’t exist. 17.4. Formal handover on Handover is a relatively formal process involving completion A commissioning checks that include �t for purpose, quality and technical compliance tests. PIM 18 Predictability in the B availability of funds for commitment of expenditures 18.1 Extent to which cash A Cash flows are monitored through a Treasury system flows are monitored that makes it almost impossible to spend more than the allocation and is updated monthly. 18.2 Reliability and Budget entities are able to plan ahead and procure works But to be taken in the context of horizon of periodic A on the basis of the annual budget appropriations up to the comments in 18.3 in-year information to limit of their appropriations only. MDAs on ceilings for expenditure commitment 3. PIM Assessment 33 Indicator/Dimension Score Brief Explanation Notes 18.3. Frequency and Signi�cant in-year amendments / adjustments to the transparency of in-year C budget are frequent to the extent that budget entities adjustments to budget might expect to suffer a negative adjustment. The allocations which are transparency of doing this is limited to the need to make decided above the level an amendment to the Budget Law in order to do so. of management in MDAs PIM 19 B Value for money in procurement 19.1. Transparency The procurement framework has improved in recent times comprehensiveness and B as it moves towards a closer approximation to EU norms. competition in the legal The prime law is the Public Procurement Law with and regulatory framework regulation being through government resolutions and guidance through the Ministry of Economy. 19.2. Use of competitive D Concerns remain about the clarity of rules that allow for procurement methods single tender procedures. The phrase ‘urgent cases’ is the phrase that gives most cause for concern as the phrase is not de�ned and open to abuse. 19.3. Participation Rates A Participation rates are good in contracts that clearly An extra dimension added to require competition. This indicates a reasonable level of the methodology. con�dence in the system from suppliers and contractors. In the event that a competitive procedure is used it seems common for at least 3 or 4 bidders for a construction tender and more for and ICT project. 19.4. Public access to There is no government wide procurement plan complete, reliable and A but government entities regularly publish bidding timely procurement opportunities along with award decisions in a timely information manner consistent with the law. 19.5. Existence of Anti-Monopoly Committee reviews complaints about Anti-Monopoly Committee an independent B procurement procedures. There was little evidence of a does not comprise of members administrative complaints culture. drawn from the private sector procurement complaints and civil society, although this mechanism not necessarily wrong. However, the score is consistent with the methodology. PIM 20 Effectiveness of internal C+ controls and internal audit of capital project expenditure 20.1. Expenditure The Treasury system requires registration of commitments The negative aspect of this is commitment controls A before they are enacted. There is no possibility of MDAs that commitments for multi-year spending more than their budget allocation for the year. projects cannot be made. This control is clearly understood and adhered to. 20.2. Compliance with It is believed that compliance with administrative rules is rules for processing and B high. Without the notable exception of capital expenditure recording transactions related to Euro 2012 the score might have been ‘A’ 34 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Indicator/Dimension Score Brief Explanation Notes 20.3. Coverage and During the base year of assessment – 2011 – there was no This function has become a quality of the internal D internal audit function. reality since Jan 1 2012. audit function 20.4. Extent of Too early to assess. management response to --- internal audit �ndings PIM 21 D+ Project Adjustment 21.1. Project progress C Financial progress reporting and monitoring is good but reports and monitoring physical monitoring against �nancial outlays is poor. 21.2. Virement between C Virement is possible between different projects in the projects same program and can be authorized either by the Ministry of Finance or the Council of ministers depending on the level of authorization. Virement between different budget entities is possible but only through an amendment to the Budget Law. 21.3. Nature of project D Many projects are regularly adjusted during adjustments implementation. Adjustments can be of a signi�cant nature and many could have been prevented through better preparation work or have been identi�ed through adequate scrutiny. 21.4. Explanation of Not scored. No scoring mechanism in variances from plan methodology. 21.5. Funding in tranches Not scored. No scoring mechanism in linked to phases methodology. 21.6. Mechanism to There is no known mechanism that would trigger a review trigger review of project D of whether a struggling project would continue to be justi�cation justi�ed, other than it being abandoned due to lack of budget support to �nish it. PIM 22 C+ Facility Operation 22.1. Asset registers C State Property Register registers �xed assets starting 2012. Potential to be B in future. The adequacy of the information inputted on to the register is yet to be determined. 22.2. Assets �t for purpose C There are a number of examples of newly handed-over assets that require further expenditure in order for them to be �t-for-purpose. 22.3. Sufficient B Generally sufficient funding is available to operate new The word ‘sufficient’ is Operational funding assets in the short term. Budget funds to do so are only interpreted to mean ‘enough allocated annually and can easily be reduced. to maintain the basic level of service. 3. PIM Assessment 35 Indicator/Dimension Score Brief Explanation Notes 22.4. Sufficient Funding for the maintenance of new assets in the O&M = Operation and Maintenance funding D short term should be minimal. Budget funds to pay for Maintenance and have been maintenance are only allocated annually and can easily be separated in to two different excluded or reduced to an unsatisfactory level. dimensions. There was evidence to suggest that maintenance of public ‘Sufficient’ maintenance funding assets in Ukraine is seriously underfunded. is taken to mean enough to maintain the asset in fair condition over its lifetime. 22.5. Monitoring of C Some service delivery agents collect information on service delivery service recipients depending on the degree of visibility of the service. Most often this is not the case. PIM 23 D+ Basic Completion review and evaluation 23.1. Policy and guidance D There is no policy or guidance regarding post project on post-project review reviews. Furthermore there appears to be no institutional understanding of the potential value of doing such reviews. Since there is no current basis for setting project objectives based on performance indicators it would not be meaningful to attempt to assess the value of a project ex-post. It was said that results should be ‘obvious’. 23.2. Completion of basic C Some projects are subject to a form of basic post project post-project reviews review but this relates only to physical input elements of the project rather than its subsequent performance / capacity to meet its objectives. 23.3. Completion of D There is no impact or investment evaluation and little Very hard to evaluate when impact evaluations comprehension of the value in doing so. there are no objectives. 36 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Indicator/Dimension Score Brief Explanation Notes PIM 248 In year budget execution reports cover general Quality and timeliness A government as a whole and are prepared according to of in-year budget program, functional, administrative and economic budget reports classi�cations. They are easily compared to the approved budget. Reports covering expenditures and revenues, as well as state debt, guarantees and obligations (liabilities and payables) are prepared regularly (monthly and quarterly). Quarterly debt reports include data on conditional and guaranteed debt obligations and liabilities. Local government reports are prepared by regional Treasury units and submitted to the central Treasury within 8 days of the end of the month, which facilitates consolidation of data. The STU subsequently prepares monthly and quarterly consolidated reports based on functional, administrative and economic budget classi�cations, including the data from local governments within 25 days of the end of the month. The single treasury account executes all budget transactions (revenues and expenditures) for all central government ministries and central spending units. PIM 259 The annual consolidated �nancial statements cover Quality and timeliness B+ central and local government revenues, expenditures, of annual �nancial government debt, �nancial assets, liabilities and statements obligations. The government’s �nancial statements omit the consolidation of the state-owned enterprises where the government holds the majority and controlling share. In Ukraine, state-owned enterprises are not considered as part of general government despite the signi�cant role they play both in terms of receiving budget allocations and transfers, and in terms of generating revenue and taxes for �nancing the state budget). The government prepares its annual �nancial statements within four months after the end of the �scal year (in accordance with Regulation #419 and Article #61 of the Budget Code). National accounting standards are applied in the production of the annual �nancial statements. National standards are mostly cash-based, with elements of accrual accounting such as reporting on assets and liabilities, debt. These standards are not compliant or aligned with IPSAS, though the government is currently developing new public sector accrual accounting standards which aim to be IPSAS-accrual compliant. The national standards have been applied consistently over time and are well known within MoF and Treasury. 8 The indicator PIM-24 is fully based on 2011 PEFA Assessment. 9 The indicator PIM-25 is fully based on 2011 PEFA Assessment. 3. PIM Assessment 37 Indicator/Dimension Score Brief Explanation Notes PIM 26 The Accounting Chamber of Ukraine (ACU) is formally The Accounting Chamber of Scope, nature and D+ subordinated to the Parliament (Rada). Its remit is to Ukraine focuses on investment follow-up of external “control� expenditure from the State Budget. programs and projects as part audit The main focus of the ACU’s work continues to be on of their performance audit. ex post compliance control of expenditure from the The annual reports for 2010 State Budget in accordance with the remit in the 1997 and 2011 have indicated a law. ACU’s remit excludes revenue, local governments, number of issues with regards extra-budgetary funds and SOEs, except to the extent that to implementation of the Euro State budget funds are concerned. The ACU has no role 2012 investment program, in relation to SOEs or extra-budgetary funds (four extra including cost overruns, project budgetary funds represent close to 20% of GDP). Some management issues and others. audit areas can only be addressed with the approval of Parliament. ACU’s overall Annual Report for the previous year is produced by the 1st of December of the current year (Article 35 of the Accounting Chamber Law, 1996), but does not include any discussion of the government’s �nancial statements. Formal response to audit �ndings and recommendations comes primarily from the audited government bodies and the Cabinet of Ministers who should reply within 15 days from receiving a report (Article 29 of the Accounting Chamber Law). The Rada discusses ACU reports on a case by case basis and “…upon consent of the Parliament…� as speci�ed in Article 34 of the Accounting Chamber Law. The ACU’s 2009 Annual Report states that bodies audited have in a number of cases taken the corrective measures required, and accepted the audit recommendations. PIM 27 The legislature’s review covers �scal policies and Legislative scrutiny of B aggregates for the coming year as well as detailed capital spending in the estimates of expenditure and revenue. The Budget Code annual budget law set responsibilities of the Parliament and Parliament Budget Committee. The Parliament according to the 2011 Budget Code also approved investment budget programs that envisage multiyear estimates, objectives and indicators. The legislature’s procedures are well established involve three readings of the annual budget law as well as prior review of broader �scal policy as part of the budget circular approval. The timing is adequate for legislature scrutiny. The legislature approves budget directions in May. The draft budget law is submitted to the Parliament by September 1. On October 15, �rst reading is conducted, based on which the Parliament send the Budget back to the Government with comments. The second reading should be done before November 20. PIM 28 The review of audit reports is conducted within 6 months Legislative scrutiny of D+ of the submission although not all reports are reviewed external audit reports in sufficient detail. The Accounting Chamber of Ukraine on capital spending (ACU) reports on the previous year’s budget execution statement, within two weeks of its receipt from the cabinet of Ministers. These reports, which are analytical and do not rest on audit examination, are formally considered by the Rada in reaching its conclusions on the execution statement. There is currently no system in place to track the receipt and handling of audit reports. In-depth hearings are not being conducted. 38 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT 4. Outline Reform Strategy 4.1. Ukraine has a number of weaknesses in Public Investment Management, some of them critical that need to be addressed through a reform program. Following the diagnosis iden- ti�ed in the preceding sections, this section suggests what might be done to deal with these weak- nesses and suggest what might comprise the key elements of a reform program. The issues identi�es include: (1) There is no coherent holistic system of Public Investment Programming and Management in place; (2) There is no clear institutional understanding of the fundamentals of public investment man- agement, the bene�ts of a well-functioning system and the costs/risks attached to the current system; (3) There is no agreed legal de�nition of a public investment project leading to institutional confu- sion about the subject matter. This results in arbitrary and subjective classi�cations creating a parallel budgeting system for public investment projects; (4) The strategy and direction of the government changes too frequently making it difficult for implementing entities to plan their projects and programs with any degree of certainty; (5) There is almost no concept of clear output based / performance based objective setting and economic option appraisal; (6) The selection process is overly politicized and under appraised for economic value; (7) Corruption contributes to the erosion of efficiency in executing public investment projects with construction costs being estimated at 23% higher than in Germany, 22% higher than the EU average and 20% higher than neighboring Poland; (8) The current dynamic nature of the civil service creates institutional instability that limits the cre- ation of real capacity in the near future. It is understood that only around 20% of those officials that took part in Bank sponsored Cost Bene�t Analysis training in 2010 are still doing the same job. 4.2. The PIM reform process faces a trade-off described by Rajaram et al (2010) – strengthening appraisal versus strengthening implementation. The assessment suggests that while improving implementation is important as evidenced by low execution rates, the scarcity of resources that are directed towards public investment, vast investment needs and currently ad hoc process for investment planning calls for rapid improvements in project selection. Thus, short term actions are focused on the immediate needs of the PIM system while medium term actions envisage a higher level of sophistication in appraisal as well as project performance monitoring. A standalone paragraph is devoted to observations on capacity development and improving the efficiency of this process. 4. Outline Reform Strategy 39 In the Short Term 4.3. Without doubt one of the most signi�cant failings of the current arrangements is the lack of a cohesive and holistic PIM system. The government should take steps to ensure that each step of a PIM system, and the system as a whole, is clearly identi�able and understood by all the actors and institutions involved. They should ensure that all roles and responsibilities are under- stood through each step of the process. The Bank, as part of its continuing support will shortly draft a ‘Step-by- Step Guide’ to the PIM process for the government. This should be used as the framework for identifying gaps in current capacity and for developing the reform agenda. In short: (a) A system should be developed along the lines of PIM international good practice; (b) A system should be implemented; (c) It should be clear and transparent to all public entities how they should proceed if they want a project to be funded or guaranteed by the government. 4.4. The government should clearly de�ne the nature of Public Investment Programs and Projects in a legally enforceable manner. The de�nition of the public investment project as a unit of analysis for decisions taken within the PIM cycle – pre screening, appraisal, selection, post eval- uation – is critical to ensure that no investment projects avoid adequate scrutiny and at the same time capital items such as office equipment and vehicles are not included. Currently, as the assessment pointed out there are construction projects, investment programs, investment projects or capital ex- penditures that are not considered “investment projects� by government agencies however �nance construction of social infrastructure. Serious weaknesses and abuse cases in PIM call for an urgent action in the short term. A formal review such as a PIP review can be immediately enforced to every project above certain �nancial threshold. 4.5. An initial screening document should be developed and published which all public entities wanting budget funds (or guarantees) for a project falling under the new de�nition, should be mandated to use. This document would contain basic information about the project, its objectives and policy aims and a rough estimation of costs (capital and recurrent) and bene�ts (both real and perceived). The document would be quickly screened initially by a central independent body such as MEDT or similar and public entities proposing well-conceived projects would then be allowed to proceed to a more detailed assessment of the project’s worth. Rejected projects at this stage would be spared the time and effort of preparing projects that might ultimately never receive funding. 4.6. Ensuring adequate capacity is a key to make the PIM system function. Work should start as soon as possible to bring forward a widespread program of capacity building in the �eld of objective setting based and output based performance targets together with option appraisal tech- niques. Capacity building shall contribute to change in a mindset required to approach the project planning and design from the point of view of the objectives to be achieved rather than inputs re- quired. At the same time, the Government should be careful when planning capacity building due to factors mentioned below (paragraphs 4.16-4.17). 4.7. In order to provide a more cohesive public investment strategy, the government should strongly consider the development of a medium-term Strategic Investment Planning tool which would initially be developed from the database of SIPs that is currently being developed by MEDT. This would allow the government to synchronize the budgeting process with current and fu- ture investment priorities. It would also better match projects with available resources – both capital 40 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT and recurrent. It should be multi-year in nature with a ‘rolling’ 5-10 year horizon and hold records on the status of all projects that have reached a pre-determined quality threshold. This planning tool should be electronic and should aim at becoming the heart of a modern PIM system. In the Medium-Term 4.8. In the event that the government agrees to develop a multi-year Strategic Investment Plan, in order for it to become a fully functioning planning tool, it will need to be developed further from the initial database. In order to improve the quality of the information contained within the data- base and for the information to be manipulated to become useable and valuable, project appraisal will need to become more developed and advanced. Suggestions made above in the ‘Short-Term’ measures will, if they are fully implemented already create better projects. Formal appraisal techniques, based on a common template issued by the government, should be fully understood by all proposing entities and adopted. Support may be required from the private sector in order to provide sufficient capacity to deliver sound project appraisals on the scale that may become necessary over time. How- ever, safeguards must be built into the system in order to avoid creating conflicts of interest.10 4.9. The current situation of MEDT effectively providing appraisals and being the entity re- sponsible for independent scrutiny should be seen as no more than a relatively short-term measure. It is far preferable to have proposing entities retain full responsibility for their own projects from preparing and proposing them to implementing them. The MEDT role other than policy and guidance should over time become one of providing the independent scrutiny to projects proposed by public entities. 4.10. To support this new appraisal and scrutiny system, the government might consider the development of a structured project quality management tool that could be similar in substance to the UK government’s Gateway Review process. The key element of this system should be inde- pendent review of appraisal. In order to be independent, a review must be free of influence from the proposing authority; therefore, for example, if an entity responsible for building highway projects, proposes a new project, it should submit its proposal to another separate entity for checking. The cooperation between the government agency owing the project, Ministry of Economic Development and Trade and Ministry of Finance is crucial for improving projects quality. Projects appraisal is a re- sponsibility of the agency initiating the project. Independent review in many countries is usually con- ducted by the Ministry of Economy or similar entity, while the project selection requires strong role of the Ministry of Finance to ensure that projects �r within available resource envelope and �scal risks are manageable. 4.11. One of the guiding principles of a new Appraisal and scrutiny system should ensure that public authorities proposing new projects demonstrate how rather than if the project would be af- fordable and to identify the assumptions that they are making to support their case. They should focus on how they would complete the project rather than to initiate it and how the subsequent operating and maintenance costs would be funded. 4.12. In order to ensure the swift (and therefore efficient) completion of projects, the Minis- try of Finance should carefully consider prioritizing the allocation of budgetary resources for agreed projects, entirely in line with the implementing authorities’ capacity to consume them 10 For example: an economic consulting �rm might be more inclined to build a positively skewed business case for a project if it be- lieves it will charge more fees from implementing the proposed project. 4. Outline Reform Strategy 41 once projects are approved and underway. The temptation to approve new projects at the ex- pense of funding existing ones should be resisted. Delayed projects tend to cost more. 4.13. Although recent improvements in the procurement regime will help, the government should investigate and monitor closely the discrepancy in construction out-turn costs be- tween Ukraine and its European neighbors. The aim should be to erode and then eliminate this discrepancy. 4.14. The government should consider how it will more closely monitor the performance of projects that the nation has invested in. Once better performance based objective setting has been ingrained in the system, this task will become easier. It should consider what independent body would be best placed to perform this task and report �ndings to the Parliament. 4.15. In order to move to a more formal structured PIM system, after a number of reforms have begun, a PIM Act should be considered that harmonizes all Laws, Decrees and other acts of gov- ernment in the �eld of PIM. It should state clearly the tasks and duties required in the identi�cation, appraisal, assessment, approval, implementation and ex-post evaluation of projects. It should specify which institution performs each duty. All future consideration of PIM policy reforms should also regulate the preparation and implementation of PPP project proposals with other form of PIPs since PPPs are merely an alternative form of implementing Public Investment Projects. There- fore the same rules of preparation, appraisal and selection should apply to all. Capacity Building / System Improvements 4.16. Training for the development of capacity in the PIM system is tricky in Ukraine.. It is becoming clear that due to the ‘dynamic’ nature of the Ukrainian civil service, officials are rapidly moving from one area of the Administration to another. It is therefore likely that officials engaged in project appraisal or project management roles, having just received training through the project may be moved on to a functional area that is no longer relevant to the subject matter. It was notable to the Assessment team that previous training in this area as recently as 2010, had not been as effective as had hoped. Only some 20% of the trainees were still doing the same job. 4.17. This raises the prospect that resources spent on training officials at the present time may not be effective, or at least might have their effectiveness seriously compromised. Instead the govern- ment should aim to ring-fence some this precious resource so that they can have some stabil- ity in the system, giving them interest and insight in to the way that efficient PIMs are prepared and implementation. Officials that are retained in the system for the medium-term will be much more mo- tivated to train new officials coming to this policy area. One possible recommendation is to establish a professional organization under the umbrella of the MEDT/MOF to continuously trace out the whole PIM processes and support the MEDT/MOF. Such organization may be more attractive for highly quali- �ed staff as it would not be subject to civil service constraints. An example of such organization is the Korean Development Institute in South Korea. Annex A. PIM Assessment 42 In addition to the quantitative core “PEFA-PIM� indicators found in Section 3, this assessment includes a color-coded table below sum- marizing the ratings for the core PIM “must have� features as just described. The table is augmented by identifying the most critical weaknesses. UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT These critical weaknesses are tagged by a “hazard indicator�. Table A: PIM Assessment – ‘Must Have’ Features11 Stage Indicators and Dimensions Findings Rating Related Recommendations 1. Guidance National development strategy; sector strategies. Whilst there are development and sector strategies, critically and preliminary there is no strategy or coherent system for managing the See Paragraphs 4.3., 4.4., 4.5., screening investments resulting from these strategies. 4.7. and 4.15. It is a critical failing. First level screening to ensure strategic alignment. Some screening takes place in order to assess technical components and that administrative procedures are correctly carried out. Officials try to align their projects with C sector and national strategies. However there is no �rst level screening related to economic need or value. Formal process for project development. There is no central guidance relating to the process required for project development that goes beyond technical and D administrative procedure. 11 Rajaram et al. 2010. Stage Indicators and Dimensions Findings Rating Related Recommendations 2. Formal project Clarity of roles in planning process. There is clarity of roles in the planning process but that appraisal appears to be because there are so few roles in what B amounts to a barely cohesive system. Formal technical guidance. Technical appraisal guidelines do not exist although See paragraphs 4.2., 4.6., 4.8. technical guidelines related to construction standards and C and 4.11. norms are in frequent and common use. Sound project appraisal. Very few projects receive any appraisal at all. The lack of capacity to do this work means that appraisals are often D done under severe time pressure which will inevitably affect quality. Proportionality of appraisal. Projects considered SIPs are appraised. Projects considered construction projects are not regardless of the size or D signi�cance of the project. Capacity in central agencies and line ministries. Appraisal capacity in central agencies and line ministries See paragraphs 4.16. and 4.17. appears critically lacking. Only MEDT has this kind of C/D capacity (5 people) and they have to cover all projects. MEDT officials carry out limited appraisal work on behalf of line ministries. Some external consultants are employed on larger projects but there appears to be a lack of economic appraisal capacity in the domestic private sector too. Annex A. PIM Assessment 3. Independent Independent reality/quality checks. There is no identi�able independent review of appraisal in review of Ukraine. See Paragraphs 4.9. and 4.10. appraisal 43 Stage Indicators and Dimensions Findings Rating Related Recommendations 44 4. Project Fixed budget calendar with sufficient time for LMs Although there is a �xed budget calendar which is normally selection and to prepare capital budgets. managed in a well-discipined way, capital appropriations C budgeting are not known until late in the process making it extremely UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT difficult for LMs and Agencies to plan ahead. See paragraph 4.12. Comprehensive guidance and capital spending There is no guidance on how to prepare capital projects ceilings. other than through technical construction/engineering D norms. Capital ceilings are not known in advance. Projects developed before submission to MOF in Most but not all projects are developed before submission budget. to MOF in the budget. It is possible to projects to be C included directly into the budget under political pressure. Integration of capital and current spending. Current spending commitments arising from capital projects are not always harmonised. C Current expenditure needed for the efficient operation and the maintenenace of the assets are easily reduced in future years. MOF checking of projects is limited to the procedurl and MOF checking of projects. administrative rather than the substantive. D Legislature has sufficient time to consider projects. The legislature has sufficient time to consider projects, but often doesn’t. B/C Legislature passes budget in timely manner. In all but one of the last three years the budget has been passed in a timely manner. B Stage Indicators and Dimensions Findings Rating Related Recommendations 5. Project Published guidelines. Guidelines exist in line ministries and are related to technical implementation norms and regulations concerning physical components of C the project. Detailed implementation plan with clear There appears not to be any centrally based support or accountabilities. advice on the methodology of project management. There D appears to be no culture of project management practices. Open competition for procurement. The Law requires the default position to be Open See paragraph 4.13. Competition but loopholes allow the system to be abused D from time to time. Effective procurement complaints mechanism. The Anti-Monopoly Commission handles and manages complaints about procurement procedures. B Commitment controls. Cash flows are monitored through a Treasury system that makes it almost impossible to spend more than the A allocation and is updated monthly. Predictability of funding. There are frequent in year adjustments to projects. Multi- year projects cannot predict whether they will receive the C required amount in years beyond the initial budget year. Often they do not receive the required amount. Regular progress reporting. Whilst �nancial reporting is good, physical progress reporting is not. C Active monitoring of progress. As above. Annex A. PIM Assessment Sound internal control and internal audit. Internal controls for �nance are good. Commitments against progress reporting are weak. Internal D audit not functioning in the assessment year. Formal project completion. Completion of assets is formal and certi�cated and includes technical testing. A 45 Stage Indicators and Dimensions Findings Rating Related Recommendations 46 6. Adjustment Constant project adjustments. Many projects are regularly adjusted during implementation. See paragraphs 4.5., 4.6. for changes Adjustments can be of a signi�cant nature and many could D in project have been prevented through better preparation work or UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT circumstances have been identi�ed through adequate scrutiny. Explanation required for variances from budget Variances appear to be accepted as a norm. and plan. D Mechanisms for project adjustment. There are no mechanisms for adjusting a project where the basic assumptions have been altered by events. D Periodic review of costs compared to bene�ts for Does not happen. major projects. D Mechanism to stop project. There is no mechanism to stop a project that appears to be on course to fail to deliver its objectives. This could be D because objective setting during project preparation is poor. 7. Facility Effective handover of assets. Assets are handed over in a competent and certi�cated operation manner. A Assets �t for purpose. There are numerous examples of projects that require further expenditure in order for them to be �t for purpose. C Asset registers. Only SIPs are included on an asset register. Since this is a recent event, the quality and the completeness of the work C is not yet known. Sufficient O&M funding. There is usually enough funding for operations during the �rst years of a new project. During this time little C maintenance funding is required but both are susceptible to regular cuts in expenditure during tougher budget years. Stage Indicators and Dimensions Findings Rating Related Recommendations 8. Ex post Basic comparison of project costs, timelines, and There is no meaningful ex-post evaluation of projects. See paragraph 4.14. assessment deliverables against budgets and plans. There appears to be no institutional understanding of D (evaluation) Formal arrangements for project evaluation against the potential value of doing such reviews. Since there is plan and appraisal. no current basis for setting project objectives based on Effectiveness or Value for Money audits performance indicators it would not be meaningful to Proportionality of evaluation. attempt to assess the value of a project ex-post. Response to evaluation �ndings. Annex A. PIM Assessment 47 48 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT Annex B. Summary of PIM Performance Indicator Set with Links to PEFA and PIM 8 ‘Must- Have’ Features Area/Indicator Relation to PEFA Relation to PIM 8 ‘must-have’ features A. PFM OUT-TURNS: Credibility of the Budget PIM 1 Aggregate capital expenditure out-turn PEFA PI-1 indicator adapted to compared to original budget capital spending PIM 2 Composition of capital expenditure out- PEFA PI-2 indicator adapted to turn compared to original budget capital spending PIM 3 Aggregate revenue out-turn compared to PEFA PI-3 indicator adapted to original budget include donor budget support PIM 4 Stock and monitoring of capital PEFA PI-4 indicator adapted to expenditure payment arrears capital spending B. KEY CROSS-CUTTING ISSUES: Comprehensiveness and Transparency PIM 5 Classi�cation of the budget PEFA PI-5 adapted by addition of project classi�cation PIM 6 Comprehensiveness of information PEFA PI-6 changed to focus on included in budget documentation capital spending PIM 7 Extent of unreported capital spending PEFA PI-7 adapted to focus on capital spending PIM 8 PIM-related inter-governmental �scal PEFA PI-8 adapted to focus on relations capital spending PIM 9 Management of �scal risks from capital Adaptation of PEFA PI-9 spending outside central government PIM 10 Public access to key information on PEFA PI-10 adapted to focus on capital spending capital spending C. BUDGET CYCLE C(i) Policy-Based Budgeting PIM 11 Investment guidance, project Not in PEFA I. Investment Guidance, Project development, and preliminary screening Development & Preliminary Screening PIM 12 Formal project appraisal Not in PEFA II. Project Appraisal PIM 13 Independent review of appraisal Not in PEFA III. Independent Review of Appraisal PIM 14 Orderliness and participation in the Same as PEFA PI-11 annual budget process PIM 15 Multi-year perspective in �scal policy and Adapted from PEFA PI-12 management PIM 16 Project selection and budgeting Not in PEFA IV. Project Selection and Budgeting C(ii) Predictability and Control in Budget Execution PIM 17 Project Implementation Not in PEFA V. Project Implementation PIM 18 Predictability in the availability of funds Same as PEFA PI-16 V. Project Implementation for commitment of expenditures PIM 19 Value for money in procurement Same as PEFA PI-19 V. Project Implementation Annex B. Summary of PIM Performance Indicator Set with Links to PEFA and PIM 8 ‘Must- Have’ Features 49 PIM 20 Effectiveness of internal controls PEFA PI-20 focused on capital V. Project Implementation and internal audit of capital project spending expenditure PIM 21 Project Adjustment Not in PEFA VI. Project Adjustment PIM 22 Facility Operation Not in PEFA VI. Facility Operation C(iii) Accounting, Recording, and Reporting PIM 23 Basic Completion review and evaluation Not in PEFA (8) Basic Completion Review and Evaluation PIM 24 Quality and timeliness of in-year budget PEFA 24 focused on capital reports spending PIM 25 Quality and timeliness of annual �nancial PEFA 25 focused on capital statements spending C(iv) External scrutiny and audit PIM 26 Scope, nature and follow-up of external PEFA 26 focused on capital audit spending PIM 27 Legislative scrutiny of capital spending in PEFA 27 focused on capital the annual budget law spending PIM 28 Legislative scrutiny of external audit PEFA 28 focused on capital reports on capital spending spending D. DONOR PRACTICES PIM D-1 Financial information provided by donors Same as PEFA D-2, for capital spending PIM D-1 Proportion of aid that is managed by use PEFA D-3 focused on capital of national procedures spending 50 UKRAINE: PUBLIC INVESTMENT MANAGEMENT PERFORMANCE ASSESSMENT