Report No. 89008 Kyrgyz Republic Public Expenditure Review Policy Notes Public Investment Management May 2014 Poverty Reduction and Economic Management Unit Europe and Central Asia Region Document of the World Bank KYRGYZ REPUBLIC - GOVERNMENT FISCAL YEAR January 1 – December 31 CURRENCY EQUIVALENTS (Exchange Rate Effective as of April 30, 2014) Currency Unit = Kyrgyz Som (KGS) US$1.00 = 53.9615 (KGS) Weights and Measures Metric System ACRONYMS AND ABBREVIATIONS ADB Asian Development Bank MTBF Medium-Term Budgetary Framework CADIICentral Agency for Development, MTDP Medium Term Development Program Investment and Innovation NSDS National Strategy on Sustainable CBEM Capacity Building in Economic Development Management OPM Office of the Prime Minister DFID UK’s Department for International PEFA Public Expenditure and Financial Development Accountability ECA Europe and Central Asia PER Programmatic Public Expenditure EU European Union Review GDP Gross Domestic Product PIM Public Investment Management GoKR Government of the Kyrgyz Republic PIP Public Investment Portfolio IMF International Monetary Fund PIUs Project Implementation Units OECD Organization for Economic Co- PPL Public Procurement Law operation and Development PPP Public Private Partnership KY Kyrgyz Republic SECO Swiss Economic Cooperation MOE Ministry of Education Organization MOF Ministry of Finance SOEs State Owned Enterprise Vice President: Laura Tuck Country Director: Saroj Kumar Jha Acting Sector Director: Roumeen Islam Sector Manager: Ivailo V. Izvorski Task Team Leader: Evgenij Najdov / K. Migara O. de Silva Policy Note Author: Paul Harnett Acknowledgements This note is part of the Kyrgyz Republic Programmatic Public Expenditure Review (PER) led by Evgenij Najdov (Task Team Leader) and K. Migara O. de Silva (co-TTL). The PER work was initiated by Orhan Niksic. Faruk Khan took over as task team leader from September 2012 to June 2013. K. Migara O. de Silva’s co-TTL-ship was uninterrupted with primary responsibility for the sector notes on wage bill management, public investments management and intergovernmental transfers. This policy note was prepared by a World Bank team comprising K. Migara De Silva (Senior Public Sector Specialist) and Paul Harnett (Consultant) with input provided by Natalia Pisareva. The team has benefited from the guidance of Alexander Kremer (Country Manager). Valuable inputs and guidance were also provided by Bakyt Dubashov and Helen Edmundson. The authors would like to thank the Kyrgyz authorities for the productive discussions, in particular the officials from the Ministry of Finance. The team is grateful to the UK’s Department for International Development (DFID) and the Swiss Economic Cooperation Organization (SECO) for co-financing this programmatic PER. i Contents 1. Executive Summary .......................................................................................................................................................................... 4 2. Diagnosis of the Public Investment Portfolio........................................................................................................................ 5 Current Status of the Portfolio ................................................................................................................................................. 5 Data ...................................................................................................................................................................................................... 8 3. Assessment of PIM in Kyrgyz Republic .................................................................................................................................... 9 Introduction ..................................................................................................................................................................................... 9 Investment Guidance, Project Concept Development and Preliminary Screening ........................................11 A Formal Project Appraisal Process and Independent Review of Appraisals ..................................................12 Independent Review ..................................................................................................................................................................14 Selection of New Projects and Continuity of Funding for Ongoing Projects through a Well Managed Budget Process..............................................................................................................................................................................14 Efficient Project Implementation and Ability to Make Project Adjustments ....................................................18 Provision for Sustainable Operation of Facilities ..........................................................................................................23 Basic Completion Review and Ex Post Evaluation ........................................................................................................23 4. Institutional and Administrative Framework for Strengthening PIM .....................................................................24 Legal and Regulatory Framework for Public Investment Management .............................................................24 Alternative Organizational Arrangements for Public Investment Management in Kyrgyz Republic ....27 Assessment of PIM Capacity and Recommendations ..................................................................................................29 5. Summary Findings, Conclusions and Recommendations of the PIM Diagnostic ................................................34 Guidance and Preliminary Screening..................................................................................................................................35 Formal Appraisal and Independent Review ....................................................................................................................35 Project Selection and Budgeting ...........................................................................................................................................36 Project Implementation and Project Adjustment..........................................................................................................36 Facility Operation ........................................................................................................................................................................37 Evaluation .......................................................................................................................................................................................37 List Annexes Annex 1. Project Appraisal Matrix .........................................................................................................................................................38 Annex 2. Summary Assessment of Kyrgyz Republic’s Public Investment Management System ...............................39 List of Boxes Box 1. Key Features of an Effective Public Investment Management System ....................................................................10 Box 2. Advantages of Preliminary Screening at Project Identification Stage .....................................................................12 Box 3. Potential Problems with Relying on Donor Appraisals ..................................................................................................13 Box 4. The Problem of Optimism Bias in Projects ..........................................................................................................................14 Box 5. Managing Multi-Year Commitments in an Annual Budget Process ..........................................................................17 Box 6. Performance Terminology Used in the Logical Framework Approach ...................................................................20 Box 7. Botswana’s Approach to Project Adjustments ...................................................................................................................22 Box 8. Project Monitoring, Re-assessment and Adjustment in South Korea ......................................................................22 Box 9. Activities of the United Kingdom National Audit Office .................................................................................................24 2 List of Figures Figure 1. Current Investment (Financial Monitoring) Management in KR .........................................................................31 Figure 2. Potential Investment (Financial Monitoring) Management in KR .......................................................................31 List of Tables Table 1. Expenditure of the PIP by Project Type ............................................................................................................................... 5 Table 2. Total PIP by Project Type ........................................................................................................................................................... 6 Table 3. Expenditure of the PIP by Donor ............................................................................................................................................ 6 Table 4. Total PIP by Donor ........................................................................................................................................................................ 6 Table 5. Total PIP by Sector ........................................................................................................................................................................ 7 Table 6. Total PIP By Type of Financing ................................................................................................................................................ 7 Table 7. Total PIP Financed by Foreign and Domestic Funding ................................................................................................. 7 Table 8. Scoring of PEFA Indicators PI-11 and PI-12 for Kyrgyz Republic .........................................................................15 3 1. Executive Summary 1. Weaknesses in the public investment management (PIM) system may limit the gains from higher public sector investments in the Kyrgyz Republic. Capital spending has averaged 6.4 percent of GDP since 2010, up from 4.6 percent of GDP between 2005 and 2009, with significant investment in the energy sector and roads. Still, it remains unclear to what extent these investment decisions reflect the country’s and sector priorities. Few projects, with the exception of donor- financed projects are subject to rigorous appraisal and there is no systematic procedure in place to monitor implementation progress. As a result, projects are often delayed or stalled and cost over- runs are frequent. Donor-financed projects, which comprise the bulk of public investment, are subject to relatively more rigorous project cycle management, however, they too face some of the same weaknesses. 2. Attempts to improve the PIM have had limited success. While the 2010 revolution introduced a number of significant institutional changes in public investment management (PIM), the system has not fully evolved to adopt these changes. As a result, a number of key institutional arrangements are not yet in place and roles and responsibilities of various entities/units/agencies remain unclear. Furthermore, PIM in the Kyrgyz Republics is seen as synonymous with the public investment program (PIP), where PIP includes donor financed projects irrespective of whether they are investment projects or technical assistance. Domestically financed investment lies outside of the PIP, although an attempt was made to unify them in 2010. 3. Based on the assessment made in this note, a sequenced plan of action is recommended, which will improve PIM and therefore the overall efficiency of public investments. The analysis suggests that ‘front-end’ processes such as preliminary screening and independent review as well as ex post assessment of projects are the weakest components of PIM in the Kyrgyz Republic, followed by project appraisal and implementation. However, given the limited capacity and resources available, the authorities should focus their efforts on some critical steps of PIM, including: a) Setting up a framework for public investment that links it to the country’s development priorities and includes full costing and prioritization of projects; b) Putting in place a transparent mechanism and guidelines within the Ministries for independent appraisal, physical monitoring and evaluation. Guidelines and procedures should apply to all projects, irrespective of funding source. . c) Consolidating PIM responsibilities in the Ministry of Finance (merging various departments which are currently dealing with public investments) and confirming its “gate-keeper� function. d) Completing the database development, which will cover donor and domestically-funded projects as well as original project data to allow regular monitoring of implementation.. e) Developing asset registers and asset management systems in the infrastructure sectors which will help identify infrastructure investment (renewal) requirements and optimal expenditure needs for facility operation (maintenance). 4. The rest of the note is structured as follows: Section 2 provides a diagnosis of the public investment portfolio, including the structure of expenditures, the quality of the data, and a quantitative assessment of the efficacy and effectiveness of the public investment portfolio. The assessment builds on previous work on public investment in the Kyrgyz Republic, in particular the 4 Public Investment Diagnostic undertaken in 2012 under the Capacity Building in Economic Management (CBEM) project. Section 3 reviews the institutional and administrative framework of PIM in Kyrgyz Republic. This draws largely from the draft Investment Diagnostic Report prepared in December 2012 using the eight key “must-have� features of a well-functioning public investments framework proposed in Rajaram et.al. (2011). Section 4 includes a summary of the findings and detailed recommendations on improving PIM. 2. Diagnosis of the Public Investment Portfolio CURRENT STATUS OF THE PORTFOLIO 5. The current institutional setup does not facilitate effective management of public investment in the Kyrgyz Republic. Before the 2010 turmoil, many public financial management (PFM) functions, including the bulk of PIM, were centralized in the Central Agency for Development, Investment and Innovation (CADII). CADII was established in October 2009 and suffered from opaque governance arrangements. This agency was responsible for the public investment budget, the management of State Owned Enterprises (SOEs) and privatization. It managed large national projects in the energy sector, agriculture, and the construction of social housing. In practice, it dominated strategic budget policy. When CADII was abolished in April 2010, many of its functions were given back to various departments under the Ministry of Finance (MoF) and to the Ministry of Economy (MoE) without establishing a clear structure for efficient cooperation. 6. As is the case in many post conflict and aid dependent countries, the primary focus of expenditure management is on the Public Investment Program which has somewhat limited coverage. The Public Investment Program (PIP) portfolio includes mainly investment, bnut also mixed projects (investment and recurrent), and some of a technical assistance (TA) nature. Although the percentage of TA projects has decline significantly from a high of 14 percent in 2008, mixed projects have increased over the period. In addition, the PIP in the Kyrgyz Republic is made up exclusively of projects – both investment and TA – financed by traditional donor (Table 1).1 Furthermore, it does not cover investment financed exclusively by domestic budget. Still, given that investment projects represent over 85 percent of the PIP portfolio at present, it would be relatively easy to transform the PIP portfolio to a true investment portfolio with a view to managing investment rather than donor projects. Table 1. Expenditure of the PIP by Project Type (Kyrgyz som) 2006 2007 2008 2009 2010 2011 2012 Investment projects 3,627.3 3,441.0 2,922.2 5,145.9 8,185.3 12,017.5 17,940.2 Mixed projects 176.2 249.7 308.3 400.8 374.3 1,233.1 2,425.2 TA projects 326.3 507.0 527.8 564.5 273.8 535.3 544.2 Total PIP 4,129.7 4,197.6 3,758.3 6,111.2 8,833.4 13,785.9 20,909.6 Source: MoF. 1 Traditional donors� in the Kyrgyz Republic are the World Bank Group, Asian Development Bank (ADB), European Union (EU), the German Gesellschaft für Internationale Zusammenarbeit (GIZ), Japan International Cooperation Agency (JICA) and the Department for International Development (DfID). 5 Table 2. Total PIP by Project Type (Percent) Percent of total PIP 2006 2007 2008 2009 2010 2011 2012 Investment projects 87.8 82.0 77.8 84.2 92.7 87.2 85.8 Mixed projects 4.3 5.9 8.2 6.6 4.2 8.9 11.6 TA projects 7.9 12.1 14.0 9.2 3.1 3.9 2.6 Source: MoF. 7. The PIP portfolio does not include projects financed by all donors. Projects financed by China, Russia, Turkey and others are not traditionally included in the PIP, although more recently some Chinese projects have been included (Table 3 and 4). Similar to other countries, information about projects falling outside of the PIP is difficult to access. Such projects should be managed within any future PIM institution to ensure consistency of appraisal and project management, and that they fall within the ambit of the government’s medium term development strategy. Similarly, PIP portfolio does not provide any information of projects which are carried under public-private partnerships (PPP).2 Given the tight fiscal constraint facing the authorities, PPPs provide an option for improving service delivery while keeping financing needs limited. However, it should be understood that PPPs are, in essence, only a form of implementing public investment projects and PPPs should be evaluated through the same system as traditional public investment projects. However, the PPP legal framework has been introduced very recently and capacity to manage PPPs is, in general, very low. Table 3. Expenditure of the PIP by Donor (Kyrgyz som million) 2006 2007 2008 2009 2010 2011 2012 ADB 1,751.2 1,581.0 1,527.1 1,905.0 1,508.7 3,249.3 3,104.9 China 0.0 0.0 0.0 1,629.8 4,658.6 5,602.1 11,976.7 EBRD 0.0 0.0 0.0 0.0 176.6 529.0 803.5 IDB 209.0 229.0 103.0 394.8 703.0 501.8 95.0 KfW 374.8 381.3 130.1 149.7 252.0 402.0 573.7 Swiss 58.9 156.6 83.8 136.3 59.1 114.2 136.0 WB 1,362.7 1,179.1 876.3 1,318.0 1,178.8 2,638.8 3,015.6 Other 373.3 670.6 1,038.1 577.6 296.5 748.7 1,204.2 Total PIP 4,129.7 4,197.6 3,758.3 6,111.2 8,833.4 13,785.9 20,909.6 Source: MoF. Table 4. Total PIP by Donor (Percent) 2006 2007 2008 2009 2010 2011 2012 ADB 42.4 37.7 40.6 31.2 17.1 23.6 14.8 China 0.0 0.0 0.0 26.7 52.7 40.6 57.3 EBRD 0.0 0.0 0.0 0.0 2.0 3.8 3.8 IDB 5.1 5.5 2.7 6.5 8.0 3.6 0.5 KfW 9.1 9.1 3.5 2.4 2.9 2.9 2.7 Swiss 1.4 3.7 2.2 2.2 0.7 0.8 0.7 WB 33.0 28.1 23.3 21.6 13.3 19.1 14.4 Other 9.0 16.0 27.6 9.5 3.4 5.4 5.8 Source: MoF. 2 There have been no capacity building efforts in the MoE on PPP. 6 8. The recent expansion in public investment has been targeted towards the energy sector. Roads were traditionally the most important investment area until the recent increase of investments in the energy sector, largely financed by China. With the significant expansion of infrastructure projects, the importance of social sectors projects in the PIP declined. For example, the share of the PIP devoted to the education sector fell from a peak of 14.6 percent in 2008 to 2.1 percent in 2012. Rural development has also seen a dramatic decrease in its share of the budget from a peak of 14 percent in 2007 to 0.3 percent in 2012. Given the importance of investments in sectors such as education, rural development and agriculture for long-term growth, the government should agree on a systematic approach to maintaining its investment in these critical areas. Table 5. Total PIP by Sector (Percent) 2006 2007 2008 2009 2010 2011 2012 Adv. Services 3.2 2.7 5.0 4.3 3.4 4.6 3.8 Agriculture 7.8 6.4 8.1 0.6 0.0 0.0 0.0 Education 3.5 10.8 14.6 8.1 2.3 3.0 2.1 Emergency 3.3 2.4 1.4 0.5 0.8 1.1 0.4 Energy 5.9 8.9 7.2 5.9 3.2 16.3 49.7 Environment 0.1 0.0 0.0 0.0 0.1 0.0 0.1 Finance 7.3 2.1 0.7 0.1 0.0 0.2 2.7 Governance 1.0 0.3 1.2 3.7 1.7 2.1 1.3 Health 2.7 1.7 2.6 3.3 3.1 1.7 1.2 Housing 0.0 0.0 0.0 0.0 0.0 6.0 2.4 Infrastructure 8.9 10.4 10.2 7.6 4.5 3.6 0.4 Irrigation 8.2 5.3 1.9 2.9 1.9 1.4 1.0 Land management 1.5 1.3 0.4 0.8 0.4 0.9 0.5 Miscellaneous 0.1 0.3 0.2 1.0 0.6 4.1 5.6 Pub. transport 0.0 0.0 0.0 0.0 0.0 0.0 0.5 Roads 34.8 28.2 27.4 51.5 73.8 49.4 24.5 Rural development 5.9 14.0 13.9 3.4 1.1 2.8 0.3 Water management 5.8 5.0 5.1 6.4 3.0 2.8 3.6 Source: MoF. 9. The changing pattern of financiers is also changing the financing structure of the PIP. The increase in the percentage of grants to 2008 and its subsequent decrease both reflects the fluctuating political situation as well as the entry of the Chinese credits in 2009 (Table 6). At the same time, the GoKR’s contribution to PIP financing (the domestic contribution) has decreased from a peak of 10.8 percent in 2007 to 2.6 percent in 2012 (Table 7). Table 6. Total PIP By Type of Financing (Percent) 2006 2007 2008 2009 2010 2011 2012 Credits 83.4 62.9 40.2 51.7 71.3 62.6 75.9 Grants 16.6 37.1 59.8 48.3 28.7 37.4 24.1 Source: MoF. Table 7. Total PIP Financed by Foreign and Domestic Funding (Percent) 2006 2007 2008 2009 2010 2011 2012 Foreign 92.4 89.2 90.9 92.6 94.4 95.5 97.4 Domestic 7.6 10.8 9.1 7.4 5.6 4.5 2.6 Source: MoF. 7 10. Exclusively domestically financed projects are managed by two Departments in the MoF, the Department of the real sector and the Department for capital investments. Both departments receive funding requests for projects from the domestic investment budget. Little or no information about projects is given to any of these departments beyond names, funding required and expected time period. Appraisal is therefore minimal, and acceptance of projects is open to political pressure. The Real Sector Department covers the non-social sector domestic investments e.g. roads, energy, agriculture. The Capital Investments Department appears to have no distinguishing reason for its portfolio and receives applications from all sectors, including the social sectors. It controls about 15 percent of the domestic investment budget. It is possible for some projects to be submitted to either or both departments. Neither of the Departments which are responsible for domestic investments has been able to provide any data for analysis. 11. The domestic investment budget is subject to parliamentary debate after budget approval. While staying within the budgeted capital expenditure ceiling, parliament has systematically re-drafted the intended budget dealing with domestic capital investment. This is partly because the government’s budget has not included a breakdown for domestic capital expenditure. This list is retained by the MoF and is regularly altered. 12. Parliament’s decisions do not necessarily respect efficiency considerations and do not appear to be consistent from one year to the next. By eliminating or cutting the budgets for on-going projects proposed by the government, parliament contributes to longer-than-planned implementation and delays in improvements in services for the public. Extended implementation usually also means increased unit costs of construction, particularly in cases where funding is interrupted for a year or more and remobilization costs are incurred. Parliament’s actions are therefore probably lowering the efficiency and effectiveness of public investment and are, at least partially, contributing to a significant overhang of slow-moving, ongoing projects that require clearing. 13. The legislature has an important role to play in a good budget process, but the Kyrgyz Republic Parliament seems to be spending a disproportionate amount of time on micro- managing a small part of the budget. Planned state budget expenditure on domestic capital expenditure was budgeted to be about som 4 billion in 2012, representing about 14 percent of total capital expenditure. It represents a small part of overall capital expenditures and an even smaller part of total expenditures. Even so, the significant changes made by parliamentarians indicate that they must be devoting a great deal of time to this very small part of the budget, time which might be more usefully spent scrutinizing the more strategic dimensions of the government’s budget proposal and focusing more on the intended and achieved results of spending rather than its detailed composition. DATA 14. As noted above, no data has been forthcoming from the two MoF departments responsible for domestically financed investments. It is likely that the data held is minimal and geared towards satisfying treasury demands for cash flow. Little or no appraisal or monitoring is carried out. It is unlikely that there will be any data that will enable comparison of originally budgeted project costs and timings with actual costs and timings. 15. PIP data relies largely on the data forthcoming from Project Implementation Units (PIUs). A PIP department in the MoF then manually assembles the data into excel spreadsheets. Actual expenditures and timings are therefore the priority and original budgets and timings are not easily accessible, though can be found as required in the original project documents held within the 8 real sector department. In summary, the database is rudimentary. One reason for this is that before 2009, an ADB sponsored database was maintained, but after the PIP was transferred to CAADI and then returned to MoF, the database was neglected and unable to be maintained. 16. Project monitoring in Kyrgyz Republic is largely concerned with financial monitoring and is largely passive. There is no process for collecting physical monitoring data. Except for donor projects, monitoring of non-financial dimensions of project progress is not happening, except on an ad hoc basis. Conceptually, monitoring is not well differentiated from financial control and inspection functions. Insofar as monitoring information is used in a planning and management capacity, it is mainly for budget preparation tasks rather than for identifying problems with individual projects and developing responses. The responsibility of the PIP Department in the MoF with respect to financial monitoring is to forward a monthly report to the budget department and the central treasury. Its responsibility with respect to physical monitoring is the collation of monthly reports from PIUs. Whilst monitoring responsibilities continue to be passive, there is little incentive to improve the database. 17. Financial monitoring can be improved. At a minimum, the financial monitoring system needs to be capable of tracking changes in: i) the disbursement profile of a project; ii) total estimated project costs; and iii) estimated completion date compared to plan. Currently, only the first of these three indicators is consistently monitored by the Ministry of Finance. Ideally, the monitoring system needs to be capable of supporting a system for managing total project costs. A proposed aid management platform, Development Gateway, is being developed at the Office of the Prime Minister (OPM) with donor assistance from the Swiss government and has the potential to cover all three indicators, though this is only at pilot phase at present. A scoping mission arrived in March/April 2012, though hasn’t returned since. This Aid Management Platform has been installed in over 20 countries and would provide the Kyrgyz Republic with an up to date management tool for not only coordinating donor aid but also, if tailored correctly, to manage public investment. It should free many MoF officers from the routine of data entry and enable them to engage in other activities such as monitoring. The reports generated would also be useful for MoF (and line ministries) management of investments. 18. It is of utmost importance that this project is implemented as soon as possible if public investment management is to be furthered. An important decision that needs to be made is where the main authority for this software should lie. The primary end-user at present is the MoF, particularly the PIP Department, so it would make sense for it to be located in MoF. 3. Assessment of PIM in Kyrgyz Republic INTRODUCTION 19. The following section benchmarks the existing systems in the Kyrgyz Republic against good international practice on effective PIM systems. This assessment has been guided by recent work on an approach for assessing the quality of public investment management systems.3 This framework delineated eight key features as being critical for an effective system as summarised in Box 1. The next part of this report is organised around these minimum features of a functioning PIM system against which Kyrgyz Republic’s current practices are assessed. The 3‘A Diagnostic Framework for Assessing Public Investment Management’ . World Bank Policy Research Working Paper No. 5397, 2010. 9 particular concerns expressed by Ministry of Economy (MoE), OPM and MoF officials involved in public investment management have been taken into account when assessing current systems. These were:  Clarity regarding PIM responsibilities;  Weakness of government policy documents post 2010;  The lack of screening;  Lack of government appraisal processes;  Strengthening project prioritisation;  Improving the ‘qualitative’ dimension of monitoring;  Lack of ex-post evaluation. 20. The section also reviews current reform efforts and provides recommendations for reforms. Against the background of the benchmarking, the second part of this assessment assesses recent proposals for improving processes, and to determine how far these proposals would go towards filling some of the gaps identified. The report concludes by examining some of the choices facing the Kyrgyz Republic in terms of the organisational arrangements for Public Investment Management (including an action plan to strengthen capacity and institutionalise procedures). This was of particular concern raised by MoE, OPM and MoF officials during discussions. Box 1. Key Features of an Effective Public Investment Management System  Strategic investment guidance, project concept development and preliminary screening : broad strategic guidance to guide sector-level decision-makers and preliminary screening to ensure project concepts meet minimum criteria of consistency with government’s strategic objectives and with the economic classification.  A formal project appraisal process: a regulated set of project preparation steps, such as pre-feasibility study and feasibility study, including preliminary design, and environmental and social impact assessments, that must be completed before a project can be approved as eligible for funding; and methods appropriate to technical capacities and to the scale and scope of a project.  Independent review of appraisal: a review by the finance ministry, a planning ministry or an independent agency is an important way of countering optimism bias - over-estimation of demand and under-estimation of costs - amongst those developing project proposals.  Project selection through a well-managed budget process: linking the process of appraising and selecting public investment projects to the budget cycle in an appropriate way even though the project evaluation cycle may follow a different timetable. Involves verification of project eligibility and priority, and close scrutiny of forward costs and their funding during budgeting.  Efficient project implementation: scrutiny for implementation realism, including organizational arrangements, procurement planning and timetable; adequate monitoring systems; and systems for managing total project costs.  Ability to make project adjustments: flexibility to allow changes in the disbursement profile – including stopping non-performing projects - to take account of changes in project circumstances identified through responsive monitoring.  Provision for sustainable operation of facilities: processes to ensure that a new facility is ready for operation and that the intended services can be delivered on a sustainable basis. Requires effective handover of management responsibility for operation and maintenance, and upkeep of robust and up-to- date capital asset registers.  Basic completion review and ex post evaluation: a systematic review of all projects upon completion to assess whether a project was delivered as specified, on time and according to budget. Introduction of a more sophisticated ex post evaluation to assess the project’s outputs and outcomes against objectives established in the design should follow. 10 INVESTMENT GUIDANCE, PROJECT CONCEPT DEVELOPMENT AND PRELIMINARY SCREENING Strategic Investment Guidance 21. Strategic guidance in the Kyrgyz Republic has been lacking in recent years, both in terms of quality and authoritativeness, though the adoption of the 2013 National Strategy on Sustainable Development could be signalling a change in this area. Given the recent political upheavals, in particular in 2010, the development of a medium term strategic framework has been fraught with difficulty. The 2012-2014 Medium-Term Development Program (MTDP) was never approved by parliament, probably reflecting the diverse policies of the broad coalition in power. The adoption of the 2013-2017 National Strategy on Sustainable Development (NSDS) is an important accomplishment in this regard. However, even the NSDS has little costing or prioritisation of activities including investments, and is poorly linked to national budgeting processes. Furthermore, there is no formal mechanism to ensure that project proposals are assessed for their compatibility with government priorities. Long-term sector strategies exist for some of the main investment sectors, though again it is unclear if these adequately reflect the strategic guidance or if they are properly costed-out and prioritized. 22. This may lead to situations in which capital investment projects do not reflect adequately the strategic considerations. Since a large share of investment is externally financed, the strategies that exist are playing only a limited part in steering donors towards priority areas.4 However, it is rare for the project proposals of donors to be tailored towards national priorities by either the line ministries or central agencies. The preparation of the PIP is essentially a collation of donor projects, rather than any attempt to steer donors towards national objectives. Short-term political considerations also continue to play an important part in deciding which domestically financed projects go forward for further development even where there is a strategy in place. Although domestic financing for projects is approved under the budget (in November/December), this is not broken down by project, but as a lump sum. Then in March, parliament approves the list of suggested projects, often with significant changes to the presented list. Positively, the government organised a high level meeting with donors in the OPM on October 18, 2012 and a high level conference in mid-2013 with a view to presenting government priorities, though again this appeared to have little detail on costing. 23. Weaknesses remain in the way strategies have been developed. Not all, if any, strategies are realistic when it comes to the assessment of resource availability for funding ‘priorities’, so that the identified investment needs often exceed feasible financing ca pacities. Lack of realism arises because strategies have often been prepared in isolation, with no overarching aggregate resource framework and with relatively weak coordination between ministries. Absence of specificity in defining investment priorities is also a problem allowing too much room for manoeuvre in the choice of projects to pursue. Because monitoring is not strong and updating is irregular, strategies become out of step with the changing fiscal realities as well. 24. Strategic investment guidance is rarely built upon a solid knowledge of the condition of assets, trends in demand for services, emerging infrastructure constraints and gaps, and funding possibilities. The sector strategy for the Ministry of Transport 2007-10 represented a move in the right direction. The response from donors after finalisation was to increase funding to roads by over six times. Unfortunately, it has not been updated since, though a new strategy is being developed at present. Elsewhere, the approach has been largely ad hoc until recently. Future 4The WB and ADB’s medium term strategies in the Kyrgyz Republic are, to an extent, based on the country’s development strategy (NSDS). 11 strategy development would be enhanced by more systematic assessment of asset condition in the sectors, including the maintenance of regularly updated asset registers and linked asset management systems. Such an approach has not yet been institutionalized in the Kyrgyz Republic. Project Concept Development and Preliminary Screening 25. A formal project concept identification and approval process is lacking. Such a process should verify the rationale for the project (including justification for government intervention) and conformity with sector plans. Project identification should be set in the context of the medium-term budget framework (MTBF) process. This will ensure that new project concepts would be assessed before entering medium-term expenditure plans, within which their detailed preparation and subsequent implementation would be programmed over a medium-term perspective according to resource availability and government policy priorities. The aim should be to promote the development of a project pipeline consistent with the medium-term macro-fiscal outlook. As noted above, the relatively recent emergence of a national strategic plan, and the weakness of sector strategies, results in the identification of projects arising from ad hoc requests from ministers or other senior government officials, as well as donor suggestions. The advantages of a preliminary screening process are set out in Box 2. Box 2. Advantages of Preliminary Screening at Project Identification Stage  Forces project promoters to clarify the project logic (the problem being addressed, the purpose of the project and the target beneficiaries) and the justification for government intervention, and to consider alternative solutions before dedicating resources to project preparation.  Project objectives can be verified at an early stage for compliance with government policy and expenditure priorities, and the project can be rejected where there is inconsistency.  The scope of the project can be verified to ensure that it represents a complete solution to the problem being addressed and not just a component of a larger project.  Where project preparation is unlikely to inform the investment decision further, screening at identification stage forces an early decision on priorities and helps to prevent resources being used up preparing unaffordable or irrelevant projects. An example would be the case of social projects where cost-benefit analysis is difficult to perform and the investment decision is more policy-based.  Early recognition of any significant recurrent financing needs allows time to develop the necessary coherence between the capital and current parts of a spending agency’s budget.  Any important design issues, including possible project alternatives, can be identified early and built into the terms of reference for studies. Alternative delivery/procurement options, e.g., PPP, can also be identified and flagged up for study.  The preliminary screening process disciplines spending agencies, by guiding them towards priority areas for investment and causing them to think in terms of developing a ‘pipeline’ of affordable projects for implementation within MTEF limits.  Monitoring and evaluation results from on-going and completed projects can be fed more easily - into the later project preparation stage. A FORMAL PROJECT APPRAISAL PROCESS AND INDEPENDENT REVIEW OF APPRAISALS Project Appraisal Process 26. Major projects that have passed the identification stage should be subject to rigorous and systematic assessment. This should include economic cost-benefit analysis (or cost- effectiveness analysis where this is more appropriate), to verify that they represent a good use of public money. The aim of the analysis undertaken for project appraisal should be to provide answers to the following questions: 12  What are the specific outcomes sought through the investment?  Are there better ways to achieve these outcomes?  Are there better uses for these resources?  Is this an appropriate role for government? 27. The lacks of a regulated project appraisal process involving systematic economic/financial analysis, raises serious questions about the quality of domestically funded capital investment in terms of economic viability and financial sustainability. Regulations from the Ministry of Transport concerning technical documentation for engineering feasibility and costing of construction projects exist and are enforced, which is an important component of preparation and appraisal. What is missing, on the other hand, are formal requirements to undertake feasibility studies involving analysis of a project’s economic viability, financial sustainability and environmental and social impacts. And there is no formal appraisal step when the findings of feasibility studies are assessed and a decision taken on whether a project qualifies to be submitted for budget funding on the basis that it represents good value for public money. It can therefore be said that the domestically funded part of capital investment is not subject to adequate and comprehensive appraisal. 28. Since a large share of capital investment is externally funded and this is subject to donor-prescribed appraisal processes, a significant portion of the project portfolio is subject to more rigorous assessment. Checks by government on the quality and consistency of donor appraisals are very limited though. In the Kyrgyz Republic’s present context of limited internal capacities for project appraisal, this is understandable, but it would be a mistake to assume that reliance on donor appraisals does not bring its own problems as summarized in Box 3. Nevertheless, the MoF does analyse the affordability of budget contributions expected from donor projects (counterpart funding etc.). On occasion the MoF will question other aspects of appraisal but this is ad hoc rather than systematic. There are no procedural safeguards to make sure that only projects that have been properly prepared and appraised may enter the budget. Box 3. Potential Problems with Relying on Donor Appraisals  Donors are not always as rigorous in their application of cost-benefit analysis or cost-effectiveness analysis as might be expected, and the quality of the analysis can sometimes be an issue.  Even when cost-benefit analysis is carried out by donors it can often lack objectivity or be performed after an investment decision has already been taken. Like domestic project sponsors, donors can also be prone to ‘optimism bias,’ i.e., a propensity to under-estimate costs and over-estimate benefits.  Different donors use different methods and different values for key parameters, like the discount rate, presenting difficulties for national governments to compare similar projects in the same sector/sub- sector. 29. A common appraisal process and methodology should be instituted and applied consistently across all projects, no matter what the funding source. Although the priority is, evidently, to undertake more rigorous scrutiny of the domestically funded program. It is worth noting that over time the proportion of investment funded domestically is likely to rise, rendering it more urgent to institute appraisal procedures. 30. Existing capacities in project appraisal methodologies represent a constraint on the development of stronger appraisal processes, but the Kyrgyz Republic currently has no systematic training program to raise these capacities. At the same time, capacity building 13 without demand for improved processes can be a fruitless exercise and stimulating demand from decision-makers for better skills must also be part of the package. INDEPENDENT REVIEW 31. There are no independent checks, either carried out internally by the sponsoring organizations (for smaller projects) or by external bodies (for larger projects), to verify the objectivity of appraisals. Such checks are important to try and counteract the problem of optimism bias, which as Box 4 indicates is a systematic flaw in project appraisal irrespective of country or period of time. One approach to dealing with this problem is for an independent institution, to carry out preliminary feasibility studies of major (e.g. over US$50 million) investments proposed (although for the Kyrgyz Republic this may be a high threshold in the current environment) by ministries as a prior step to allowing them to proceed with their own full feasibility studies. Another approach is to contract consultancy firms to perform independent reviews of major project proposals on its behalf, both at preliminary screening and appraisal stages. Box 4. The Problem of Optimism Bias in Projects Optimism bias is the systematic tendency for project costs to be under-estimated and for project benefits to be over-estimated. A research group on large infrastructure at Aalborg University (Denmark) tried to quantify the extent of the problem and offer explanations, working from a large sample of major transport projects. This research (summarized in Flyvbjerg, 2005) finds that:  Significant cost overruns (in real terms) were seen across a sample of 258 major transportation projects, irrespective of country, continent or transport mode, and with no tendency to diminish. Nine out of ten projects had a cost overrun and the average (real) cost overrun was: o 45 percent for rail o 34 percent for bridges and tunnels o 20 percent for roads  From a sample of 208 rail and road projects: o 9 out of 10 rail projects had overestimated traffic with actual passenger traffic being 51 percent lower on average than forecast o Interestingly, traffic for roads tended to be under-estimated, with actual traffic being on average 9.5 percent higher than forecast Because errors are systematically biased in one direction poor techniques cannot be an explanation for the bias. Inherent over-optimism on the part of planners is also dismissed because the sample covers 70 years and objective planners would be expected to have noticed this tendency and corrected for it. The favoured explanation is that project promoters and planners systematically ‘cook the books’ in order to make projects look better than they are and obtain funding approval. The paper goes on to identify accountability and critical questioning as the key weapons in the war on deception and waste and to recommend that the, frequently unreliable, claims about costs, benefits, and risks made by planners, engineers, architects, economists, and administrators ‘…should be carefully examined by independent specialists and organizations.’ Source: ‘Policy and Planning for Large Infrastructure Projects: Problems, Causes and Cures’, Bent Flyvbjerg, World Bank Policy Research Working Paper, no. 3781, 2005. SELECTION OF NEW PROJECTS AND CONTINUITY OF FUNDING FOR ONGOING PROJECTS THROUGH A WELL MANAGED BUDGET PROCESS Features of Kyrgyz Republic’s Budget Process for Capital Budgeting 32. According to the Public Expenditure and Financial Accountability (PEFA) assessments, the Kyrgyz Republic has in place a reasonably well-managed budget process 14 but a weak multi-year perspective, including sector strategies, and inadequate planning of investments. It appears that the development of the MTEF has also been hampered by recent year’s political changes. As a result, linkages between the capital and recurrent budgets are weak with little future planning of resultant recurrent costs from projects which leads to insufficient recurrent budgets and a lack of efficiency in use of investments. There is a well-established budget preparation calendar and clear and timely guidance on budget preparation. Despite the existence of a elements of a medium-term budget framework, ministries have not developed a project pipeline. Recent departures from the calendar and process have been caused by political events, rather than by systemic problems. Table 8. Scoring of PEFA Indicators PI-11 and PI-12 for Kyrgyz Republic Policy Based Budgeting 2009 2005 PI-11 Orderliness and participation in the budget process B B Existence of an adherence to fixed budget calendar B B Clarity/comprehensiveness of and political involvement in the guidance on the A D preparation of budget submissions (budget circular or equivalent). Timely budget approval by the legislature or similarly mandated body (within D A the last three years). PI-12 Multi-year perspective in fiscal planning, expenditure policy and budgeting D+ C+ Preparation of multi-year fiscal forecasts and functional allocations. C C (Dimension not relevant) Existence of sector strategies with multi-year costing of recurrent and C D↑ investment expenditure. Linkages between investment budgets and forward expenditure estimates. D D↑ Source: PEFA 2005 and 2009. 33. The PIP department of the Ministry of Finance carries out a limited ‘gate-keeping’ function with respect to donor projects, verifying only that projects have the necessary engineering studies, cost estimates and approvals. Government resolution of 2007 moved the responsibility for PIP co-ordination from MoF to MoE but MoE has so far not played any significant role in PIP. One major reason for this is its lack of capacity as well as flows of information from the line Ministries and MoF. Implementing the government resolution would require that MoE is authorized to be the primary gatekeeper of the PIP, as well as other investment projects. This would require line ministries to undergo a screening process in their own ministries first and foremost, to be followed by appraisal by MoE (see also para 73). However, the ‘gate-keeping’ responsibilities of MoF, referred to above, differs from the primary gatekeeping responsibility of the MoE. In the latter case, it only refers to MoF verifying that projects presented in budget submissions have been appraised and approved by the relevant authorities and reviewed independently by the Ministry of Finance. In the Kyrgyz Republic’s case, the absence of a formal project appraisal process (as discussed above), together with capacity constraints, prevents the government from performing a fuller role with respect to domestically funded projects. It has occurred that projects can be agreed outside of the budget process, as in the case of the Ministry of Energy agreeing a US$389 million loan with the Chinese Exim Bank. Build on the government resolution of 2007 which moved the responsibility for PIP co-ordination from MoF to MoE. Despite this resolution, MoE has not played any significant role in PIP since 2007. It lacks capacity as well as flows of information from the line Ministries and MoF. To implement the government resolution it is essential that MoE is authorized to be the primary gatekeeper of the PIP, as well as other investment projects. This would require line ministries to undergo a screening process in their own ministries first and foremost, to be followed by appraisal by MoE 15 Weaknesses in Capital Budgeting: An Analysis of Recent Capital Investment Budgets 34. Despite some positive features of the budget process, for domestic capital budgets prioritisation and continuity of funding for ongoing projects are still problems. Comparison of the government’s budget as submitted to parliament with the approved budget provides an insight into inefficiencies in budget decision-making. It should, however, be recognised that the political landscape has been particularly turbulent during this time with negative consequences for budgeting. 35. While staying within the budgeted capital expenditure ceiling, parliament has systematically re-drafted the intended budget dealing with domestic capital investment. This is at least partly because the government’s budget when approved does not include a breakdown for domestic capital expenditure. This list is retained by the MoF and is regularly altered. 36. Parliament’s decisions do not necessarily respect efficiency considerations and do not appear to be consistent from one year to the next. By eliminating or cutting the budgets for the on-going projects proposed by the government, parliament ensures that implementation takes longer than planned and that improvements in services for the public are delayed. Extended implementation usually also means increased unit costs of construction, particularly in cases where funding is interrupted for a year or more and remobilisation costs are incurred. Parliament’s actions are therefore probably lowering the efficiency and effectiveness of public investment.5 As a result of Parliament’s decisions, there is a significant overhang of slow-moving, ongoing projects that require clearing. 37. The legislature has an important role to play in a good budget process, but the Kyrgyz Republic Parliament seems to be spending a disproportionate amount of time on micro- managing a small part of the budget. Planned state budget expenditure on domestic capital expenditure was budgeted to be about som 4 billion in 2012, representing about 14 percent of total capital expenditure.6 It therefore represents a small part of capital expenditure and an even smaller part of total expenditure. Even so, the significant changes made by parliamentarians indicate that they must be devoting a great deal of time to this very small part of the budget, time which might be more usefully spent scrutinising the more strategic dimensions of the government’s budget proposal and focusing more on the intended and achieved results of spending rather than its detailed composition. 38. By comparison to domestically funded projects, parliamentary influence over state budget expenditure on capital investment within the PIP has been relatively limited. Comparison of the draft submitted to Parliament by the government with that eventually approved by parliament indicates relatively small changes to the projects included and virtually no change to the aggregate expenditure. However, this may be changing as the parliament is taking a more active role in scrutinizing donor financed projects. An example of this is the parliamentary delay on the approval of a World Bank Financial Sector Development Project which includes requirements for the privatisation of a state bank, though this is not a capital investment project. 39. The MoF attempts to hold the line as far as directing resources towards completion of ongoing projects is concerned. It seems clear that the pressure to start new projects at the expense of continuity of funding for ongoing projects, which seems to be a significant problem for 5 A recent example is the road from Issyk-Kul to the Kazakh border, which had reached the stage of being asphalted but not completed and then delayed for a year. 6 Though it should be remembered that much of the PIP is not capital investment although at this stage is included as part of capital in Kyrgyz budgeting. 16 domestically financed investment, arises in line ministries and not in the Ministry of Finance. The MoF has also postponed projects in recent years as there has been a shortage of domestic funds to fulfil domestic financing obligations in donor projects. Possible Responses to Inefficiencies in Capital Budgeting 40. It is not infrequent for the government and parliament to have differing priorities when it comes to public domestic capital expenditure. Government will emphasis efficiency, continuity and adherence to the government’s strategic plan. Parliamentarians often lobby for projects in their own constituencies. There is no easy solution to these differences, though a step in the right direction would be for a resolution to be passed which ensured that only projects which had passed the screening and appraisal process should be entered into the budget, except in exceptional circumstances. Increased information about projects’ lifespans and costs which should be made easily accessible with current software plans should also assist in the continuity of funding for on-going projects. 41. A mandatory requirement to reintroduce stalled projects could act as a deterrent to disrupting financing and prioritising new projects over ongoing projects. Stalled projects should be defined to include not only those where work is suspended, but also those where disbursement falls below pre-defined percentage of the financial plan. Re-assessment could include a requirement to resubmit major projects. Requiring written justification from spending agencies for inserting a new project into its budget at the expense of efficient implementation of ongoing projects could also have a deterrent effect and would, at least, make the practice more transparent. Currently, Kyrgyz Republic does not have a system/methodology of eliminating poorly performing projects and cleaning up exercises are carried out on an ad hoc basis (see also para 59). Box 5. Managing Multi-Year Commitments in an Annual Budget Process There is often an apparent contradiction between the multi-year character of major projects and the annularity of the budget. How to handle multi-year contracts in the context of an annual budget is often seen as presenting difficulties. The most inefficient solution, thankfully avoided by most countries, is to forbid the signature of multi-year contractual commitments. This inevitably leads to implementation inefficiencies and higher unit costs for capital investments. The alternative of making multi-year appropriations brings with it problems of budgetary discipline. Most countries find an intermediary approach. The stability of the policy environment, the political-economic context and the legal framework can have an important bearing on what works and how well. The approach frequently adopted is to allow multi-year contractual commitments, but to make annual, lapsing appropriations (as in Moldova). Where there is sound contract law, a judiciary that is fully independent of the executive and impartial procurement procedures (so that litigants are not discriminated against), there are likely to be significant financial penalties - on top of the efficiency considerations - from decisions to interrupt financing for ongoing multi-year contractual commitments. Generally, governments in these contexts choose automatically to renew funding for multi-year projects - except in cases where projects are obviously failing or there is extreme fiscal stress. Contractors are ready to enter into long-term contracts with government, because they are confident in the stability of the system and of priorities, and because, ultimately, they know they will be able to enforce their contracts. In the political-economic context of Kyrgyz Republic, these conditions may not necessarily pertain and continuity of funding becomes an issue. Some EU Member States use ‘authorizations for forward commitments’ or ‘commitment appropriations’ for multi-year capital investment projects. ‘Authorisations for forward commitments’ authorise commitments to be made over a multi-year period, but annual appropriations are still then required to make payments. ‘Obligations-based appropriations’ also cover multi-year programmes, but are authorisations to pay as well as commit, and are rarer. In some countries, authorisations for forward commitments are presented to parliament as part of multi-year estimates accompanying the budget (with an indicative schedule of forward payments), giving an effective instrument to control and manage the implementation of investment projects, 17 and assisting with analysis of the budget. In presentational terms, this would be much like the information notes on the investment program presented in the Government’s draft 2012 budget, but with more legal force if presented as part of the budget (usually as an annex). As one example, France’s organic budget Law (the ‘LOLF’) distinguishes between commitment appropriations and annual cash-limit appropriations, with an implicit acknowledgment that commitment appropriations may extend beyond one year for projects (‘In the case of an investment transaction, a commitment authorisation covers a consistent whole able to be implemented or executed without any additions’). Sources: ‘Managing Public Expenditure: A Reference Book for Transition Economies’, R. Allen & D. Tommasi, OECD, 2001. 42. With most investment financing coming from donors, the Kyrgyz Republic does not experience major problems in managing multi-year financial contracts in the context of an annual budget, as much of the financing is from donors. However, it is likely to become more problematic if domestic financing increases. This is not an uncommon issue and different countries have different ways of approaching it (see Box 5). At present most domestically financed projects are small in size and therefore not multi-annual, though the example of the road in footnote 5 above shows that problems can arise. In the Kyrgyz Republic, it is possible to enter into multi-year commitments, but resources are only appropriated annually and appropriations lapse at the end of the year. This is a frequent arrangement, but it can cause significant problems in the Kyrgyz context because of unstable priorities, so that multi-year projects end up competing against new projects for continued financing. In practice, entering into multi-year contracts seems to be more difficult than the law would appear to indicate and the very strict policy of closing the books at year-end, so that no commitments can be paid after 31st December, can cause problems with works contracts where the time between submission and eventual payment of stage-payments for completed works can often be lengthy because of verification procedures. EFFICIENT PROJECT IMPLEMENTATION AND ABILITY TO MAKE PROJECT ADJUSTMENTS Efficient Project Implementation Project Implementation Manual and Procedures 43. Kyrgyz Republic has no manual that would formalize project implementation procedures. Structured guidance, including organizational arrangements for proper project management and appropriate monitoring systems (see below), would be advantageous for efficient project implementation. Procurement7 44. The Kyrgyz Republic’s public procurement process is undergoing significant transformation. The Kyrgyz Republic’s 2004 Public Procurement Law (PPL) No 69 created a solid legal basis for public procurement. The PPL established a system where implementation of procurement procedures is decentralized to public entities, with a regulatory body, the State Agency for Public Procurement and Material Reserves at its center to coordinate the system. However, under the previous government’s reorganization of PFM, the State Agency was abolished in October 2009. A new Department for Public Procurement methodology was established under the MoF, and the procurement supervision function was not ascribed to any agency. Training also suffered as it was handed to private contractors and the level of training reduced. As such, budget entities were solely responsible for procurement (although MoF officers sit on committees), seriously undermining the quality of public procurement, especially given that the public sector 7 Sources for this section are the 2010 Joint Economic Assessment by donors. 18 internal audit is weak. Changes being made in 2014 are reintroducing the role of a stronger regulator of public procurement and introducing a number of good international practices on procurement procedures. Even once the new PPL is approved; there will be a need for extensive capacity and institution building. 45. The MoF Procurement Unit started publishing contract award information to bring more transparency to the procurement system. The e-procurement portal currently provides information on ongoing and completed procurement activities with information provided by budget users on voluntary basis, though with the new PPL currently in parliament its use will be mandatory. The process for redressing tender violations remains unclear, though the 2014 changes are introducing an independent appeals process that can mitigate risks somewhat. Given that public procurement carries a risk of high corruption, appropriate oversight and transparency will need to be established as a priority. The MOF recognizes this priority, but its capacity and current legal basis are the main constraints for immediate improvements. 46. The capacity to effectively implement the new public procurement framework will need to be upgraded significantly. Currently, the MoF public procurement unit includes only five staff, whose major responsibility concerns methodology of public procurement. Recently, the unit began to request the minutes of tender commission meetings in an attempt to check accuracy of the tender process; however, the unit’s capacity clearly is not sufficient to do this promptly and with acceptable quality. The treasury is developing a database of contracts, but there is an institutional gap in monitoring the adherence to the procurement rules. The Chamber of Accounts, the supreme audit institution of the Kyrgyz Republic, reviews procurement process only during the regular assessment of budget organizations once every two years. Considerable training will also be needed for procurement officers across the public administration to effectively implement the new law. Financial control and internal audit8 47. A strict central financial control over expenditures is practiced in the Kyrgyz Republic. Fiscal discipline takes priority over predictability of funds for budget units. This is good for aggregate budget discipline, but the rigidities may damage the efficiency and effectiveness of some expenditures, particularly capital investment, which does not have “protected item� status when the monthly budget commission sits to decide on which demands for cash will be met (see ‘Project Adjustments’ below). At present the treasury system operates through an interim software solution that is linked to the central bank, generates some reports and allows for the electronic transfer of funds over som 1 million (about US$20,000). A protracted effort to develop a fully integrated treasury information system, also supported under a World Bank project, was eventually cancelled in late-2013. 48. Cash planning is not always efficient and cash rationing is not uncommon leading to payment of invoices being put on hold.9 As well as possible implementation inefficiencies, e.g., contractors slowing works because of late payment, this probably also increases the cost of capital investment, as contractors build the cost of payment delays into their bids. 49. Internal audit, which takes the form of soviet style revision and checking, is confined to a small number of ministries, departments and agencies. A Law on Internal Audit was adopted in 2009 but with limited progress in implementation, though reform efforts have been renewed recently. However, the concept of internal audit has yet to be fully absorbed, with most auditing still being related to compliance, rather than also extending to financial and performance 8 Source for much of this section is the 2009 PEFA (PI-20 and PI-21 and their dimensions). 9 PEFA 2009. 19 auditing. There is a basic understanding of economy, efficiency and effectiveness concepts, but implementation in everyday work is still to be achieved and more training for auditors is required. At this stage therefore, the impact of internal audit on the qualitative dimensions of capital investment implementation can be expected to be very limited, whereas compliance with rules can be expected to be high. Active Monitoring of Financial and Non-financial Performance of Projects 50. Project monitoring in Kyrgyz Republic is largely concerned with financial monitoring and is largely passive. Except for donor projects, monitoring of non-financial dimensions of project progress is not happening, except on an ad hoc basis. Conceptually, monitoring is not well differentiated from financial control and inspection functions. Insofar as monitoring information is used in a planning and management capacity, it is mainly for budget preparation tasks rather than for identifying problems with individual projects and developing responses. The responsibility of the PIP Department with respect to financial monitoring is to forward a monthly report to the budget department and the central treasury. Its responsibility with respect to physical monitoring is the collation of monthly reports from PIUs. 51. Financial monitoring can be improved. At a minimum, the financial monitoring system needs to be capable of tracking changes in: i) the disbursement profile of a project; ii) total estimated project costs; and iii) estimated completion date compared to plan. Currently, only the first of these three indicators is consistently monitored by the Ministry of Finance. Ideally, the monitoring system needs to be capable of supporting a system for managing total project costs. A proposed aid management platform, Development Gateway, is being developed at the OPM and has the potential to cover all three indicators, though this is only at pilot phase at present. It is possible that this software will be transferred to MoF. 52. The PIP Department of the MOF currently monitors the non-financial performance of donor projects on a monthly basis. It collects and analyses this information using a specific form which implementing agencies (typically PIUs) must return each month, which is then backed up with quarterly meetings between the PIP Department, PIUs, Ministerial Departments responsible for PIU coordination, and donor representatives. In depth meetings are also held on an annual basis. Such monitoring is typically backed up with analyses of annual audit reports of donor projects. 53. While monitoring is evidently a weak point in the Kyrgyz Republic’s PIM system, ambitions concerning its immediate improvement need to be kept realistic and in line with an appropriate sequencing of system improvements. Good indicators and a baseline are required in project design or else there is nothing to monitor against. Project planning improvements are therefore likely to be a prerequisite. The concepts and terminology of the logical framework approach could be a good basis for a project performance monitoring system, but the approach would need to be introduced into project design first. A hierarchical summary of the project’s objectives and quantitative indicators to verify their achievement offers a framework for monitoring (and later ex post evaluation). Box 6 illustrates the hierarchy of objectives in the logical framework, for which corresponding monitoring indicators need to be identified during design. Good implementation plans are also required for monitoring progress: this seems a more realistic possibility in the near term. Box 6. Performance Terminology Used in the Logical Framework Approach  Overall objective/goal: the higher level strategic objective to which the project contributes, usually at the level of the sector. 20  Purpose: summarizes the impact of the project and is the reason why it is being carried out. Usually expressed in terms of the achievement of sustainable benefits for the target group.  Project outputs/results: what the project will deliver, i.e., the specific achievements of the project for which the project management can be held directly accountable (provided they have been provided with adequate resources). For public infrastructure, project outputs will correspond to the facilities to be created. This is not the same as budgetary outputs, which are the goods and services delivered by the government.  Activities: how the outputs/results will be produced, i.e., the actions undertaken by the project organization to deliver the outputs/results. 54. A distinction needs to be made between monitoring progress in project implementation and monitoring performance once a project starts operating. These are different levels of the hierarchy in Box 6. Implementation monitoring involves tracking progress in carrying out project activities, focusing on the important milestones in the project implementation plan, and in delivering a capital project’s physical outputs. Post-implementation monitoring involves tracking the achievement of the project purpose, i.e., the flow of benefits to the target groups through improved public service delivery (volume, quality and accessibility of services) resulting from realisation of the project. Different types of information and different skills in interpreting and analysing the information are required in each case. Monitoring project activities and outputs is more straightforward than monitoring success in achieving the project purpose. 55. Clarity about what needs to be monitored at each level in the system and at what level of detail is also required. Monitoring information required by a project manager is different from that required by the responsible spending agency, which is in turn different from that required by a Ministry of Economy or Finance. In the case of the Kyrgyz Republic, monitoring at the level of spending ministries does not yet seem to be well developed, so the Ministries of Economy or Finance may need to take the lead in developing improved systems. Over time though, monitoring can be expected to become more decentralised to spending ministries: few central ministries in the countries with more advanced PFM systems involve themselves in centralised monitoring of the overall investment program on a project-by-project basis, though there is often some focus on the progress of major projects. 56. The definition of the scope of a project is also important in the context of monitoring . Central ministries cannot dedicate scarce human resources to monitoring the implementation of small projects and individual contracts. This is where grouping objects into programmatic projects can be useful (see the discussion of the proposed regulation on public investment below): rather than monitoring the status of the replacement roof for the primary school in a certain village, for example, a central ministry ought probably to be more interested in how many primary schools have been rehabilitated within a more broadly defined primary school rehabilitation project, and whether these achieved project outputs are according to plan and within budget. For major projects, central ministry monitoring of implementation would be expected to be at a more detailed level, focusing on achievement of the most important milestones in the project implementation plan. Ability to Make Project Adjustments 57. The Kyrgyz Republic’s strict line-item budgeting system does not allow much flexibility when it comes to in-year project adjustments, though counterpart funding can be redistributed between donor projects. Limited flexibility is provided by the frequent supplementary budgets. Some in-year flexibility, with clear rules and limits, is generally desirable, so that resources might be transferred from slower to faster moving projects within a ministry’s 21 overall capital ceiling, thus optimizing overall implementation (see Box 7). This kind of flexibility is effectively blocked in the Kyrgyz Republic by strict rules on virements, a very detailed line-item budget and appropriations defined by several classifications simultaneously. In practice, any reallocations have to be made through supplementary budgets, which are enacted two to four times a year. Box 7. Botswana’s Approach to Project Adjustments A successful approach that used by Botswana has been for Parliament to approve the total estimated cost of each project and the total capital spending ceiling for each ministry for the coming year. This allows funding to be switched (with the approval of the finance ministry) between projects in-year so that an under-spend on a slow moving project can be switched to a faster moving project where implementation might be moving faster than planned, thus enhancing the ability to make project adjustments. Control of total project costs is maintained by approval of the total estimated cost, which becomes the focus of parliamentary attention (rather than the annual slices). If a project’s total estimated cost needs to be revised, the change comes under parliamentary scrutiny and requires approval through the budget. 58. Supplementary budgets are often used to transfer resources away from poorly disbursing projects that entered the budget without adequate checks. Invariably, these projects are projects that had been inserted into the budget during parliamentary scrutiny and amendment. As such, they would not necessarily have represented government priorities. 59. As indicated earlier (para 41), systems for stopping poorly performing or irrelevant ongoing projects are not in place and this tends to be carried out through infrequent, ad hoc cleaning up exercises. Weak monitoring and a lack of a total project cost management system (see example from South Korea in Box 8) makes it difficult to identify which projects should be considered for closure. The stopping of donor projects is almost always as a result of donor intervention, typically on their integrity departments’ recommendation. Box 8. Project Monitoring, Re-assessment and Adjustment in South Korea South Korea has developed 3 instruments for identifying and reacting to changes in project conditions that risk undermining the forecast economic returns from the investment. Total Project Cost Management System (TPCM) The TPCM is used to monitor closely the total cost of major multi-year projects (construction period>2 years; civil works > USD 30 million; architectural projects >USD 10 million) and to prevent significant cost escalation during planning and implementation. In 2009, 1,116 projects were being monitored through the TPCM. The system is based on strict principles limiting the justifications for cost increases and the authority to agree such increases. Since introduction in 1999, it has successfully reduced requests and approvals for increases in total project costs. Re-assessment of Demand Forecast (RDF) Demand forecasts for major projects are re-examined when there are important changes in the project environment. These might be:  Cancellation of a large-scale international event or new city development plan that impacts directly on demand for project services.  Proposals for an alternative way of meeting demand that competes directly with the project.  Demand forecasts become out of date (taken to be more than 5 years since the forecast was made). The aim is to re-examine the continued adequacy of the demand forecast in the light of changes to conditions and assumptions. The RDF can be carried out during any phase of the project cycle from planning to construction. 22 Re-assessment Study of Feasibility (RSF) A RSF is performed for projects in the TPCM if:  The total project cost for a project increases by more than 20 percent in real terms (excluding land acquisition) compared to the previously approved cost.  When the demand forecast for a project falls by 30 percent or more (on the basis of the RDF). When these conditions are met then the preliminary feasibility study (PFS) has to be repeated using the new costs (excluding sunk costs, where construction has commenced) and/or demand forecasts. On the basis of the results of the new PFS, a decision is taken on whether to continue or stop the project. Ways of reducing the size and cost of the project are examined to achieve continued viability and avoid cancellation. Sources: Presentation by Korean Development Institute to International Conference on Public Investment Management in Hanoi, 2010; and ‘Public Investment Management Functionality’, presentation given in Mongolia by James Brumby, World Bank, 2010. PROVISION FOR SUSTAINABLE OPERATION OF FACILITIES 60. By government resolution, The Kyrgyz Republic has a handover process for new facilities by which they are verified as being consistent with the original design specification and as being fit for purpose. There are formalised procedures for acceptance and coming into service of new facilities, which cannot be used until this is completed. 61. However, asset registers are not well developed. There is a long way to go before the Kyrgyz Republic has a fully developed system of asset registers that are kept up-to-date with the recording of new assets and the updating of values for existing assets. There is even further to go before asset management tools are developed. At present all asset information is submitted to the President’s office. 62. Although a framework for integrated budgeting of recurrent and capital expenditure exists in the form of the MTBF, it is by no means certain that operating and maintenance expenditures for new facilities are being systematically planned for. PEFA 2009 gives a score of D to dimension “iv� (‘linkages between investment budgets and forward expenditure estimates’) of PI-12. Planning improvements in the road sector have resulted in a more integrated approach to budgeting, but such practices have not infiltrated the whole of government. BASIC COMPLETION REVIEW AND EX POST EVALUATION 63. Ex post evaluation remains one of the weakest components of the Kyrgyz Republic’s PIM system. This issue has recently been highlighted in parliamentary budget hearings, with members of parliament (MPs) rightly concerned about the impact of public investment spending. At present, there is no systematic evaluation of completed projects to determine whether they represent an efficient and effective use of public resources, either by government or donors. This is probably not surprising at this stage of development of the PIM system, but evaluation should become increasingly important as the country moves towards performance-based public management and budgeting (as is foreseen in the adoption of a program budgeting approach). Good practice is to begin by carrying out evaluations of major projects, probably beginning with a focus on problem projects where lesson learning is potentially greatest. Evaluation can then be extended to all major projects and a sample of smaller projects as capacities develop. 64. The possibility of introducing a requirement for a formal project completion report should be considered. This would determine whether a project has been delivered on time, on 23 budget and to the correct specification and would differ from certification/acceptance of works. It should be more informative and analytical, looking at such things as the original vs. the achieved budget and the achieved time-line vs. the planned and offering explanations and lessons learnt. A project completion report may be seen as a first step towards full ex post evaluation, which may be an ambitious requirement in the near-term. 65. The Kyrgyz Republic’s Chamber of Accounts can be expected to take an increasingly important role in the evaluation of public investment projects. Amongst its responsibilities is the evaluation of implementation of republican and local budgets, off-budget and special funds. This includes all projects under the PIP. These capacities can now be expected to develop further so that independent assessment of the performance of public investment projects (using risk-based sampling) eventually becomes part of the PIM system. However, at present the capacity of the Chamber of Accounts (COA) appears to be limited, and therefore unlikely to embrace a wide portfolio in the near future. Nevertheless, evaluations by a country’s supreme audit institution can be a powerful way of driving through system improvements (see Box 9). Box 9. Activities of the United Kingdom National Audit Office The National Audit Office (NAO) is the Supreme Audit Institution for central government in the United Kingdom. It is an independent body with no judicial authority reporting directly to Parliament. Under the 1983 National Audit Act, the NAO is able to report on the economy, efficiency and effectiveness of public spending - summed up as ‘spending less’, ‘spending well’ and ‘spending wisely’. About 60 value-for-money reports are presented to Parliament each year on all aspects of expenditure, so only a small number of public investment projects are audited (financial audit of projects is picked up through the financial audits of the organisations implementing them). Problem projects or innovative projects from which lessons can be learned are therefore chosen. Reports are made available to the public and press releases are issued on the day of publication. As a result of the tendency to focus on problem projects, the reports can sometimes make gloomy reading, but the intention is to assist in improving future performance. Three recent reports relating to public investment in the transport sector are:  Estimating and Monitoring the Costs of Building Roads in England (March 2007) : focuses on explaining the significant cost increases experienced in the road building programme compared to original estimates and makes recommendations on how better to estimate and then control costs.  The Modernisation of the West Coast Main [Railway] Line (November 2006): explores how a poorly performing project was eventually turned around by improving project management and cost control, reducing design standards, relying more on proven technology and phasing in capacity increases. London Underground [Railway] PPP: Were they good deals? (June 2004) : looks at three contracts using innovative procurement approaches for securing long term investment in service improvements using PPPs. The report drew attention to the significant contingent liabilities faced by the public sector, which were subsequently realised when the private partners went into liquidation. 4. Institutional and Administrative Framework for Strengthening PIM LEGAL AND REGULATORY FRAMEWORK FOR PUBLIC INVESTMENT MANAGEMENT Legal Basis for the Public Investment Management System 66. The legal roles and responsibilities for the Government in respect of the budget are set out in the Constitution of the Kyrgyz Republic from 2010. The other major legislation is: the Law on the Basic Principles of Budget Management 1998 (updated in 2009); the Law on Treasury; 24 the Law on the Chamber of Accounts 2004 (updated in minor ways in 2008). There is also a Law on Procurement and a new Budget Code currently in the parliamentary approval process. 67. The usual practice among countries in a similar position to Kyrgyz Republic is to provide for the necessary powers for public investment management through the budget systems law. This can either be done succinctly by granting the necessary powers to the government or the finance ministry to issue regulations and methodologies to define the system (as in the examples of Croatia and Turkey), or through a more extensive definition of the public investment management system within the overall budget system (as in the examples of Kazakhstan and Croatia). It is recommended in the Kyrgyz Republic that a ‘light’ approach be adopted so as not to run the risk of the amendment being rejected as a substantive change to the original draft law. This would also have the advantage of leaving greater flexibility to adapt systems in the future on the basis of experience. 68. There is usually no specific basis in primary legislation for the public investment management system in countries with more advanced performance management arrangements (Australia, the United Kingdom or South Africa for example). These countries are characterised by decentralised responsibilities for expenditure management existing alongside strong accountability arrangements, including well-developed internal audit. Generally, there is a strong and independent external audit institution with the necessary human and financial resources to verify ‘proper use’ (see Australia for example) and with a focus on performance rather than compliance (although its coverage will tend to be selective). However, this approach is not universally applied (or applicable) and is probably not suitable for Kyrgyz Republic at the present time - although it could offer a potential future orientation - because of differences in administrative culture, legal tradition and political context. Regulation of Public Investment Management Areas where the regulation could benefit from strengthening 69. There are a number of areas where the draft regulation would benefit from strengthening. This would ensure that a system is established that addresses the weaknesses identified in the diagnostic review above and is fully in line with the good practice framework described at the beginning of this chapter. The following suggestions for improvement are considered to be the most important:  Define a consistent non-financial performance terminology to be used in project design – to summarize the project rationale and define suitable performance indicators - and then as a basis for more systematic monitoring and evaluation.  Allow grouping of similar, small projects into larger programmatic projects as a way of avoiding centralized micro-management and as a basis for a more strategic prioritization process.  Introduce greater flexibility in the application of different analytical methods to reflect the nature of project benefits, the financial importance of the project and the availability of the necessary skills. Lower value projects should be subject to less sophisticated analysis, particularly considering the scarcity of the necessary skills in Kyrgyz Republic. Introducing strong proportionality will improve the overall efficiency of the public investment management system. Still, general guidance should be developed in advance on which methods could be used under which circumstances. This guidance could state: 25 (i) the basic norms of the analytical tools and parameters; (ii) the scope of each sector or size method; and (iii) basic rules on how to manage them.  Distinguish more clearly between the project screening and project selection functions and clarify the roles of the MoF, OPM and MoE in relation to these two. In the context of selection, ensure greater consistency between the regulation and the new budgeting methodology.  Confirm the Ministry of Finance’s role as ‘gatekeeper’ to the budget, verifying that projects proposed for budget funding have indeed passed through the necessary screening and approval processes.  Extend the regulation to cover regular monitoring of implementation progress for projects, specifying the minimum requirements for monitoring financial and non- financial progress. This would allow early identification of problem projects and the formulation of timely responses, including compulsory re-evaluation of significantly over-budget projects. 70. The draft regulation and the refinements suggested above will not resolve all identified issues, but could go some way to reducing the scale of the problem by requiring all projects to follow the same route through a disciplined screening process before being proposed for the budget. This will prevent unprepared projects from being ‘parachuted’ into budget and taking up fiscal space that would be better made available to ongoing projects or projects that are ‘ready to go’. Grouping smaller projects into programmatic projects and applying a strict definition of a project to avoid funding isolated project components will also help.10 It is, however, not cost effective to dedicate scarce human capacities to applying sophisticated analytical methods and multi-stage approval procedures to micro-projects of the kind that parliamentarians usually propose: it is sometimes better to accept small inefficiencies of this nature and to focus on getting the bigger strategic investment decisions right. Recommendations for Improvement of PIM Administration 71. Whilst the analysis above identifies a myriad of weaknesses within the PIM framework in the Kyrgyz Republic, it is important to prioritise improvements. A basic rule of thumb in this regard is to introduce improvements at the earliest levels of PIM i.e. identification and screening, and then introduce other improvements over time. In addition, it is necessary to coordinate improvements with appropriate institutional strengthening, and recognise available capacity. The following recommendations are therefore presented in order of importance:  Ensure the development of a comprehensive database of investment projects; both donor and domestic funded. The database should include original project data so that eventual outcomes can be tracked against original plans as well as revised.  Training should be instituted in identification and appraisal. This could be initiated with short courses as provided by the University of Central Asia (UCA) in Dushanbe in 2012.  Introduce a project identification process. Ministries should institute project committees and internally screen project ideas before presentation to a central body. 10 A project is a group of activities with a clearly defined purpose and outputs which are implemented over a fixed time schedule. It should encompass all the activities and resulting outputs required to deliver sustainable benefits to the identified target beneficiaries (service users). 26  A central body (MoF or an Investment Agency) should screen project proposals, both donor and domestically funded. This could start with a simple project appraisal matrix as described below which could be developed into a more sophisticated matrix over time. Such a matrix should apply to both donor funded and domestically funded projects equally.  The domestic investment budget should be approved under the budget law including a breakdown of all projects to be included.  Strategic documents at national and then sectoral level should be developed and fully costed-out to guide investment priorities.  GoKR should encourage the participation of civil servants in project development activities of donors especially in areas of cost benefit analysis, so that capacity can be developed.  A project completion report should be instituted and required for all projects.  Independent appraisal of large-scale projects should be considered, using independent consultancy firms.  Project monitoring should be developed hand in hand with donors at first. Emphasis should be put on ensuring that output indicators are well developed so that future monitoring can be easily effected.  The eventual recurrent costs of projects should be linked to the medium term budget.  There should be a mandatory requirement to reintroduce stalled projects unless a decision has been taken to close the project.  Procurement requires strengthening as per the latest PEFA report, in particular in the encouragement of open competitive bidding.  The ad hoc monitoring of domestically financed projects should be considered for a limited number of years.  Donors should be encouraged to assist in the evaluation of projects post implementation, particularly larger projects. ALTERNATIVE ORGANIZATIONAL ARRANGEMENTS FOR PUBLIC INVESTMENT MANAGEMENT IN KYRGYZ REPUBLIC Overview 72. There is no ‘correct’ model of organisational arrangements for public investment management. The most suitable model will depend on country circumstances and what is politically feasible. There are a variety of different organizational forms for leading and performing the key functions of an effective public investment management system. In general, more centralized models, like Chile, South Korea or Belarus, can be distinguished from more decentralized models, illustrated by the role of the Central Expenditure Evaluation Unit in Ireland’s finance ministry. The latter model tends to be associated with the more advanced performance management systems. In the more centralized models there is a stronger role in the quality control of the investment program for central financial agencies; in the more decentralized models, the central financial agencies have a role that can be seen as closer to quality assurance, i.e., ensuring 27 the presence, effectiveness and application of quality control systems in the spending agencies responsible for planning and managing public investment projects. The quality control – quality assurance distinction is helpful, but not that clear-cut in practice: even in the most decentralized system, central financial agencies will generally quality control the biggest projects. 73. For the Kyrgyz Republic at the present time, many of the non-financial aspects of PIM are not being carried out and financial aspects require strengthening. The Ministry of Finance needs to keep and strengthen its role in the financial control of projects prior to and during implementation. This leaves the question of where those non-financial aspects should be carried out. There are general system weaknesses, insufficiently developed accountability for performance and weak project preparation and assessment capacities in spending agencies. This applies to monitoring as well, where some form of centralized monitoring of the investment program is appropriate for the same reasons. The government should be active in developing improved procedures and methods, and supporting their introduction across spending agencies. As time goes on, the quality assurance function would become more important. Essentially there are three options: a) As indicated above (para 33), build on the government resolution of 2007 which moved the responsibility for PIP co-ordination from MoF to MoE. Despite this resolution, MoE has not played any significant role in PIP since 2007. It lacks capacity as well as flows of information from the line Ministries and MoF. To implement the government resolution it is essential that MoE is authorized to be the primary gatekeeper of the PIP, as well as other investment projects. This would require line ministries to undergo a screening process in their own ministries first and foremost, to be followed by appraisal by MoE. Clarity regarding the responsibilities and flow of information between MoE, OPM and MoF would be essential. Capacity building in MoE would be a high priority, building on the existing capacity in the Investment Department, which until now has been orientated towards private rather than public investment such as the development of public-private partnerships (PPP) and the implementation of the ADB financed Investment Climate Improvement Programme. It would also be beneficial if the domestically financed investments were also appraised by the same department within MoE, using similarly developed processes. The question of responsibility for monitoring and evaluation of projects also requires clarity. PIP monitors (both financial and physical) donor projects, but does not perform evaluations. There is no formal process for the monitoring and evaluation of domestically financed projects. b) Restore PIM to the MoF. Again, this would require a significant degree of capacity building to implement the required techniques of appraisal and monitoring and evaluation. The current PIP unit concentrates on the financial aspects of appraisal. With respect to monitoring, it performs both financial and physical monitoring. Domestically financed investments would require significant capacity building in appraisal, physical monitoring and evaluation. It should be noted that if the proposed database was developed, then capacity would be freed in the PIP department that could be used for PIM. c) Create a separate body responsible for PIM. This could possibly work under the OPM’s Macroeconomic Framework and Investment Policy Council or another separate body. One advantage of this approach would be to reap economies of scale in using all or most of the staff of the PIUs in one centralised body. Again, if such a path were taken, it would require clear guidelines for responsibilities and information flows between this body and the MoE, OPM and MoF. 28 74. Over time, it is likely that the line ministries will increasingly perform more of the screening, appraisal, monitoring and evaluation functions, and the function of the central ministry/agency would be more orientated to ensuring that these functions have been carried out correctly. Still, in the context of the Kyrgyz Republic at present, it is important to centralise operations given the low capacity throughout government. It could therefore either be recommended to centralise operations under the MoF or under a “new� agency. Either option should consider the streamlining of PIUs so that PIU capacity can be used across all projects rather than individual projects. ASSESSMENT OF PIM CAPACITY AND RECOMMENDATIONS 75. Three institutions provide some capacity in the field of PIM. Ministry of Economy has capacity in the Investments Department. Ministry of Finance has capacity in three Departments: PIP, Real Sector and Capital Investments. The Office of the Prime Minister has an Economy and Investments Department. Line ministries (not including donor financed PIUs) have little capacity to prepare, screen, monitor or evaluate investments. 76. The Investments Department in MoE primarily works on private investment. It currently implements an ADB project “Investment Climate Improvement Programme�. It also prepares legislation and procedures for the development of public private partnerships in the Kyrgyz Republic. Finally it responds to requests in assisting with various investment-related tasks within government e.g. visiting trade missions. There are 11 members of the department, plus two more working in an intern capacity. At the time of the drafting of this note, the Head of department position was unfilled, with the Head of investment promotion acting as Head of department. The department also has an aid coordination division which coordinates donor aid which falls outside the ambit of the PIP, such as most technical assistance projects. The investment promotion division has 5 staff as well the Head. The aid coordination division has 3 staff as well the Head. The two Heads of division have expertise in the field of public investment management. However, the rest of the staff have varying levels of expertise and experience picked up in assisting with various investment related tasks and research papers written. High turnover of staff has also inhibited the accumulation of expertise in the Department. 77. The Ministry of Finance financially monitors public investment in three separate departments. Donor financed investment is monitored in the PIP division of the department of public investment and TA. It also collects physical monitoring information from PIP projects. Domestically financed investment is divided between the real sector department and the capital investments department, which assumes responsibility for various projects in all sectors, including all social sector projects. Whilst all departments have an obligation to report to the Budget Department, there is little linkage of investment planning and budget preparation. 78. The PIP division is relatively young with limited capacity. All the staff are relatively new to PIP issues with the exception of the Head of division. Five sector specialists collect data from their ascribed sector and assemble it on excel spread sheets, facilitating requests for disbursement schedules from Treasury and budget planning data from the Head of budget. Other responsibilities include attending donor meetings on any aspect of projects in their sector, and attending tenders for projects they have responsibility for. Physical monitoring information is also collected monthly and used in association with financial monitoring information at project review meetings every quarter. The relationship with donors is largely passive given the lack of capacity to analyse project documents. There are currently about 60 projects under PIP with a value of about 29 US$2 billion. The expenditure for 2012 is expected to be about US$500 million. The division has no formal link to sector specialists in other departments dealing with public investments. 79. The real sector department is responsible for most domestic investment in the sectors of: agriculture, transport, energy, emergencies, forestry and ecology. It also covers budget issues in other sectors that do not include an investment budget e.g. minerals agency. The five sector specialists receive project proposals at any stage during the fiscal year from their respective ministries, and check for affordability. At present the preparation of proposals is weak. There is no formal process for submitting proposals. Moreover there is no requirement for line ministries to submit their list of project proposals to MoF at the start of budget planning, though some sector specialists insist on this. The specialists mostly focus on financial monitoring, though ad hoc physical monitoring also occurs. The department is subject to political pressure to approve projects irrespective of their affordability, strategic importance, or degree of project preparation. 80. Integration of the recurrent and capital budgets is very limited, even institutionally. For example, the social sector department is responsible for the recurrent budget in the health, education and other social sectors. There are 12 staff, four sector specialists in health and social affairs, and six in education/sport. However, they have no responsibility for domestically financed investment which is solely the responsibility of the capital investments department. There are no formal communications between the two departments, indicating a segregated budget. 81. The department of capital investments is responsible for capital investment in a variety of sectors, both real and social, though their budget is significantly less than that of the real sector department. In addition, the department also channels stimulating grants to local authorities, which are generally for maintenance rather than capital expenditure. Four specialists are responsible for the financial monitoring of projects. The quality of project proposals from line ministries is similar to those received by the real sector. Stimulating grant approvals reflect the quality of project proposals, but also political influence rather than need. There is no monitoring beyond the receipt of local authority reports. There is no formal coordination of activities with the activities in the real and social sectors departments. 82. The three separate departments report to the Head of budget, where affordability and financial monitoring is aggregated. Whilst the Budget department is primary, and has authority to request information, it is not directly in control of the three departments responsible for investment. As a result investment planning is seriously disaggregated in the Kyrgyz Republic, with little central management. There are no fundamental reasons for the capital investments’ department projects in transport/roads appears to be separate from those of the real sector; the nature of projects there being not fundamentally different. Whilst the emphasis is on donor coordination rather than investment management, there is a rationale for PIP to be a separate department, but when this orientation changes, then PIP also should be reformed, shedding its non- investment projects. Figure 1 indicates current investment management in MoF. The budget department is central and requests and receives information on investment from three separate departments with almost no formal information flows between any of the three departments feeding the Budget department. 83. The move from PIP to PIM has occurred in many countries. It represents an increased ownership and management of investment by national governments. Donor coordination still occurs but the focus is on the management of investment whether donor or domestically financed. If PIP was to be reformed in the Kyrgyz Republic then the investment elements would be retained within a new investment department, whilst the technical assistance and other “non-investment� projects would be taken out. The new investment department would also take responsibility for 30 domestically financed investment projects. Procedures for both types of projects would be exactly the same, though larger projects would be subject to more rigorous screening, appraisal, monitoring and evaluation. A potential investment (financial monitoring) management framework for the Kyrgyz Republic is presented in Figure 2. The divisions of the real sector and capital investments could be merged into a single domestic investment division. Sector specialists in both domestic and donor investment should communicate, particularly during budget planning in assessing which projects represent priorities under government policy. Figure 1. Current Investment (Financial Monitoring) Management in KR PIP Budget Dept. Real Sector Capital Investments Source: Authors. Figure 2. Potential Investment (Financial Monitoring) Management in KR Budget Department Domestic Donor Investment Investment Source: Authors. 84. The proposed unification of the Ministry of Finance’s investment responsibilities is, at present only reflective of financial monitoring, with important decision yet to be made regarding the remaining aspects of PIM. The other essential aspects of investment Management (screening, appraisal, physical monitoring and evaluation) are yet to be placed within any institution. It should be noted that the paper does not recommend a specific PIM model for Kyrgyz 31 Republic. For instance, it could be centralised in the MoF or split between MoF and MoE (with MoE taking responsibility for at least appraisal, but possibly physical monitoring and evaluation as well), or split between MoF (financial monitoring) and a specialised agency. While, the decision as to where the functions of investment management should be carried out is of utmost importance, any ministry or agency tasked with PIM will require:  Significant training.  Responsibility for the proposed Development Gateway database.  Development of regulations (and possibly laws) and procedures for PIM.  Clear information flows with the budget department of the Ministry of Finance.  Clear understanding of the Government’s Policy.  Primacy over line ministries and other budget agencies in demanding investment plans (including procurement), monitoring data, and evaluations. Rejection of project proposals can also be made if procedures are not followed.  Primacy over donors in demanding Kyrgyz Republic government procedures are followed in their reporting. 85. As noted earlier, given weak capacities in budget units, a centralized model is recommended at this stage. Simply addressing quality control issues will not suffice for the Kyrgyz Republic at this stage, though through the leadership of a central ministry or agency, capacity could be developed in line ministries over time. 86. The departmental structure for PIM would be similar irrespective of where responsibility was placed. It is likely that staff would be allocated along sectoral lines. The key to the establishment of the department would be to provide it with the responsibility as the sole agency that can approve projects. This would require it to have a set of procedures for staff to follow, laid down by government regulation. These may well develop over time but, at first, could include a simple appraisal form such as the one in Annex 1. Such an appraisal form may also need to be accompanied by evidence that the budget unit has followed internal processes in approving the project. This may include a Project Concept Note, evidence of the unit’s project meetings and minutes as well as full project documentation for design so that the appraisal process can function. Any projects which do not fulfil these criteria would be ineligible for funding under the budget. 87. Training would be required to initiate PIM procedures. Staff would require courses on understanding the project cycle as well as aspects of PIM such as a course given by the UCA in Khorog, Tajikistan in summer 2012.11 Longer courses such as a Masters may also be appropriate for senior members of staff. Staff should be included in project cycle activities of the donors, sensitising them to donor processes as well as all technical stages of the project cycle. A study visit to experience PIM in other countries may also be beneficial, such as Turkey. Realistic Plan for the Development of PIM in MoF or Independent Agency 88. This plan would require clear delineation of responsibilities (for example: project proposer; appraiser; reviewers; gate keeper and implementing agency). In addition it will also require the following steps to be followed. http://www.ucentralasia.org/downloads/SPCE%20Khorog%20Project%20Appraisal%20- 11 %202%20July%202012.pdf. 32 Step 1: Nominate MoF or an independent agency as the institution responsible for the appraisal of all investment projects. Step 2: Formal Guidelines and Procedures for project appraisal should be subject to government regulation. Critical elements of these guidelines are:  No project can be forwarded for funding purposes to MoF without approval.  Budget agencies are required to submit projects to MoF or independent agency in the format designated, including donor funded projects.  Thresholds for the level of detail of documents required for appraisal are set; for example: o projects above US$50 million would require a feasibility study; o projects above US$2 million would require the calculation of costs and benefits or cost effectiveness12; o projects under US$250,000 would not require cost-benefit analysis (CBA) or cost effectiveness (CE) but could indicate simple ratios e.g. costs of a classroom per student, o projects under US$50,000 would be required to be aggregated with other similar projects except in special circumstances.  Budget agencies are required to forward costed Sector Strategies on an annual basis.  MoF is required to issue an investment ceiling during budget preparation.  MoE or the agency is responsible for preparing a list of projects within the budget ceiling provided by MoF and providing this to MoF by the deadline described in the budget circular. The list will reflect decisions by line ministries (as long as the list of projects proposed is in line with budget ceiling and meeting all requirements) or government decision.  All Sector strategies and projects must be in line with both national and sector strategies.  Once the budget is approved, MoF is responsible for updating the database of investment projects (expected to be a web-based Development Gateway with facility to include domestically financed projects). Step 3: Staff are recruited or redeployed and trained in the formal guidelines and procedures. Just for purposes of appraisal, it is likely that staff are required to cover all major investment sectors (transport, health, education, mining, agriculture, other social sector, local government (possibly), energy). Other sectors with occasional investment projects could be ascribed to a related specialist e.g. forestry and ecology could be paired with agriculture. A critical part of training is that of 12 The main difference in the two concepts are that cost effectiveness analysis relates costs to specific measures of program effectiveness (for example: cases treated in health or drop-out rates in education) while cost benefit analysis relates the costs to the monetized value of program benefits. In other words higher cost- benefit analysis does not always translate into cost effectiveness. A hospital could be built in the most efficient manner (optimal resources, time, technology are used within the allowed budget) but may be located in an inaccessible place which will hardly be utilized by the citizen (hence lower cost effectiveness). 33 determining SMART (Specific, Measurable, Achievable, Relevant and Time bound) outputs. If these are not developed at appraisal stage then monitoring and evaluation will be difficult or impossible. Step 4: All required forms are developed for appraisal. Step 5: Line ministry and other budget agencies are invited to MoE to explain the new procedures for the acceptance of projects. They are also briefed on how to follow required procedures in their own institutions e.g. the use of a Project Concept Form, the necessity of forming project committees, the establishment of a two-stage procedure to design projects (project identification and design). Step 6: In collaboration with MoF, determine the timetable for project appraisal for the following year’s budget so that deadlines can be issued to line ministries for submissions. 89. Finally, it should be noted that capacity at budget unit level should also be addressed. Whilst regulations can require line ministries and others to submit project proposals in required formats, it is likely that training in project design will be required at budget unit level. In particular, the identification of SMART output indicators is of crucial importance and often proves problematic without training and experience. 5. Summary Findings, Conclusions and Recommendations of the PIM Diagnostic 90. A number of components of PIM need to be improved in the Kyrgyz Republic in order to increase efficiency of public investments. Annex 2 summarises the findings of the PIM diagnostic, comparing the situation in Kyrgyz Republic against the institutional benchmarks in the diagnostic framework. A score on a five-point scale is suggested against each benchmark. These scores are also colour coded from red to green, with red indicating dimensions of greatest concern where Kyrgyz Republic is farthest from good practice. Taking the arithmetic mean of the scores under each of the eight ‘must-have’ features of a functioning PFM system allows a general overview of Kyrgyz Republic’s PIM system to be obtained. The ‘heat’ chart derived from these aggregate scores helps identify those areas that are of particular concern in Kyrgyz Republic’s case . ‘Front- end’ processes - preliminary screening, and independent review – together with ex post assessment are shown to be weakest elements in Kyrgyz Republic’s PIM system, followed by project appraisal, implementation and the capacity to adjust to changes in project circumstances. Heat Chart of PIM in Kyrgyz Republic Strategic guidance and preliminary screening Formal project appraisal Independent review of appraisal Project selection and budgeting Project implementation Adjustment for changes in project circumstances Facility operation Ex post assessment (evaluation) 34 GUIDANCE AND PRELIMINARY SCREENING 91. The Kyrgyz Republic needs to strengthen its strategic planning processes by developing a more integrated framework for strategic planning. Such a framework should ensure greater consistency between strategic plans, involve regular review and updating, and link planning to realistic estimates of medium- to long-term resource availability. The development of an overarching infrastructure strategy might be worth considering, provided its development is coordinated with sector strategy development. The development of asset management tools will help in identifying needs and feeding these into a strategic planning process that is more firmly based on evidence of the condition of the country’s assets. 92. The Kyrgyz Republic requires a rigorous preliminary screening process. This will involve an initial assessment of the project concept and a formal decision to proceed to detailed project planning and appraisal and should be introduced as an important first step in screening out poor project concepts or those that do not fit with expressed national or sector priorities (see Box 2). Allowance needs to be made in Kyrgyz Republic’s budget system for more strategic, ‘programmatic’ projects that group together similar small projects with the same objective . This will encourage planning and budgeting to be more strategic. However, it would be important that a clear definition of the program is developed including eligible scope and other characteristics (scale /type) to reduce the risk of misuse. FORMAL APPRAISAL AND INDEPENDENT REVIEW 93. Project appraisal should be formalised and made more rigorous and involve systematic analysis of the economic feasibility and financial sustainability of public sector projects. In effect, this means bringing appraisal of domestically funded projects into line with externally financed projects, for which this is already a usual requirement. On the basis of a feasibility study (or its equivalent) a formal appraisal decision should be a requirement, involving sign-off by officials responsible for promoting and authorising projects. 94. The Kyrgyz Republic should adopt a common appraisal methodology, applicable to all projects irrespective of funding source. This methodology could be based on existing methodological guidelines such as those prepared in the PIU for the roads sector. The sophistication of methods should be made proportional to the importance of the decision, i.e., scale and complexity of the project, and to the availability of skills. At first, it is recommended that the threshold for full feasibility studies involving cost benefit analysis (or cost effectiveness analysis, where applicable) should be set relatively high, so that closer scrutiny is directed at the more important decisions and so that scarce analytical capacities are used to best effect. Simpler approaches can be used for less significant investments. On the basis of experience and as capacities develop, the threshold can be lowered. 95. Independent review of appraisal findings and the appraisal decision is a critical element in many countries’ PIM systems and Kyrgyz Republic needs to consider how best to replicate this. The choice is between review by the Ministry of Economy or Finance or by a demonstrably independent peer reviewer. The former two would require developing the necessary skills in the chosen ministry and preparing its staff to take on a stronger challenge function. Overall, project appraisal skills must be developed across government in a coordinated fashion, so that the new skills are actually applied in decision-making. 35 PROJECT SELECTION AND BUDGETING 96. Further development of Kyrgyz Republic’s MTBF process will be beneficial for public investment management, creating a framework for planning, prioritisation and implementation of multi-year infrastructure projects and for their sustainable operation and maintenance after delivery. At a technical level, the MTBF is already part of the budgetary processes, but it still lacks the political buy-in that is crucial for credibility. It is recommended that the Kyrgyz Republic continues its efforts in this direction. Project prioritisation within the MTBF/budget preparation process will become easier if more rigorous preliminary screening and appraisal are introduced. This will focus attention at budget time more on the fit of projects with national and sector policy objectives on practical programming issues (see Box 5). 97. The relationship between the executive and the legislature in defining the capital part of the budget poses needs to be clarified. Currently, parliament spends quite some time in debating the allocation of the capital budget ceiling months after this ceiling has been approved. Though MoF has the details of which projects make up this ceiling, parliament often changes the allocation significantly. 98. Furthermore, ability to effectively implement multi-year project is undermined by uncertainty over budget availability in outer years. Developing a capital baseline – estimated funding requirements for ongoing projects - at the beginning of the budget preparation process is recommended as one element of the solution. New projects should then compete against each other for funds remaining within capital ceilings once the capital baseline is met. 99. The gate-keeping role in the budget process needs to be reinforced. The ministry responsible for appraisal needs the power to turn down projects at budget time when they have not been through the proper appraisal process beforehand. This should apply equally to projects inserted into the budget by Parliament, although obtaining parliamentary consent for this might be difficult. PROJECT IMPLEMENTATION AND PROJECT ADJUSTMENT 100. Broader reforms in public financial management will be important for strengthening project implementation. This includes better cash planning, strengthened financial control and developments of an integrated treasury information system, further development of internal audit, and more competitive public procurement. Specifically in relation to public investment, the Kyrgyz Republic needs to consider developing a government-wide project implementation manual to guide project managers on how to organise, manage and report on project implementation. 101. Reporting on project implementation needs to be enhanced. Financial reporting on project implementation should be improved so that there is better reporting on changes in the total estimated cost of ongoing projects and in the disbursement profile. Reporting on non-financial indicators of implementation performance - which need to be defined during project preparation - also needs to be instituted to beyond the PIP portfolio. More centralised monitoring of project reports is one option to be considered, but monitoring for its own sake serves little purpose. A responsive monitoring system, which identifies problems early and then ensures that solutions are being implemented, should therefore be developed in order to improve implementation efficiency. For this to be effective, project managers also need to be given adequate flexibility to make adjustments to projects in order to react to changed conditions, the suggestion being that they enjoy too little flexibility at the moment. 36 102. Appraisal procedures need to build in re-appraisal of major projects where total estimated costs increase substantially or where there is expected to be a significant shortfall in estimated project benefits. Mechanisms for stopping failing projects or projects where value for money is no longer assured should be put in place to replace the intermittent reviews of the project portfolio that occur now. FACILITY OPERATION 103. Development of asset registers and asset management systems in the infrastructure sectors should be pursued. These will help in identifying infrastructure investment (renewal) requirements and optimal expenditure needs for facility operation (maintenance). EVALUATION 104. Ex post evaluation is often one of the last areas of a PIM system to develop. Partly this is neglect, but partly it is also because evaluation is very difficult without meeting some minimum planning requirements so that there is a baseline and forecasts against which to measure performance. The priority should therefore be to achieve minimum standards in project preparation and appraisal first. A critical feature in this regard is the careful choice of output indicators. All the same, limited reviews on project completion, particularly larger projects, should become standard to identify reasons for deviations between the project as designed and as delivered, and learn lessons for future project design. 105. As with appraisal, evaluation needs to be proportionate. It is recommended that evaluation should begin with a small number of major projects and then extend to a risk-based sample of lower value projects as time goes on. It is unlikely to ever be worthwhile undertaking evaluation of every project. 37 Annex 1. Project Appraisal Matrix Project Name Applicant Ministry Amount APPRAISAL CRITERIA A1. PROJECT STRUCTURE SCORE Max 20 Is the project purpose well defined and is it deliverable by the project? 5 Are project outputs and activities sufficient and necessary to meet the purpose? 5 Will the indicators measure project success? 5 Is there potential for significant long-term or wider impact? 5 Justification for Score: (in box below which expands) A2. RISK SCORE Max 8 Has a risk assessment been completed and does it cover all significant potential 4 risks? Have clear risk management strategies been identified and will these be sufficient? 4 Justification for Score: (in box below which expands) A3. BUDGET SCORE Max 10 Have all resources for the project been identified, including recurrent cost liabilities, 3 & are they covered in the budget? Is the project financial profile (cash flow) in line with the activity schedule proposed 3 Overall, does the project offer value-for-money? 4 Justification for Score: (in box below which expands) 38 Annex 2. Summary Assessment of Kyrgyz Republic’s Public Investment Management System Stage Indicators and Dimensions Findings Rating 1. Guidance and National development strategy; A comprehensive strategic development document was only recently adopted in the 1 (Red) preliminary sector strategies form of the National Strategy on Sustainable Development. Earlier efforts under the screening. framework of the Medium Term Development Programme proved largely ineffective. This left a proliferation of sector strategies with no overarching framework or hierarchy. Encouragingly, a number of sector strategic documents are increasingly being aligned with the NSDS though this is yet to be systematically ensured. Credible investment strategies exist in important infrastructure sectors, but their influence on actual investment proposals seems to be muted, though better in the donor sphere than domestic financing. First level screening to ensure strategic alignment Investment planning remains largely ad hoc and is not yet based on systematic recording 1 (Red) of asset condition within dynamic asset management systems. There is no formal project concept development and preliminary screening process. 2. Formal project Clarity of roles in project A government resolution of 2007 transferred PIP co-ordination from MoF to MoE. Since 1 ( Red) appraisal. planning process then, MoE has not assumed any significant role in the PIP. Project planning is decentralized to line ministries and spending agencies. PIM procedures are not formalized and the Role of Ministry of Economy in the project appraisal process is not properly codified. Formal technical guidance There is no formal technical guidance on planning and appraisal methods applying to the 1 (Red) whole of government. More systematic guidance exists for some sectors, e.g. roads. Sound project appraisal A large share of capital expenditure is IFI/donor-funded and is subject to rigorous 3 (Yellow) appraisal. Projects relying on domestic budget funding alone are not subject to Proportionality of appraisal systematic appraisal. No formal proportionality. By default appraisal is proportional - because donor-funded 2 (Yellow / Red) Effective coordination and projects are higher value - but this is not a procedural requirement. scrutiny of donor-funded projects Donor reporting to MoF is harmonised under the PIP, though some donors do not report. 2 (Yellow / Red) Donor-funded projects are not necessarily aligned with government priorities, reflecting Capacity in central agencies and the weakness of the current MTDP. line ministries Capacities in project appraisal are weak throughout government. Lack of demand for 1 (Red) good appraisal from decision-makers does not foster a good environment for strengthening. 3. Independent Independent reality/quality There is no independent peer review to assess the objectivity of appraisal findings. 1 (Red) review of checks Externally-financed projects are subject to internal scrutiny by the relevant IFI/donor, appraisal. but are not independently reviewed by government. 4. Project Fixed budget calendar with The budget calendar is fixed in law and gives adequate time for central government 4 (Green/ selection and sufficient time for LMs to budget entities to prepare their capital budgets (31st May, when budget circular is issued, Yellow) 39 Stage Indicators and Dimensions Findings Rating budgeting. prepare capital budgets to 1st July). Budget preparation begins with a strategic phase leading to the issuance of ceilings by the MoF (30th April). Adherence to the calendar has been poor in recent years, essentially for political and not technical reasons. Comprehensive guidance and Guidance on preparing capital requests is included in the budget circular. Central 5 (Green) capital spending ceilings government budget entities are issued with separate ceilings for capital and current expenditure. Projects developed before Domestically funded projects are sometimes poorly prepared prior to submission, and 3 (Yellow) submission to MOF in budget serious preparation only begins once budget funding becomes more certain. Donor- financed projects are in a better state of development. Integration of capital and Kyrgyz Republic’s MTBF process favours integrated budgeting. However, this is not 1 (Red) current spending implemented. Expenditure on maintaining the existing capital stock is not always prioritized over new infrastructure spending and the operating and maintenance costs of completed projects are rarely taken into account explicitly during budgeting. Well-structured budget In theory, the design of Kyrgyz Republic’s MTBF process should be supportive of multi- 2 (Yellow/ preparation to consider the year budgeting for projects. In practice, ongoing domestic projects seem to have to Red) multi-year nature of compete afresh each year for continuity of funding. Often new projects seem to be infrastructure projects favoured over ongoing projects and implementation delays appear common. Budgeting for foreign-financed projects runs more smoothly. MOF checking of projects There is adequate time in the budget calendar (1st July-1st September) for the MoF to 2 (Yellow/Red) exercise an active gate-keeping role before the government submits its budget to Parliament. This it done as far as verifying technical aspects of projects and required approvals. However, it does not have the power to check projects ‘parachuted’ into the budget during parliamentary proceedings. The budget includes a ceiling for domestically funded capital projects but not a list of projects allowing for a major debate three months after budget approval as to which projects should be included. Many projects receive cuts in funding. Legislature has sufficient time to By law, government submits its budget to Parliament by 1st September for approval 3 (Yellow) consider projects before 31st December, leaving more than adequate time for parliamentary scrutiny. However, approval is only of the global amount of financing. In March/April parliament debates the proposed list of domestically financed projects with significant changes in recent years. The role of Parliament in micro-planning the domestic investment budget is excessive. Legislature passes budget in The budget was passed late in few years during the last five year period, reflecting the 3 (Yellow) timely manner recent political changes in Kyrgyz Republic and the nature of the coalition government. 5. Project Published guidelines There are no published project implementation guidelines for the whole of government. 1 (Red) implementation Detailed implementation plan 40 Stage Indicators and Dimensions Findings Rating with clear accountabilities Implementation planning is not systematic and is made difficult for multi-year projects 2 (Yellow/ Red) by funding shortfalls. Likewise, it is difficult to assign and enforce accountabilities for project delivery when managers face unpredictable financing (at least for domestic projects). Open competition for procurement Open competition is the default option in the procurement law but is only used for about 2 (Red/ Yellow) 20 percent of contracts over the threshold. Legally defined domestic preferences are an impediment to obtaining the full benefits of international competition. Effective procurement complaints mechanism There is no complaints procedure. 1 (Red) Commitment controls There is limited computerised commitment control. Most budget units maintain manual 2 (Red/ Yellow) records. The move towards a modernised Information System has been slow, with an interim solution at present enabling large transactions to be sent on line. Predictability of funding Project managers depend on monthly cash allocations. Most donor financed projects 2 (Red/Yellow) have “protected� status, with predictability thereby improved. Cash rationing is not uncommon leading to late payment of some invoices (and possibly higher costs overall). Regular progress reporting Reporting on progress is mainly financial reporting. Financial reporting is limited to in- 2 (Red/Yellow) year disbursements, and changes in total estimated project cost and estimated completion date are not systematically reported. PIP collects physical monitoring data from PIUs, and together with financial data uses this for making recommendations during quarterly project review meetings. Active monitoring of progress Monitoring is largely passive and not well differentiated from financial control and 2 (Yellow/ Red) inspection functions. Systems for early recognition of problems and their solutions are not in place. Sound internal control and internal audit Commitment controls are weak though payment controls are strong. Internal audit is 2 (Yellow/Red) not carried out. Formal project completion Formal procedures exist for sign-off on project completion – certification of works and 3 (Yellow) use-permits - and these are followed. These fall short of an assessment of implementation performance. 6. Adjustment Constant project adjustments Narrowly defined budget appropriations, tight virement rules and relatively inflexible 2 (Yellow/ Red) for changes in financial controls mean that it is difficult to ‘fine-tune’ expenditure plans in-year. In project practice, changes can only be implemented when supplementary budgets are voted. circumstances. Explanation required for There is no formal process for explaining variances in project disbursements compared 1 (Red) variances from budget and plan to plan or budget. 41 Stage Indicators and Dimensions Findings Rating Mechanisms for project Projects tend to be tightly circumscribed and adjustment by project managers is not adjustment easy. 2 (Yellow/ Red) Periodic review of costs Except for externally-funded projects, total project costs for multi-year projects are not compared to benefits for major closely monitored. No cost-benefit analysis is done anyway for entirely domestically 2 (Yellow/ projects funded projects. There is no formal process for reviewing forecast project performance Red) in the light of significantly increased costs or decreased benefits. Mechanism to stop project There is no institutionalized mechanism for stopping poorly performing or irrelevant projects. In practice, there have been periodic sweeps of the project portfolio to identify 3 (Yellow) and cancel stalled projects by decree. This leads to much delay in taking action. Any donor projects stopped have been as a result of donor integrity agencies’ recommendations. 7. Facility Effective asset handover There is an effective handover process. 5 (Green) operation. Assets fit for purpose Assets are verified as fit for purpose on handover and no concerns about the quality of 5 (Green) contracted works were expressed. Asset registers Asset registers and asset management systems are poorly developed. 2 (Yellow/ Red) Sufficient O&M funding No evidence of shortages of funding for operating expenses. Underfunding of 3 (Yellow) maintenance for existing and new assets remains an issue. The lack of linkage of the PIP to O&M within the MTBF is a concern. 8. Ex post Basic comparison of project No formal assessments of project implementation performance at the central level. 1 (Red) assessment costs, timelines, and deliverables (evaluation). against budgets and plans Formal arrangements for project No formal arrangements for ex post evaluation. No dedicated evaluation units. evaluation against plan and 1 (Red) appraisal Effectiveness or Value for Money Court of Accounts has developed some capacity in performance auditing and has audits undertaken project audits. Likely to become increasingly important as capacities 2 (Yellow/ Red) develop. Proportionality of evaluation. No ex post evaluation, even for donors 1 (Red) Response to evaluation findings. There are no arrangements in place to ensure evaluation findings and recommendations 1 (Red) are acted upon 42 References Rajaram, A., T. M. Le, N. Biletska and J. Brumby (2010). “A Diagnostic Framework for Assessing Public Investment Management�, Policy Research Working Paper 5397, The World Bank 43