85009 This report was prepared by a core team comprised of Marcel Ionescu- Heroiu, Sebastian Burduja, Ha Thi Thu Vu, Graham Glenday, Victor Giosan, Manuela Moț, Dumitru Sandu, Elena Iorga, Nicolae Țarălungă, Adina Vințan, Nolin Huddleston, Paula Tufiș, and Florian Gaman. In- depth case study analyses were prepared by the European Policies Research Centre (University of Strathclyde, Glasgow, UK). The report benefited from thoughtful comments by peer reviewers Lewis Raymond Hawke, Cătălin Păuna, and Shilpa B. Pradhan. The team would like to thank Elisabeth Huybens and Elisabetta Capannelli for the advice, support, and guidance provided throughout the development of this report. The team would also like to thank the counterparts in the Ministry of Regional Development and Public Administration and in the Regional Development Agencies (particularly Gabriel Friptu, Bogdan Țigău, Laurențiu Caprian, and Ionuț Trincă) for the timely feedback, the support offered in the elaboration of this study, and the excellent collaboration throughout. Special thanks also go to the 150+ interviewed beneficiaries of the Regional Operational Programme for volunteering their time and for sharing highly useful suggestions. The findings, interpretations, and conclusions expressed in this report do not necessarily reflect the views and position of the Executive Directors of the World Bank, the European Union, or the Government of Romania. February 20, 2014 ii Table of Contents Executive Summary .............................................................................................................................................. ix I. Introduction ........................................................................................................................................................ 1 Objective and Scope .............................................................................................................................................1 Methodology.........................................................................................................................................................3 The Public Investment Management (PIM) Framework ...................................................................................4 Primary and Secondary Sources .......................................................................................................................9 Audiences ............................................................................................................................................................10 Structure of the Report .......................................................................................................................................11 II. Background on Romania’s ROP ........................................................................................................................ 13 A. ROP 2007-2013 ...............................................................................................................................................13 B. ROP 2014-2020 ...............................................................................................................................................17 III. Review of ROP Project Selection Models (2007 – 2013) .................................................................................. 23 Analysis of the 2007 – 2013 ROP Project Selection Models ...............................................................................25 Investment Guidance, Project Development & Preliminary Screening ..............................................................29 Investment Guidance ......................................................................................................................................29 Project Development ......................................................................................................................................31 Preliminary Screening .....................................................................................................................................31 Formal Project Appraisal .....................................................................................................................................32 Independent Review of Appraisal .......................................................................................................................34 Evaluation Procedures ....................................................................................................................................35 Evaluators .......................................................................................................................................................35 Inputs from application ..................................................................................................................................35 Evaluation Criteria ..........................................................................................................................................36 Project Selection, Detailed Technical Design, and Budgeting .............................................................................38 Project Selection .............................................................................................................................................38 Detailed Technical Design ...............................................................................................................................39 Budgeting .......................................................................................................................................................40 IV. ROP Practices in Select EU Countries .............................................................................................................. 41 Synthesis of Case Studies ....................................................................................................................................42 Project Size and Relevance to PIM ..................................................................................................................42 Evolution not Revolution.................................................................................................................................43 Motivations behind State-Budget-Funded PIM Projects vs. EU-Funded Projects ...........................................43 Short-Term Measurement of Success .............................................................................................................43 Initiative Overload? ........................................................................................................................................44 Top-Down, Bottom-Up....................................................................................................................................45 Simplifying Appraisal and Selection Techniques .............................................................................................46 Recommendations on EU Good Practices Relevant for Romania .......................................................................48 Options for Building Capacity .........................................................................................................................51 Service Contract Projects ................................................................................................................................52 V. Basic principles for an efficient project selection model .................................................................................. 55 Balancing impact and absorption .......................................................................................................................55 Defining impact ...................................................................................................................................................57 Understanding the entire complexity of a project selection model ...................................................................61 VI. Setting the foundation for improved project selection models in Romania .................................................... 63 Strategic Priorities for Romania’s Economic Development ................................................................................63 The World Bank’s Recommendations on Nurturing Romania’s Competitive Cities ........................................64 iii The Europe 2020 Strategy: Priorities for Romania’s Development.................................................................69 National-Level Priorities for Romania’s Development ....................................................................................71 Adjusting selection models according to project size .........................................................................................73 Tailoring projects to regional needs ...................................................................................................................78 VII. Proposals for Project Selection Models ......................................................................................................... 85 ROP 2014-2020 Project Selection Models: Comparative Scenario Assessment ..................................................85 Scenario 1- “Status Quo+”...................................................................................................................................88 Scenario 2 - “Semi-competitive” .........................................................................................................................90 Scenario 3 - “Competitive” ..................................................................................................................................95 VIII. Improving the Current Evaluation and Selection System ............................................................................ 101 Recommended adjustments for all proposed selection models ......................................................................101 Strategic assessment of Integrated Development Plans ..............................................................................101 Introduction of Applications for Pre-Qualification (APQ) .............................................................................103 Elimination of Cost-Benefit Analysis for Small and Medium-sized Projects..................................................105 Introduce, promote, and build capacity for the adoption of the FIDIC contract forms ................................109 Evaluation recommendations .......................................................................................................................111 Proposed Improvements to the ROP 2007-2013 Evaluation and Selection System .........................................115 Administrative evaluation grid .....................................................................................................................116 Eligibility evaluation grid ..............................................................................................................................116 Technical and Financial Evaluation Grids .....................................................................................................117 Proposals for new evaluation and selection grids ............................................................................................134 IX. Identifying Strategic Regional Projects (Type 1) ............................................................................................ 135 Why Regional Strategic Projects are important ................................................................................................135 How to define regional priorities ......................................................................................................................136 Addressing some of the shortcomings of large projects...................................................................................138 Risk of slowing down absorption ..................................................................................................................139 Risk of wasting resources .............................................................................................................................140 Risk of lacking the proper institutional frameworks and adequate capacity ...............................................141 X. Pre-allocating Funds for County Residences and County Councils (Type 2) .................................................... 143 Why pre-allocation for county residences and county councils is important ..................................................143 How to pre-allocate funds at the sub-national level ........................................................................................145 Strategy and planning framework in Romania for cities and counties .........................................................145 Capital Investment Planning as a tool to help formulate and implement projects ......................................148 Proposed framework for evaluating development strategies ......................................................................150 What should be the pre-allocation for each county residence and county council .........................................154 What kind of projects should receive pre-allocations under Type 2 ................................................................158 Addressing some of the shortcomings of pre-allocating funds to sub-national entities ..................................162 Risk of slowing down absorption ..................................................................................................................162 Risk of not having enough funds for 40 county residences and 41 county councils .....................................168 Risk of lack of fiscal space for further capital expenditures..........................................................................169 XI. Monitoring and Evaluation .................................................................................................................... 177 The Performance of the ROP 2007-2013 ..........................................................................................................177 Best-Practice Elements of a Functional M&E Framework ................................................................................182 Program and Project-Level M&E.......................................................................................................................183 An M&E Dashboard for ROP 2014-2020 ...........................................................................................................185 Better Performance Management for the ROP ................................................................................................186 The Performance Framework for the Regional Operating Programme........................................................187 Setting ROP Goals .........................................................................................................................................190 iv Managing for Performance across the Regional Operational Programme ..................................................193 Long Term Maturation of the ROP Capabilities ............................................................................................195 Measuring the overall performance of the ROP ...............................................................................................195 XII. Conclusions .................................................................................................................................................. 197 Bibliography ...................................................................................................................................................... 201 Annexes ............................................................................................................................................................. 207 Annex 1. The 2007-2013 ROP Allocation ..........................................................................................................208 Annex 2. 2007 – 2013 ROP Structure ................................................................................................................218 Annex 3. Scheme of ROP Project Selection Process .........................................................................................226 Annex 4. Analysis of Regional Committees for Strategic Evaluation and Correlation (RCSECs) .......................227 Annex 5. Analysis of Integrated Urban Development Plans (PIDUs) ................................................................229 Annex 6. Analysis of Regional Development Plans for the period 2014 - 2020 ................................................232 Annex 7. Criteria used by RDAs to identify regional strategic projects ............................................................234 Annex 8. Content of Feasibility and Detailed Technical Design in Romania .....................................................238 Annex 9. Interview Guide..................................................................................................................................243 Annex 10. Selected Case Studies – Project Selection Models in 8 EU Member Countries ...............................245 Germany .......................................................................................................................................................245 Poland ...........................................................................................................................................................255 Estonia ..........................................................................................................................................................265 Italy ...............................................................................................................................................................275 Slovenia ........................................................................................................................................................284 United Kingdom ............................................................................................................................................289 Ireland...........................................................................................................................................................296 Lithuania .......................................................................................................................................................299 Annex 11. Categorization of investments funded through the ROP 2007-2013 ..............................................301 Annex 12. Schemes for proposed scenarios .....................................................................................................307 Annex 13. Ministry of Finance prioritization criteria, according to GO 88/2013 ..............................................332 Annex 14. ROP 2007-2013 screening procedures for Integrated Development Plans (IDPs) ...........................338 Annex 15. ROP 2007-2013 funds accessed by public authorities by August 2013 (by axis and by county) .....341 Annex 16. The availability of development strategies at the county residence and county council level .......344 Annex 17. Capital Investment Planning/Case study: Nis, Serbia ......................................................................362 Annex 18. Draft Proposal for ROP 2014-2020 Axes ..........................................................................................365 Annex 19. ROP 2007-2013: Synthesis of types of investments and eligible beneficiaries ................................369 Annex 20. Financial Situation for County Residences and County Councils .....................................................370 Annex 21. Economic and financial appraisal criteria ........................................................................................381 Annex 22. Limitations to the use of the Cost-Benefit Analysis .........................................................................385 Annex 23. ROP Performance Crosswalk based on US example ........................................................................389 Annex 24. The LHDI as an Overall Measure of Public Investments for the ROP’s M&E ...................................397 Steps in computing the LHDI .........................................................................................................................399 Computing algorithm....................................................................................................................................400 Validating the LHDI .......................................................................................................................................404 Annex 24A. Design and use of the LHDI in applied research in Romania .....................................................407 Annex 24B. Comparability of LHDI across time ............................................................................................413 Annex 24C. LHDI at local and county levels, 2002, 2011 and 2012 ..............................................................419 Annex 25. The new European Commission requirements on Project Selection models ..................................422 The new approach of the EC on programming, monitoring, and evaluation ...............................................422 Annex 26. Proposed evaluation and selection grids for the ROP 2014-2020 ...................................................426 Grids proposed for the Status Quo+ .............................................................................................................426 Grids proposed for the Semi-Competitive and Competitive Scenarios ........................................................485 v List of Figures Figure 1. A delicate balancing act ..................................................................................................................................2 Figure 2. Eight must-have features in public investment management ........................................................................4 Figure 3. Submitted ROP applications vs. allocation (100%) .......................................................................................15 Figure 4. Submission of ROP applications per development region (compared to allocation of 100%) .....................16 Figure 5. Relative performance of EU structural programs in Romania (as of November 30, 2013) ..........................16 Figure 6. The current structure for 2014-2020 programming of EU structural funds in Romania ..............................19 Figure 7. Duration of Selection Process by Phase, 2008 – 2011 (in months) ..............................................................24 Figure 8. General scheme of the strategic framework for 2007 – 2013 ROP projects ................................................30 Figure 9. Percentage of EU funds paid by the European Commission, in June 2013 ..................................................55 Figure 10. ROP: Balancing Absorption and Impact ......................................................................................................56 Figure 11. The Local Human Development Index at the locality level, for 2002 .........................................................59 Figure 12. The Local Human Development Index at the locality level, for 2011 .........................................................60 Figure 13. Investments in road infrastructure do not seem to have a real developmental impact ............................60 Figure 14. A framework for devising an optimal monitoring and evaluation system for project selection ................62 Figure 15. Typology of optimal interventions (Competitive Cities, World Bank) .........................................................66 Figure 16. Tailoring selection models based on the project type ................................................................................74 Figure 17. Optimal selection models should reflect the degree of project complexity ..............................................77 Figure 18. In Spain regions are categorized in a more nuanced manner ....................................................................78 Figure 19. The Regions in the East and the South of the country have lower basic infrastructure endowment than regions in the West .............................................................................................................................................79 Figure 20. Quality of Life investments show a concentration of investments in the most dynamic cities regionally .81 Figure 21. Less developed regions have benefited more from investments in roads than more developed regions 82 Figure 22. Life standard investments seem to follow the expected pattern (higher in lagging regions) ....................83 Figure 23. A framework for defining an optimal selection model for Romania’ s ROP ................................................85 Figure 24. The three scenarios considered propose a range of different adjustments to the status quo model .......86 Figure 25. The WHAT, HOW, and WHO of project selection .......................................................................................87 Figure 26. The introduction of an APQ phase appears to be a win-win for all sides involved ..................................105 Figure 27. Selecting the right type of contract (i.e., FIDIC color) is relatively simple ................................................110 Figure 28. ROP 2007-2013 Financing Requests and Commitments by Axis, by then end of 2012 ............................139 Figure 29. Total ROP 2007-2013 funds accessed by public authorities by August 2013 ...........................................144 Figure 30. Putting the CIP within the local context ...................................................................................................148 Figure 31. A 12-step process to implementing the CIP .............................................................................................149 Figure 32. Option 1 for evaluating city/county development strategies...................................................................151 Figure 33. Total budget figures for county councils and county residences in 2012 ................................................157 Figure 34. Degree of dependence on earmarked grants for county councils and county residences, in 2012.........157 Figure 35. Number of projects submitted under Axis 1 by individual cities ..............................................................165 Figure 36. Value of projects submitted for funding under Axis 1 by individual cities ...............................................165 Figure 37. Total budgets for sub-national entities, in 2012 .......................................................................................166 Figure 38 How fast (in days) ROP projects move through the system ......................................................................167 Figure 39. Technical and Financial Evaluation Scores for ROP 2007-2013 projects ..................................................168 Figure 40. An outlook on how County Residences perform over time on the Glenday-Giosan indicators ...............173 Figure 41. An outlook of how County Councils perform over time on the Glenday-Giosan indicators ....................173 Figure 42. An effective M&E system is like a good dashboard: simple and easy to read ..........................................183 Figure 43. A possible dashboard for the ROP's M&E system ....................................................................................186 Figure 44. Two Phases for ROP Performance Framework .........................................................................................187 Figure 45. Timeline for Performance Framework......................................................................................................190 Figure 46. Strategic Decomposition of ROP Goals .....................................................................................................190 Figure 47. The Programming stages the ROP has to go through ...............................................................................192 Figure 48. Maturity Model View of RDA Operations .................................................................................................193 Figure 49. Simplified comparative assessment of proposed Project Selection Models ............................................198 vi Figure 50. The Mechanics of the LHDI .......................................................................................................................397 Figure 51. Human development level of the counties, LHDI 2012 ............................................................................406 Figure 52. Human development level of the counties, LHDI 2002 ............................................................................406 List of Tables Table 1. Key components of prefeasibility and feasibility studies .................................................................................7 Table 2. Public investment in Romania, 2007 – 2012* ................................................................................................13 Table 3. The allocation (% from the European Structural and Investment Funds - ESIF) ............................................20 Table 4.Indicative proposals by type of investment and institutional structure .........................................................20 Table 5. Duration of selection process, 2011 ..............................................................................................................24 Table 6. Overview of the 2007 – 2013 ROP Selection Models (except Axis 6 – Technical Assistance) .......................26 Table 7. Diagnosis of the 2007 – 2013 ROP selection procedures according to the PIM framework .........................27 Table 8. Comparison of appraisal components between Romanian and international standards .............................33 Table 9: Overview of Selected Case Studies ................................................................................................................41 Table 10. Absorption of EU Funds by Operational Programme*, by July 2013 ...........................................................57 Table 11. Typology of ROP interventions based on the World Bank’s Competitive Cities framework ........................68 Table 12. Europe 2020 targets and Romania's performance to date ..........................................................................70 Table 13. ROP projects contracted as of August 2013.................................................................................................75 Table 14. Large projects contracted, by August 2013, under the ROP 2007-2013 ......................................................75 Table 15. Contracted ROP projects by axis (by August 2013) ......................................................................................76 Table 16. Minimum and maximum funding caps for ROP 2007-2013 projects ...........................................................77 Table 17. Proposed selection procedures for Scenario I according to the Public Investment Management Framework ..................................................................................................................................................88 Table 18. Improving IDP evaluation criteria ..............................................................................................................102 Table 19. A comparison of CBA and CEA highlights the latter’s superior fit with the current ROP ...........................107 Table 20. Performance of different OPs at the end of 2012 .....................................................................................140 Table 21. Prudent Fiscal Space for Capital Expenditures over 2014-2020, ...............................................................155 Table 22. Share of Capital Expenses that go to EU investments ...............................................................................157 Table 23. Investment priorities for county residences and county councils .............................................................160 Table 24. An Outlook of the Glenday and the Giosan Indicators, by Region .............................................................170 Table 25. EXAMPLE Performance Framework Indicator by Axis ...............................................................................189 Table 26. Europe 2020 Romania Development Targets ............................................................................................191 Table 27. ROP Planned Allocation by PAs and Years (in EUR) ...................................................................................209 Table 28. Predictors of Human Local Development in Romania, 2012 .....................................................................405 Table 29 .Local human development index by counties and residence, 2002 and 2011 ..........................................419 Table 30. Predictors of LHDI 2012 by rural and urban residence ..............................................................................420 Table 31. Leading and lagging localities in Romania by LHDI values .........................................................................421 vii List of Abbreviations CBA Cost-Benefit Analysis CCRD Consultative Committee for Regional Development CEA Cost-Effectiveness Analysis CF Cohesion Fund MA Central Managing Authority RCSEC Regional Committee for Strategic Evaluation and Correlation CSF Common Strategic Framework DR Development Region DTD Detailed Technical Design EAFRD European Agriculture Fund for Rural Development EMFF European Maritime and Fisheries Fund ERDF European Regional Development Fund ESF European Social Fund FS Feasibility Study IB Intermediate Body ICPA Inter-institutional Committee for Partnership Agreement IDA Inter-municipal Development Association KAI Key Area of Intervention LGU Local Government Unit MA Managing Authority MEF Ministry for EU Funds MRDPA Ministry of Regional Development and Public Administration MRDT Ministry of Regional Development and Tourism NARMPP National Authority for Regulating and Monitoring Public Procurement NGO Non-Governmental Organization OP Operational Program PA Priority Axis PFS Prefeasibility Study PIDU Urban Integrated Development Plan PSM Project Selection Model RCP Regional Committee for Planning RD Development Region RDA Regional Development Agency RDP Regional Development Plan RDS Regional Development Strategies ROP Regional Operational Programme viii Executive Summary Romania has the opportunity to benefit from significant EU structural assistance through the Regional Operational Programme (ROP). In preparation for the 2014 – 2020 ROP, a key task at hand is to enhance not only the absorption rate, but also the impact of ROP-funded investments. One critical component in this process is improved project selection models to help boost the quality of ROP investments, have a stronger linkage to strategic development priorities, and focus on getting the highest “value for money” projects. The proposed selection models for the ROP 2014 -2020 should build upon the strengths and correct the weaknesses of the 2007-2013 framework, as well as incorporate best practices from other EU countries, appropriately tailored to the Romanian context and capacity. Background The Regional Operational Programme (ROP) is one of seven operational programs (OPs) financed through EU funds during the 2007-2013 programming period. The EU dedicated EUR 3.7 billion to the ROP to support Romania’s regional development in the post-accession period. This amounts to 19% of the total structural funds available through the National Strategic Reference Framework (NSRF) between 2007 and 2013, making the ROP the third largest program after transportation (23.7%) and environment (23.5%). In 2011, projects financed under the ROP accounted for nearly 27% of projects financed by external grants and more than 12% of total public investment. The total allocation of the 2007-2013 ROP is more than EUR 4.5 billion, of which the EU contributes nearly 82%. It is important to note from the start that the ROP has been a top performer for Romania in terms of absorption rates, despite a complex and heterogeneous portfolio of projects. Because of its broad focus on regional development issues, the ROP addresses a wide-ranging set of needs at the level of each region, from urban development to transport infrastructure, social infrastructure, business development, and tourism. In this context, the ROP has achieved the best absorption rates out of all structural programs (as of November 30, 2013), particularly when it comes to payments from the EC to Romania, which stand at 44% of the total available ROP allocation. Relative performance of EU structural programs in Romania (as of November 30, 2013) 120% 118% 110% 103% 100% 85% 82% 80% 78% 80% 60% 48% 45% 46% 44% 40% 40% 33% 30% 27% 22% 22% 21% 19% 18% 18% 20% 0% ROP Environment Transport Competitiveness Administrative Human resources Technical capacity assistance % contracted of total ROP funds available % paid to beneficiaries % paid by EC to Romania Source: Ministry of European Funds (www.fonduri-ue.ro) ix The ROP has also performed relatively well when compared to similar Operational Programmes from other Member Countries. As the table below indicates, although absorption rates were relatively lower compared to the other OPs at the end of 2013, the Romanian ROP had one of the highest commitment rates. This indicates that the ROP system functions relatively well up to the project implementation stage, where most delays in absorption are caused by systemic issues (e.g., an inefficient public procurement system, poor quality technical documentation, property rights issues, etc.). Performance of different OPs at the end of 2012 Commitments Absorption by Operational Programme by end of 2012 end of 2012 ROMANIA – Regional Operational Programme 96% 34.5% GERMANY – Nordrhein-Westfalen ROP 95.1% 47.3% ESTONIA – OP for the Development of the Living Environment 87.7% 54.3% SLOVENIA – Strengthening the Regional Development Potential OP 86.6% 69.5% UK – West Wales and the Valleys Convergence Programme 103% 34% POLAND – Slaskie Regional Operational Programme 88.4% 48.5% That said, the commitment and absorption rates should be neither the only, nor the most important indicators for the ROP’s success. A high rate of absorption may be a sign that projects have been chosen for financing simply because they were ready to be implemented, rather than for their potential contribution to development objectives. In Romania and more broadly across the EU, policymakers and key stakeholders should pay increased attention to the impact of EU-funded investments in terms of achieving social and economic goals – which, after all, is the ultimate aim of such interventions. Lessons from other Member States One of the first steps in elaborating an improved selection model is to assess the approaches of other Member States. This way, best practices can be collected and improved upon to generate even better outcomes for the ROP. In the full report, project selection models from 8 Member Countries (Germany, Poland, Estonia, Italy, Slovenia, the UK, Ireland, and Lithuania) are analyzed to identify potential suggestions for enhancing the ROP selection model for the 2014-2020 programming period. The operational programmes analyzed were chosen based on their similarities to the Romanian ROP. A number of key lessons can be drawn from the analysis of these eight case studies: 1. Each Member State used unique approaches for project selection, with some focusing on the pre-selection of a number of large strategic projects (e.g., Lithuania), some focusing primarily on competitive calls (e.g., Ireland, Estonia), and yet others, particularly the larger countries (e.g., Germany, Italy, Poland), deploying a mix of methods. 2. The evaluation and selection systems used by these countries are by-and-large simpler than the system used under the ROP 2007-2013. For example, most countries did not use the Cost- Benefit Analysis (CBA) for evaluating projects. Germany was one of the exceptions, and it only used CBA for large strategic projects. Similarly, the ROP’s evaluation and selection grids appear much more granular and detailed than comparable frameworks in other Member States. 3. For the most part, the case studies analyzed revealed a keen focus on impact and also used weights for individual criteria to differentiate between more important and less important evaluation factors. x Recommendations for improving the ROP Project Selection Model The improvement of the ROP 2007-2013 Project Selection Model, through a stronger focus on impact, can be achieved in a number of ways, keeping the following recommendations in mind: 1. Do not sacrifice ABSORPTION for the sake of IMPACT. Having projects that generate development impact is the main underlying aim of the ROP. Projects that lack actual impact are equivalent to a waste of resources and may in fact lead to high operation and maintenance costs that will pose a burden on limited local resources. However, having a few impactful projects and low absorption rates is not an optimal situation either. The ROP should balance absorption and impact in a legitimate and functional selection system Absorption Impact • Appropriate focus for 2007-2013 to reduce • Higher priority for 2014-2020, after the risk of funds decommitment 2007-2013 learning curve • In hindsight, optimal decision to prioritize • Impact and value for money as key quantity and speed objectives In addition to absorption and impact, an optimal selection framework also depends on: • Legitimacy: How credible is the system? • Capacity: How functional is the system? Romania showcases the poorest absorption rates of any Member State, as an average across all its Operational Programmes. In fact, Romania is some distance away (14 percentage points) from the second worst performer – Bulgaria. This is obviously not an ideal situation. Average absorption rates by Member States, in June 2013 Source: http://ec.europa.eu/regional_policy/thefunds/funding/index_en.cfm xi Low absorption means that substantial and critical resources are not used in a timely, effective manner, which also implies that resources are wasted. In some ways, this may be a worse situation than having an insufficient focus on impact. For example, even if the funds are not spent on the newest, most impactful projects, but on the rehabilitation of existing infrastructure (to a large extent, this is what ROP funds have been used for and will continue to finance in 2014-2020), this may decrease operation and maintenance costs for current infrastructure. Thus, even though the funds may not generate a significant development impact, they will help reduce costs and improve the quality of life for people – an important achievement in of itself. 2. There should be a purposeful focus on a number of strategic projects, with an appropriate allocation for such types of investments. Currently, the ROP primarily focuses on small and medium-sized projects, with a total share of commitments for large projects of less than 4% of available funds. Of course, the fact that a project is large does not make it necessarily impactful. Also, there are very small projects that can have very large impact (e.g., well-targeted information campaigns). However, when it comes to infrastructure development, it is usually the larger projects that generate the most significant outcomes. The ROP 2007-2013 primarily focused on small and medium-sized projects (situation as of August 2013) Number of Average value of Total value of % of total projects project (in RON) projects (in RON) contracted projects Small Projects (<1.5 mln RON) 1,952 841,553 1,642,712,364 5.74% Medium-Sized Projects (1.5 - 100 mln RON) 1,554 16,750,343 32,497,284 90.51% Lower-middle-sized projects 1,279 8,017,773 10,254,732,124 35.8% Upper-middle-sized projects 275 56,976,793 15,668,618,334,76 54.71% Large Projects (>100 mln RON) 9 119,471,996 1,075,247,961 3.75% TOTAL 3,515 137,059,204 27,915,616,807 100% Data source: MRDPA The longer preparation time that large projects entail can be countered by simplified application and selection procedures for small projects. For example, the ROP 2007-2013 processed over 1,950 small projects, which underwent similar evaluation and selection procedures as larger projects, but represented only 5.74% of allocated funds. Simply put, evaluation and selection procedures should reflect the value, complexity, and expected risks/costs/benefits of a particular investment. This should ensure that the overall evaluation and selection system remains functional and has enough capacity to accommodate peak-time demand. 3. Key strategic projects should be drawn from integrated development strategies. Such strategies should represent the overall developmental vision for the area covered and should not be drafted for the sole purpose of attracting EU funds, but present needs, priorities, and projects regardless of the ultimate financing source. New strategies should only be drafted when addressing an area that is not already covered by an existing strategy (e.g., new metropolitan areas) and reflect, if available, the development vision set in the strategies of constituent administrative units. Where integrated development strategies already exist (as is the case of Growth Poles and Urban Development Poles), they should only be updated. xii To ensure consistency in approaches across development strategies, a number of key items should be required (e.g., to have a clear vision and objectives, a list of projects to help achieve those objectives, and corresponding funding sources). Also, to ensure a proper prioritization of projects, tools such as Capital Investment Planning (CIP) could be used to ensure that limited budget funds are targeted at key strategic projects and are not spent in an opportunistic manner, while ongoing operations and maintenance (O&M) costs remain reasonable. Sub-national entities have limited budgets for capital expenditures and they should spend these budgets wisely Integrated approaches also entail that EU and state-budget-funded projects are not treated differently or separately, but that they are part of a single, coherent development vision and help achieve the same overall goals. This means that similar evaluation and selection criteria and procedures should apply to such investments, regardless of the funding source (i.e., EU funds or state-budget financing). 4. The ROP 2014-2020 programmatic document can play a strategic role in steering beneficiaries toward a set of core types of investments. While local and county authorities define priorities on their own, as they know well the needs of the communities they represent, the ROP can help guide them toward the type of investments that are likely to have the highest impact. The Competitive Cities report makes a number of recommendations in this respect, taking Romania’s current development level into consideration. It is also important to keep in mind the main policy aims and thematic objectives set in the Europe 2020 Strategy. This is a key document that sets the priorities for the EU’s development as a whole, but obviously the needs of individual Member States may vary considerably. As such, priorities should be adapted to the Romanian context, with an enhanced focus on those objectives that are likely to be more relevant for Romania. xiii Key investment priorities for Romania (as identified in the Competitive Cities report) Shorten the distance to large markets globally by improving infrastructure and International encouraging cross-border flows of people, capital, and ideas Intervention level Improve connections between leading and lagging areas within Romania to enable Regional efficient concentration of resources and spillover effects Improve connective Foster good institutions infrastructure between cities (basic services infrastructure, & surrounding areas Local education, health, land markets, etc.) Promote quality-of-life investments Design and implement targeted measures for marginalized groups Lagging Leading Area Area Level of economic development 5. Principles of fiscal prudence should be applied for the ROP 2014-2020, ensuring that beneficiaries do not overburden their financial capacity to support projects. For example, beneficiaries should ensure that, once they make capital investments, they also have the necessary budgetary resources to cover the associated operation and maintenance costs. The fiscal health of individual beneficiaries can be assessed with the help of the Glenday Indicator and/or the Giosan Indicator. These two indicators are explained in more detail in the full report. For large strategic projects, which also require substantial co-financing from the State budget, it is important to have the Ministry of Public Finance (MoPF) involved to ensure that funds are spent in an efficient and sustainable manner. The MoPF can ensure that basic fiscal prudence principles are followed and it could also set up a system for prioritizing and coordinating the development of large projects at the national level. xiv Some sub-national entities have spent more on capital investments than is sustainable in the long term, as indicated by the Glenday and Giosan Indicators Glenday Indicator for county residences Glenday Indicator for county councils Giosan Indicator for county residences Giosan Indicator for county councils 6. An optimal Project Selection Model should also have a good Monitoring and Evaluation (M&E) system to facilitate continuous improvement. A good M&E system serves several purposes. For xv one, it helps track the performance of a specific project and the extent to which this project has reached its output indicators – this is a mandatory task that the European Commission requires of all Operational Programmes. However, in addition to what the EC asks for, a good M&E system can help evaluate if projects, and the overall program that finances them, have achieved planned outcomes and if they have had an actual development impact. In particular, it can provide information on whether the project selection model is appropriate and has delivered on its intended goals, making sure that the ROP finances investments that have the biggest impact – the same way a farmer selectively breeds crops using the strongest seeds from the last harvest. A comprehensive M&E framework can assess not only the degree to which impactful projects are selected and prioritized, but also whether the programme delivers optimal absorption rates, a functional system with appropriate capacity to process applications, and a high level of credibility among applicants and beneficiaries. A proper M&E system with optimal feedback loops Source: European Commission. 2006. “Indicative Guidelines on Evaluation Method” 7. While keeping in mind the need to ensure a stable and predictable playing field for all applicants and beneficiaries, the ROP 2014-2020 should feature flexible approaches, as needed. For example, even if most of the available funds will be pre-allocated (for strategic projects) or allocated using a First-In First-Out (FIFO) approach, the programme could adopt a competitive selection model for one priority axis or one key area of intervention (e.g., business infrastructure). For example, many Member Countries use competitive calls for private beneficiaries (e.g., SMEs) or for the development of business infrastructure. Another option is to enable the ROP’s shift to a competitive approach after a mid-term claw-back (e.g., if some beneficiaries have not used pre-allocated funds in due time). xvi Proposals for Project Selection Models With the above recommendations as key guidelines, three project selection models are proposed for the ROP 2014-2020. The development of the scenarios followed a framework for assessing various options based on six fundamental requirements (efficiency, effectiveness, clarity, fairness, transparency, and feasibility) and four key objectives (absorption, impact, legitimacy, and capacity). A framework for defining an optimal selection model for Romania’s ROP OPTIMAL PROJECT SELECTION MODEL Evaluation  Selection  Contracting OBJECTIVES 1 2 3 4 ABSORPTION IMPACT LEGITIMACY CAPACITY CORE Does the model Does the model Is the model Is the model feasible ensure a high optimize for impact, legitimate for all given current absorption rate? value for money? stakeholders? capacity? A EFFICIENCY: Quick, smooth process that ensures an optimal absorption rate B EFFECTIVENESS: Selected projects optimize for impact and value for money REQUIREMENTS FUNDAMENTAL C CLARITY: Clear selection procedures and requirements minimize the room for interpretation D FAIRNESS: Fair selection criteria apply to all submissions E TRANSPARENCY: Transparent procedures enable full visibility and accountability of all parties F FEASIBILITY: The institutional system is able to properly implement the model The figure above conveys precisely this message, making the case that an optimal project selection model for the next ROP will need to accomplish all four targets. Put differently, a model may be great at promoting quick absorption of EU funds, but it may fail to deliver value for money and resources could be wasted on low-impact interventions. Similarly, if a model is not legitimate (i.e., with clear and transparent evaluation criteria and fair selection procedures), not enough ROP applicants will demonstrate interest in the program, so there will be an inherent limitation on what the ROP can accomplish. Last but not least, any proposed change needs to be more than just a theoretical framework that promises to deliver great results in the abstract; indeed, selection models have to work in practice, keeping in mind the system’s current and expected capacity constraints. Three scenarios are worth considering in terms of the overall project selection model:  Scenario 1 (“Status Quo+”): The main characteristics of the system remain unchanged, with a few tweaks (e.g., excluding the full Cost-Benefit Analysis from the formal assessment, making the Detailed Technical Design optional at the evaluation phase, etc.) to correct current weaknesses. This option capitalizes on the ROP system’s accumulated experience and leverages the benefits of predictability. Essentially, the two selection procedures remain the same: pre- xvii allocation for growth poles and urban development poles (with the potential inclusion of all county seats as UDPs); and first-in-first-out for all other projects.  Scenario 2 (“Semi-Competitive”): This variation applies different selection methods based on project size and beneficiary type. Project size tends to be a good proxy for both complexity and potential impact. Under this scenario, large regional projects (type 1) would enjoy dedicated funding. Medium-sized investments by counties, growth poles, and urban development poles (type 2) would be selected in two phases: (1) dedicated funding for a limited period of time; and (2) selection based on FIFO for the remaining pool of funding allocated to these types of beneficiaries. For other cities, as well as for private and non-profit applicants (type 3), the FIFO rule would apply, regardless of whether the project is small (under 1.5 million RON) or more significant (over 1.5 million RON). Ultimately, this is a “semi-competitive” model, as a “middle- of-the-road” type of option, although applications are not compared or scored against ea ch other under either FIFO or the dedicated funding model, regardless of the type of project under consideration.  Scenario 3 (“Competitive”): This scenario maintains the FIFO selection rule for a limited set of projects, namely the smallest ones under 1.5 million RON, while also setting aside dedicated funding for the largest regional projects. For counties, growth poles, and urban development poles, the funding would be allocated in two phases, much like for the previous scenario, with a key difference: the phase 2 reallocation could be done on the basis of competitive ranking for projects under each axis. The three scenarios considered feature a range of different adjustments to the status quo model The figure above summarizes the main features of the three options for each applicable type of project. As this report argues, choosing an optimal selection model is a challenging task and may involve several quintessential tradeoffs: between impact (effectiveness) and absorption (efficiency); between needed xviii changes and predictability/continuity; and between ideal outcomes and practical feasibility given capacity constraints. Importantly, the current work does not propose selection models that vary by sector. Instead, the size of a proposed intervention is the key variable that determines how a project should be evaluated, selected, and contracted. This is in line with broader thinking on integrated interventions (something the ROP is uniquely positioned to deliver on because of its focus on the broad field of regional development) and a move away from sector-specific procedures. In fact, there is value in putting forth standardized selection models and criteria that allow for easy comparisons across projects, regardless of their thematic focus. In the main report, the three scenarios presented above are discussed in more detail. The figure below gives a quick summary of the key strengths and weaknesses of these scenarios. Some of the main features of these options are discussed in greater depth in the next sections of the report. Simplified comparative assessment of proposed Project Selection Models Note: A larger black-fill in the pie-charts indicates the degree to which each individual scenario fulfils a specific criterion xix I. Introduction 1. The Romanian Ministry of Regional Development and Tourism (MRDT) (now the Ministry of Regional Development and Public Administration – MRDPA) has signed an agreement with the World Bank for five technical assistance projects. The aim of this technical assistance program is to determine how cities in Romania can become more competitive, sustainable, and inclusive, and to help the MRDPA develop its administrative capacity to better manage policy and investment work. The technical assistance program includes the following five projects, which were designed to be complementary in nature, with cross-cutting findings and recommendations: (1) Enhanced spatial planning as a precondition for urban development; (2) Upgrade growth poles strategic planning and economic impact; (3) Assessment of the communication and collaboration between the Managing Authority (MA) and Intermediate Bodies (IB) of the Regional Operational Programme (ROP) and facilitation of proactive and direct support for beneficiaries; (4) Technical assistance for the identification of project selection models; and (5) Elaboration of integration strategies for poor areas and disadvantaged communities. 2. This report focuses on the identification of optimal selection models (project #4 listed above), which is critical for improving the efficiency and impact of the Regional Operational Programme (ROP) during the 2014-2020 budgeting period. This is a very timely endeavor, as Romania is expected to finalize in early 2014 the Partnership Agreement with the European Commission (EC). By summer/fall 2014, key documents like the programmatic ROP 2014-2020 and the Framework Document for Implementation will also need to be published in their final form, including a definitive proposal for selection models. The time to act is now in order to avoid subsequent changes to the rules of the game, which has been one persistent criticism of the current ROP and of other operational programs in Romania. This principle is particularly relevant for the criteria that lie at the foundation of how projects are selected: once the first ROP calls for proposals are launched, the selection system should maintain the same core principles and criteria to ensure a fair and equal treatment of all applicants. Objective and Scope 3. More specifically, the current technical assistance seeks to support the Managing Authority of the Regional Operational Programme (MA-ROP), part of the MRDPA, with the identification and recommendation of improved, innovative selection models for the ROP. The stakes are high: selection models will guide decisions on the types of projects that will receive funding and how they will be evaluated to ensure optimal scope, objectives, and expected impact. The new selection models have to align with the strategic development priorities and indicators of the next programming period, but must also remain clear, simple, and easy to communicate to all interested stakeholders (e.g., potential beneficiaries, experts, evaluators, MA and IB staff, etc.). Above all, they must guarantee a fair and transparent evaluation and selection process, with high credibility among ROP applicants. 4. Optimal project evaluation and selection depend on balancing two priorities: absorption and impact. Regarding the former, Romania remains, as of January 2014, the weakest performer in terms of EU funds absorption, although for the ROP in particular the risk of funds decommitment remains relatively low. Nonetheless, new or updated selection models should not come at the expense of efficient spending of available funding during the next cycle (quantity of interventions), as Romania cannot afford to miss out on any non-reimbursable financing in the context of tight investment budgets and pressing development needs. At the same time, building on the experience accumulated to date, 1 the country has the opportunity to shift some of the focus toward impact (quality of interventions), which can come from (a) improved selection of impactful projects that deliver high value for money, once they are submitted for evaluation (the focus of the current work) and (b) improved programming and pipeline development to successfully take more impactful concepts from the design through the application, implementation, and post-implementation phases (a topic covered by the World Bank’s report “ROP 2.0: Facilitation of Proactive and Direct Support for Applicants and Beneficiaries of the Regional Operational Programme 2014-2020”). At the EU level, the emphasis on impact is also growing, often in conjunction with procedural simplifications that are meant to reduce the burden on beneficiaries and also increase the overall speed of accessing EU funds. In any case, the question of balancing the two desiderata, quality and quantity of EU-funded interventions, is a key, recurrent theme of the current work. Figure 1. A delicate balancing act Absorption Impact • Appropriate focus for 2007-2013 to reduce • Higher priority for 2014-2020, after the risk of funds decommitment 2007-2013 learning curve • In hindsight, optimal decision to prioritize • Impact and value for money as key quantity and speed objectives In addition to absorption and impact, an optimal selection framework also depends on: • Legitimacy: How credible is the system? • Capacity: How functional is the system? 5. This report provides an assessment of the ROP’s current selection models, reviewing strong points and areas for improvement. The bulk of the current work focuses on the status quo system and identifies those components that decision-makers should consider adjusting in the near future, as well as strengths to build upon. The goal is to provide a clearer picture of the degree to which submitted projects can contribute to the achievement of the Regional Operational Programme’s objectives. This report also draws on experiences in several other Member States, selected based on their positive overall performance and similarities to the Romanian ROP, as well as on best practices at the global level. 6. The scope of this work covers primarily the evaluation and selection stages in a project cycle, although broader implications exist in terms of project generation, implementation, and post- implementation. Indeed, there are significant synergies with the broader portfolio of projects under the World Bank Regional Development Program in Romania and findings are meant to be consistent across all analyses performed for the MRDPA. For one, the Competitive Cities report (activity #1 listed above) defines the main directions and strategic priorities of Romania’s urban and regional 2 development, which can play a key role in the formulation of criteria for selecting ROP projects that contribute to the country’s inclusive and sustainable growth. Second, the assessment of the growth poles policy (activity #2) can guide the prioritization of investments – including, for instance, in energy efficiency – in the seven major cities that are likely to continue to benefit from dedicated financing for the ROP 2014-2020. Third, the poverty mapping project (activity #5) helps identify where disadvantaged communities are, as well as optimal ways for integrating them, including through future ROP assistance. Finally, the report on the MA-IB collaboration and beneficiary support (activity #3) is arguably the closest in scope to this analysis of selection models. It provides an assessment of the ROP across the entire project cycle, evaluating how the “backend” systems function between the Managing Authority and the Intermediate Bodies and how effectively they assist applicants and beneficiaries in designing and completing ROP-financed projects. As such, the current study includes references to these complementary work streams, whenever relevant, although an integrated, comprehensive reading of the full package of analysis is recommended for having a complete picture of the program. Methodology 7. The first guiding principle of the current analysis, as for all functional reviews, including the concurrent assessment of the MA-IB collaboration and beneficiary support, is the following: first do no harm. The intention is not to provide feedback for the sake of changing the current project selection models, particularly because the ROP is still, as of January 2014, Romania’s top performing operational program in terms of absorption. As such, recommendations for adjustment are only useful insofar as they address the system’s weaknesses, while preserving its strengths. There is also an implicit cost of making potentially sweeping changes that would derail the currently functioning mechanisms and generate increased risk of funds decommitment. Put differently, a high level of predictability is valuable, in of itself, as a way to provide a stable environment with clear rules for all the actors involved in the financing and implementation of ROP projects, in a general context that has been too often characterized by repeated changes and variability. 8. At the same time, in-depth case studies of other Member States (chapter IV) provide a range of options and ideas for how the ROP evaluation and selection system can be improved. These case studies cover the whole spectrum of the evaluation and selection cycles, and present different options for a more efficient and effective management of the ROP. This comparative assessment reviews operational programmes in Germany, Poland, Estonia, Italy, Slovenia, the United Kingdom, Ireland, and Lithuania, distilling best practices for the Romanian context and drilling as much as possible into how other Member States evaluate and select projects (models, criteria, scoring grids, etc.). 9. Last but not least, in-depth discussions with representatives of the MA-ROP, the Regional Development Agencies, beneficiaries, and other stakeholders have helped fine-tune the recommendations made here. Such consultations were organized periodically and included workshops with the MRDPA and site visits to all eight development regions in Romania. The team presented preliminary proposals for improved project selection models to MA counterparts, RDAs’ technical staff, as well as key ROP beneficiaries from around the country. The feedback received has been extremely valuable and is reflected throughout this report. 3 The Public Investment Management (PIM) Framework 10. The analytical part of the report is structured around the Public Investment Management (PIM) framework developed by the World Bank.1 An efficient PIM system must have the following eight features for each stage of the project cycle (Rajaram et. al. 2010), with their corresponding elements: (1) Investment Guidance, Project Development, and Preliminary Screening: guidance on national and sectoral policy priorities, a formal process for project development, and first-level screening for strategic alignment. (2) Formal Project Appraisal: quality of proposals presented to justify new investment spending. (3) Independent Review of Appraisal: review process of project proposals. (4) Project Selection and Budgeting: decision on priority projects to be funded. (5) Project Implementation: actual construction of physical assets. (6) Project Adjustment: monitoring of project implementation and adjustments, as necessary. (7) Facility Operation: use of assets for service delivery. (8) Basic Completion Review and Evaluation: ex-post collection of data on actual total cost and timeline compared to the proposal, and selective evaluation of project results. 11. This framework requires adaptation to the ROP project selection model, which covers only the first four features of the project cycle management. These are as follows: (1) project guidance, project development, and preliminary screening; (2) formal project appraisal; (3) independent review of appraisal; and (4) project selection, detailed project design, and contracting. The figure below synthesizes the eight must-have features of the PIM framework, highlighting the fact that the scope of the current project, along with the specific context of the Romanian ROP, have called for an analysis focused on the first four phases (in dark blue), while the subsequent stages (in light blue) are covered in the concurrent World Bank assessment of the MA-IB collaboration and beneficiary support. Figure 2. Eight must-have features in public investment management Pre-feasibility Feasibility CE CBA Regulatory Detailed project Basic completion Project requirements design review development 1 2 3 4 5 6 7 8 Guidance Formal Appraisal Project Project Service Project Implementation & Project Review Selection & Changes Delivery Evaluation Screening Appraisal Budgeting Focus of current report 1 Rajaram, Anand, Tuan Minh Le, Nataliya Biletska, and Jim Brumby (2010), Framework for Reviewing Public Investment Efficiency (Washington, DC: World Bank Policy Working Paper, No. 5397). 4 12. The next paragraphs provide more in-depth explanations of the four aforementioned features to set the stage for the assessment of the ROP system, which follows in Chapters II and III. Each of the subsections below includes two main components: the first describes and defines applicable concepts under the relevant stage in the project cycle, explaining their role in ensuring, ultimately, that the highest-quality projects are selected for financing; the second lists the key dimensions that help assess whether a particular mechanism is functional and effective. 1. Project guidance, project development, and preliminary screening 1A. Concepts:  Investment Guidance: A high-level, authoritative statement of the government’s economic and social vision and priorities can help align investment planning, channeling funding to the highest-value areas while weeding out low-priority, low- impact projects. This might take the form of a national/regional development plan or other medium-to-long-term strategic document. More detailed sector/subsector or regional/local level strategies, including some indication of priorities and objectives, can provide additional effective guidance for investment planning.  Project Development: There should be a formal process for project development. Governments and agencies initiating projects for public investment should prepare a project profile with basic information, including the relevant strategic priority, the specific problem to be addressed, the project’s objectives, main activities, expected results, and a rough estimated budget. In addition, it is important to weigh different options for addressing the targeted problem through the proposed project versus other alternatives (e.g., private-sector intervention, etc.) and to also undertake thorough demand, supply, and gap analyses.  Preliminary Screening: First-level screening of all project proposals ensures that they meet the minimum criterion of consistency with the government’s strategic goals. A project that fails to meet this initial test should be rejected, making it unnecessary to subject it to further evaluation and detailed appraisals. An appropriate institutional arrangement to ensure preliminary screening of all major project proposals is a critical must-have feature to enable the system’s resources (e.g., MA/IB teams, evaluators, etc.) to focus only on the promising, eligible applications. 1B. Dimensions:  A regional economic and social strategy is in place to help guide the prioritization of capital spending decisions, supported by detailed strategies (at least for major sectors).  Initiators submit a project profile for a central body/line ministry/regional agency’s initial review.  A preliminary screening process is in place for testing project proposals for basic consistency with the government’s existing policies and strategic guidance. 2. Formal project appraisal 2A. Concepts:  Quality of project preparation and appraisal: The quality of project preparation and analysis is a critical ingredient in any PIM system. Weak project preparation is very hard to remedy at later stages of the cycle; by contrast, thorough preparation and evaluation can make a substantial difference in terms of how smooth the 5 implementation phase goes and how high the project impact ultimately is.2 In many countries, including Romania, project preparation is relatively weak and there is a serious lack of in-line capacity for project appraisal. The more serious problem, however, is lack of demand from decision-makers for better project appraisal. This is illustrated by the current underutilization of relatively large technical assistance funding to improve the quality of project appraisal.  Project appraisal: Projects or programs that meet the first screening test should be subject to the appraisal of their viability, which requires a feasibility analysis. Its objective is to answer the essential question of whether the initiator should proceed with a project. At this stage, a proposal will be consistent with the government’s priorities, but that does not necessarily mean that it is feasible and optimal per se. This process may require a regulated set of project preparation steps, such as prefeasibility and feasibility studies, including preliminary design and environmental and social impact assessments, which must be completed before a project can be approved for funding. Projects involving non-standard procurement, such as public private partnerships (PPPs) and bundled “resources for infrastructure” projects should be subject to the same appraisal process as standard public investments, and their costs and benefits should be compared against a public sector comparable project.  Cost-benefit analysis: While cost-benefit analysis (CBA) can be complex and involve contentious judgments about the valuation of costs and benefits, it is generally accepted that, because major public investment decisions involve scarce resources and trade-offs between competing claims, decision-makers should be provided with technical advice on the net value of the choices they face. These costs and benefits should ideally be society-wide (i.e., including positive and negative externalities) rather than just focusing on fiscal or financial costs and benefits. At the same time, CBA should be linked to actual project size and also reflect the available capacity of beneficiaries and evaluators to implement and assess such a methodology. Virtually all projects under the ROP 2007-2013, regardless of their size, had to have a CBA submitted along with the formal application, although the EU requires the CBA only for projects over 50 million Euro. No ROP project has passed that threshold in the 2007- 2013 programming period and it is likely that this will not change under the next ROP.  Prefeasibility/feasibility studies: The prefeasibility study helps identify relevant alternatives before undertaking a full-fledged feasibility study, and enables decision- makers and technical staff to find out early whether a proposed project is worth undertaking. The feasibility study takes the prefeasibility analysis further by compiling relevant data, refining project outputs and outcomes, outlining and analyzing the selected solution for achieving project objectives, and carrying out various background assessments, including environmental and social impact analysis. It helps narrow the scope of a project to identify an optimal option for preliminary design. The table below summarizes the main differences between the two outputs. 2 Harberger (2005) argues that improved project evaluation that has a marked impact on the rate of economic productivity can permanently increase a country’s growth rate. 6 Table 1. Key components of prefeasibility and feasibility studies Prefeasibility Study Feasibility Study  Data gathering (geographic, climate,  Compilation of all relevant data socio-economic, and technical)  Project alternatives  Alternative technologies for the project  Major risks (including institutional and  Detailed estimate of costs and benefits budgetary) for a selected alternative  Comparison of alternatives (technical  Preliminary design design and engineering, socio-economic costs and benefits)  Recommended project alternative  Detailed risk assessment  Preliminary estimate of project costs and  Detailed sustainability assessment benefits  Regulatory requirements  Environmental impact assessment  Identification of lacking information for  Social impact assessment the Feasibility Study  Portfolio of projects: It is helpful to maintain a portfolio of the appraised projects. This can help not only to track how many projects have been selected, but also allows for revisiting rejected or lower-scoring projects later on, when underlying circumstances change and they are more likely to generate net positive benefits or there are sufficient available resources to justify the intervention. Hence, all appraised projects should be recorded in a project database ranked by priority for budget consideration. 2B. Dimensions:  Clarity of roles in project preparation: There should be defined attributions for potential beneficiaries, IBs, and the MA.  Specific procedures for project preparation and appraisal: Properly sequenced and disciplined procedures, with formal decisions at key stages (including the decision to proceed to detailed preparation) should be properly applied to relevant project types.  Availability of technical guidance: Clear, comprehensive, and practical technical guidance on the substantive requirements for preparing and appraising new proposals (e.g., a manual appropriate to the existing capacity of the target audience, with specific successful examples) should be widely accessible. The guidance should cover standard public investment delivery modes and, where relevant, non-standard modes such as PPPs. It should provide practical guidance on proportionality of appraisal.  Quality of project appraisal: This includes several key questions. 7 o Is the need for the project clearly identified and well justified? Is the project consistent with the role of the government in the economy? Is the investment based on a systematic assessment of future infrastructure needs and a regular assessment of the current condition of existing infrastructure? o Are the project’s objectives clearly specified and fully consistent with national/regional/local priorities, and with sector and sub-sector strategies? o Are alternative options to meet the project’s objectives clearly identified and comparatively examined? Do the project’s outputs meet set objectives? Is the most promising option analyzed in detail? o Are project costs fully and accurately estimated? Are project benefits likely to justify the costs? Have other potential impacts of the project been identified and adequately assessed? o Has all the required financing been identified? o Have relevant stakeholders been adequately consulted?  Scope and proportionality of appraisal application: Appraisals should be applied comprehensively as well as proportionally, depending on projects’ scope and sector. 3. Independent review of appraisal 3A. Concepts:  Independent review: Optimism bias amongst those developing project proposals – under-estimation of costs and over-estimation of benefits – is well-documented. In a review of 258 transport projects in 20 countries, Flyvberg et. al. found that nine out of ten projects underestimated costs by an average of 28%.3 While project proposing units often have the best information on which to base their new project proposals, they may have weak incentives to produce high quality proposals aligned with the government’s strategies and they may lack technical capacity in project design and analysis. It is therefore a necessary good practice for all projects to be subjected to “challenge” and careful analysis by an agency that is independent of the project- proposing units. However, independent review can also be conducted selectively to focus on the most significant (e.g., high value, high risk, high complexity) projects. 3B. Dimensions:  Project appraisals: Project proposing units or external agencies can formally undertake the development of appraisals (e.g., feasibility studies). An external agency or department should carefully assess the quality and objectivity of such appraisals.  Independent review: All project appraisals, especially for large projects, should be subject to independent review. It is also important how this process is conducted and by whom.  Evaluators: The independence of evaluators, as well as their competence, should be guaranteed through proper selection and contracting procedures.  Criteria: The evaluation criteria should be practical and properly tailored to the type and size of projects considered and there should also be full transparency around how investment proposals are assessed. 3 B. Flyvberg, M. Holm, and S. Buhl, (2007). The average cost overruns varied across transport sub-sectors, with rail projects exceeding estimates by an average of 42%, tunnels and bridges by 34%, and roads by 20%. 8 4. Project selection and budgeting 4A. Concepts:  Project selection: It is generally much more difficult and costly to halt a project after it has started than to redesign, defer, or reject a project prior to initial budget funding. Poor project selection is a common weakness in PIM systems, particularly in developing countries where the urgent need for public investment can overwhelm the call for careful analysis of alternatives. The decision to grant funding represents the last key (and sometimes only) selection gateway. While political considerations or genuine emergencies (including the need to avoid the decommitment of EU funds) can result in pressure to rush a project into contracting and implementation without adequate appraisal, these instances should be exceptional.  Efficient investment: The key factors for efficient investment are good decisions in the choice of projects, active management of the asset portfolio (including through disposals of inactive or irrelevant projects, as needed), and a budgetary process that ensures recurrent funding to operate and maintain existing assets. The latter is especially important for donor-funded projects that create assets, while operation and maintenance costs are assumed to be borne by the government. Forward costs of investment projects and their funding should be reviewed systematically by both sector ministries and the Ministry of Finance during the budget’s preparation. The budget preparation process should be well structured through a careful interface with the project planning cycle, as well as through close coordination among officials responsible for capital and current spending. Before new projects are selected, sufficient funds provisions need to be made for the costs of ongoing (incomplete) projects, so that their efficient rate of physical implementation is not interrupted by lack of financing. In some cases, insufficient funding is allocated to ongoing projects due to political pressures or due to the lack of information on funding requirements, resulting in unfinished investments, high costs, and overall inefficient public spending.  Multi-year budgets: Given the multi-year nature of most capital projects, provisions in the financial management laws and regulations are needed to provide beneficiaries with budget authority to commit expenditures beyond the current budget year, so that they can enter multi-year contracts (e.g., for construction of large infrastructure assets). In addition, larger projects may involve project preparation expenditures that span several fiscal years. In some countries, there is a separate central budget line to finance the costs of project preparation. 4B. Dimensions:  Transparent criteria for project selection  Capital and current spending fully integrated  Quality of detailed project design  Ongoing projects receive sufficient funding  Multi-year funding authority Primary and Secondary Sources 13. Observations and recommendations in this report are based on multiple sources. First, this work builds on past and concurrent analyses, including the World Bank’s 2010-2011 functional review of the Romanian Ministry of Regional Development and Tourism, as well as on other technical assistance work undertaken for the current MRDPA (as referenced earlier). Second, the team conducted substantial desk review and analysis of available materials: specific to the ROP (e.g., 9 programming documents, evaluation grids, applicant guides, progress reports, relevant legislative acts, etc.); on regional development and structural programs in Romania more generally (e.g., the draft Partnership Agreement, Regional Development Plans, Integrated Development Plans, etc.); and related to the broader European framework (e.g., Europe 2020, the Position of the Commission on the Development of Partnership Agreements, the Common Strategic Framework 2014-2020, etc.). Second, there was a substantial effort to gather information on the management and implementation of structural funds in other EU Member States, with a specific focus on selection models, which forms the basis of Chapter IV. This was supplemented, whenever feasible, by phone conversations with relevant actors in other countries. 14. In addition, the team organized several missions to Romania, specifically focused on selection models, engaging in in-depth conversations with both the central Managing Authority within the MRDPA and with Intermediate Bodies and beneficiaries across the country. More specifically, in the first part of the assessment, missions were organized to four of the eight Regional Development Agencies, covering the following regions: North-West, West, North-East and Bucharest-Ilfov. Annex 9 includes a detailed Interview Guide. In the second phase of the analysis (i.e., after the completion of the intermediary report), missions were organized to all eight regions and included discussions with the MA, the RDAs, ROP beneficiaries, and other stakeholders. As part of this effort, and under the broader World Bank Regional Development Program in Romania, a pool of 200+ counterparts contributed with insights and suggestions for improving the ROP. Interviews and discussions focused primarily, but not exclusively, on the MA-IB-beneficiaries triad, also covering key actors like the National Authority for Regulating and Monitoring Public Procurement (NARMPP), financial institutions, consulting firms, contractors, etc. Audiences 15. The main audiences of the current report are senior managers from the Ministry of Regional Development and Public Administration (MRDPA), as well as officials from the European Commission (EC) responsible for the ROP’s oversight. Other key stakeholders who may benefit from this assessment include IB managers and staff, as the main providers of the ROP’s interface with beneficiaries (under the program’s current setup), as well as MA and IB teams from other operational programs. Given its expected role in the 2014-2020 period, the Ministry of European Funds is also a primary target of the current work, and there is hope that the proposed selection models will also inspire other operational programs to enhance the mechanisms for project evaluation. A broad distribution and discussion of the findings is desirable, including among beneficiaries, who should develop a deep understanding of proposed changes that will affect the selection of projects by the next ROP, along with the reasons for adjusting the models used for 2007-2013. Last but not least, given that there is a relative lack of information on optimal selection models for structural funds in Member States more generally, this work can contribute to broader thinking on how to evaluate and select projects that receive co-financing from EU funds. 16. Above all, this analysis will produce positive results if it is accepted, endorsed, and “owned” by the MRDPA leadership and staff and by decision-makers across the Romanian Government. The good news is that the MA team has the decision-making power on how the next ROP will look like in terms of selection models. The hope is that key actors within the MRDPA will continue to discuss the report’s assessment and internalize recommended actions, pushing for their full-fledged implementation. As noted earlier, to the extent that lessons from the ROP apply to other structural programs in Romania and to other EU Member States, and in many cases the principles underpinning selection models are similar (e.g., drive for impact, efficiency, etc.), the application and/or adjustment 10 of this report’s observations and recommendations to a broader context would be much welcomed and encouraged. Structure of the Report 17. This report focuses on three key tasks: assessment of the status quo, comparison with other EU Member States, and recommendations for the ROP’s improvement. It is structured accordingly. Following this introductory section, Chapter II provides background information on the Romanian ROP, particularly useful for audiences who are less familiar with the program. Further, Chapter III narrows in on the ROP’s project selection models for 2007-2013, providing an in-depth analysis of their strengths and weaknesses. As referenced before, this assessment is based on the public investment management (PIM) framework developed by the World Bank. Chapter IV reviews practices in eight selected EU Member States. Applicable lessons from their best practices are recommended for improving project selection models in Romania. Chapter V discusses some basic principles that should be taken into consideration when developing an evaluation and selection framework. Chapter VI discusses key building blocks (e.g., a sound Regional Development Strategy) that need to be in place to enable an evaluation and selection model to work efficiently and effectively. Chapter VII proposes an overall framework for project selection models for the 2014 – 2020 ROP, with a discussion around three potential scenarios (Status Quo+, Semi-Competitive, and Competitive) for the MA-ROP’s consideration. Chapter VIII goes into detailed recommendations and proposes a number of changes to the current evaluation and selection system – changes that would apply to all three scenarios discussed in this report. Chapter IX and Chapter X discuss multiple issues pertaining to the recommendation for pre- allocating funds for certain types of projects – an issue that is particularly poignant for the Semi- Competitive and Competitive scenarios. Chapter XI proposes a number of options for a competitive selection model. Last but certainly not least, Chapter XII tackles a critical issue that is often overlooked – the proper monitoring and evaluation of the project selection model to establish whether it has achieved the desired goals and functioned within the set parameters. Chapter XII concludes. 11 II. Background on Romania’s ROP 18. Romania continues to face important challenges in regional development, particularly in terms of “shortening” the distance to the markets in Western Europe, which attract roughly 70% of the country’s exports.4 At the same time, Romania has the opportunity to benefit from significant EU structural assistance through the Regional Operational Programme (ROP). In this process, one critical component is to improve project selection models for the next ROP, including the evaluation process and criteria, while maintaining strong linkages to the strategic development priorities of the future programming period. The following sections provide basic information on the structure, strategic priorities, and implementation of the ROP, for both 2007-2013 and 2014-2020. A. ROP 2007-2013 19. The Regional Operational Programme (ROP) is one of seven operational programs (OPs) in Romania financed through EU funds and its importance for the country’s development can hardly be overstated. The EU dedicated 3.7 billion EUR to the ROP to support Romania’s regional development in the post-accession period. This amounts to 19% of the total structural funds available through the National Strategic Reference Framework (NSRF) between 2007 and 2013, making the ROP the third largest program after transportation (23.7%) and environment (23.5%). In fact, based on the draft Partnership Agreement, its size is expected to increase significantly for 2014-2020, in part due to its previous success. The ROP involves a diverse portfolio of projects, from urban development to transport infrastructure, social infrastructure, tourism, and technical assistance. In 2011, projects financed under the ROP accounted for nearly 27% of investments supported through external grants and more than 12% of total public investment in the country. Qualitative data from interviews with local administration representatives (e.g., Mayors, County Council Presidents, etc.) have confirmed that the ROP plays a key role in supporting Romania’s development needs, particularly at the city and county level. In fact, many of the local priorities are set to align with the key goals of the ROP. Table 2. Public investment in Romania, 2007 – 2012* Investments Projects Annual Financed Annual Payment Year GDP Total Expenses Non-Financial through ROP Total under Assets External Planned the ROP Gants 2012 136,557 41,787 30.6% 3,678 2.7% 3,632 2.7% 2,547 1.9% 806.9 330.5 2011 129,269 48,468 37.5% 5,440 4.2% 5,140 4.0% 2,537 2.0% 677.0 534.6 2010 124,127 47,959 38.6% 4,601 3.7% 4,601 3.7% 1,719 1.4% 636.8 151.0 2009 118,271 45,555 38.5% 5,469 4.6% 5,469 4.6% 560 0.5% 546.8 355.3 2008 139,762 51,354 36.7% 6,301 4.5% 6,301 4.5% n/a n/a 500.7 3.0 2007 124,654 40,918 32.8% 4,318 3.5% 4,214 3.4% n/a n/a 408.8 0.0 *Note: All figures are in million Euros (where applicable) **Source: Team’s calculation based on data from ROP - Annual Implementation Reports 2009, 2010, and 2011, budgetary execution reports from Ministry of Finance 2007-2012, and other information from the Romanian National Bank and the National Institute for Statistics 4 GDP/capita data show that regions closer to the EU’s hard development core tend to be more developed. For a more detailed discussion of these aspects, see the report on “Competitive Cities,” World Bank, 2013. 13 20. The ROP programmatic document defines strategic and specific objectives that are generally aligned with sound principles of economic growth. The program seeks to support the economic, social, territorially balanced, and sustainable development of Romanian regions, according to their specific needs and resources, by focusing on urban growth poles, the business environment, and basic infrastructure. The vision is to transform Romanian regions, particularly the ones lagging behind, into more attractive places to live, work, visit, and invest in. Based on this, the following specific objectives were set: (1) to increase the economic and social role of urban centers, adopting a polycentric approach, in order to stimulate a more balanced development of regions; (2) to increase accessibility within regions and in particular the accessibility of urban centers and their connection to surrounding areas; (3) to increase the quality of social infrastructure of regions; (4) to increase the competitiveness of regions as business locations; and (5) to increase the contribution of tourism to the development of regions. Concentration, connectivity, basic institutions and services, quality of life – these are cornerstone concepts of proper policies promoting sustainable and inclusive economic development (albeit with spatially uneven benefits in the short term, but faster, stronger growth and overall convergence in the long term), as noted in the World Bank’s World Development Report 2009 and also in the 2013 Competitive Cities study. 21. The ROP’s allocation across Romania’s eight development regions is inver se proportional to GDP/capita, with lagging regions, which have greater needs and sometimes lack even basic infrastructure, receiving more funds.5 In addition, the ROP has 6 Priority Axes (PAs) that consist of 15 Key Areas of Intervention (KAIs), as described below. Error! Reference source not found. provides further details on activities, eligible beneficiaries, target groups and target areas, submission timeline, submission types of project application forms, and technical and financial evaluation criteria for each of the ROP’s KAIs. Below is a summary of the ROP’s PAs and their corresponding priorities:  Priority Axis 1: Support for sustainable development of urban growth poles. This aims to increase the quality of life and to create new jobs in cities by rehabilitating the urban infrastructure, improving services, including social services, as well as by developing business support structures and entrepreneurship. Investments are concentrated in those cities that act as regional and / or local growth poles and spread the development into the surrounding areas, giving priority to growth poles located in regions and counties with lower level of development in terms of GDP and unemployment. KAIs for PA1: PA 1.1. Integrated urban development plans; PA 1.2 Support for the investments in the energy efficiency of the buildings.  Priority Axis 2: Improvement of regional and local transport infrastructure . This aims to increase the accessibility of regions and the mobility of population, goods and services, in order to foster sustainable economic development. There is one KAI for PA2: PA 2.1.: Rehabilitation and modernization of the county roads and urban streets networking including construction/rehabilitation of ring roads.  Priority Axis 3: Improvement of social infrastructure. This axis aims to create the premises for better access of the population to essential services, contributing to the achievement of the European objective of economic and social cohesion by improving infrastructure for health, education, social services, and public safety in emergency situations. There are 4 KAIs: PA 3.1.: Rehabilitation, modernization, and equipping of the health services’ infrastructure; PA 3.2.: Rehabilitation, modernization, development & endowment of social services infrastructure; PA 3.3.: Improving the equipment of the operational units for public safety interventions in emergency situations; 5 See Annex 1 for the ROP’s planned allocation by priority axis, KAIs, development regions, and year. 14 PA 3.4.: Rehabilitation, modernization, development, and endowment of pre–university, university, and lifelong learning infrastructure.  Priority Axis 4: Strengthening the regional and local business environment. This priority axis aims to set up and develop business support structures of regional and local importance, rehabilitate industrial sites, and support regional and local entrepreneurial initiatives, in order to facilitate job creation and sustainable economic growth. KAIs for PA4 are: PA 4.1.: Development of sustainable business support structures of regional & local importance; PA 4.2.: Rehabilitation of unused polluted industrial sites and preparation for new activities; PA 4.3.: Support the development of micro-enterprises.  Priority Axis 5: Sustainable development and promotion of tourism. This aims mainly to promote the country’s cultural heritage and natural resources with tourism potential, as well as to improve the quality of accommodation and leisure tourist infrastructure, in order to increase the regions’ attractiveness, develop the local economies, and create new jobs. KAIs for PA5 are: PA 5.1.: Renovation and sustainable use of cultural heritage sites and set-up/modernization of related infrastructure; PA 5.2.: Creation, development, and modernization of tourism infrastructure for sustainable use of natural resources and for increasing the quality of tourism services; PA 5.3.: Promotion of tourism potential and set-up of needed infrastructure to increase Romania’s attractiveness as a travel destination.  Priority Axis 6: Technical assistance. The objective is to provide support for the transparent and efficient implementation of the program. KAIs include: PA 6.1.: Support for the implementation, overall management, and evaluation of the ROP; PA 6.2.: Support for the advertising and information activities related to the ROP. 22. The ROP’s importance and relevance in the context of Romania’s pressing development needs is also obvious in the high rates of applications, far exceeding the available allocation, across all regions and nearly all axes. The two figures below summarize this information and the ROP Annual Implementation Report (AIR). As of December 31, 2012, the AIR shows good results: 95% contracting for a total of 5.67 billion Euros, with all regions exceeding 85%; 1.44 billion Euros paid to beneficiaries, of which 1.29 billion from the European Regional Development Fund (ERDF); 1,131 finalized projects, with 292.5 million Euros as the ERDF contribution; and the positive examples could go on.6 Figure 3. Submitted ROP applications vs. allocation (100%) 6 For more complete statistics at the 2011 level, see Annex 1 – Table 7. 15 Figure 4. Submission of ROP applications per development region (compared to allocation of 100%) Source: ROP Managing Authority: ROP – Annual Implementation Report 2012 (draft) 23. In this light, it is important to note from the start that the ROP has been a superior performer in the Romanian context, in terms of absorption rates, despite a complex and heterogeneous portfolio of projects. Because of its broad focus on regional development issues, the ROP addresses a wide-ranging set of needs at the level of each region, from urban development to transport infrastructure, social infrastructure, business development, and tourism. In this context, the ROP has achieved the best absorption rates out of all structural programs (as of November 30, 2013), particularly when it comes to payments from the EC to Romania, which stood at 44% of the total available ROP allocation. Of course, this is a quantity-focused indicator and does not mean that the selection models per se have been flawless (and, in fact, as noted throughout this report, there is room for improvement) or that the impact of financed projects was maximized. But the principle still holds true that any changes going forward should not affect the program’s positive results (e.g., absorption, overall beneficiary satisfaction, etc.). Figure 5. Relative performance of EU structural programs in Romania (as of November 30, 2013) 120% 118% 110% 103% 100% 85% 82% 80% 78% 80% 60% 48% 45% 46% 44% 40% 40% 33% 30% 27% 22% 22% 21% 19% 18% 18% 20% 0% ROP Environment Transport Competitiveness Administrative Human resources Technical capacity assistance % contracted of total ROP funds available % paid to beneficiaries % paid by EC to Romania Source: Ministry of European Funds 16 24. The ROP’s broad scope and diversity inevitably translate into a complex evaluation and selection framework, with criteria and grids adapted to each major intervention area. Even so, while it may be challenging for applicants and beneficiaries to navigate through the process, their satisfaction with the program is generally very high, as evident in both qualitative interviews and in responses to a survey conducted in April-May 2013.7 Specifically, out of 470 respondents, 62% qualified the support received from Intermediate Bodies during the evaluation stage as “excellent” or “very good,” while 12% were dissatisfied. The numbers related to beneficiaries’ satisfaction with MA support provided during project evaluation are similar: 68% characterized it as “excellent” or “very good” and only 13% thought it was “unsatisfactory” or “poor.” Such results are encouraging and tend to reflect beneficiaries’ broader attitudes vis-à-vis the MA and the IB staff, which they routinely characterize as professional, competent, and helpful. With few exceptions, interviewed subjects praised the fact that selection models and criteria are transparent and described in sufficient depth in the Applicant Guides, which helps them design projects that are more aligned with the objectives of the ROP. 25. At the same time, field research and qualitative survey inputs also point to potential improvements of current project selection models. The subsequent chapters of this report go into great detail for each potential shortcoming. As a preview, it suffices to note that in some cases the current criteria and procedures are considered too elaborate and lengthy (e.g., the cost-benefit analysis is required even for projects where it is hardly applicable or where the expenses related to implementing a proper CBA are excessive compared to the total value of the investment); in other instances, the models tend to prioritize simpler applications that pass a certain threshold and, based on the first-in-first out rule, receive funding to the detriment of more complex projects that require substantial time for concept design and application writing. These are precisely some of the issues that have motivated the current effort to identify and propose improved ROP selection models that would build on the system’s strong points while addressing its weaknesses. B. ROP 2014-2020 26. The ROP for the 2014-2020 budgeting period is, as of January 2014, in the programming phase, contingent on the conclusion of higher-level agreements between the EU and the Romanian Government. At the European level, the flagship document guiding the Union’s future development is the Europe 2020 strategy, which established three priorities around smart, sustainable, and inclusive growth, each supported by several flagship initiatives (e.g., Youth on the move, European platform against poverty and social exclusion, etc.). Five fundamental objectives cover employment, research & development, climate change and energy efficiency, education, and poverty, backed by specific targets like 3% of GDP invested in R&D and at least 20 million people fewer in or at risk of poverty and social exclusion. While Europe 2020 significantly shapes EU-level thinking, its objectives remain vague, and it is not entirely clear how they were selected as critical to the Community’s development at this point in time. Given the critical importance of Europe 2020 and other related documents, which ultimately influence how structural funds are managed and what kinds of investments they can support (and, hence, how selection models look like), there is a need to ensure better consultations at the country level to bring in the voices of potential beneficiaries and citizens. In other words, there need to be more bottom-up processes to highlight needs and priorities of different Member States, and ensure that they are fully reflected in EU-level strategic documents. 7 These data come from a survey conducted as part of the “ROP 2.0: Beneficiary Support Mechanisms” World Bank technical assistance project. 17 27. Mirroring the main priorities of the Europe 2020 strategy, a legislative package was adopted by the EC on October 6, 2011, establishing 11 thematic objectives. These include: strengthening research, technological development, and innovation; enhancing access to, use, and quality of information and communication technologies; enhancing the competitiveness of small and medium- sized enterprises, the agricultural sector (for the EAFRD), and fisheries and the aquaculture sector (for the EMFF); supporting the shift toward a low-carbon economy in all sectors; promoting climate change adaptation, risk prevention, and management; protecting the environment and promoting resource efficiency; promoting sustainable transport and removing bottlenecks in key network infrastructures; promoting employment and supporting labor mobility; promoting social inclusion and combating poverty; investing in education, skills, and lifelong learning; and enhancing institutional capacity and an efficient public administration. 28. Based on these goals, the EC presented the Common Strategic Framework (CSF), designed to enable more impactful and integrated spending of structural funds.8 As noted by the Commission, “national and regional authorities will use this framework as the basis for drafting their 'Partnership Contracts' with the Commission, committing themselves to meeting Europe's growth and jobs targets for 2020.”9 The next steps in the process are the drafting, evaluation, negotiation, and signing of the Partnership Agreement (PA) with each Member State. This is the document that is meant to replace the National Strategic Reference Framework 2007-2013 for the next programming period. The PA aims to set out an integrated approach for territorial development supported by all the CSF Funds and includes objectives based on agreed indicators, strategic investments, and a number of conditionalities. As of October 2013, a first draft PA is open for public debate and key stakeholders have been asked to provide feedback to the Romanian MEF. 29. Previously, the EC had prepared position papers customized for each Member State, summarizing the Commission’s views on funding priorities, challenges, and recommendations for 2014-2020. This document has been the best indication of what the EC expects from Romania in terms of how it will use available structural funds over the next seven years. As it specifically notes in its introduction, “the purpose of [the] position paper is to set out the framework for dialogue between the Commission services and Romania on the preparation of the Partnership Agreement and Programmes.” The paper goes on to say that “the Commission proposes to group and limit EU funding to key challenges outlined in [the] position paper.” Indeed, the position paper has been rightly cited in interviews with both MA and RDA representatives as the basis of current programming efforts. 30. Partly as a result of internal political changes, partly due to delays in the adoption of the EU budget, Romania has started the programming for the 2014-2020 cycle relatively late, particularly compared to the 2007-2013 period. In June 2012, the Government approved a “Memorandum on the Approval of Actions and Documents for the Preparation of Accessing and Implementing European Funds in 2014-2020.” Based on this framework, the Ministry of European Funds (MEF) created a set of methodologies that describe the main elements of programming documents, as well as the stages of the planning process, including methods for analyzing development needs, formulating strategies, and consulting stakeholders. Like the Ministry of European Integration in 2007-2013, the MEF is responsible 8 These include: the European Regional Development Fund (ERDF), the European Cohesion Fund (CF), the European Social Fund (ESF), the European Agriculture Fund for Rural Development (EAFRD), and the European Maritime and Fisheries Fund (EMFF). 9 See official press release related to the CSF proposal (http://europa.eu/rapid/press-release_IP-12-236_en.htm) 18 for the overall coordination of the programming process, including related technical assistance projects and, additionally, is in charge of developing the draft Partnership Agreement (PA). The Government has responded to EC requests in this area promptly: for instance, four days after the EC presented its position paper, the Government adopted a note on fulfilling ex-ante conditionalities and, on January 31, 2013, it had a first discussion with the EC on the position paper and its implications. This dialogue has continued over the past months and the PA is expected to be approved by the EC in early 2014, according to the official schedule. This timeframe is still relatively delayed compared to other Member States (for example, the Czech Republic, Belgium, Poland, and Slovenia all aimed to finalize their PAs at the end of 2012).10 31. The overall consultation system for developing Romania’s priorities for 2014 -2020 is similar to the previous programming period and involves bottom-up inputs through thematic working groups. At the national level, an Interinstitutional Committee for Elaborating the Partnership Agreement (ICPA) draws from the work of 12 sub-groups: 10 thematic consultative committees (TCCs), spanning topics like transport, environment, education, health, tourism, administration and good governance, etc.; and 2 committees on regional development (CCRD) and territorial cohesion (CCTC). Each committee is coordinated by central authorities (i.e., line Ministries). The CCRD is most directly tied to the future ROP, although some of the other sub-groups (e.g., education, health, tourism, territorial cohesion, etc.) may well influence the program’s priority axes and strategic directions. The CCRD is the main link between the national and the regional levels, and MA-ROP ensures its Secretariat. Figure 6. The current structure for 2014-2020 programming of EU structural funds in Romania 32. The two tables below provide further information about the proposed allocations by operational program and types of investments for 2014-2020, as included in the May 2013 preliminary draft of the PA. The ROP benefits from 6.99 billion Euros, significantly more than for the current programming period, which would also make it the largest operational program (although the Large Infrastructure OP is a close second). As for investment types and priorities, it is likely that they will remain largely the same compared to the 2007-2013 period (e.g., urban development; social and 10 “A Balancing Act: Managing the Programmes, Closure, and Preparations for 2014-2020,” IQ-Net Review Paper No. 31 (1), November 2012. 19 health infrastructure; energy efficiency; business environment). There is no indication, as of July 2013, of the relative split of funding among the various intervention areas. In general, there may be further changes to the data below, to be determined for the final version of the PA that will be signed between the EC and the Romanian Government, likely before the end of 2013. Table 3. The allocation (% from the European Structural and Investment Funds - ESIF) by Operational Program (billion EUR) EU Policy Operational Program Amount % from total EU cohesion policy Large Infrastructure OP 6.98 27 Human Capital OP 2.18 8 Administrative Capacity OP 0.96 4 Regional OP 6.99 27 Competitiveness OP 1.35 5 Technical Assistance OP 0.3 1 European Territorial Territorial Cooperation Program Romania-Hungary 0.6 2 Cooperation Territorial Cooperation Program Romania-Bulgaria Common Agricultural Policy Rural Development OP 6.6 25 Fisheries OP 0.22 1 Source: Draft Partnership Agreement for Romania (May 31, 2013) Table 4.Indicative proposals by type of investment and institutional structure Operational Managing Program Fund Type of Investments Intermediate Body Authority (MA) (OP) CF - road infrastructure TEN‐T Department of Infrastructure Projects and Foreign Investments (DIPFI) - railway TEN-T Ministry of - harbors TEN-T (navigation to be Transport (MT) decided) Large - public utilities services (water and used Ministry of Infrastructure water, waste management) Environment and OP - risk prevention and climate (structural Climate measures: dikes, coastal areas and Change(MSCC) non‐structural measures of risks, including draught), Nature 2000, biodiversity, etc. - energy efficiency in industry Ministry of Ministry of (production, transport and distribution Economy (ME) European for renewable energy, cogeneration Funds thermo and electrical systems) Human Capital ESF -employment, social inclusion, Min. of Labor, OP strengthening institutional capacity of Family, Social National Agency for Employment Protection, and Elderly (MLFSPE) -education and training Ministry of National Education (MNE) Competitive- ERDF -research, development and innovation Ministry of National ness OP (including for large enterprises) Education (MNE) 20 Operational Managing Program Fund Type of Investments Intermediate Body Authority (MA) (OP) -Digital Agenda (broadband and different Ministry for IT infrastructure and interoperability ) Informational Society Technical ERDF -horizontal technical assistance for Assistance OP management of structural and - investment funds Ministry of Administrative ESF - strengthening administrative capacity Regional Capacity OP - modernizing the judicial system Development - e‐government (e‐health, e‐justice, - and Public e‐education, etc.) Administration - management system (MRDPA) - cadaster Regional OPs ERDF - local road, railway and harbor Regional infrastructure Development - urban /local development (including Agencies (RDAs) public transportation and related infrastructure cultural heritage, sport facilities, multifunctional facilities) - energy efficiency (including rehabilitation of networks if needed) - education and social infrastructure - health infrastructure - competitiveness and business environment for SMEs, including financial instruments European ERDF -Territorial Cooperation Program Territorial Romania ‐ Hungary Cooperation - Territorial Cooperation Program OPs Romania – Bulgaria Ministry of Rural EARDF - investments in agriculture and rural Payments Agency Agriculture and Development development for Rural Rural Program OP Development and Development Fisheries (PARDF) (MARD) EARDF - public infrastructure in rural areas Ministry of Regional (roads, water, used water-waste Development and community center) Public Administration Fisheries OP EMFF - sustainable fisheries and aquaculture investments (both private and public) Direct Payment EAGF Source: Draft Partnership Agreement for Romania (May 31, 2013) 33. At the regional level, as during the preparation of the 2007-2013 programming period, there are also eight RCRs – Regional Committees for Elaborating the Regional Development Plans (RDPs). They maintain consultative roles and are coordinated by the region’s respective Regional Development Agency (RDA), which also represents the RCR in the Inter-institutional Committee for Elaborating the Partnership Agreement (ICPA). The main functions of the RCR are to supervise and endorse: the social and economic analysis used to design the RDP; the strategy and the main objectives for the development of region; the proposed/planned financial program assigned to the RDP; the draft of the 21 RDP; the monitoring reports of the RDP; and the list of strategic projects identified at the level of the development region. For its part, the MA-ROP has issued a Methodology for the Planning of Regional Development to guide the preparation of regional development plans.11Each RCR brings together a wide range of social and economic partners, academic experts, and key future beneficiaries (e.g., city halls, county councils, etc.), organized into working groups. Though there is some variation across the eight development regions, WGs generally cover the following areas: urban development; energy efficiency; business environment; disadvantaged communities; and regional infrastructure. 11 Metodologie privind Planificarea Dezvoltării Regionale 2014-2020 22 II. Review of ROP Project Selection Models (2007 – 2013) 34. The ROP selection process includes three main stages: first, the administrative conformity and eligibility assessment; second, the technical and financial evaluation; and third, the preparation and analysis of the technical project (if required) and the pre-contracting/contracting phase. A full graphical description of this relatively complex process is presented in Error! Reference source not found., covering a hypothetical project that requires a technical project (TP) / detailed technical design. For some ROP projects, such as those covering the promotion of tourism, there were no actual infrastructure works involved, so a TP is not needed. The paragraphs below describe in greater detail the various components of the three-staged ROP selection process:  The first stage covers the administrative conformity and eligibility assessment, and should not exceed 25 working days. The procedure allows for an exceptional extension of no more than 5 working days, which means that, in exceptional cases, the maximum duration should be 30 working days (6 weeks or 1.5 months). o Step 1.1: “Checking the administrative conformity and eligibility” should last for a maximum of 4 working days. The expert in charge of this task can ask for 2 clarifications, at most, although each round can include multiple questions. The maximum time allowed to finalize this step, including the time for the beneficiary to prepare the answers and for the expert to analyze them, is 15 working days. o Step 1.2: “Checking based on the ‘4 eyes’ principle” involves a second expert in charge of the verification, with up to one round of clarifying questions (as before, there is no limit on the total number of questions asked within this round). This step must be complete within 8 working days, which includes the time for the beneficiary to prepare answers and for the expert to analyze them. o Step 1.3: “Checking the process” is completed by the head of the unit in charge with the administrative conformity and eligibility verifications. The maximum duration allowed is 2 working days.  The second stage, the technical and financial evaluation, should last for a maximum of 86 working days (around 17 weeks or 4.3 months) and has three steps, as described below. o Step 2.1: “The Technical and Financial Evaluation” is performed by a team of 3 people (2 independent evaluators and 1 thematic expert), typically appointed by MA-ROP and endorsed by the RDA. In some cases, the RDA has directly appointed these independent experts to speed up the process and surpass certain blockages at the MA- ROP level. Specifically, the MA has had difficulties in conducting the public procurement procedures related to the selection of evaluators at the national level, mainly because of objections formulated by the National Authority for Regulating and Monitoring Public Procurement (NARMPP). The maximum duration allowed for this step is 35 working days, including, at most, 2 requests for clarification. o Step 2.2 (if needed): “The Exceptional Evaluation” is required when during the first step the two independent evaluators cannot agree on the final result. In this case, the IB asks the MA to appoint a third independent evaluator. The allotted timeline for this step is 28 working days, including 1 maximum request for clarification. o Step 2.3: The MA-ROP checks and endorses the technical and financial evaluation report. The maximum duration allowed for this step is 23 working days, during which the MA has the opportunity to clarify any pending issues. 23  The third stage has two major steps. o Step 3.1 (if required): “Preparation and Analysis of the Technical Project” involves the beneficiary submitting this document to be verified by the IB. The TP preparation can take up to six months, after which IB experts have to verify the conformity of the TP. In exceptional situations (e.g., significant discrepancies between the TP and the feasibility study), the IB can request the MA to appoint a technical expert to verify the content of the TP and recommend changes. This selection process generally applies to all PAs, with several exceptions for some particular PAs such as PA 4.3., where beneficiaries are not required to submit technical projects (interventions may not include a construction component). o Step 3.2:“Pre-contracting/contracting” includes site visits, preparation of contracting documents, and the signing of the financing contract by all parties. 35. In practice, it has taken significantly longer than the required norm to complete certain steps of the selection process. For example, in 2011, the real duration for the evaluation process took 135.5% versus the planned duration. Most of the delays happened at the stage of technical project submission and verification, and pre-contracting/contracting. Interestingly, as evident from the figure below, the actual average duration of the selection process was longer every year. In 2011, it increased by 21.4% compared to 2010 and by 87% compared to 2008. The duration for the technical and financial evaluation in 2011 increased by 82.6% compared to 2010 and by 133.3% compared to 2008. This is a reflection of the number of applications submitted for consideration, but can also suggest an increase in the complexity of applicable procedures. The only phase for which processing became slightly more efficient is the conformity and eligibility check, performed entirely by the RDAs. Table 5. Duration of selection process, 2011 Duration in practice Phase Formal timeline Duration in practice vs. norm Conformity and Eligibility 1.5 months 1 months 66.6% Technical and Financial Eval. 4.3 months 4.2 months 97.7% Technical Project 6 months 9.1 months 151.7% Pre-contracting/ Contracting 2 months 4.4 months 220% Total 13.8 months 18.7 months 135.5% Source: ROP Managing Authority: ROP Annual Implementation Report 2011 Figure 7. Duration of Selection Process by Phase, 2008 – 2011 (in months) Source: ROP Managing Authority: ROP – Annual Implementation Report 2011 24 Analysis of the 2007 – 2013 ROP Project Selection Models 36. This section presents findings from the analysis of the 2007 – 2013 ROP project selection models, according to the first four stages of the eight-must-have PIM framework, as described in the methodology section. It first discusses why the current ROP project selection models were chosen, then presents a summary table of the findings, followed by a detailed discussion. 37. Within each region, two primary ROP selection models are currently in use. The first applies to projects proposed by growth poles (GPs) and urban development poles (UDPs) under Priority Axis 1.1 and its key feature is the dedicated or pre-allocated funding. The second model covers all other priority axes and types of beneficiaries and involves selection through the First-In-First-Out (FIFO) rule, as explained below. There is also a third selection model that applies only to ROP Axis 6 – Technical Assistance, for which the beneficiaries are the MA and the RDAs, and funds are meant to cover expenses related to the proper implementation and promotion of the program. Since in practice there are no evaluation procedures or formal project applications under this axis. 38. The first selection model (“dedicated”) ensures pre-allocated funding to each GP and UDP. The choice to rely on predetermined amounts is in line with EU requirements for integrated projects in large urban settings. The strategy formulation, planning, and identification of integrated projects occur through the Urban Integrated Development Plans, as prepared by each municipality interested in accessing ROP funding under the GP or UDP priority areas. The list of GPs and UDPs was determined at the national level and only cities formally included under each category have been able to apply for funding available under the corresponding axis/key area of intervention. On the one hand, dedicated funding has worked well for administrations that needed to rely on a predetermined budget to support planning and project preparation (e.g., financing of complex technical documentation), and the general idea of integrated projects is correct and powerful in terms of potential impact. In that sense, the ROP was an opportunity for local administrations to draft and implement development strategies based on clear priorities and projects, a process that has been valuable in guiding cities’ growth and in developing planning capacity at the local level. Indeed, many localities across Romania had lacked strategic planning documents before they became mandatory as a precondition for accessing EU funds through the ROP Axis 1.1. On the other hand, dedicated funding also means that there is an extended period of time available for project submission and completion. There is a risk of delays in realizing investments if the municipality is not well motivated to pursue the preparation and implementation of projects. In addition, the model only requires meeting minimum standards of quality, rather than selecting the highest-impact projects, a broader weakness of the current ROP system, whereby projects do not compete (and are not scored) against each other. 39. The second selection model, based on the First-In-First-Out (FIFO) rule, involves potential beneficiaries submitting applications for funding on a continuous basis, under various axes, until the budget allocation is exhausted by contracted projects and the respective axes are closed. The reasons for this FIFO approach have not been explicitly stated as a policy with a formal description of the rationale, but the benefits of this framework are relatively straightforward. A FIFO-based selection model encourages the accelerated submission of projects, especially those that are already prepared and in the pipeline at the beginning of the cycle. It also allows for continuous processing of the proposed projects, which reduces the risk of peak-time over-burdening of MA/IB capacity. This highly structured approach to selecting projects also limits external disruptions to quick and fair processing. Overall, the FIFO approach promises to encourage higher absorption of ROP funds in a given time period (i.e., 2007-2015, following the n+2 rule). While FIFO processing of projects has some 25 advantages, it also has a number of weaknesses. First, projects are approved as they come through the pipeline if they pass the minimum required points on evaluation criteria, rather than selecting them for the highest expected impact. “Early-risers” may submit projects that meet minimum standards, thereby implicitly excluding better projects that take longer to design and, thus, are generally submitted later, after available funds are already depleted. Second, accelerating project preparation and submission can lead to: (a) lower quality applications and technical documentation; and (b) possible delays in processing (i) when clarifications and corrections are required for the project to be eligible, (ii) the project has to be improved to meet minimum standards, or (iii) corrections are required to remove inconsistencies between technical designs and feasibility studies. Table 6. Overview of the 2007 – 2013 ROP Selection Models (except Axis 6 – Technical Assistance) Selection Fund Selection Project Body & Applicant Allocation Selection Type Model Size Payment Authority Authority Non-competitive Selection  Dedicated allocation for each applicant for the whole ROP programming cycle  Projects must be included Pre- ex-ante in Integrated Growth Poles, Central MA selection < 20 Development Plans (IDPs) Urban allocates funds (axis 1.1 for million Central MA  Projects are evaluated and Development to each GPs and EUR selected in the order their Poles applicant UDPs) application was received  Projects that have a final score above 3.5 out of 6 get funded; projects scoring 0 on any criterion are disqualified Non-competitive Selection  Projects are evaluated and Counties, Central MA selected in the order they urban centers, allocates funds are received < 20 other local to each region, FIFO  Projects that have a final million administrations, for each Central MA (all others) average score above 3.5 EUR private sector priority axis out of 6 get funded; entities, and and key area projects scoring 0 on any NGOs of intervention criterion are disqualified 40. The table below summarizes the main findings of the analysis of the 2007 – 2013 ROP project selection models, according to the PIM framework. The detailed discussion of the findings follows subsequently. 26 Table 7. Diagnosis of the 2007 – 2013 ROP selection procedures according to the PIM framework ROP PIM Stage 1 PIM Stage 2 PIM Stage 3 PIM Stage 4 Phases and Investment Guidance, Project Project Appraisal Independent Review of Appraisal Selection, Detailed Technical Design Procedures Preparation, Preliminary Screening & Contracting STRENGTHS: + strategic framework is available + clear eligibility guidelines WEAKNESSES: Guidelines -no formal strategic prescreening and Calls for And lack of key sector indicators Applications - lack of coordination among OPs - low quality IDPs (for Axis 1.1) - closed list of projects lacking strategic prioritization (for Axis 1.1) -ROP design does not reflect priorities and needs specific to each region WEAKNESSES: -lack of long-term O&M cost estimates - poor-quality CBAs (no specific guidelines, no alternatives considered) Application -CBA not cost-effective for all projects submission -poor-quality technical documentation due to underfunded preparation costs and lowest-price award criteria for public procurement procedures STRENGTHS: WEAKNESSES: Admin. and +detailed, transparent evaluation - majorissues with ownership eligibility criteria and verification grids documents (undetected before this check stage in the process) STRENGTHS: STRENGTHS: Technical +detailed, transparent evaluation + high degree of independence of &financial criteria clearly communicated to evaluators (especially when hired at evaluation applicants the national level) 27 ROP PIM Stage 1 PIM Stage 2 PIM Stage 3 PIM Stage 4 Phases and Investment Guidance, Project Project Appraisal Independent Review of Appraisal Selection, Detailed Technical Design Procedures Preparation, Preliminary Screening & Contracting WEAKNESSES: WEAKNESSES: - too many evaluation grids - generally low quality of evaluation - lack of standardized guidance to due to: underfunding; procurement evaluators on how “deep” to go into using least cost criterion; lack of their assessments (e.g., conformity holding evaluators accountable; delays check vs. verifying assumptions in hiring evaluators by the MA behind CBA) - lack of verification of eligibility of expenditures in feasibility studies - inability of applicants to improve project at feasibility study stage STRENGTHS: +clear, transparent selection criteria + reduced risk of political interference + good for non-strategic, small projects + easier management of peak demand + accelerated application rates + movement of most-ready projects Selection get quickly to implementation WEAKNESSES: - prioritization of simpler projects to detriment of complex, strategic ones - increased risk of errors, clarifications - no guarantee of best value for money and highest impact generated Verification WEAKNESSES: of Detailed -low quality of DTD Technical -lack of technical verification of DTD Design -no/weak accountability of designers WEAKNESSES: Financial -no clear mechanism for order of Contracting contracting, delays in signing 28 Investment Guidance, Project Development & Preliminary Screening Investment Guidance 41. The ROP strategic framework for 2007-2013 consists of a number of key documents:  The National Development Plan (NDP) is the overall strategic framework on structural funds, theoretically also including national policy, but practically driven by EU-level policies and the need to absorb EU financing. The NDP was designed by the Authority for Coordinating Structural Instruments (ACSI) in the Ministry of European Funds, with the support and consultation of other ministries, especially those managing Operational Programs (OPs) and the Ministry of Finance. A national development strategy, covering the country’s growth vision, regardless of financing sources, is not yet available.  The National Strategic Reference Framework (NSRF) details the NDP in terms of operational programs, including objectives, actions, resources, and eligibility. The NSRF was also designed by ACSI and, for 2014-2020, will be replaced by the Partnership Agreement (PA) that is currently negotiated between the EC and the Government of Romania. The mandate for developing the PA 2014-2020 has stayed with the Ministry of European Funds (MEF).  The Regional Operational Programme 2007-2013 is the main strategic and programmatic document in the current ROP system. It is developed by the Management Authority for the ROP in the Ministry of Regional Development and Public Administration (MRDPA), with the consultation of Regional Development Councils (RDCs), Regional Development Agencies (RDAs), other sector ministries to leverage linkages (e.g., the Ministry of Agriculture and Rural Development, the Ministry of Transport, the Ministry of Environment, etc.). This process was supervised by ACSI within MEF.  Regional Development Plans are designed by Regional Development Agencies, in consultation with local and county authorities from the region, as well as with other socio-economic partners. The RDPs are eventually approved by the Regional Development Councils (RDCs).  The Framework Document for Implementation (FDI) provides details for every ROP priority axis in terms of specific objectives, main activities, target groups, resources allocated, eligible beneficiaries, and eligibility and evaluation criteria. The FDI is developed by the MA-ROP with the consultation of RDCs and RDAs, as well as other line ministries, and under the supervision of the MEF and the Certification and Payment Authority (CPA) within the Ministry of Finance. The Monitoring Committee of the ROP discusses and eventually endorses the evaluation criteria. Eligibility guidance is clear and provided through the Implementation Framework Document (See Annex 2). For each priority axis and key area of intervention there are clearly defined activities, eligible applicants, target groups and target areas, timeline for submission, and technical and financial evaluation criteria.  Applicant Guides (AGs) are designed by the MA-ROP, in consultation with RDAs, and are also endorsed by MC-ROP. Based on interviews from the “MA-IB Collaboration and Beneficiary Support” World Bank project, beneficiaries appreciate AGs as very useful and comprehensive. They include detailed information on the application process, procedures, timeline, and requirements, including contact information at the MA and IB levels. AGs remain a key instrument for assisting potential and actual beneficiaries. 29 Figure 8. General scheme of the strategic framework for 2007 – 2013 ROP projects National Development Program Authority for Coordinating the Structural Instruments (ACSI) – Ministry of European Funds National Reference Strategic Framework Authority for Coordinating Structural Instruments (ACSI) – Ministry of European Funds Regional Operational Programme Regional Development Plans Managing Authority – Ministry of Regional Regional Development Councils Development and Public Administration Regional Development Agencies Framework Document for Implementation Managing Authority – Ministry of Regional Development and Public Administration Applicant Guides for every Priority Axis Management Authority – Ministry of Regional Development and Public Administration Project Development Eligible Applicants 42. In the FIFO selection model, a major problem in the strategic guidance for project development is that neither the ROP’s programming, nor the Regional Development Plans, reflect the specific priorities and needs of each region. RDPs are driven by EU-funded operational programs (OPs), including the ROP, for which the strategic directions are set at the national level in the form of financing 30 axes and KAIs. Indeed, there is little flexibility for regions to tailor the ROP funding to the specific local/regional context and the only dimension that varies across regions is the absolute monetary allocation of funds (to the benefit of less developed regions, as noted earlier). Consequently, ROP projects do not necessarily contribute to the areas of highest value for money and do not always generate the highest potential impact for their regions. Moreover, RDPs are generally ineffective due to the following reasons: first, they tend to be vague, with too many priorities and objectives; second, there is a lack of measurable targets and indicators for outcomes and impact; third, activities are not always clearly linked to the objectives; fourth, RDPs are not mandatory to implement and there is no system to monitor their progress; finally, the lack of a formal regional administrative tier implies that it is hard to hold stakeholders accountable for reaching RDP goals and to ensure the much needed coordination for successful project implementation. In some sense, project identification appears more geared toward meeting ROP requirements rather than achieving actual development needs. 43. As for the dedicated funds model (PA 1.1 for GPs and UDPs), investment guidance relies primarily on the Integrated Development Plans developed by each city. The pre-determined funds allocation may encourage planning and integration of investments by applicants. It is also good for growth poles and urban development poles to have a project pipeline with guaranteed financing. However, the pre-allocation of funds to each applicant creates incentives for delays in project submission and lower-quality projects. The idea that money will wait for an applicant for the duration of the ROP cycle (depending on the “n+2” or “n+3” rule) creates disincentives for quick submission of project applications. Moreover, guaranteed financing of projects passing a certain threshold may make applicants focus less on quality. At the same time, there has been a tendency for core cities to absorb most of the available funding, without properly investing in surrounding localities. In fact, reaping the benefits of concentrating investments in a few economic engines (i.e., growth poles and urban development poles) depends on whether planners and local authorities use the funds to further territorial integration and improve connectivity. But the current ROP selection model does little to ensure that investments within metropolitan (functional urban areas) are truly integrated. Project Development 44. The Framework Document for Implementation (FDI) and the Applicant Guides (AGs) play key roles in guiding project development, regardless of the applicable selection model. Both documents are comprehensive and transparent. But when developing proposals, applicants generally tend to focus more on how to get projects qualified for funding rather than on how to solve specific problems or address communities’ needs. Preliminary Screening 45. Preliminary screening is absent from both the FIFO and the dedicated model. Regarding the former, the Regional Committee for Strategic Evaluation and Correlation (RCSEC) used to screen projects in each development region for strategic alignment to local needs, but the role of this body has become limited due to several flaws in its design. The strategic screening by the RCSEC was conducted only after the technical and financial evaluation of projects, so it was not a preliminary test per se, meaning that some projects could go through the whole processing system before the RCSEC’s decision on whether they should receive financing. The body also lacked: clear criteria and procedures for member selection; a balanced structure for member composition; clear criteria and detailed procedures for strategic prioritization. In this context, the RCSEC’s mandate was limited to a purely consultative role; in practice, this meant that the committees would no longer be functional. Annex 4 provides a more detailed analysis of RCSECs. 31 46. As for the dedicated model applicable to PA 1.1 for growth poles and urban development poles, projects must be in the closed list attached to an Integrated Development Plans (IDPs) prepared by each applicant. Generally, however, the IDPs were not well-prepared: problems were poorly defined and not supported by clear evidence; there were also complicated strategic chains, meaning that it was hard to transform strategies into actions; finally, as with RDPs, there is a lack of clear prioritization and integration. Annex 5 is dedicated to a more complete analysis of IDPs and a full analysis of current shortcomings is available in the IDP study, part of the World Bank’s Growth Poles assessment (2013). 47. At the same time, coordination at this early stage, albeit of critical importance, is generally lacking. For one, it is deficient among policymakers at central and local government levels. For example, local governments in the West development region got funding from the ROP and completed the modernization of some schools that were later closed by the Ministry of Education. Also, there is little coordination among OPs and the projects they finance. A road funded under the ROP may be completed on time and then the pipes underneath may need to be replaced as part of a project under OP Environment. Such situations are relatively common across all regions in Romania, reflecting the need for improved integrated planning. 48. Last but not least, the use of indicators in planning and project development is either lacking or of poor quality. Applicants are under the impression that the more they include in that section, the higher the likelihood of their application getting approved. They can then be held responsible for reaching these goals, even though sometimes they are completely unmeasurable or unrealistic. Instead, applicants should be encouraged to select from a pre-defined list of indicators, aligned with program- and axis-level targets. At a minimum, chosen indicators should abide by the “SMART” set of criteria: specific, measurable, attainable, relevant, and time-oriented. There is also a lack of consistent use of output and performance indicators that can be used to screen relevant projects. Formal Project Appraisal 49. Internationally, there are different meanings and definitions of a prefeasibility study, but the most common one is that it contains everything in a feasibility study except the level of certainty on the parameters based on currently available data and estimates. The key issue is identifying major missing data that need to be acquired and the main risks and design alternatives. Where projects are more common, such as road rehabilitation work, then there may be no significant difference between a prefeasibility and feasibility study. In Romania, all investment projects, independently of the financing source, should have standard technical documentation, set by the Government Decision (GD) 28/2008. Annex 8 provides further details on the content of prefeasibility and feasibility studies according to Romanian regulations. The results for comparing this structure with the international practice are summarized in the table below. 50. The components of project appraisal in Romania, as defined by Government Decision (GD) 28/2008, are generally similar to international practice, but the major problem is that the quality of the technical documentation submitted by applicants is general low. At one level, there is a lack of needed in-house experts within local governments to prepare good projects. Contracting out all the skills related to project preparation also proves to be problematic. For example, it took much time for some RDA staff to train consultants hired to assess projects and the value of the training is often lost because each project has a different consultant. At the same time, local governments lack the capacity to verify the quality of technical documentation and, what is more, do not hold project verifiers accountable for approving poor-quality outputs. 32 Table 8. Comparison of appraisal components between Romanian and international standards Prefeasibility Study Romanian Feasibility Study Romanian Key Components Recognized Standard Key Components Recognized Standard Internationally GD 28/2008 Internationally GD 28/2008  Data gathering (geographic,  Compilation of all relevant Yes Yes climate, socio-economic, and data technical)  Project alternatives  Alternative technologies Yes Yes for project  Major risks (including institutional  Detailed estimate of costs and budgetary) No and benefits for a selected Yes alternative  Comparison of alternatives  Preliminary design (engineering, socio- economic Yes No costs and benefits)  Detailed risk assessment  Recommended project alternative Yes Yes No (partially  Preliminary estimate of project Yes (without  Detailed sustainability covered by costs and benefits benefits) assessment CBA)  Regulatory requirements  Environmental impact Yes Yes assessment  Identifying missing information for  Social impact assessment No No Feasibility Study 51. If internal capacity is low, high-performing external resources are also hard to access. Available appraisal consultancy services vary greatly between large cities and small towns, which affects the quality of the technical documentation. Large cities have a better access to good consultants because there are simply more consultants with better qualifications available and large cities also have a higher financial capacity for hiring them. In addition, the market is fairly opaque. There are no easy ways to determine whether a consultant is good and many beneficiaries get stuck with poor services, which is the reason behind two recommendation included in the report on ROP beneficiary assistance: namely, the creation of a transparent, accessible platform for peer reviews of consulting services; and the publication of the names of consulting firms corresponding to various projects financed by the ROP by adding these data to the database of beneficiaries currently available on the MRDPA’s website. 52. A related issue is the underfunding of project preparation costs, which is deeply tied to the Eligible Expenses Order (EEO) and currently applicable public procurement norms.12 For the ROP, the list of eligible expenses is defined through a common order (“the Eligible Expenses Order” – EEO) by the Minister of Regional Development and Public Administration and the Minister of Finance.13 The current EEO establishes a maximum cap of 5% for expenses related to project design and technical assistance (Article 3), which has contributed to the low quality of technical documentation and construction work. 12 The need for appropriate project preparation costs was discussed in more in-depth in the MA-IB Collaboration reports. It was indicated there that making project preparation costs eligible for EU funding is critical in order to enhance absorption (e.g. fewer preparation and implementation delays) and to improve impact (though better quality projects). 13 See, for instance, “Monitorul Oficial al României,” No. 327, from May 11, 2011 33 International benchmarking suggests that this artificial cap should be eliminated or raised to at least approximately 10% of the project’s value. In addition, procurement criteria are mainly based on the “least cost” criterion and do not include strict standards on minimum qualifications of independent evaluators. In theory, going beyond the lowest cost for selecting a winning bid is allowed, but in practice beneficiaries note that the NARMPP has been reticent to accept technical criteria of selection. The World Bank 2013 report on ROP 2.0: Facilitation of Proactive and Direct Support for Applicants and Beneficiaries of the Regional Operational Programme 2014-2020 goes into greater detail on these issues and provides specific recommendations for improvement. 53. Two additional factors influence the quality of technical documentation: first, the FIFO selection model itself; and second, the failure to incorporate operation and maintenance (O&M) costs. Under the FIFO rule, applicants rush to submit applications before funds are depleted, so some projects are of minimum quality, just sufficient to pass the necessary threshold to obtain the financing. Also, the sustainability of operation and maintenance expenditures is not always taken into account. Local governments have a limited capacity to assess the impact of operational expenditures generated by investment projects on the medium and long-term local budgets. There is also a weak capacity of the local governments to support O&M needs because there is no multiyear local budgeting planning. For their part, independent evaluators have had difficulties in assessing the real financial capacity of the applicants and make the connection between financial statements and the actual budgetary situation. 54. Finally, cost-benefit analyses (CBAs) are required but not relevant for all projects under the current ROP, and their quality remains generally poor. According to the technical and financial evaluation grids, all qualified projects must have an internal rate of return (IIR) equal to or above 5.5% and the benefit-cost ratio equal to or above 1. This requires that all project applications must complete a CBA, which exceeds both Romanian and EU standards that normally require that only projects of over 25 million EUR in the environment sector and of over 50 million EUR in other sectors have to complete a CBA. For smaller projects, conducting a proper CBA can take up a significant share of the available budget. At the same time, CBAs are particularly difficult to conduct for projects in the social, education, and health sectors; more generally, the data that are often needed to justify assumptions are lacking or remain hard to access by applicants. Furthermore, CBAs do not include alternatives and, given that both the FIFO and the dedicated-funding selection models remain non-competitive, the tool itself is of limited use since there is no prioritization of projects based on CBA results. In some cases, the results themselves are manipulated to obtain the required ratio to get projects qualified for funding, and few evaluators really question the calculations and the assumptions behind certain numbers. In principle, CBAs for small projects should be substituted by a limited number of performance indicators of outcome, cost-efficiency, and/or cost-effectiveness. Independent Review of Appraisal 55. Broadly speaking, the quality and duration of project evaluation depends on evaluation procedures, evaluators, inputs from the application, and the evaluation criteria. In this sense, the assessment of ROP project proposals is equivalent to the independent review of appraisal, as understood within the World Bank PIM framework. The evaluation is done by IB staff and independent evaluators, and eventually reviewed and approved by the MA. IB staff verifies the administrative conformity and eligibility. Independent evaluators conduct the technical and financial evaluation. There are detailed procedures with deadlines, evaluation criteria, evaluation grids, scoring guidelines, and “audit tracks” (piste de audit). All these are prepared by the MA in consultation with IBs and can be updated during the ROP implementation, if deemed necessary. 34 Evaluation Procedures 56. Regulations on evaluation procedures are detailed, transparent, and published in the Applicant’s Guide. Therefore, all potential applicants know how their project applications will be assessed, which is a positive feature of the system and one that should be maintained for the 2014-2020 programming period. 57. At the same time, there have been significant delays in the evaluation process, as shown earlier, particularly during the last few phases (technical project verification and contracting).It is true that much time is lost in administrative arrangements between evaluation steps. For example, a report by RDA West shows that the average duration for the evaluation process was 272 days, of which 136 days was spent at the MA level, 51 at the RDA, 8 days with independent evaluators, and 77 days with applicants. This time lost due to administrative arrangements suggests that the further decentralization of tasks is needed, in line with different project types and sizes (specifically, smaller, simpler projects should be mostly handled by RDAs, with minimal MA involvement).Delays have also resulted from the fact that in 2012 the MA could not contract evaluation firms for nearly one year because of NARMPP concerns about selection criteria. The temporary solution was that the MA delegated this task to RDAs, which could complete procurement procedures faster and with fewer constraints (per the smaller value of the contracts put out for bidding). Last but not least, an issue that is widely covered by the “ROP 2.0: MA-IB Collaboration and Beneficiary Support” reports is the system’s bureaucratization and heavy reliance on hard-copy documentation. The switch to an e-ROP would generate significant benefits in terms of lowering costs and increasing processing speed (i.e., reduced need for storage, easier filing and retrieval, etc.). Evaluators 58. There is a lack of good quality evaluators due to the fact that procurement selection criteria generally emphasize the least cost and lack standards to ensure quality evaluations. Procurement contracts do not have criteria on minimum expertise and responsibility of evaluators. A complementary solution for improving the quality of evaluations and holding evaluators accountable for their work requires stronger contracts with proper incentives. For instance, some part of the payment could depend on whether the project is successfully implemented and delivers the promised impact. Unfortunately, there has not been a thorough, systematic analysis of the causes for why ROP projects fail during implementation. Such a study would hopefully generate a set of clear standards for evaluators, pointing to specific issues and flagging problematic areas. It would also enable an informal appraisal of different evaluators, allowing the MA and the IBs to request first and foremost those who have already demonstrated their competence, while possibly disqualifying evaluators who have made grave errors in the past. A useful add-on to this recommendation would be to require all firms providing evaluators to have professional insurance so that any mistakes resulting from evaluation can be properly sanctioned and a sense of accountability can be fostered among all evaluators of ROP projects. Finally, there are also complaints that consulting companies sometimes offer low daily fees and thus cannot attract highly qualified evaluators. This is because the MA is not legally allowed to regulate a minimum payment to evaluators in the procurement document. Inputs from application 59. The quality of project proposals/applications is generally low, as already noted in an earlier section. The cause is the lack of internal capacity and the inability of some applicants to contract qualified external experts and hold them accountable for results. Lower-quality applications also imply more work at the MA/IB level, as many clarifications are often needed to complete the project’s file with 35 all the required documents, ensure that objectives and timelines are clear and feasible, and generally reach a point where the proposal can be properly evaluated. One idea is to have simpler procedures for applicants to update or correct errors or add missing documents to an application, as long as deadlines are upheld and there are no major changes to the core project objectives and activities. Evaluation Criteria 60. The selection process has resulted in a rejection rate of 35% (as of June 14, 2013), which means that roughly a third of the proposals fail. The highest rejections rates are in PA 4.1 (business support structures) at 59%, PA 5.2 (tourism infrastructure) at 45%, and PA 4.3 (micro-business support) at 41%. The breakdown of rejection rates between various phases (e.g., between the first stage of eligibility and administrative conformity screening and the second stage of selection based on the technical and financial evaluation) was not available. 61. One general issue is that the evaluation criteria are not linked to regional development strategies, but only to overall ROP program objectives. The criteria for administrative conformity and eligibility verification and for technical and financial evaluation differ among priority axes/key areas of intervention, but not among regions. 62. During the first stage of eligibility and administrative conformity screening, the main problem is related to demonstrating applicants’ ownership, especially through cadaster documents. RDA officials noted problems with confirming the legal ownership of land and infrastructure, which has been a major cause for rejection of applications at this stage. Another problem is generated by some overlap in the criteria used for the eligibility verification and for the technical and financial evaluation, namely those referring to equality of opportunity, environmental protection, energy efficiency, and public procurement. 63. A first area of assessment in the evaluation grids refers to a project’s relevance and importance – or, put differently, its contribution to reaching ROP targets.  Strategic screening or contribution of the project to the realization of the ROP goals: For regional or urban integrated development projects, there is a lack of sector performance indicators for each priority axis to allow for objective screening and evaluation of projects. These indicators should be used in screening whether this is in the context of pre-screening of large integrated projects (as discussed further below), or in the context of implicit screening through points awarded for features of sector-specific urban or local projects. Where applicable, projects should be identified in the context of regional development strategies and plans, urban development plans, or other sector plans. For example, a specific road project should be included in a road development or rehabilitation plan of a particular local/county administration, so that the priority of the particular road segment proposed for financing can be assessed against that broader strategic framework. The selection of projects to be included in a plan should be prioritized in term of the same sets of performance indicators that are used to subsequently screen, evaluate, and select projects. Currently, there is no assessment of how a project aligns with regional development strategies and plans. Such an assessment would also increase the relevance of RDPs and enable greater collaboration and ownership among actors involved in the plan’s implementation.  Importance of the project to the region: The importance of the project is currently assessed by an economic CBA criterion based on a discount rate of 5.5%. However, decision-makers may consider dropping this, as it is difficult and costly to conduct CBAs in the case of public sector 36 health, education, and transportation projects, as well as for a whole range of smaller-size investments. Furthermore, the results of a CBA can be manipulated to meet or beat the threshold and it is difficult for an evaluator to audit the quality of a CBA without significant effort. This evaluation of the quality of any CBA bottom line estimate of net benefits is also made more difficult by the lack of detailed guidelines on how to perform an adequate financial and economic CBA for specific types of projects. An alternative would be to maintain the CBA, particularly for larger projects, while training applicants and holding evaluators accountable for assessing CBAs and validating applicants’ assumptions behind the numbers. Finally, the financial evaluation of projects is a good feature. However, private and public-sector projects should be assessed somewhat differently: for the former, the focus should be on financial feasibility, while for the latter it should be based on the need for grant and budget support to achieve viability and sustainability (including consideration of O&M costs).  Project benefits based on technical design: Currently, the quality of the technical solution is evaluated separately from the benefits or importance of the project. But, in many cases, the actual benefits expected to be derived will depend upon the exact nature of the technical solution. For example, for road rehabilitation or improvement projects, the increased speeds of traffic flow (or reduced travel times) (Criterion 1.3 for PA 2.1) and the gains in road safety (Criterion 1.2 for PA 2.1) depend upon the types of design features of the road included in the proposed project. Specifically, the nature of improvements in a particular road (surface, quality, camber, width, lane structure and markings, etc.) needs to be assessed first and only then should the scores be assigned for expected gains in travel time and safety. 64. A second area of assessment of selection criteria covers the quality, maturity, and sustainability of a project. The following four observations stand out:  Documentation coherence: This criterion is relevant, but the actual method for how the documents and data are evaluated should be adjusted. Currently, this criterion focuses on whether: the data are sufficient, accurate, and justified; the analysis of alternatives is comprehensive; there are realistic operating cost estimates and assumptions related to commitments; the corrections (related to externalities and fiscal costs) are correctly applied in the economic analysis; the estimates (general and itemized) are clear, comprehensive, and realistic; and the design is complete and aligned with the written portion of the application. Going forward, it is recommended that this criterion focus on assessing the coherence among the Application Form (AF) and its supporting documents, and the technical documentation (e.g., pre-feasibility/feasibility studies). Any changes to the main documents should be finalized at this stage to avoid adjustments – and possible penalties/corrections – during the stage of detailed technical design or worse, later on, during implementation.  Relevance and quality of the technical solution: Currently, this criterion is analyzed in terms of technical documents’ correlation and completion. It is recommended that the analysis concentrate on the proposed technical solution and its alignment with available sustainable and innovative solutions. The technical solutions should focus on the material and financial requirements for maintenance and operations over the future, and avoid merely replicating old, outdated designs. As proposed later in this report, as well as in the “ROP 2.0: MA-IB Collaboration and Beneficiary Support” reports, the evaluation of technical solutions by an expert should include feedback on ways to improve the technical aspects of the project. In addition to a field visit, this may also require granting bonus points to applications that have already identified improved solutions that are known to be cost-effective in specified 37 circumstances (e.g., types of road rehabilitations suitable for major urban roads versus minor urban roads versus major intercity highways versus minor rural roads). Extra points could also be available for other innovations that are expected to be cost-effective over the long run, with clear guidelines for evaluators on how to grant these additional rewards.  Capacity to implement projects: The current criteria focus on the technical and staffing capacity of an applicant organization to implement and manage a project over its life cycle. There is also a focus on the adequacy of the budget plus grants to cover both the ineligible and eligible costs of the investment. These are all positive features that should be maintained. The one area for improvement relates to giving proper consideration to operation and maintenance (O&M) costs to ensure continued service delivery over an investment’s lifecycle. In the case of private commercial projects, a business plan is required and should include a clear reasoning for why the project is financially feasible over a specified time horizon based on the net present value of projected cash flows. In the case of public sector projects, the cash flows costs of operations and maintenance over a specified time horizon need to be identified and the source of the available funds demonstrated. For example, the public entity needs to show that it can currently fund the operation and maintenance of its existing investments and is either running fiscal surpluses or is expected to have revenue growth or expenditure savings in future years to cover the added costs required to sustain a proposed investment. Chapters VIII and X provide further details related to this point. 65. All in all, there is a lack of differentiation of evaluation procedures and criteria applicable to different project types and sizes. Smaller projects should go through more straightforward evaluation procedures and less demanding criteria to avoid unnecessary burdens and delays in selection and contracting. Larger investments need more rigorous strategic screening and quality check to ensure their effectiveness and to avoid delays in implementation due to errors in project preparation and design. Project Selection, Detailed Technical Design, and Budgeting Project Selection 66. A project’s actual selection in the 2007 – 2013 ROP occurs when the MA decides whether a particular proposal gets funded. At this stage, the MA approves or rejects applications based on one clearly established criterion: a project with an overall score equal to or greater than 3.5 out of 6 points total for the technical and financial evaluation qualifies for ROP co-financing. Importantly, the current selection process is non-competitive, i.e., projects are not compared nor compete against one another based on their relative quality. The non-competitive selection has both advantages and disadvantages, as noted below. 67. On the one hand, the non-competitive selection has the following advantages:  It helped reduce outside interference with project selection by minimizing political bargaining over project portfolios. No project entered the selection stage without proper evaluation and no application was approved if it failed to pass the agreed 3.5 threshold.  In the absence of clear and effective regional development strategies and plans, the non- competitive selection was suitable for non-strategic projects because there was no pressing need to compare them against one another in terms of their impact. 38  It helped with the rapid, efficient selection of small projects where the competition in terms of relative quality is not cost-effective. Because of self-imposed constraints, the ROP has funded projects under 20 million EUR. Moreover, the ROP has many priority axes and key areas of intervention, and thus financial resources are divided into small fixed amounts. The existing 13 PAs/KAIs within 8 development regions have resulted in 104 fixed small allocations.  The non-competitive selection avoided deadlines in project submission and evaluation, which facilitated central management of human resources by reducing the variance in workload and minimizing the risk of over-burdening capacity during peak demand. It also reduced the selection workload by avoiding comparison among projects, i.e., the same evaluators did not have to review the full batch of proposals to compare them against each other.  It speeded up project applications by allowing projects to be selected on a first-in-first-out (FIFO) basis. This was suitable for getting ready-to-go projects closer to implementation more quickly. These are the projects that were already prepared before the launch of the 2007 – 2013 ROP, partially through the PHARE pre-accession program. Increased speed also resulted from incentivizing applicants to send their proposals before funds were at risk of depletion.  This selection type also gave priority, in some ways, to the applicants who were most proactive, engaged, and efficient, ceteris paribus. In other words, for projects of equal complexity and time needed for preparing the application, the beneficiaries who moved faster got funding first. 68. On the other hand, the non-competitive selection has the following disadvantages:  It did not allow for a selection of projects based on regional development priorities, which limited the effectiveness and impact of the ROP. This is because the selection did not compare projects against one another in terms of alignment with regional development goals. Moreover, the technical and financial evaluation criteria did not include points for fit with regional strategic priorities.  The model did not ensure the highest value for money of funded investments, which limited the efficiency and impact of the ROP. This is because the selection did not compare projects against one another, meaning that, in principle, a project of lower impact and quality could get funded at the expense of better ones coming through the pipeline later on, simply because the former submitted the application before the latter. This scenario is particularly likely given that lower- quality, lower-impact projects tend to take less time to prepare and submit.  Projects were only required to pass a relatively low threshold (i.e., 3.5 out of 6 points), so the key differentiating criterion was often the duration of putting together an application. As such, there were few incentives to submit high-quality projects. Actually, projects rushed into application submission were often of low quality due to a higher risk of errors and weaker technical solutions. Consequently, the current ROP has funded a relatively large number of small projects of lower quality, which slowed down the effectiveness and efficiency of the selection process, and caused delays and other issues in later stages, including project design and implementation. Detailed Technical Design 69. After projects are approved for funding, they enter into the detailed technical design (DTD) stage, also called “the Technical Project.” The applicant is required to prepare the DTD within a maximum of six months and then submit it to an IB for verification, including site visits. There have been delays in DTD completion due to the low quality of projects at the feasibility study stage. The quality of DTDs is also generally low, which has caused further delays in project implementation, slowing down the ROP’s absorption rate. The low quality of DTDs is due to a number of reasons. First, there is a shortage 39 of highly qualified designers (civil engineers). Second, designers are not held accountable for design flaws, although increasingly beneficiaries have learned to draft stronger contracts that penalize mistakes by designers, including by requiring professional insurance schemes. Third, local governments generally lack in-house technical capacity to verify the quality of a DTD, and “project verifiers” (external technical experts) who are supposed to be contracted by the beneficiaries themselves are often paid by the designers, so they have little incentive to perform a thorough review. For their part, evaluators typically check only the administrative conformity of the DTD. Budgeting 70. After the DTD’s verification, the financing contract is prepared and signed by the MA, the IB, and the ROP beneficiary. The MA has had the good practice of over-contracting available funds for particular axes or KAIs. The total value of funded projects can exceed available funds by up to 20%. This is because there is a high probability of savings/corrections during the implementation of projects. The adjustments come from financial corrections, non-spent project budgets, and under-contracted procurement. 71. As another area for improvement, it is worth noting that the budget for each project is typically too detailed, using segmented quantities of materials and prices (“indicatoare de norme de deviz”) versus the more common consolidated items (“articole comasate”). This, together with generally low quality DTDs, has resulted in too many budget amendments during the project implementation phase, along with frictions between project beneficiaries and their contractors, which caused further delays in project completion and slowed down the absorption rates for the ROP. The “ROP 2.0: Beneficiary Support Mechanisms” report discusses this issue in greater detail. 40 IV. ROP Practices in Select EU Countries 72. As part of the current study, several EU Member States – Germany, Poland, Estonia, Italy, Slovenia, the United Kingdom, Ireland, and Lithuania – were chosen for the purpose of reviewing comparable models for project selection and summarizing best practices and replicable lessons for Romania. These countries were selected because of various reasons: (1) they have a good absorption rate and are among the star performers among new Member States (e.g., Estonia, Lithuania, Poland, and Slovenia); (2) like the ROP, they have well-developed programs targeted at cities (e.g., Italy, Poland); or (3) they use innovative approaches to project selection (e.g., Germany). The UK was chosen because its project identification, appraisal, and selection tools within the overall PIM system are often considered to represent international good practice. Ireland has been a long-term beneficiary of EU funding, is regarded as a star performer, and it has used the opportunity to consistently improve its PIM system, enabling it to take greater advantage of opportunities available. It is also consistently referenced as a success story for its recent, enduring growth and speedy recovery after the crisis. Table 9: Overview of Selected Case Studies Country / Operational Programme Selection model Intervention theme Estonia – NOP for the Development of Open calls Business infrastructure Living Environment Germany – Nordrhein Westfalen RCE ROP Competitive calls Business technology and research infrastructure Italy – Lazio RCE ROP Competitive calls in the context Integrated urban development of Integrated Local Development Plans Poland – Śląskie Convergence ROP Pre-selected projects in the Urban revitalization context of Sub-Regional Development Programs Slovenia – Strengthening Regional Competitive calls Business technology Development Potentials OP United Kingdom – West Wales and the Open calls in the context of Sustainable infrastructure Valleys Convergence ROP Strategic Frameworks investments Ireland – Border, Midlands and Western Open calls Regional development Regional OP Lithuania Strategic pre-allocation Economic growth 73. The purpose of this case study analysis is to do a wide sweep of available models within the EU and identify lessons and best practices that may be applied in Romania. The idea is not to patch together proposals and recommendations from models used by other countries, but to identify solutions that are more efficient and better suited for the Romanian Regional Operational Programme. For example, most of these countries have project selection models that are much simpler and more streamlined than in the case of Romania. They also provide examples of how more strategic and impactful projects could be selected. 41 74. Individual case studies are discussed in more detail in Annex 10. To the extent possible, a similar analysis structure was followed for all case studies, although data availability in some countries did not always permit this. A synthesis discussion on these case studies is included below and references to these case studies are made throughout the rest of the report. Annex 25 includes an interesting analysis on the new European Commission requirements regarding project selection for the 2014-2020 Programming Period. Synthesis of Case Studies 75. One of the first things to notice from these case studies is the fact that all countries analyzed here take different approaches for selecting EU funded projects. There are as many evaluation and selection systems as there are countries. There are, however, many common points as well. For example, most analyzed countries use a combination of competitively selected projects, pre-selected projects, or projects selected on a first-come-first-serve basis. Some countries, such as Lithuania or Poland, have leaned stronger toward pre-allocation and a focus on strategic projects, whereas countries like Germany or Slovenia place more value on competitive selection. It is important to note that strategic frameworks (at the level of geographies or sectors) are in several cases a defining feature of the selection model, notably in Italy (Lazio), Poland (Śląskie) and the UK (Wales). All of them provide lessons regarding the efficiency and effectiveness of project selection models that could be useful for Romania as it plans the next programming period. 76. Another thing to note is that Member States have different priorities for funding and this is reflected in the make-up of what comprises the Operational Programs (OP) within each NSRF. For example, in Ireland, environment projects are included in the ROP, whereas in Romania there is a separate OP Environment. In Slovenia, the OP Environment also includes Transport. And Lithuania does not have an ROP at all, due to its relatively small size, as explained earlier. Therefore, true “apples to apples” comparison is almost impossible. The paragraphs below cover the most relevant themes emerging from this comparative assessment. Project Size and Relevance to PIM 77. The relevance of the PIM approach varies when it comes to the different selection models and project types under review. In most cases, there is a close thematic link with the activities undertaken under the Romanian ROP, notably regarding major (pre-selected) projects in the field of urban development and infrastructure investments. However, some activities within programs, such as training initiatives or business support, would certainly fall outside the scope of what might normally be defined as PIM, and can be more properly considered to be service contracts requiring advanced skills in procurement practices. Much of the contemporary PIM lexicon such as the ‘PIM eight must-have framework’ does not fit comfortably with these small-scale activities given that it refers to processes that are more commonly deployed on major projects. This is also why, for the purpose of this study, the PIM framework has been adapted to fit the specific context of the Romanian ROP. 78. Regardless of project size, ultimately the same basic principles apply: need and justification, objective setting in outcome terms/measurable indicators of success, option appraisal, and a comparative selection methodology. In fact, it should be noted that the UK government’s Green Book, which is commonly thought to refer to capital projects alone, actually refers to all spending decisions by government departments. The caveat is that, simply put, smaller-size projects should go through simplified procedures. In particular, running a required Cost Benefit Analysis (CBA) would be an unnecessary burden for all but the larger projects. Given the often highly devolved nature of spending 42 EU funds (except Estonia, Lithuania, and Slovenia), the skills and capacity needed for complex appraisals would be either missing or relatively expensive to hire on a consultancy basis, as the example of the 2007-2013 ROP has shown. 79. The most common form of value assessment in smaller projects is cost effectiveness, ultimately tested through a free and open procurement process. There have been significant challenges in this area in the Romanian context, but the country is not at all an exception. For example, in a report commissioned by the European Commission in 2012 regarding Lithuania’s use of Structural Funds, the authors stated: “The public procurement process remains the main risk factor for the financial implementation of OPs.”14 It should be stressed that, in the case of these smaller projects, the phrase “procurement practices” does not refer only to the administrative process of tendering, but should specifically include the careful alignment of objectives and outcomes, and contribution to the option appraisal. A useful indicator of how many interventions using EU funds are major public works versus smaller projects is to investigate how many were awarded as “works contracts” and how many were “service contracts.” Evolution not Revolution 80. It should be noted that whereas the UK’s modern PIM system has evolved over at least 30 years and Ireland’s over around 20 years, and other countries (e.g., Germany, Italy) also have well- established project selection systems, Romania’s PIM system is still in an early development stage. The need for reform has been recognized through a series of interventions, not least of which is the current engagement with the World Bank. That said, many of the basic components of a modern PIM system (with some notable exceptions such as performance-based ex-post evaluation) are already in place in Romania. This begs the question: why is the system not working as efficiently as in other countries, particularly Lithuania and Poland that have had more or less the same reform lead time as Romania, but have performed much better in terms of funds absorption? The portfolio of projects within the World Bank’s Romania Regional Development Program, along with other technical assistance work (including, recently, for the Ministry of European Funds, the World Bank report on “Improving the National Framework for Preparing and Implementing Public Investment Projects”), seeks to provide answers and solutions to precisely this issue. Motivations behind State-Budget-Funded PIM Projects vs. EU-Funded Projects 81. The motivations behind projects differ depending on the source of funding (i.e., EU vs. state budget). There is one important reason for this: at least in Member States that recently joined the EU, nationally funded PIM projects often have to compete for funding – there are more projects than funds. However, the persistently slow absorption of EU Structural Funds by (newer) Member States suggests that there are sometimes more funds than readily achievable projects. This might suggest that the competitive tension between projects competing for limited funds that should drive quality in project appraisal and selection might not be as effective in EU-funded projects. Applicant behavior is also likely to be distorted when the driving motivator is to “absorb” funds (see below). Short-Term Measurement of Success 82. The capacity to deliver or ‘readiness to go’ appears to be a key determinant in selecting projects. For example, in the case of Wales (UK), one of only two high-level selection indicators is known as “Certainty Factors.” For the 2014-20 period, Welsh authorities consider to identify key ‘backbone’ 14 Klaudijus Maniokas, Agne Miseliuniene: Expert evaluation network delivering policy analysis on the performance of Cohesion policy 2007– 2013 Year 2 – 2012 Task 2: Country Report on Achievements of Cohesion policy Lithuania. 43 projects to ensure certainty, but with scope to select more innovative and risky projects later in the period. In the Italian region of Lazio, a higher score is assigned to Local Development Plans that include projects with available or expected authorizations and certificates. This might be seen, on the one hand, to be a sensible measure to ensure that good projects that may incur potential delays (e.g., permits, land acquisition) do not crowd out other projects. On the other hand, this may also motivate officials to approve projects that broadly fit the criteria in order to improve the seemingly all-important absorption rate, as argued in greater depth below. Generally, short- or long-term indicators can be used to measure success. During a program period – of whatever source, EU, national-budget, or otherwise – the most common short-term performance measures are “on time” and “on budget.” While these are important considerations, of more significance is the longer-term measure of success: Did the project achieve its intended goals? Or, more realistically: To what extent did the project achieve its intended goals? For projects under implementation during a program period, those questions remain to be answered until an ex-post performance evaluation can be conducted based on a suitable period of stable operation or even after the completion of the operation. Therefore, the EU uses absorption rate as a form of on-time, on-budget measure to assess short-term achievements. 83. Absorption and impact are not the same. While in the short term a country program with an absorption rate of 40% might be seen as a poor performer relative to another country with a 50% absorption rate, a longer-term measure might show that the 40% was invested more wisely than the 50%. Caution should therefore be exercised in exaggerating the importance of absorption as a measure of success. The rush to spend before the money tap is turned off at the end of a programming period is clearly a risk factor in considering how well public money is invested. But the potential for distortion in the selection criteria because of this incentive structure, along with associated behavioral dynamics, should also be carefully considered. As of now, there is no readily identifiable correlation between the quality of public investment systems and the rate of absorption of EU Funds. Initiative Overload? 84. Notable in the Operational Programmes looked at in this study is an overwhelming array of policy initiatives, both at an EU and national levels. Correlation is challenging, if not impossible. Also, the high number of programs may act as a deterrent to would-be applicants and cause delays by those who prepare projects so that they can adjust them in order to “tick the boxes.” The question should be considered: Do too many policy initiatives ultimately choke-up project preparation and encourage a culture of box ticking in order to improve the chances of selection? As a case in point, below is a list of policy initiatives mentioned in the policy context in one of the Irish ROPs – and Romania faces a similarly long list for 2014-2020:15  Cohesion policy  Lisbon and Gothenburg processes  The European Employment Strategy  The Lisbon Agenda  The EU Sustainable Development Strategy  Lisbon Community Program  Regional Competitiveness and Employment Objective  National Reform Program  NDP 2007-2013 15 Border, Midland and Western Regional Operational Program, Supporting and Enabling Dynamic Regions EU Regional Policy 2007-2013 44  Barcelona Target and the Barcelona Objective  National R&D Action Plan  Strategy for Science, Technology, and Innovation 2006-2013  Program for Research in Third Level Institutions (PRTLI) and Science Foundation Ireland (SFI) Initiatives  National Spatial Strategy 2002-2020  Toward 2016 – The 10-year Framework Social Partnership Agreement, 2006-2015  The Enterprise Strategy Group Report (2004) ‘Ahead of the Curve’  National Action Plan for Social Inclusion 2007-2016 In less centralized settings such as the UK and Ireland, this is not a major issue but can still be seen as an obstacle to overcome. Alignment with policies may also be used in the scoring / selection grids, although the degree of alignment with policies is more important than full compliance on every count. 85. It is important to note that project selection is not always particularly closely linked to strategic objectives and that identifying strategic projects of high quality is not always an easy task. Looking at individual case studies, the competitive call approach used in Nordrhein-Westfalen has not allowed for sufficiently strong strategic alignment – it has been recommended that criteria should already be defined at the planning stage in order to make them more strategic. In other cases, targeted project selection can be hampered by objectives that are not clearly defined; also, the hierarchy of strategic documents is not always obvious (e.g., Estonia). In the case of Wales, it was found to be challenging to implement strategic targeting without becoming too prescriptive; and equally problematic was to find a balance between eligibility and strategic criteria. There is therefore scope to select more strategic projects no matter which selection approach is used. Top-Down, Bottom-Up 86. These policy initiatives and other high level strategies such as the Lisbon Agenda, as well as national government policies, can incentivize potential beneficiaries to look for projects that fit the bill. While some notable major projects such as roads or broadband infrastructure can be readily identified as national or regional priorities, many of the smaller projects take time to move through various stages of the process (from concept to implementation and post-implementation) due to the often bewildering web of interwoven policies. Therefore, good communications and, in particular, strong stakeholder consultation regarding the opportunities available and the strategies which they must address, are key for guiding potential beneficiaries. What follows is a strong bottom-up approach to project identification. It might be reasonable to assume that although stakeholder consultations can add time to the process, they can ultimately lead to better targeted and more relevant projects. This reasoning might not be as obvious in some countries where the pressure to absorb the funds is the dominant influence. 87. Across the EU, there appears to be a strong desire to delegate responsibility for project selection and implementation to local bodies. Spain is a typical example in that it devolves around 66% of capital expenditure to subnational bodies. Germany and Italy take a similar approach based on the decentralized management and implementation of Structural Funds programs. Even in the UK, where devolved capital expenditure is less notable, there is practically no involvement from any department of the central government in any of the decision-making regarding the identification, appraisal, and selection of projects financed from structural funds. Matching funds also come from devolved budgets. In Ireland, despite its relatively small size, other than nationally significant projects, the identification, appraisal, and selection are entirely the responsibility of a devolved regional body with the national 45 government taking almost no role other than laying down the strategy and ensuring legality in the way the money is spent. Therefore, the institutional distance between a particular problem and the proposed solution is a short one, an argument that is often made in favor of decentralization (including in Romania in the context of the government’s regionalization initiative). Poland has already taken steps toward greater regionalization with delegating the management of ERDF OPs to the regions, although the center still retains a significant degree of control. The exceptions to this observed norm in case study countries are Estonia, Lithuania (with the highest rate of absorption) and Slovenia, but the small size of the countries makes them more analogous to a region in any case. 88. In the case of Lithuania, a top-down approach is more common. 75% of funds are targeted at state and regional projects contained in national strategy documents. Other funding areas with a top- down approach to project identification demonstrate how easy it is to weaken the link between objectives and desired outcomes. For example, some 10% of the funds are targeted at R&D, despite the fact there is no institution (such as a National Research Council) with the required competencies to grant such funds on an objective basis. None of the local universities or institutes scores well in international rankings. This raises the question: are some priorities selected in order to achieve a certain input-based target, such as “amount of money spent on R&D,” rather than on the effectiveness of the program itself, expressed in outcomes/impact? If the priorities are overstated in a way that diminishes the possibility to properly appraise and implement them, this may imply that the projects proposed: are likely to be fewer in number; are likely to be weaker due to the lack of capacity to review and challenge them; and/or are likely to be weaker due to the pressure to select inadequate projects to meet an otherwise artificial or unrealistic target. 89. Lithuania also shows a keen tendency to rely on projects with dedicated, pre-allocated funding. In early 2007, the Lithuanian government had already pre-approved the implementation of some public projects whose planned expenditures would be reimbursed from structural funds. 65 such projects were in the implementation phase and two projects (the development of the Vilnius and Palanga airports) were already completed by 2008. This shows the willingness of the Lithuanian Government to begin using new EU assistance as soon as possible and it is has probably been a factor in its high absorption rate to date. In 2007-13, a large share of EU funds has been used strategically for pre- selected state and regional projects. This is in sharp contrast to the previous programming period, when almost all assistance was considered “opportunity financing” available to many types of applicants through open calls for proposals. This shift from opportunity financing to strategic delivery entails the risk of politicization at various levels and requires strong delivery bodies in the public sector. In order to respond to this new challenge, the Lithuanian authorities created a Central Project Management Agency. Simplifying Appraisal and Selection Techniques 90. There are different approaches to project selection in terms of involved procedures and criteria. Weighting and scoring techniques are, for example, used in the German Land of Nordrhein- Westfalen, Estonia, and Slovenia. In the Polish Śląskie Voivodship and the Italian Lazio region, projects are selected following mainstream procedures after they are “pre-identified” in the framework of urban development strategies. The systems usually involve several categories of criteria (program-level, priority-level, measure-level and, if applicable, call-specific). If projects achieve the same scores, they can be further prioritized in line with domestic objectives (e.g., in Estonia, preference is given to the project that receives the highest score for impact on the relevant measure objective; in Nordrhein- Westfalen, the project with the best score for sustainable development wins, etc.). Care needs to be 46 taken that the complexity of the appraisal system is kept as low as possible, in order for criteria to be properly applied. Put differently, the system should not be and it should not feel “over-engineered.” 91. The World Bank Working Paper No. 161, which discusses PIM in new Member States, argues that “the quality of project appraisal practices is difficult to assess accurately.” 16 The current assessment confirms this view. Of the countries analyzed, only the UK and Ireland have clear, transparent, nationally applicable, and consistently applied appraisal guidelines. With the exception of Lithuania, the actual selection of projects is most often left to local authorities, including the choice of selection criteria. There appear to be two broad categories of selection criteria in use: the capacity of a project proposal to contribute to the achievement of strategic objectives or policy initiatives; and the degree to which the project is ready to be implemented. As Working Paper 161 found, it is unclear what impact the use of various selection tools has on the actual choices made – they still leave plenty of room for discretion.17 92. Some countries apply different selection criteria for various project categories in the program. In Poland, strategic and administrative criteria (including drafting necessary documentation and the readiness for implementation) are identical for all potential beneficiaries of a given type of project. This means that selection criteria will differ from program to program and from priority area to priority area, but the rules within each category will be the same. While this allows the possibility of targeting precise criteria to meet specific requirements and objectives, it reduces overall transparency. Poland is also one of the countries that encourage a competitive approach to projects through calls for proposals with pre- announced criteria. Such exercises are highly comparable to procurement competitions and the skills required to run them effectively are similar. It should be stressed that this refers to the “good practice” interpretation of procurement rather than the heavily administrative-focused version practiced in Romania. 93. Wales offers one of the more interesting ideas for selection good practice. Since it also operates a competitive-based call for projects (except for the larger, pre-identified infrastructure projects), it finds that it has more proposals than funds, therefore a more managed system is used to guide proposals through the process, starting with informal discussions between the interested applicant and the MA. This can be useful in terms of ensuring that projects are first of all of general interest and that they are prepared in a high-quality, compliant manner. Support is provided throughout the process by the MA, which has created a position called the “Priority Controller” – an official whose job is to prioritize and rank proposals using a relatively simple selection matrix that allows for assigning a numerical score. Priorities can be ranked before a final decision, reducing the risk of interference. The challenge to achieve both absorption and effectiveness 94. Ensuring absorption requires active management through a range of possible techniques. Examples include a two-stage (pre-qualification-type) procedure applied in the German Land of Nordrhein-Westfalen, the Polish Śląskie Voivodship, the Italian Lazio region, and (as an option) in Estonia. This helps target interventions better and can lead to time-savings later in the project cycle, although care has to be taken that this does not become a cumbersome and time-consuming process (a one-stage procedure may be preferable for small projects or certain target groups, e.g., SMEs). Also, it is crucial to avoid overlap between the initial and the full assessment stages, making sure that nothing is 16 “Public Investment Management in the New EU Member States. Strengthening Planning and Implementation of Transport Infrastructure Investments,” p. 13 17 Ibid., p. 14 47 missed out that might jeopardize project selection / contracting at a later stage. There are also examples of using sanctions, for example in Śląskie where unused funding was to be reallocated to other project types, or of rewarding performance by setting aside a reserve for the best-performing Local Development Plans in the Italian Lazio region. In general, there is often a negative correlation between absorption and effectiveness, with high-impact projects often having long lead-times (e.g., Śląskie). 95. A crucial aspect in enhancing program effectiveness is linked to the quality of regional strategies. As discussed, a number of case study regions under review work in a federal / decentralized administrative setting (Germany, Italy, UK) or at least with a decentralized Structural Funds management (Poland), which means that their domestic strategies and OPs usually capture regional priorities and needs. Additional efforts to enhance coherence have been made in Śląskie (pre-selected projects for integrated urban development were found to have been more effective than other project selection approaches) and Lazio (where there has been a strong link between Local Development Plans and OP objectives). 96. Lastly, it is important to note that none of the countries analyzed have identified “perfect” evaluation and selection systems, and many face difficulties in the management of EU funds even after many years of experience. While there are no silver bullets, there is always scope for improvement, even if systems and procedures work smoothly. Also, in order to introduce or apply certain approaches or criteria the necessary pre-conditions need to be met in order for them to be successful. Recommendations on EU Good Practices Relevant for Romania 97. Romania needs to perform better than in the 2007-2013 funding period if it wants to increase effective spending of structural funds and, at a minimum, get closer to the EU average absorption for 2014-2020. However, political pressure from Brussels to improve its poor absorption rate will undoubtedly translate into some temptations to rush into projects that appear to be good options simply because they are “ready to go.” Classic effects of this include unqualified assumptions in the appraisal and the deferral of issues until the implementation phase. As argued repeatedly throughout this report, effective public spending is not necessarily the same as effective public investment. Adequate management controls with appropriate capacity to understand and implement them are the best way to prevent abuses. 98. There are a number of areas for improvement to the system of identifying, appraising, and selecting projects that can be drawn directly from the practices of better performing EU Member States. Although these can be mimicked in guidance, policy, and through regulation, they may amount to very little in substance if the implementing capacity is sufficient. The following recommendations are important to highlight:  Assess the nature of the projects in existing and future OPs – selection procedures will need to be appropriate to the nature of the intervention. It has become clear that there are essentially two types of interventions financed by structural funds: o The first type can be referred to as PIM-style projects: infrastructure-based projects that require prefeasibility studies and feasibility studies and need to address complex issues such as construction permits, environmental permits, land acquisition, and the like. They also need complex financial and economic appraisals. They can take years to develop, followed by a similarly lengthy period of implementation. Contractual and payment conditions can also be complex. The entire process requires a skill set that is 48 not always found in public administration. The projects are often pre-identified at a national or regional level and EU funding offers an affordable way to implement them. o The second type of intervention, typically involving a much larger number of projects, can be referred to as service contracts. These are projects that are more short-term and typically smaller in value (e.g., sometimes under 10,000 USD). They are much more likely to be based on an opportunity funding model where public authorities issue a call for proposals based on certain criteria. The respondents are generally smaller entities without the skills to engage in complex exercises, and therefore require close support in order for them to fully understand and comply with (ideally, simpler) regulations.  The selection methodology should be more aligned with the nature of envisaged projects, i.e., PIM-style investments or service contracts. Larger projects should be subject to a CBA appraisal, while smaller ones should rely more on cost effectiveness assessment. It therefore becomes difficult to compare, contrast, and therefore prioritize between the two types of projects. A way of comparing the relative values of all projects, regardless of their type, would be useful.  The nature of the institutions that support different types of interventions needs to be carefully considered. The skills required, although complementary, are quite different. The Romanian Government needs to consider options for achieving this significant capacity improvement. Selection techniques are only as good as the capacity to implement them. Some of the options for accomplishing these goals are discussed at the end of this section.  The overwhelming tendency is to delegate identification and preparation of projects to regional bodies. The institutional distance between problems and solutions is thus becoming shorter. Regional or local bodies are much more aware of the issues they face than distant central institutions. Projects initiated and developed at the local level are far more likely to have relevance and win the support of local beneficiaries and affected citizens than those promoted by centralized bodies. Having said that, regional and local entities need technical support to enable them to achieve their projects. Again, some of the options to do this are discussed below.  Romania should put project achievability at the heart of the appraisal and selection process. The Bank’s May 2013 Report, “Romania - Improving the National Framework for Preparing and Implementing Public Investment Projects,” identified this as an issue to be addressed and further pointed out that those charged with reviewing and challenging proposals seem to not take the issue seriously enough. The Gateway Review system in the UK shows that this consideration is core to the appraisal process itself, rather than a separate issue to be dealt with in the run-up to implementation or even further down the road. However, this can be a double- edged sword: on the one hand, it might be seen as a tool for rushing through projects that are easy to achieve but are of little value; on the other hand, when done within the context of a fully effective system, good projects should be assessed to check whether the necessary permits, licenses, approvals, governance arrangements, and skills are in place in order so that the project, if approved, can proceed smoothly.  The Romanian Government should also review the number of policies and initiatives that require compliance with in order to make a proposal acceptable. It should focus on essentials. 49 Also, there should not be pass/fail indicators, but the degree of compliance should be assessed and used as part of an appraisal and selection methodology. At the same time, policy initiative overload should be avoided. Too many high-level policies may simply lead to a culture of box ticking. The Bank’s May 2013 Report, “Romania - Improving the National Framework for Preparing and Implementing Public Investment Projects” found that the Romanian PIM system is burdened with a control rather than an enabling culture among regulatory bodies, a conclusion supported by the research undertaken as part of the “ROP 2.0: MA-IB Collaboration and Beneficiary Support” evaluation. This makes many officials focus on dogmatic compliance. In such an environment, a plethora of strategies and policies, which officials feel obliged to fulfill, is likely to lead to delays in preparation in order to make the projects “fit” the policies. The more policies a project must comply with, the greater the risk of conflict; it is not possible for all of them to be compatible.  Develop a good communications strategy that encourages the generation of good project ideas. In order to put as many minds to work on thinking up good project proposals (a form of innovation in of itself that is much aspired to by all), the strategies and objectives of an OP should be explained openly to as many stakeholders as possible on a regular basis, particularly at the start of a programming period. The possibilities opened up by the availability of funds should be understood by as wide of a range of stakeholders as possible. Some critics suggest that EU communications strategies are overly focused on marketing the benevolence of the Union instead of explaining how people themselves can be part of the process. A good communications strategy, if implemented well, should engage people at the local level to create and submit their own proposals. The more people are involved in this process, the more project ideas will be brought forward, and the more likely it is that good projects will be selected in a competitive setting.  Decision-makers in Romania should consider instituting the role of a Priority Controller, following the Welsh model. Certainly in the case of smaller interventions where a competitive, opportunity funding system might be proposed, the need for a standard selection methodology would indicate the need for an arbiter, possibly at the level of the MA, to ensure consistency of application and fairness in the assessment scores. The Priority Controller (PC) would also assess priorities in terms of overall affordability within the OP and, where there are more proposals than funds, the PC could select project proposals on the basis of best fit. The PC would also ensure that project proposals do not compete with, duplicate, or contradict any other interventions in the OP or elsewhere in the public domain in the regional or local area.  Where evaluation panels or juries are used (e.g., Estonia, Nordrhein-Westfalen), the transparency of selection processes is perceived to be high. Evidence from case studies shows that, in order to guarantee high-quality funding decisions, a number of conditions need to be fulfilled, including: careful composition of panels (members need to cover technical, economic and strategic aspects, any bias must be excluded, (some of the) members need to know the local context); and detailed guidance for panel members (scoring methods to be used, clear definition of indicators). In addition, the availability of all necessary documentation through web portals has promoted transparency and certainty with respect to selection procedures (e.g., Estonia, Italy).  Prepare projects in advance of the beginning of the programming period and approve match funding. Lithuania made a good start to 2007-2013 by having a number of strategic projects 50 already prepared and approved for match funding at the national level. This meant that once the program was underway, only the approval of the allocation from Brussels was required before beginning the actual implementation phase. This seems to be a logical and practical step to improve absorption rates in Romania too, particularly when larger PIM style projects are envisaged. It is worth noting that Romania tried to do something similar, preparing technical documentation funded by the central MRDPA in the pre-accession period – this backfired in some cases, as the technical designs were of poor quality and sometimes did not fit the reality on the ground. As a concept, advance preparation is desirable, as long as it is properly implemented and aligned with actual needs and priorities at the local level; otherwise, it will only lead to lengthy delays during the implementation phase. Options for Building Capacity 99. In particular, there are several options worth considering for strengthening the capacity of Romanian stakeholders to manage projects and especially PIM-style investments. Three in particular are worth considering: (1) create a Program Implementation Unit with appropriately skilled and experienced staff on a “task and finish” basis to provide support to beneficiaries in preparing and implementing projects; (2) hire more staff in the Managing Authority to fulfill the same role as in (1); and (3) hire external capacity at the level of beneficiaries and focus efforts on them becoming an intelligent customer of external providers, instead of trying to achieve all tasks internally. The paragraphs below summarize the pros and cons of these various options. 1. Program Implementation Unit (PIU) o Advantages / Benefits:  A dedicated unit focused on one main task;  Additional skills, not always found in the government sector such as project management, can be brought in. o Disadvantages / Risks:  Remoteness from beneficiaries;  Potential lack of focus that would otherwise be present if the tasks were the responsibility of the beneficiaries;  Perceived lack of ownership among beneficiaries and ultimate operators of the project;  Potential for resentment if remuneration is greater in a separate PIU (which it may well need to be);  Resistance to create additional institutions unless PIU would be already part of the MA, in which case it would be hobbled by existing pay-scales making recruitment difficult. 2. Hire more Staff in the Managing Authority o Advantages / Benefits:  Centralized human capital resource that can supplement the skills and local knowledge of the beneficiaries. o Disadvantages / Risks:  Remoteness from the beneficiaries;  Lack of ownership of beneficiaries / ultimate operators; 51  Difficult to pay additional staff outside normal pay-scales would make recruitment difficult and may also generate conflicts. 3. Hire External Capacity in direct support of the beneficiaries o Advantages / Benefits:  Where funds are available – good quality advice;  Direct and targeted support to specific projects;  Can be planned so that availability matches requirement. o Disadvantages / Risks:  Requires a procurement process that may delay the availability of that capacity, particularly in the event of legal challenges against award procedures;  High cost. Service Contract Projects 100. As for service contract style projects, the six alternatives presented below should be considered as options for strengthening the system’s capacity: 1. Employ procurement specialists in a PIU o Advantages / Benefits:  A one-stop shop for the preparation and implementation all projects related to EU structural funds;  Single point of reference for the MA, allowing greater and simpler management control. o Disadvantages / Risks:  Pre-supposes the existence of a PIU;  Would be seen as remote and out of touch with regional and local issues;  May face a lack of co-operation from regional entities;  Difficult to engage with local stakeholders in order to understand the requirements, possibly leading to weaker projects. 2. Employ (more) procurement specialists in the MA o Advantages / Benefits:  MA is able to exert full management control over the process and take responsibility for errors;  PIU would need procurement experts anyway in order to fulfill its role in PIM style projects. o Disadvantages / Risks:  Would be seen as remote and out of touch with regional and local issues;  May face a lack of cooperation from regional entities;  Difficult to engage with local stakeholders in order to understand the requirements possibly leading to weaker projects. 3. Employ procurement specialists in regional entities o Advantages / Benefits:  Brings the solution closer to the problem; 52  Full engagement at an operational level. o Disadvantages / Risks:  Requires a recruitment exercise;  Requires additional funding;  Difficulty in sourcing appropriately skilled people in the regions. 4. Train existing officials in the required procurement skills o Advantages / Benefits:  No need to hire extra people. o Disadvantages / Risks:  Temptation to simply add the responsibility to other existing duties, leading to unsatisfactory service and lack of focus;  Possible lack of interest in the subject matter among appointed staff;  Pre-conceived ideas may lead to subjective treatment of proposals;  Takes time and money to engage in a training program that could be meaningful;  Lack of previous experience means that mistakes in the early days would be inevitable, despite training. 5. Engage NARMPP to act as an agent o Advantages / Benefits:  Existing pool of knowledge on procurement rules and procedures;  Readily available institution with established authority on the subject. o Disadvantages / Risks:  It is not clear whether NARMPP has the legal authority to act as agent;  Relatively poor reputation of NARMPP may lead to local bodies refusing to work with them;  Suspicion of motives due to the existing ‘control’ mentality of some NARMPP staff. 6. Hire consultants to provide support when required o Advantages / Benefits:  Where funds are available – good quality advice;  Direct and targeted support to specific projects;  Can be planned so that availability matches requirement;  Largely insures against the risk of making errors that lead to legal challenges. o Disadvantages / Risks:  Costly;  Requires a procurement process that may delay the availability of capacity, particularly in the event of legal challenges to tender procedures. 101. In sum, one of the most important issues to address in designing a suitable appraisal and selection model for Romania is to quantify the nature of proposed interventions. As noted repeatedly 53 throughout this chapter, there are two types of intervention in ROPs across the EU: strategic level, PIM- style projects; and opportunity funding projects based largely on service contracts. The relative balance between the two types needs to be understood in order to operate effective programs. 102. It seems clear that, for larger projects involving infrastructure, support may be required either through hiring consultants or from a centralized support institution. But this should not imply centralized bodies taking on the responsibility for preparation and implementation. This was a situation found in the environment sector in Romania in the early part of the 2007-2013 programming period that led to institutional conflict, misunderstandings, and poor preparation leading to delays and low implementation rates. 103. For the smaller service contracts, there is no reason why, provided that the right tools and training are available, local entities should not be capable of identifying and implementing solutions to their own problems. This is a system that appears to serve well other EU Member States. The degree to which they require external support and the nature of that support remains to be determined in each case. 54 V. Basic principles for an efficient project selection model Balancing impact and absorption 104. It is obvious that while the impact of EU funded projects should play a bigger role in the next programming period, basic absorption considerations should not be ignored. More specifically, it does not matter much that projects are designed to be more impactful if funds are not properly absorbed to fund these projects. It is therefore important to start the discussion on absorption before talking about impact. 105. Romania has had, consistently, the poorest absorption rates of any Member State. In fact, Romania is some distance away (14 percentage points) from the second worst performer – Bulgaria. This obviously brings to the fore the need to improve absorption capacity in tandem with improving the impact of selected projects. Among other things, this means that better selection criteria cannot be designed without also thinking about streamlining processes and improving institutional capacity to implement projects. As was shown in the MA-IB Collaboration report, this also means dealing with a number of systemic issues (e.g., the public procurement system, technical documentation quality, etc.), which are not necessarily linked to the ROP. Figure 9. Percentage of EU funds paid by the European Commission, in June 2013 Source: http://ec.europa.eu/regional_policy/thefunds/funding/index_en.cfm 106. As such, the decision to focus on higher impact projects should not be taken without ensuring that the right mechanisms are in place to ensure proper absorption of EU funds. In the Romania Functional Review: Regional Development and Tourism, the World Bank argued that there is a need to balance abortion and impact within every programming period, while ensuring the adequate legal and institutional frameworks are in place to prevent systemic issues from affecting either proper absorption and/or impact (see figure below). 55 Figure 10. ROP: Balancing Absorption and Impact Source: World Bank. 2011. Romania Functional Review: Regional Development and Tourism 107. Consequently, for the 2014-2020 Programming Period, Romanian authorities, and Managing Authorities in particular, will have a hard task ensuring that absorption quota are met, while also selecting investments that will have the most significant possible impact. It is hard to make a strong case for higher impact, when absorbed funds sometimes do not even cover the country’s contribution to the EU. This also means that new project selection models have to not only enable a more strategic selection of projects, but also ensure that funds are absorbed at a fast enough rate. This may also mean that project selection models have to be adjusted according to the make-up of individual operational programmes. For example, OP Transport finances primarily large scale projects with a significant impact, while the Human Resources OP focuses primarily on smaller scale projects. The Regional Operational Programme, the focus of this analysis, finances a mix of large scale (strategic) and small scale projects. 108. For the 2007-2013 Programming Period, the Regional Operational Programme (ROP) was a top performer in terms of absorption rates, in the Romanian context. Of the 7 operational programmes it had received the highest level of reimbursements from the EC by July 2013 – both in absolute and in relative terms. Reimbursements had reached around 30% of allocated funds by July 2013 and continued to growth throughout the rest of the year, reaching 44% (nearly 50% higher) by the end of November 2013. 56 Table 10. Absorption of EU Funds by Operational Programme*(as of January 31, 2014) Allocated Contracts/ Payments to Value Current Reimburse- Share of funding for Financing Beneficiaries declared absorption ments from allocated Operational 2007-2013 Decisions to the rate the EC funds Programme EC reimbursed by the EC RON (bln) RON (bln) RON (bln) EURO % EURO (bln) % (bln) Regional 17.7 29.8 9.3 1.8 44,9% 1.65 41.5% Operational Programme Environment OP 19.7 32.9 7.5 1.2 27.1% 1.1 23.8% Transport OP 19.8 27.5 4.8 1.4 31.2% 0.88 19.8% Competitiveness 11.4 24.6 4.9 0.9 36.4% 0.45 17.5% OP Human Resources 15.5 15.3 8.1 1.0 29.2% 0.95 27.3% OP Administrative 0.92 1.3 0.44 0.1 50.6% 0.08 40.1% Capacity Building OP Technical 0.76 0.83 0.22 0.06 33.9% 0.04 20.7% Assistance OP 85.9 149.9 35.2 6.5 mld TOTAL (19.5 mld. (34.1 mld. (8 mld. EUR 33.5% 4.96 26.5% EUR) EUR) EUR) Source: Ministry of European Funds *Note: This table does not include sums absorbed through the National Rural Development Programme 109. Not only did the ROP have the highest absorption rates of all OPs, but it also had a high contracting rate. Basically, of the EUR 3.72 billion allocated for 2007-2013, ROP contracting reached over 110% of allocated sums. This is critical information when discussing the importance of project impact. Focusing on impact requires a competitive selection of projects, but a competitive selection is possible only if there is actual competition between projects. If the value of submitted projects is smaller than allocated sums, or if the quality of submitted projects is low, selecting impactful projects is a futile undertaking. Defining impact 110. Many reports and books have been written about “impact” and the topic would merit a separate treatment. Depending on what type of impact is sought, there are different approaches that may be considered for efficient project evaluation and selection. Economic impact may warrant a more pecuniary approach (e.g., Cost Benefit Analysis, Internal Rate of Return, annual cash flows, etc.); social impact may be focused more on the benefits accrued by poor people or communities in general; environmental impact may try to estimate net environmental benefits (e.g., green-house gas reductions). At a more general level, development impact can be measured through a number of standard indicators and methods (e.g., GDP per Capita, Human Development Index, Input-Output Analysis, Computable General Equilibrium Models, etc.). 111. The level at which impact is measured also plays an important role, both in the method chosen to measure impact and in terms of the effort spent on measuring impact. Obviously, more sophisticated methods and more time should be spent on the selection of large projects with a 57 proportionally large expected impact. For example, deciding which highway to finance requires that a proper transport and regional development strategy/plan is in place. For medium-sized projects, it may be more expedient and efficient to have a standardized evaluation and selection method, which allows a boiler-plate analysis for a number of projects – this could involve both batch evaluation (e.g., looking at Cost Benefit Analysis or financial sustainability over the expected life of the project), or evaluation by project type (e.g., separate analysis done for transport projects, for educational infrastructure projects, for business development projects, etc.). For small projects, which usually have a low anticipated impact and tend to come in large numbers, it may be most efficient to simply look at basic qualification criteria (e.g., eligibility and basic technical and financial evaluation) and to select them for financing based on the first-in first-out (FIFO) principle, rather than through a competitive selection. 112. Thus, depending on the level of impact expected, evaluation and selection methods can range from the elaborate and sophisticated to the rather simple. All in all, it is important that selection models apply different selection criteria depending on the size and anticipated impact of the project. As indicated earlier, large-scale high-impact projects would ideally have a local/regional/national strategy/plan at their foundation, which would clearly evidence how the proposed strategic project can help achieve local/regional/national objectives. For medium-sized projects, a competitive selection process would have to apply a standard evaluation and selection method, as there may be more requests for funding than actual funds. For small projects, a FIFO method may be the best option, as small projects tend to come in high numbers, they tend to have a low overall impact, and complicated evaluation and selection methods may simply slow the process down – and, by extension, reduce absorption rates. 113. A discussion about impact should also touch on the type of impact that is expected. Generally, economic theory talks about direct impact, indirect impact, and induced impact. A direct impact accrues directly in the sector addressed. For example, a new investment in connective infrastructure may shorten travel times between two cities by 50%. An indirect impact has a positive or negative effect on other related sectors. Thus, an improvement in transport infrastructure could help improve the business environment in the two cities by enabling firms located there an easier access to a larger labor pool. Finally, the induced impact relates to the positive or negative effect on non-related sectors. For example, the fact that the economy is doing better as a result of new infrastructure means that people in the region will have more disposable income, and they will spend it in local businesses (e.g., supermarkets, cinemas, restaurants, bars), which will provide another positive economic boost. 114. Of course, measuring direct, indirect, and induced impact in practice is easier said than done. In the US, there are companies that have specialized in measuring the direct, indirect, and induced impact of growth in particular economic sectors. The total impact usually tracks revenue growth and growth in employment. Such models are based on the Input-Output framework and are quite data intensive and cumbersome to put together. They also have a number of shortcomings that are relatively well-known. For example, such models cannot track negative effects of growth in one sector (e.g., the growth of the CD market has led to the virtual disappearance of the audio cassette market). Also, such models need to be calibrated on a regular basis to properly respond to changing economic conditions – an undertaking that is usually quite expensive and technical in nature. 115. Given that impact is so hard to assess even with the help of complex and well-established economic models, it becomes clear that assessing the impact of public investments is no easy task either. In fact, there are no foolproof models (at least none that the authors know of) for correctly assessing the impact of public investment projects, and the reason for this is quite simple: public 58 investment projects usually hope to achieve social, economic, and environmental outcomes that are usually provided sub-optimally (or not at all) by the private sector, and that usually are hard to quantify and measure. For example, it is hard to put a number on the economic and social benefits (as well as the negative side-effects) of a new road project. One could not tell with certainty how much of the economic growth the road project was responsible for, how many new jobs it helped create, of how many poor people managed to rise out of poverty because of it. One can look at concrete outcomes such as the improved travel time between two places, or the effective output of such an effort (i.e., a new road has been developed), but when it comes to impact per se it is challenging to isolate the pure effects of such investments from all other possible variables. 116. Indeed, outside factors (e.g., global economic performance, the performance of neighboring countries) are strong determinants of internal performance. The fact that the global economy, or the EU economy, is doing well, may be the chief factor in triggering a positive impact of a road investment project. Similarly, with the global economy is performing poorly, most public investment projects may have just a modest developmental impact. 117. The difficulty of assessing the impact of a particular project can easily be assessed by looking at development data for Romania. In the Competitive Cities report, the team made use of a Local Human Development Index (LHDI), developed by Romanian sociologist Dumitru Sandu. The LHDI maps below show the development performance of individual localities in Romania from 2002 to 2011. As can be seen, there is a strong geographical pattern to development in Romania (as in most other countries in fact). The localities that have developed the most are those that are close to the West (where 70% of Romanian exports go), those that have large demographic and economic mass (e.g., București), and those that are close to places with large demographic and economic mass. Annex 24 (and 24A, 24B, and 24C) describe in detail the LHDI methodology and, moreover, how it could be deployed as part of the ROP’s monitoring and evaluation (M&E) system. Figure 11. The Local Human Development Index at the locality level, for 2002 Data source: Dumitru Sandu Note: The blank spots indicate localities for which no data were available 59 Figure 12. The Local Human Development Index at the locality level, for 2011 Data source: Dumitru Sandu Note: The blank spots indicate localities for which no data were available 118. A quick glance at road investment projects indicates that there is no real correlation between the areas where new roads have been built and the areas that have registered the most significant development in the past decade. Moreover, as the maps below indicate, some of the areas that would benefit most from investments in road infrastructure (e.g., Timiș County) still have a relatively low density of roads, whereas some counties where market dynamics are less strong (e.g., Botoșani) have both a high road density and have benefited from significant investments in road infrastructure. Figure 13. Investments in road infrastructure do not seem to have a real developmental impact Data Source: National Institute of Statistics 60 119. This is the first lesson in devising proper project selection and evaluation models: public investment projects should attempt to enhance positive market externalities, while mitigating negative market externalities. If we were to use the example presented above, market dynamics seem to indicate that the highest need for road investment projects is in the West of the country and around the larger cities in Romania. Consequently, one prioritization factor for new road development projects could be the ease of access they enable to Western markets and the markets of large Romanian cities. Understanding the entire complexity of a project selection model 120. When devising a project selection model, it is important to keep in mind that every project has a particular impact. The construction of a new road may not necessarily generate an economic boost in the area where it is implemented, but it may have a positive impact nonetheless – even if just a modest one. A new road project employs a number of local people and generates additional revenues for some local companies (e.g., concrete manufacturers). The key task for the effective selection of projects is to assure that not only the direct impact is maximized, but also the indirect and induced effects (which are often harder to measure). 121. It is also important to be mindful of the potential negative impact of a project. Not every new investment is necessarily an economic boon. For example, a new road project, like the one we talked about above, may in the long-term prove to be a drain on local public resources, requiring high maintenance costs. As such, a project that was expected to have a positive development impact may in fact have a negative effect in the long run. This is why it is critical for new public investment projects to be mindful of existing market dynamics. And this is why the elaboration of a project selection model requires a good understanding of the entire complexity of project selection models. 122. Every good project selection model needs to have a good strategy at its foundation. The purpose of a strategy is to identify ways of harnessing positive market externalities and addressing negative market externalities. A good strategy will provide the back drop for preparing a program (e.g., an operational programme), with a set of clear objectives and a specific action plan. The latter would include a concrete list of desirable projects, each with a list of necessary inputs attached to them. Inputs would in turn generate a number of outputs (e.g., a new road of a number of kilometers), which in turn aim to achieve a number of outcomes/results (e.g., faster travel time between two economic centers). Finally, the achieved outcomes/results would help generate a strong impact (e.g., local/regional/ national development). 123. Moreover, this chain of actions should be provided with appropriate feedback loops to determine what has worked and what has failed. These feedback loops are also essential for continually improving evaluation and selection models, as further explained in Chapter XI. Thus, the objectives defined should be relevant for the needs/problems identified in the sector strategy. The inputs should be devised efficiently (e.g., with the lowest possible costs for the highest possible benefits) to achieve quality outputs. In turn, the achieved outcomes/results would be a measure of the effectiveness of set objectives, while the achieved impacts would serve as a measure for the utility and relevance of the sector strategy. Obviously, a weak link in this chain of actions may negatively affect the whole system. 124. The implications for the Regional Operational Programme 2014-2020 are that a sound regional development strategy should be in place before devising the programme itself. As of October 2013, the Ministry of Regional Development and Public Administration was still working on its regional development strategy. Consequently, there is no back-drop to use in this analysis. Given that there are 61 no clear objectives defined for the new programming period, for the purpose of this analysis we will draw on a number of key strategic documents, prepared by the World Bank, the European Commission, and the Romanian Government. The next chapter focuses specifically on these issues. Figure 14. A framework for devising an optimal monitoring and evaluation system for project selection Source: European Commission.2006. “Indicative Guidelines on Evaluation Methods: Ex-Ante Evaluation”. 62 VI. Setting the foundation for improved project selection models in Romania Strategic Priorities for Romania’s Economic Development 125. There is wide agreement that Romania has a unique potential for achieving sustainable and inclusive growth, but there is no consensus on a specific set of strategic priorities for the country’s development. In fact, particularly in recent years, policymakers at both the EU and the national level have prepared an overwhelming number of plans, strategies, position papers, memoranda, communications, and other “key documents”. In many ways, thorough planning is a positive feature of the system and results from a justified desire to achieve the most impact with the least resources committed or, in the spirit of a recent concept promoted by the EU and the broader international aid community, “maximize value for money.” This inclination very much applies to the European Commission, as the party accountable for how structural funds are spent and for the results they deliver. It also holds true for the Romanian Government, which has limited budgetary resources relative to the country’s development needs at this point in time. Nevertheless, having too many strategies can backfire in the absence of close coordination and prioritization of objectives, making it harder to design and implement a coherent vision for development based on actual needs on the ground and expected impact of potential interventions. In addition, enforcing accountability mechanisms and carrying out proper monitoring and evaluation can be challenging without clearly defined strategic directions, goals, activities, and indicators. 126. As it relates to the current work on project selection models for the Regional Operational Programme (ROP), the issue of defining and operationalizing Romania’s development priorities is quintessential. Indeed, it would be impossible to decide on a typology of projects or to recommend a particular set of evaluation and selection criteria without agreeing, ex ante, on the overall policy directions. To limit the scope of this engagement to a more manageable task, it is useful to note that the Ministry of Regional Development and Public Administration (MRDPA) focuses on a specific range of issues – they are critical aspects of Romania’s development, but do not cover every single sector that may contribute to this endeavor. More specifically, based on Government Decisions (GDs) 1 and 512/2013, the MRDPA coordinates policy in the following areas: regional and urban development, territorial cohesion, cross-border cooperation, public works, housing, national and local administration, public services, industrial parks, and “programming, coordination, monitoring, and control of non- reimbursable EU assistance for relevant activities under its mandate.”18 For its part, the ROP naturally covers some of the same key areas, as is to be expected of a program administered by the Managing Authority under the MRDPA. As noted throughout this report, the ROP finances a range of complex infrastructure projects, primarily in urban settings, as well as smaller investments for strengthening the private sector’s development. 127. More specifically, the ROP seeks to support the economic, social, territorially balanced, and sustainable development of Romanian regions, according to their specific needs and resources, by focusing on urban growth poles, the business environment, and basic infrastructure. The program’s vision is to transform Romanian regions, particularly the ones lagging behind, into more attractive places to live, work, visit, and invest in. Based on these goals, as spelled out in the ROP’s programmatic document, the following specific objectives were set: (1) to increase the economic and social role of urban centers, adopting a polycentric approach, in order to stimulate a more balanced development of 18 See www.mdrap.ro. 63 regions; (2) to increase accessibility within regions and in particular the accessibility of urban centers and their connection to surrounding areas; (3) to increase the quality of social infrastructure of regions; (4) to increase the competitiveness of regions as business locations; and (5) to increase the contribution of tourism to the development of regions. Concentration, connectivity, basic institutions and services, quality of life – these are cornerstone concepts of sound policies aimed at promoting economic development, as explained below and as noted at length in the World Bank’s 2009 World Development Report and also in the 2013 study on Romania’s Competitive Cities. 128. In this context, this brief summary reviews the potential strategic directions for Romania’s economic development, as reflected in a series of flagship documents, with the aim of orienting the conversation toward a set of clear, consistent criteria for the prioritization of interventions. First, keeping in mind powerful synergies with the World Bank’s broader ongoing Regional Development Program in Romania, the initial sections below summarize the framework presented in the Competitive Cities report, prepared for the MRDPA as part of the advisory services on “Enhanced Spatial Planning as a Precondition for Urban Development.” This is based on a well-known World Bank methodology presented in the 2009 World Development Report and adapted to Romania’s context. Second, this chapter presents EU-level priorities that apply to Romania as one of the 28 Member States, focusing most notably on the Europe 2020 strategy, which is often cited as the main document guiding the Union’s development path. Third, a review of the national-level dimension, as reflected in the draft Partnership Agreement (PA) that will shape how Romania spends structural funds available for the 2014- 2020 budgeting cycle, helps put things into perspective. The hope is that this effort will begin to shed light on a unified, coherent framework for looking at Romania’s development priorities and the MRDPA’s role in supporting them, including through the next phase of the ROP. Keeping in mind this goal, this chapter’s final section draws parallels between the various key documents assessed and presents a set of priorities that can further guide the strategic prioritization of investments at local, regional, and national levels. The World Bank’s Recommendations on Nurturing Romania’s Competitive Cities 129. The 2013 World Bank report on “Competitive Cities” makes an extensive case for enabling Romania’s sustainable and inclusive development by allowing all people to pursue opportunities and maximize their productive potential. In particular, the role of cities, as an economy’s primary engines, is fundamental to this endeavor. The key argument goes back to an understanding of economic growth as a product of two factors: population and productivity. Since Romania’s demographic decline leaves little hope for an economic growth based only on rising population figures, boosting individual productivity is the key task at hand. The history of advanced economies teaches that a country’s development goes hand in hand with urbanization – a process that involves, essentially, the concentration of human and economic resources around various centers of activity. The greater the mass of a country’s cities and the closer they are to each other and to other large markets abroad, the better the prospects of high and sustainable economic growth. But beyond regions, cities, and economic sectors, the fundamental premise of this work is that an economy is the sum of its people. A successful economy is therefore one that enables people to be the most productive they can be or, in simpler terms, to fulfill their individual dreams and potential for their own benefit and for the good of society at large. 130. The types of optimal measures needed to promote economic development vary based on the type of targeted area. On one end of the spectrum, there are leading areas that experience strong growth and an accelerated concentration of resources. These generally form around major cities (per the arguments presented above, leading areas are inherently and increasingly urban) and are growing, 64 typically in every sense of the word: their economy is dynamic; unemployment is low and, moreover, human capital is in high demand; incomes rise and people have an increasing ability to spend more on goods, services, and leisure; new residential neighborhoods and industrial areas push the city’s boundaries, often along existing infrastructure (e.g., roads and railroads). This virtuous cycle of prosperity continues to draw in more competitive people and companies, expanding the local economy, at least until the costs of further concentration (in specialized jargon, also known as diseconomies of scale) begin to outweigh the benefits of this process. Data show that in Romania the urbanization process is in full swing and will likely continue for some time before it reaches a turning point and, indeed, a new hierarchy of cities is emerging.19 131. At the other end of the spectrum, there are lagging areas, which struggle to break through a vicious cycle of poverty, low economic competitiveness, lack of education and skills, and possibly marginalization. Such places have few professional opportunities, public institutions and services often function poorly on account of a crumbling infrastructure (e.g., lack of running water, inadequate sanitation, poor healthcare services, low-quality schools, etc.), and many people struggle with poverty and, in special cases, discrimination and isolation. Even the best-qualified people have a hard time in finding adequate employment because there are few profitable firms, if any, so they often move away in search of better opportunities. People who are less mobile (e.g., children, the elderly, other vulnerable groups, low-skilled workers, etc.) are left behind, often struggling with poverty and the lack of basic necessities. Needless to say, those who are born and grow up in a lagging area typically have a much weaker shot at fulfilling their potential and boosting their productivity, which ultimately represents a cost for the entire society. As noted above, the challenge is to create the conditions for all people, in all areas, to fulfill their aspirations. 132. While most policymakers easily understand the differences between various areas as presented above, there is little consensus on the best solutions for supporting development. Indeed, most of the time, the tendency is for governments to artificially redistribute resources from growing cities to less developed ones, along with top-down incentive schemes for people and firms to relocate to lagging areas. In practice, such interventions run the risk of slowing down a country’s overall growth by weakening precisely the engines that push the national economy forward, while redirecting scarce resources toward projects with improbable benefits. Again, what is needed is a careful look at the historical evolution of advanced economies: in short, uneven development is inherent and smart policymakers learn to embrace and enable the process, rather than try to oppose or alter it. While in the short term some degree of internal divergence is inevitable, with some areas growing faster than others, eventually the benefits of development extend across the country and lead to similar standards of living regardless of location, which is the essential key to unlocking increases in productivity and overall economic growth. In other words, if the desideratum is a growing, sustainable, and resilient economy, where all people have the opportunity to realize their potential and maximize their productivity, there is a proven path to get there the fastest. The World Bank’s 2009 World Development Report and the 2013 work on Competitive Cities, specific to Romania’s context, make these arguments persuasively and at length. 19 Essentially, data from the 2012 Census confirms predictions based on the Zipf rank-size rule: Bucharest is emerging as a clear primate city in terms of economic and demographic mass, followed by Cluj-Napoca and Timișoara, which have pulled ahead of other growth poles. This has happened as a result of free market dynamics, after the seven growth poles came out of the centrally planned system at similar development levels in 1990. This argument and its implications are presented at length in the two 2013 World Bank reports on Competitive Cities and Growth Poles: The Next Phase. 65 133. In the context of the current project, this framework has direct implications on the optimal types of ROP-financed interventions at the local, regional, and international level, as illustrated in the figure below. At the local level, lagging regions should invest in fostering good institutions, ensuring that all people have access to the range of basic public services (e.g., running water, sewage, gas, electricity, trash collection, education, health, etc.) that allows them to focus on productive activities, as opposed to spending time on securing essential needs. This is in line with the desideratum to create similar living standards across the country, which also often resonates with political decision-makers. Leading cities, for their part, should channel resources toward improving connectivity with surrounding areas to continue to increase their economic mass and strengthen the virtuous concentration economic and demographic mass. At the same time, these areas should invest in increasing the quality of life for residents, so that they can attract and retain the qualified human capital to fill the innovation jobs that the local economy needs to prosper. Finally, both leading and lagging areas should invest in local interventions targeted at enabling and empowering marginalized groups to pursue educational and professional opportunities, reducing discrimination and allowing them to share in the broader benefits of Romania’s development. 134. As for the regional and international levels, the main aim of ROP-financed interventions should be to improve the connectivity between different areas within Romania and between domestic and foreign markets. Regionally, investments in connective infrastructure between leading and lagging areas can enable more people to relocate in growing areas, where feasible, adding to the mass of growing cities and strengthening their potential for growth. At the same time, less developed areas share the benefits of development through spillover effects and, on account of better connective infrastructure, get to retain some of their residents and limit the demographic decline, even though people may be commuting to work in bigger cities in the region. Figure 15. Typology of optimal interventions (Competitive Cities, World Bank) Shorten the distance to large markets globally by improving infrastructure and International encouraging cross-border flows of people, capital, and ideas Intervention level Improve connections between leading and lagging areas within Romania to enable Regional efficient concentration of resources and spillover effects Improve connective Foster good institutions infrastructure between cities (basic services infrastructure, & surrounding areas Local education, health, land markets, etc.) Promote quality-of-life investments Design and implement targeted measures for marginalized groups Lagging Leading Area Area Level of economic development 66 135. The same principles apply internationally, where improved connectivity essentially means that people, capital, services, goods, and ideas can travel more easily between different markets. Better infrastructure means lower costs for Romanian exporters, as well as cheaper imported goods for those products that cannot be produced competitively on the domestic market. At the same time, Romanians can explore opportunities on global markets that are more closely aligned with their career preferences, while local companies can potentially expand their labor pool and attract qualified employees from abroad. In particular, migration is not necessarily a bad thing. As argued extensively in the Competitive Cities report, Romanians abroad contribute to the country’s development with money transfers (in the form of remittances), direct capital investments (e.g., starting new businesses, based on practices learned elsewhere), and sharing of knowledge and ideas (e.g., studying/working abroad and returning home to apply the lessons learned). There is one caveat to the arguments above: in strategic sectors such as healthcare, a massive exodus of qualified labor can create problems, particularly in the short term, and in such exceptional circumstances government intervention may be needed to motivate people to continue to work in Romania to preserve the quality of public services rendered. Generally speaking, however, improved mobility and connectivity are positive features and should be nurtured by addressing potential obstacles through appropriate measures (e.g., improved infrastructure, better airline connections, lower trade barriers, etc.). 136. The Regional Operational Programme (ROP) is an essential instrument for achieving the development vision laid out above. It can fund a range of infrastructure projects that improve connectivity at both local and regional levels (e.g., roads, bridges, metropolitan public transport, etc.), particularly when coordinated with the EU-level TEN-T network and investments from the OP Transport. At the same time, the ROP can support various quality-of-life projects, from the rehabilitation of public parks and historic buildings to the creation of pedestrian areas, bicycle lanes, and parking infrastructure. In lagging areas, the program can be fundamental in revamping available social infrastructure (e.g., schools, hospitals, retirement centers, utility networks, etc.), particularly in conjunction with OP Environment. Last but not least, the ROP has the potential to empower marginalized groups by connecting them with opportunities (e.g., updated schools, better spatial integration of Roma neighborhoods, etc.), while keeping in mind the need to combine hard and soft measures to effectively combat discrimination and promote true empowerment of disadvantaged communities. The table below provides a few additional examples of potential ROP projects that contribute to Romania’s development by promoting urbanization, bringing people closer to opportunities, and lowering barriers against labor and capital mobility. 137. Importantly, all decisions to fund a particular project should be based on actual needs and expected impact on the ground, which will vary on a case-by-case basis. In other words, the framework above does not justify investments in any project that, say, connects any city to its neighboring area. Clear, persuasive data are needed regarding demographics and the mobility of people across localities, along with trends across time, economic flows, etc. This observation holds for all types of proposed investments, across all axes and priority areas: just because a particular idea/project fits under a general heading in the abstract does not necessarily mean that it actually generates significant impact and contributes to the country’s development more than other available opportunities. 138. Equally important, regions can have different needs, even within the broader categories of leading/lagging areas. The Competitive Cities report shows, for instance, that the economy of Cluj- Napoca is much more localized than that of Timișoara, so the former may benefit from more quality-of- life investments in central areas, while the latter could prioritize enhancing connectivity with 67 neighboring areas. Ultimately, local communities know best what their most pressing needs are, and the selection of ROP projects for 2014-2020 should ensure, at a minimum, that: (a) proposed projects have a significant development potential; and (b) there is a sufficient justification of need, based on accurate data and in-depth analyses. 139. The priorities above are aligned with a vision of public institutions catalyzing Romania’s sustainable and inclusive development, in line with the country’s current stage of development. To summarize: a first priority is to improve connectivity and accessibility, enabling people to identify and take advantage of opportunities in Romania and abroad; second, the government should nurture good, functioning institutions for the provision of basic public services, as a measure that enables a similar living standard for people in leading and lagging areas, ensuring that everybody has proper access to good schooling, well-functioning land and housing markets, affordable healthcare, and basic public utility services like running water, sanitation, solid waste management, heating, etc.; in addition, special measures should target marginalized and minority groups to mitigate the persistent factors that limit their mobility (e.g., discrimination, ethnic and language barriers, etc.). Table 11. Typology of ROP interventions based on the World Bank’s Competitive Cities framework Intervention level Key development objective Potential ROP projects (select examples) A. Improve local connectivity  Expand public transport to metropolitan between major cities and their area (where density warrants it) surrounding areas  Upgrade rolling stock (e.g., buses, trams)  Build park-and-ride facilities B. Promote investments in  Investment in pedestrian areas, bicycle cities’ quality of life lanes  Rehabilitate historical centers  Expand/upgrade public parks  Investment in cleaner public transport 1. Local systems C. Nurture good institutions in  Rehabilitate school infrastructure lagging areas  Renovate and equip local hospitals  Upgrade social centers and homeless shelters D. Integrate marginalized  Build overpass to connect marginalized groups through targeted community to a city’s economic center measures (in conjunction with  Expand bus network to Roma “soft” interventions through neighborhood other OPs)  Finance SMEs in marginalized communities E. Improve connections  Build regional road connecting multiple 2. Regional between leading and lagging counties areas  Upgrade major commuting railroad F. Shorten the distance to  Improve local/regional connections to 3. International global markets TEN-T network 68 The Europe 2020 Strategy: Priorities for Romania’s Development 140. At the European level, there are a range of key documents that guide the Union’s short -, medium-, and long-term development, across several different areas. Other reports that are part of the World Bank Regional Development Program, particularly the assessment of Romania’s current growth poles policy, provide a review of the EU’s applicable framework in the area of regional development and spatial planning. This section focuses on a more limited, well-defined spectrum, looking at the fundamental EU-level thinking that has a direct bearing on how Romania’s ROP should evaluate and select projects for the 2014-2020 programming period – the core focus of the current report. 141. The primary framework guiding the European Union’s future development is the Europe 2020 strategy, adopted in March 2010, with the direct aim of promoting smart, sustainable, and inclusive growth. From the get-go, this document describes three interconnected and mutually reinforcing priorities:  Smart growth: an economy based on knowledge/education, innovation, and a “digital society” with performing information and communication technologies;  Sustainable growth: a green, competitive, and energy-efficient economy that is well-equipped to respond to climate-related challenges;  Inclusive growth: an economy characterized by higher employment rates, a competitive labor force with adequate qualifications, and effective poverty-reduction measures. Several flagship initiatives support each of these objectives, ranging from “Innovation Union” (as a way to refocus R&D efforts on emerging challenges) to “Youth on the Move” for promoting the mobility of students and trainees. Other examples include: “A Digital Agenda for Europe,” “Resource Efficient Europe,” “An Industrial Policy for the Globalization Era,” “An Agenda for New Skills and Jobs,” and the “European Platform against Poverty.” Each initiative defines a specific set of aims, as well as a list of actionable to-do items for both the Commission and the Member States, ensuring a basic reporting system to strengthen accountability and measure progress. 142. Based on the same three overall directions described above, EU-level policymakers have set several quantitative indicators for the Union as a whole and for each Member State. The table below summarizes information on the agreed targets, including Romania’s current “gap” along various dimensions.20 The country has already exceeded requirements on one dimension, renewable energy, thanks to a generous incentive scheme based on green certificates, which was actually softened in 2013 as a result of pressure from industrial groups claiming that the high energy prices, in part on account of the subsidies granted to renewable technologies, were hurting their competitiveness and created the risk for bankruptcy and massive layoffs. On all other Europe 2020 targets, Romania still has some way to go, including in critical areas like research and development (R&D), education, income, and employment, though there is still time to reduce the recorded gaps. 143. The reasons behind the adoption of the Europe 2020 priorities are complex. The first section of the strategy, entitled “a moment of transformation,” summarizes the main challenges that the Union faces in the current context, particularly in comparison with other global competitors: a structurally lower economic growth rate, pointing the need for improved R&D and innovation; low employment rates, particularly among the older segments of the population (i.e., 46% of the 55-64 years-old group has jobs, compared to over 62% in Japan and the United States); accelerated demographic decline, 20 Current data on Romania is from 2011/2012 and is based on the Draft Partnership Agreement, May 31, 2013. 69 putting pressure on social security systems; relatively frail post-crisis global finances; and the evolution of climate change, which requires drastic adjustments. The strategy, however, also points out a number of key strengths specific to the EU, including the fact that Member States’ economies are deeply interconnected (and Romania’s makes no exception to this, with over 70% of exports heading to Western Europe) and that collective coordination has already proven its functionality and added value during the recent crisis. Table 12. Europe 2020 targets and Romania's performance to date Reduction of Energy Early population at Employment RDI Renewable efficiency school Tertiary risk of poverty, rate (% of energy reduction leaving education social exclusion Indicator (%) GDP) (%) (%) (%) (%) (# people) EU target 75 3 20 20 10 40 20,000,000 RO 70 2 19 24 11.3 26.7 580,000 target RO 63.8 0.48 51.84 20.79 17.4 21.8 240,000 current Current 6.2 1.52 -32.84 3.21 6.1 4.9 340,000 Gap Source: Romania’s Draft Partnership Agreement, May 31, 2013 144. These observations are certainly pertinent, although it is not entirely clear how the specific objectives of Europe 2020 were selected as the most critical to the Community’s development for this point in time. The mechanisms through which EU decision-makers narrowed in on this particular list of priorities appear to have been more top-down than participatory, despite the fact that Europe 2020 significantly shapes EU-level policy design and implementation, including structural funding. At the same time, the strategy’s operationalization ultimately depends on stakeholders on the ground (e.g., ROP beneficiaries) generating high-impact projects and completing them, which suggests that they should be closely involved from the beginning of the process. Of course, higher participation can come at the cost of lower efficiency, but as the EU moves forward it will pay off to bring into the wider conversation stakeholders at all levels, not just to create stronger ownership but also to ensure that the agreed priorities actually reflect the most pressing needs on the ground. This would also help orient applicants and beneficiaries of EU structural funds in a space that often seems overly complex, in terms of both policy programming (i.e., there are many EU and national-level documents, with their own priorities) and application/implementation procedures related to accessing funds. 145. Because the Europe 2020 priorities are relatively broad, operationalizing them at the level of structural funds like the ROP is not an easy task. One may argue that ROP-funded projects contribute across the board toward all seven targets, from employment to energy efficiency, school attendance, and poverty reduction. There are of course other OPs that seem more directly aligned with particular themes, including OP Environment, OP Competitiveness, and OP Human Resources Development. By design, the ROP targets multiple components of Romania’s regional development: for example, updated school infrastructure can contribute to a reduction in early school leaving and an increase in energy efficiency; similarly, a business purchasing a new equipment to expand its offering can translate directly into more and better jobs, in conjunction with a reduction in poverty rates; and targeted projects for business infrastructure development can also promote R&D spending and the growth of new, innovative ventures. The point of these examples is that the ROP may not fit neatly under distinct categories of the 70 Europe 2020 framework, which is to be expected from a program that is meant to promote regional and urban development through different available means (i.e., it does not have a pure “thematic” focus). That said, the ROP is still a very powerful instrument for supporting Romania’s smart, sustainable, and inclusive growth and has an advantage in the possibility to support integrated investments as part of broader local development strategies. 146. Mirroring the main priorities of the Europe 2020 strategy, a legislative package was adopted by the EC on October 6, 2011, establishing 11 thematic objectives. These include: strengthening research, technological development and innovation; enhancing access to, use, and quality of information and communication technologies; boosting the competitiveness of small and medium-sized enterprises, the agricultural sector (for the EAFRD), and fisheries and the aquaculture sector (for the EMFF); supporting the shift toward a low-carbon economy in all sectors; promoting climate change adaptation, risk prevention, and management; protecting the environment and promoting resource efficiency; promoting sustainable transport and removing bottlenecks in key network infrastructures; promoting employment and supporting labor mobility; promoting social inclusion and combating poverty; investing in education, skills and lifelong learning; and enhancing institutional capacity and an efficient public administration. Again, ROP-funded projects have the potential to positively impact many of these areas. 147. Ultimately, the Europe 2020 strategy provides broad guidelines for what Member States should prioritize in their development, while openly noting that it does not seek to promote a “one - size-fits-all approach,” given that countries have particular needs.21 It would be hard to compare the German and the Romanian economies, for instance, and generally there are still structural differences between new EU members in the East and advanced economies in the West. The next section turns to the national-level framework that helps identify the strategic priorities guiding Romania’s development broadly and, in particular, the ROP’s programming, evaluation, and selection. National-Level Priorities for Romania’s Development 148. Within the large set of strategic documents guiding Romania’s development, the draft Partnership Agreement (PA) plays the fundamental role in the programmatic and institutional set-up of the 2014-2020 budgeting period. Another useful resource on spatial planning and territorial development is the World Bank study on “Enhanced Spatial Planning,” which argues in favo r of improved coordination and reduction of duplication among various strategic frameworks developed at the local, county, regional, and national level, while reviewing the various elements of these plans. For the purpose of the current work, the main focus remains on the PA and particularly on its sections dedicated to sustainable urban development. 149. In general, the draft PA follows the logic and structure of the Europe 2020 strategy, drawing explicitly on its thematic objectives and commitments, as recommended in a template put forth by the EC. Accordingly, the PA summarizes the main types of investments that are expected to be funded through the Regional OPs: local road, railway, and harbor infrastructure; urban/local development (including public transportation and related infrastructure, cultural heritage, sport facilities, multifunctional facilities); energy efficiency (including rehabilitation of networks); education and social infrastructure; health infrastructure; and competitiveness and business environment for SMEs. In particular, the draft document notes that a prioritization of investments – and the corresponding 21 See Europe 2020, p.9 71 funding allocation – follows from Romania’s specific needs at this point in its development, although it is not entirely clear (at least in this early draft) how the various areas will rank against each other and how the limited funding allocation will look like eventually. 150. The PA’s sections on territorial development include a number of concepts also reflected in the World Bank’s Competitive Cities framework, as well as in multiple EU-level documents. Common themes include: the importance of economic geography (e.g., Romania’s peripheral position in the EU); polycentric development and spill-over effects facilitated by strong urban-rural partnerships; connectivity between people, goods and services, and capital, with special efforts to overcome natural barriers (e.g., the Carpathian Mountains); the protection of natural and cultural resources as ingredients for sustainable development; and equitable access to public services (“services of general interest”) that are vital for citizens’ welfare. In particular, the PA highlights this last area, i.e., ensuring equal access to public services, as a “fundamental right” and “the most important component” in the fight against poverty. 151. The PA also includes a brief sub-section dedicated to the key topic of sustainable urban development, listing a number of guiding principles for 2014-2020 investments. There is a notable focus on two elements: (1) customization of investments based on the type of urban settlement (e.g., growth poles, development poles, small towns, and tourist resorts); and (2) the need to design and implement integrated interventions that go beyond a particular physical objective (e.g., a historical building, a public park, etc.) and create multiplier effects within a broader vision for local and regional development. The May 2013 draft version is still relatively vague on priority areas, noting several potential criteria for selection of intervention/project sites: concentration of historical buildings for downtown areas; concentration of population and age of buildings for residential neighborhoods; quality of buildings, crime rates, and lack of public utilities for deprived neighborhoods; and former industrial or military sites for brownfield redevelopment. The idea of customizing interventions to the specific type of urban settlement is sound and policymakers should expand it even further: as argued in the 2013 analyses on Competitive Cities and Growth Poles: The Next Phase, optimal investments may vary even within a single category of urban settlements (e.g., growth poles). As for the focus on integrated investments, this is very much in line with EU-level thinking, manifested most prominently through the introduction of the Integrated Territorial Investment (ITI) instrument for 2014-2020. This is also consistent with World Bank recommendations, which repeatedly call for designing projects within coherent, comprehensive development strategies, irrespective of the source of financing (i.e., as a response to the community’s needs, not simply as a precondition for accessing EU funds). 152. Equally important, the document lists several priorities for promoting sustainable urban development, many of them well aligned with recommendations put forth by the World Bank in the Competitive Cities report. The PA mentions the following:  Support the development of cities’ economies by increasing their economic and demographic mass. This can be accomplished by shortening distances between major cities and their surrounding areas to ensure dynamic flows of people, capital, and ideas. A few other priorities mentioned under this heading cover the strengthening of the business environment through support for the services and manufacturing sectors, as well as through the redevelopment of former industrial sites (a major intervention area for 2007-2013 as well) that often enjoy a central location and are usually already connected to utility networks. The recognition of the importance of nurturing cities’ economic mass shows a deep understanding of urbanization dynamics and is also the main message promoted by the 2013 World Bank reports on Romania’s regional development. 72  Improve urban areas’ environmental quality. This covers increasing the energy efficiency of the dwelling stock, but also adopting measures to promote additional green spaces and a reduction in car traffic. Enhanced non-motorized mobility goes hand in hand with pedestrian- and bicycle- friendly areas, although the need to build more parking spaces (particularly around residential areas) is also featured prominently. The document does not fully address the tension between facilitating private vehicle use (through improved roads and parking, the target of many ROP projects between 2007 and 2013) and promoting non-motorized transportation for improved quality of life and health of the population.  Support the development of basic infrastructure for Romanian cities. Without going into specifics, the PA rightly notes that the availability of public infrastructure is key for the optimal functioning of cities, influencing their accessibility and their attractiveness for investment, tourism, and other productive activities. Still, it is unclear what types of investments would fall under this broad category and where they would apply the most (e.g., in a small, mono- industrial town with crumbling infrastructure vs. a large, booming economic center).  Promote social inclusion and urban mobility (including for disabled persons). This covers integrated and coordinated interventions in neighborhoods struggling with poverty and marginalization, both in downtown areas (e.g., historical centers) and in peripheral, isolated neighborhoods. The draft PA does not describe specific interventions, but merely highlights the need to design integrated strategies in the area of social inclusion. It also specifically mentions the need to support the mobility of disabled people to enable their access to opportunities. A potential gap in this draft version – and a primary area of focus in the World Bank reports – is the need to involve marginalized communities in how investment needs and projects are defined, designed, and implemented, focusing not just on physical infrastructure but also on soft measures to promote engagement and empowerment of these groups.  Increase the quality of the public space and ensure an attractive urban landscape. Much like the Competitive Cities report, the PA notes that quality-of-life investments (for example, in the waterways crossing Romanian cities) are fundamental to retaining talent in growing urban areas. This is related to the above-cited priority on the environmental quality of urban areas. 153. Overall, there is almost full convergence between the draft PA and World Bank recommendations, as they apply to urban settings and the future Regional Operational Programme. This, along with the fact that the PA is, by design, well aligned with the Europe 2020 strategy, means that it is easier to agree on a list of regional development priorities that would form the basis of the next set of ROP selection criteria. At least at a macro level, it is safe to say that there is an emerging consensus around the need to support: connectivity around major economic centers and between leading and lagging areas; basic functioning institutions, meaning equal access to public services for all Romanians, regardless of where they live; integration of marginalized groups; and quality-of-life investments for continuous improvement of urban environments and retention of a talented worker pool. Adjusting selection models according to project size 154. Not all projects are created equal. Some projects are likely to have a larger impact (usually, but not necessarily, the larger projects), while others will have more limited effects. Consequently, it is important to distinguish between different types of projects when project selection models are devised. 73 155. For the purposes of this analysis, and for the ROP 2014-2020, we propose that three types of projects be considered: large projects, medium projects, and small projects. Large projects will be those that exceed in value 100 million RON (this is the cut-off used to distinguish strategic projects in the recent Government Emergency Ordinance 88/201322). Medium-sized projects will be considered that have a value between 1.5 million RON and 100 million RON. Given that medium-sized projects cover a large spectrum, a further refinement is proposed, to include: lower-middle-sized projects (with a value between 1.5 million RON and 30 million RON) and upper-middle-sized projects (with a value between 30 million RON and 100 million RON). Small projects have a value below 1.5 million. 156. In an ideal, best-practice scenario, selection methods would be tailored for projects based on both size and complexity level. Thus, large projects, of a strategic nature, would ideally be included in a comprehensive sector development strategy and would receive pre-allocated funding. Upper-middle sized projects (e.g., those of strategic importance at the regional/local scale) would ideally be featured in regional/local level strategies (for example, local or county development strategies in the case of Romania) and may be subject to competitive selection (especially if many such projects are submitted). Lower-middle sized projects may or may not be part of regional/local development strategies, and could be subject to competitive selection, depending on the objectives that are hoped to be achieved. Small projects usually come in very large numbers and have a limited local/regional impact, so it would be optimal to apply simplified selection procedures in their case. 157. For the ROP 2007-2013, the largest share of contracted projects included medium-sized investments – 90.5% of the total value of contracted ROP projects. Among those, it was the upper- middle-sized projects (with a value between 30 million RON and 100 million RON) that dominated, with 274 contracted projects, an average value of nearly 57 million RON, and around 55% of the total value of the contracted projects. There were only 9 large projects, with an average value of around 120 million RON, and a 3.75% share of the total value of contracted projects. 5.74% of contracted funds were allocated to small projects, with an average value of around 841,500 RON. Figure 16. Tailoring selection models based on the project type 22 GO 88/2013 on Budgetary fiscal measures in order to fulfill some commitments agreed upon with international bodies and for the modification and completion of some legislative acts 74 Table 13. ROP projects contracted as of August 2013 Number of Average value of Total value of % of total projects project (in RON) projects (in RON) contracted projects Small Projects (<1.5 mln RON) 1,952 841,553 1,642,712,364 5.74% Medium-Sized Projects (1.5 - 100 mln RON) 1,554 16,750,343 25,923,350,459 90.51% Lower-middle-sized projects 1,279 8,017,773 10,254,732,124 35.8% Upper-middle-sized projects 275 56,976,793 15,668,618,334 54.71% Large Projects (>100 mln RON) 9 119,471,996 1,075,247,961 3.75% TOTAL 3,515 8,148,310 28,641,310,783 100% Data source: MRDPA 158. Overall, there seems to be an under-representation of large, strategic projects within the ROP 2007-2013. Given the amounts that were allocated to the ROP, and given the issues this operational programme is supposed to address, it is surprising that only 9 large projects, with a share of 3.75% of total funds, were selected. Moreover, none of these 9 projects were truly regional in nature. As the table below indicates, all of these large projects were implemented by individual county councils and local authorities. All of the 9 projects were road rehabilitation and/or modernization projects (under Axis 2 of the ROP) – primarily because the Applicant’s Guides only permit Axis 2 projects to receive funding of over 100 million RON. 159. For the 2014-2020 ROP, it will be critical to also focus on truly regional, strategic projects. For the 2007-2013 programming period, there was no formal administrative body at the regional level, which could act as a beneficiary and implement regional projects. For the 2014-2020 programming period, this situation will likely endure although there are plans to move forward on the regionalization process in 2014 (adopting these potential changes to the EU funds’ management framework will take some time). In either case, it is important to think early on about solutions to implementing large regional strategic projects. Even if the regionalization process will not be complete, it is important to enable the development and implementation of cross-county projects (either by means of Inter- Community Development Associations, partnerships, or through other administrative vehicles). Table 14. Large projects contracted, by August 2013, under the ROP 2007-2013 Total Value Project Beneficiary Region (in RON) Rehabilitation of County Road 606 County Council Mehedinți South West 138,114,003 Modernization of road infrastructure in Piața Sudului Local Council București Bucharest-Ilfov 127,140,762 Rehabilitation of County Road 151 County Council Bistrița Năsăud North West 126,610,050 Modernization of Dorobantilor Boulevard Local Council Brăila South East 125,055,687 Modernization of County Road 582 County Council Caraș Severin West 121,137,662 Rehabilitation of County Road 701 County Council Teleorman South 119,905,413 Rehabilitation and modernization of County Road 411 County Council Giurgiu South 109,216,190 Modernization of County Road 687D County Council Hunedoara West 105,371,104 Rehabilitation of County Road 108 County Council Sălaj North West 102,697,091 Source: MRDPA 75 160. Medium-sized projects, particularly upper-middle-sized projects, would ideally be part of local/county development strategies. These are projects of considerable value, and it would be important to provide a sound justification for their financing. Funds may be pre-allocated for such investments (the way it was done for Growth Poles and Urban Development Poles under the 2007-2013 ROP), or the projects may be selected on a competitive or non-competitive (as currently) basis – provided a number of eligibility, technical, and financial requirements are met. As the table below indicates, the average project value financed through Integrated Urban Development Plans for the 2007-2013 Programming Period was below 30 million RON – 29 million RON for Growth Poles, 16 million RON for Urban Development Poles, and 14 million RON for Urban Centers. In part, these low averages are explained by the relatively low maximum project value thresholds set by the ROP Applicant Guides. Table 15. Contracted ROP projects by axis (by August 2013) No. of Average value of Total value of Axis Sub Axis projects projects (in RON) projects (in RON) DMI 1.1 GP 129 28,524,891 3,679,710,914 Axis 1 - Integrated Urban DMI 1.1 UDP 91 15,542,470 1,414,364,751 Development Plans DMI 1.1 UC 256 14,005,494 3,585,406,581 Axis 2 - Roads DMI 2.1 146 43,659,582 6,374,299,003 DMI 3.1 78 16,032,505 1,250,535,376 Axis 3 - Educational, Health, DMI 3.2 202 2,991,212 604,224,773 and Social Infrastructure DMI 3.3 16 33,204,571 531,273,141 DMI 3.4 231 8,246,624 1,904,970,064 DMI 4.1 75 27,250,498 2,043,787,360 Axis 4 - Business DMI 4.2 7 41,720,674 292,044,718 Development DMI 4.3 1,672 899,292 1,503,615,986 DMI 5.1 80 20,468,194 1,637,455,492 Axis 5 - Tourism DMI 5.2 111 22,130,440 2,456,478,882 Development DMI 5.3 354 1,977,702 700,106,605 Source: MRDPA 161. Small projects should be selected following a simplified procedure. Currently, the largest number of small projects is funded under KAI 4.3 (Support for micro-enterprises), with the average intervention totaling around 900,000 RON. All other axes had projects that were, on average, above the 1.5 million RON threshold. 76 Figure 17. Optimal selection models should reflect the degree of project complexity 162. Finally, focusing on large, medium, and small projects, the way they are defined in this report, may require the MA-ROP to revise the caps it currently imposes on individual projects. As the table below indicates, the ROP 2007-2013 is primarily geared toward medium-sized projects, with the maximum cap for small projects set at 1,700,000 RON, and with the minimum cap for large projects set at around 85,000,000 RON. These minimum and maximum caps are just as good as the one proposed above (the differences are relatively small). However, the MA-ROP may consider adjusting the minimum threshold for large projects to 100,000,000 to follow the Ministry of Finance’s classification, and to facilitate a coherent, unified approach at the government level. The threshold for small and upper- middle-sized projects may also be adjusted, depending on the needs of the MA-ROP and depending on the structure of the ROP 2014-2020. Table 16. Minimum and maximum funding caps for ROP 2007-2013 projects Minimum and maximum funding caps Axis Sub Axis Types of projects finances (in RON) Rehabilitation of urban infrastructure 1,700,000 - 90,000,000 DMI 1.1 GP Axis 1 - Integrated Urban Development Plans DMI 1.1 UDP Business development 1,700,000 - 85,000,000 DMI 1.1 UC Social infrastructure 350,000 - 3,500,000 Rehabilitation and modernization of Axis 2 - Roads DMI 2.1 3,000,000 - 134,000,000 roads DMI 3.1 Health infrastructure 700,000 - 85,000,000 Axis 3 - Educational, Health, DMI 3.2 Social infrastructure 350,000 - 3,500,000 and Social Infrastructure DMI 3.3 Emergency situation equipment 10,000,000 - 50,000,000 DMI 3.4 Educational infrastructure 500,000 - 67,000,000 DMI 4.1 Business infrastructure 1,700,000 - 85,000,000 Axis 4 - Business DMI 4.2 Brownfields Redevelopment 1,700,000 - 85,000,000 Development DMI 4.3 Support for micro-enterprises 100,000 - 3,000,000 DMI 5.1 Cultural heritage tourism infrastructure 1,700,000 - 85,000,000 DMI 5.2 Natural heritage tourism infrastructure 700,000 - 85,000,000 Axis 5 - Tourism Development DMI 5.3 Tourism promotion 170,000 - 1,000,000 (an 100,000,000 upper cap is allowed for the promotion of Romania's tourism) 77 Tailoring projects to regional needs 163. For the 2007-2013 programming period, all eight regions in Romania were treated the same. All of them had a GDP per Capita that was below 75% of the EU average and, as such, all fell in one category, without any further differences based on their relative development level. As the analysis in the Competitive Cities report has shown though, there are stark differences between regions in Romania. The most notable “outlier” is the Bucharest-Ilfov Region, which before the 2014-2020 programming period has registered significant growth and is now considered to be a developed region. The other seven regions are considered to be less developed, and there are no transitioning regions yet (the third category). Even so, there are important differences between the seven less developed regions. 164. The West Region, the Center Region, and the North West Region have higher overall economic development levels than the North East Region, the South Region, or the South West Region. As such, they have a different set of needs and priorities. Consequently, it may pay to have an additional level of refinement when looking at regions and regional priorities. For example, Spain has four types of regions for the 2007-2013 Programming Period (as defined by the EU): Convergence, Phasing out, Competitiveness, and Phasing in. This, along with a national level commitment to decentralize responsibilities and customize interventions based on local/regional needs, has allowed Spain to approach EU funds’ priorities in a more nuanced way. Even if the same does not apply to Romania, it will be important to take regional differences into consideration when designing the ROP 2014-2020. Importantly, this does not presuppose the existence of a formal administrative level; even under the current set-up, the ROP can be designed as a national-level programme with interventions and priorities customized to fit the profile of each region. Figure 18. In Spain regions are categorized in a more nuanced manner Source: Guadalupe Melgosa, The Spanish Ministry of Administration and Public Finances. PowerPoint Presentation (titled “The Management of ERDF Funds”) held on September 24, 2013. 78 Figure 19. The Regions in the East and the South of the country have lower basic infrastructure endowment than regions in the West Source: National Institute of Statistics 79 165. As noted in the earlier section, the Competitive Cities report identified four key priorities as critical for Romania at its current development stage: - Connective infrastructure for the country as a whole (particularly around urban centers and between leading and lagging areas); - Investments in basic living standards in less developed regions; - Targeted measures for marginalized communities; and - Quality of life investments in the most dynamic and competitive cities. 166. The Competitive Cities report further argues that while connective infrastructure is needed in the country as a whole, it is of utmost importance in the more developed regions in the West of the country. This would allow the dynamic cities in those regions to have better access to the rich markets in the West, while at the same time connecting a larger population to the opportunities that these cities offer, promoting urbanization and long-term growth. 167. To take the discussion from the Competitive Cities report a step further, we propose that a different funding approach be taken for less developed regions (mainly the North East, South West, South East, and South) and for more developed regions (mainly the West, Center, and North West). București-Ilfov already is in its own category as Romania’s most developed region, and will most likely have its own operational programme. Therefore:  In lagging regions, priority should be given to: 1) Investments in basic standards of living (e.g., 40% of available funding); 2) Connective infrastructure (e.g., 30% of available funds); 3) Measures targeted at marginalized communities (e.g. 20%); and 4) Investments in quality of life in cities (e.g., 10%).  In leading regions, priority should be given to: 1) Connective infrastructure (e.g., 40% of available funds); 2) Investments in quality of life in cities (e.g., 30%); 3) Measures targeted at marginalized communities (e.g., 20%); and 4) Investments in basic standards of living (e.g., 10%). The main premise of this proposal is that lagging regions in Romania have an under-endowment of basic services infrastructure (e.g., water, sewage, sanitation, education, health, social care) and also have fewer funds at their disposal for investments in basic needs (see, for example, the maps below). At the same time, the more developed regions in the West have both a better basic public services infrastructure endowment and larger local/regional budgets for investment in such infrastructure. What they lack, however, is a proper transportation network that could further benefit their growing economies – e.g., by improving access to the rich markets in the West and by improving accessibility to the most dynamic cities in the region. Of course, regional allocations also need to match Europe 2020 objectives and Romania has to hit the national-level targets in an effective and timely manner; that said, there are no fundamental incompatibilities between the proposed framework and the country’s commitments at the EU level. 168. The way funds were allocated and spent for the 2007-2013 programming period indicates that little attention was paid to different regional needs and priorities. This would have allowed the most developed cities in those regions to have better access to the rich markets in the West, while at the same time connecting a larger population to the opportunities these cities offer. To allow us to better compare investments made under the ROP 2007-2013 to actual regional needs, in Annex 11 we have tried to assign to each type of activity funded a code corresponding to: connective infrastructure (CI), 80 life standards (LS), marginalized communities (MC), and quality of life (QL). Most activities funded by ROP fall neatly in these categories, with the mention that there was only a limited focus on marginalized communities, and there was also an extra category focused on business development (primarily under Axis 4), which will not be discussed here in great depth. 169. To simplify matters and help focus the analysis, we have primarily looked at Axes 1, 2, and 3 of the ROP 2007-2013. Axis 1 generally falls under the Quality of Life Investments category; Axis 2 focuses squarely of Connective Infrastructure; and Axis 3 primarily deals with improvements of Life Standards. We have then compounded total contracted ROP funds under these three axes and have mapped them at different administrative levels. 170. Quality of life investments show a clear pattern and a focus on the counties with the strongest economic centers within the region. This is the result of targeted measures under the Growth Poles policy, which has pre-allocated funds for the most dynamic cities in Romania. This approach, although it may be viewed as top-down, has enabled a more strategic allocation of funds to the cities that have the strongest polarization potential at the regional level. 171. Nonetheless, some inefficient allocation of funds does exist under Axis 1. For example, the counties of Sibiu, Mureș, and Bihor are quite urban, have strong economic engines (Sibiu, Târgu Mureș, and Oradea have larger economies than growth poles like Iași and Craiova, although they have a smaller population), and would most likely have benefited more from urban investments than counties such as Maramureș, Mehedinți, or Botoșani. Part of the reason for this occurrence is that while growth poles and urban development poles had pre-allocated funds, urban centers (basically all other cities) had to compete for funds. Inevitably, some cities have been more adept at accessing funds under Axis 1 than others. Figure 20. Quality of Life investments show a concentration of investments in the most dynamic cities regionally 81 172. When it comes to investments in roads, there also seems to be a mismatch between needs and investments. Less developed regions in the East and South have been more adept than more developed regions at attracting funds for the rehabilitation and modernization of their roads (Axis 2 is also the axis with the fastest absorption rate). However, good connective infrastructure is likely to have the strongest positive effects in the more developed regions of the West, banking on the positive market dynamics there. And, as we have seen earlier, these more developed regions in the West seem to paradoxically have a less dense road infrastructure endowment than less developed regions. 173. For the 2014-2020 programming period, it will be important to identify strategic regional infrastructure projects, which will enhance existing positive market dynamics. These projects should have a comprehensive strategy at their base – e.g., the Regional Development Plans – and should clearly indicate how they could help enhance positive regional market dynamics (e.g., better connecting people to centers of growth and better connecting centers of growth to the rich markets in the West). Figure 21. Less developed regions have benefited more from investments in roads than more developed regions 174. Life standard investments do seem to follow the expected pattern, with a higher concentration in less developed regions and counties than in the more developed ones. As in the case of road infrastructure investments, this may reflect the fact that less developed regions have received a more generous funds allocation, and the fact that local/county authorities in less developed regions may 82 have been more proactive at accessing EU funds. In any case, for 2014-2020, it may be worth considering a more generous funds allocation for life standard investments in less developed regions (which have a net deficit in this respect over more developed regions), and a lower allocation for more developed regions. Figure 22. Life standard investments seem to follow the expected pattern (higher in lagging regions) 83 VII. Proposals for Project Selection Models ROP 2014-2020 Project Selection Models: Comparative Scenario Assessment 175. In choosing improved project selection models for the ROP 2014-2020, there is value in comparing the main features, strengths, and weaknesses of multiple potential scenarios. The starting point is, naturally, the previous analysis of the current system, which has exposed both positive features and areas that require adjustments. But before looking at each scenario individually, this chapter proposes a framework for assessing the various options based on six fundamental requirements (efficiency, effectiveness, clarity, fairness, transparency, and feasibility) and four key objectives (absorption, impact, legitimacy, and capacity). 176. The figure below conveys precisely this message, making the case that an optimal project selection model for the next ROP will need to accomplish all four targets. Put differently, a model may be wonderful for promoting quick absorption of EU funds, but it may fail to deliver value for money and resources could be wasted on low-impact interventions. Similarly, if a model is not legitimate (i.e., with clear and transparent evaluation criteria and fair selection procedures), not enough ROP applicants will demonstrate interest in the program, so there will be an inherent limitation on what the model can accomplish. Last but not least, any proposed change needs to be more than just a theoretical framework that promises to deliver great results in the abstract; indeed, selection models have to work in practice, keeping in mind the system’s stringent capacity constraints. Figure 23. A framework for defining an optimal selection model for Romania’s ROP OPTIMAL PROJECT SELECTION MODEL Evaluation  Selection  Contracting OBJECTIVES 1 2 3 4 ABSORPTION IMPACT LEGITIMACY CAPACITY CORE Does the model Does the model Is the model Is the model feasible ensure a high optimize for impact, legitimate for all given current absorption rate? value for money? stakeholders? capacity? A EFFICIENCY: Quick, smooth process that ensures an optimal absorption rate B EFFECTIVENESS: Selected projects optimize for impact and value for money REQUIREMENTS FUNDAMENTAL C CLARITY: Clear selection procedures and requirements minimize the room for interpretation D FAIRNESS: Fair selection criteria apply to all submissions E TRANSPARENCY: Transparent procedures enable full visibility and accountability of all parties F FEASIBILITY: The institutional system is able to properly implement the model 85 177. Based on these general principles, three scenarios are worth considering in terms of the overall project selection model per se:  Scenario 1 (“Status Quo+”): The main characteristics of the system remain unchanged, with a few tweaks (e.g., excluding the full Cost-Benefit Analysis from the formal assessment, making the Detailed Technical Design optional at the evaluation phase, etc.) to correct current weaknesses. This option capitalizes on the ROP system’s accumulated experience and leverages the benefits of predictability. Essentially, the two selection procedures remain the same: pre- allocation for growth poles and urban development poles (with the potential inclusion of all county seats as UDPs); and first-in-first-out for all other projects.  Scenario 2 (“Semi-Competitive”): This variation applies different selection methods based on project size and beneficiary type. Project size tends to be a good proxy for both complexity and potential impact. Under this scenario, large regional projects (type 1) would enjoy dedicated funding. Medium-sized investments by counties, growth poles, and urban development poles (type 2) would be selected in two phases: (1) dedicated funding for a limited period of time; and (2) selection based on FIFO for the remaining pool of funding allocated to these types of beneficiaries. For other cities, as well as for private and non-profit applicants (type 3), the FIFO rule would apply, regardless of whether the project is small (under 1.5 million RON) or more significant (over 1.5 million RON). Ultimately, this is a “semi-competitive” model, as a “middle- of-the-road” option, although applications are not compared or scored against each other under either FIFO or a dedicated funding model, regardless of the type of project under consideration.  Scenario 3 (“Competitive”): This scenario maintains the FIFO selection rule for a limited set of projects, namely the smallest ones under 1.5 million RON, while also setting aside dedicated funding for the largest regional projects. For counties, growth poles, and urban development poles, the funding would be allocated in two phases, much like for the previous scenario, with a key difference: the phase 2 reallocation is done on the basis of competitive ranking for projects under each axis, based on either quantitative (scenario 3A) or qualitative (scenario 3B) criteria. Figure 24. The three scenarios considered propose a range of different adjustments to the status quo model 86 The figure above summarizes the main features of the three options, for each applicable type of project. As this report argues, choosing an optimal selection model is a challenging task and may involve several quintessential tradeoffs: between impact (effectiveness) and absorption (efficiency); between needed changes and predictability/continuity; and between ideal outcomes and practical feasibility given capacity constraints. 178. Importantly, the current work does not propose selection models that vary by sector. Instead, the size of a proposed intervention is the key variable that determines how a project should be evaluated, selected, and contracted. This is in line with broader thinking on integrated interventions (something the ROP is uniquely positioned to deliver on because of its focus on the broad field of regional development) and a move away from sector-specific procedures. In fact, there is value in putting forth standardized selection models and criteria that allow for easy comparisons across projects, regardless of their thematic focus. 179. Using the PIM framework, this report offers an assessment of the current ROP’s selection model based primarily on a project’s typical cycle (from preparation through implementation). This chapter builds on these insights and organizes them under two primary headings: (1) minimum recommended improvements, which apply horizontally to all three scenarios (including the “Status Quo+”); and (2) comparative strengths and weaknesses of each of the three project selection scenarios, taking into account the specific features of non-competitive vs. competitive selection, FIFO vs. pre- allocation, etc. Under both sections, this assessment applies the same overall PIM approach to show corresponding changes for each different stage in a project’s cycle. 180. A complementary way of segmenting recommendations in this section is separating “the WHAT” from “the HOW” and “the WHO.” Specifically: the selection model and criteria per se (i.e., “the WHAT,” as covered more extensively through the three different scenarios presented under the second section); the process and procedures through which the model delivers on its objectives of choosing optimal projects (i.e., “the HOW,” covered primarily in the next section on minimal adjustments); and the key stakeholders who essentially spin the wheels of the project selection model in practice (i.e., “the WHO,” covering the MA, IB, and beneficiary staff working on ROP projects). The figure below provides several examples that fit under each category. Figure 25. The WHAT, HOW, and WHO of project selection 87 Scenario 1- “Status Quo+” 181. Scenario 1 starts from the premise that one should not change a system that is working well. The Regional Operational Programme is the best performing OP and the project selection system that is currently in place has been continuously improved and has been adapted to challenges, needs, and problems that have arisen throughout the process. As such, the current system is well known by all parties involves (the MA, the RDAs, and the beneficiaries that have accessed ROP funds). Continuing with the current selection system basically means that there will be no lost time trying to learn and adapt to a new system. This, as well as the accumulated experience in previous years, will save time in the implementation of the selection system, and will ultimately help drive up absorption. 182. A detailed description of Scenario 1, following the PIM framework, is provided below: Table 17. Proposed selection procedures for Scenario I according to the Public Investment Management Framework Public Investment Management Framework Stage 1 – Investment Guidance, Stage 2 Stage 3 Stage 4 ROP Project Preparation & Preliminary Project Appraisal Independent Review of Selection & Contracting Procedures Screening Appraisal Project Preparation:  Permanent calls  Prepared by MA and approved by MC  By all ROP beneficiaries  Implicit strategic screening realized through eligibility and evaluation criteria for Axes 2-5 Strategic Screening : Guidelines and  Regional Development Plans by MA Announcement  Urban Development Integrated Plans (only for Growth Poles and for Urban Development Poles) by IB/RDA (Planning/Programming Unit)  Based on objective measurable criteria and clear procedures (proposed by the MA and approved by the Monitoring Committee of the ROP) Content:  Feasibility Study  Technical Studies  Cost Effectiveness Analysis  Operation and Application Maintenance Costs Submission  All are submitted in digital copies Project Preparation Costs: Direct beneficiary responsibility Administrative Centrally prepared evaluation grid & Eligibility Verification Centrally prepared evaluation grid Independent evaluators: Project passing certain Technical and  Selected by MA threshold (minim) score Financial  Payment sufficiently by category of evaluation Evaluation funded by MA criteria gets funded  Procurement criteria Pre-allocation and FIFO 88 weighed more on quality selection principles and experience than costs  Mandatory tasks: desk review, scoring, site visit, and making recommendations for project improvements in writing and in meetings with applicants  Evaluation criteria reflecting the objectives of ROP Axes and including the criteria from GEO 88/2013 Applicant improves Improvement project at feasibility of Project at study stage according to Feasibility the recommendations of Study Stage independent evaluators Verification of Independent evaluators project verify the improvement of improvement project at feasibility study & final stage and give final scores evaluation.  Financing Contract is signed between MA and applicant  Then applicant selects between the following options: Option 1: Red FIDIC: Two Financing contracts in sequence: Contracting first contract for design and second contract for build Option 2: Yellow FIDIC: Applicant contracts both design and build with a single company Option 3: Green FIDIC for small projects 183. The advantages and disadvantages of this scenario are discussed below, looking at four key criteria: absorption; impact; legitimacy; and feasibility. CRITERIA ADVANTAGES DISADVANTAGES  This scenario may help drive absorption,  Two key systemic issues which significantly affect maintaining the strengths of the actual system and absorption rates are not addressed: public the capacity that has been built in 7 years of procurement and the administrative/management implementation. There will be no time lag from capacity of beneficiaries. implementing a new system.  The pre-allocation system can reduce absorption  The new IDP screening procedures will not rates because beneficiaries do not feel the pressure ABSORPTION negatively impact the absorption rate because each to design and implement projects right away (as region has to screen a limited number of IDPs (one under the ROP 2007-2013 Axis 1 for Growth Poles for the growth poles; and 1-2 for urban and Urban Development Poles). development poles).  A focus primarily on small and medium-sized  Limiting the use of Cost-Benefit Analysis (CBA) only projects may bog down the system and generate to large projects may have an important effect on excessive bureaucracy. 89 CRITERIA ADVANTAGES DISADVANTAGES smoothing the process of submitting project applications. The Cost-Effectiveness Analysis (CEA) could be much easier to implement.  Introducing the red, yellow, and green FIDIC will have an important role in improving the project design and implementation.  Extending the use of e-procedures will have a direct positive effect on smoothing the submission and communication system.  A more strategic screening of IDPs can improve  The FIFO approach to project selection does not their quality and the overall impact of these local encourage the selection of large strategic projects development programs. This is an important step, (these usually take more time to design and as there are currently no incentives and penalties contract). for poorly designed IDPs.  The FIFO system does not permit a strategic focus  The CEA approach puts a higher emphasis on the on regional priorities and the selection of projects value-for-money principle, and since it is easier and with significant potential to boost development cheaper to use than the CBA approach, it may regionally. IMPACT encourage higher quality proposals. In the same  The proposed system is primarily geared toward vein, the CEA approach is easier to verify by absorption and there is usually a trade-off between evaluators and less susceptible to “window- measures that encourage absorption and those that dressing” strategies often used in CBA approaches. encourage impact.  The FIDIC system can help improve the quality of project design and implementation.  Enabling larger projects to be funded, can also lead to a greater overall impact of the programme.  The advantages of the current system in terms of  The pre-allocation of funds under Axis 1, particularly beneficiary and public trust remain unchanged. The the pre-allocation for Urban Development Poles, positive features of the FIFO approach are raises questions about criteria used to designate the preserved. UDPs. LEGITIMACY  The marginal improvements proposed largely have a positive effect on the relationship between the MA, the RDA, and the beneficiaries. As such, stakeholders may view this new model as a more solid approach than the current state of affairs.  Institutional systems will not be modified  The proposed marginal changes may require some significantly from the actual system – changes are adjustment (e.g., the introduction and marginal. This means that the current team implementation of e-communication procedures, FEASIBILITY (provided there is low staff turnover) would have FIDIC, etc.). little time lag in adjusting to the selection system,  The FIFO system tends to limit political interference 184. A general scheme of Scenario 1 is provided in Annex 12. Scenario 2 - “Semi-competitive” 185. Scenario 2 starts from the premise that a significantly higher impact can be achieved if funds are pre-allocated for a number of strategic projects. To achieve maximum impact, it is proposed that funds be targeted at regions, counties, growth poles, and county capital cities. The decision to extend 90 the list of cities with pre-allocated funding form urban development poles to county capitals came out of the analysis for the Growth Poles projects, which indicates that in economic terms there are often small differences between cities of smaller size. Moreover, there are county capitals, such as Slatina, which have a higher economic output than urban development poles like Brăila, Satu Mare, Bacău, or Suceava. County capitals also serve as administrative centers and centers of service provision. We have also decided to include county councils among the administrative units that receive dedicated funding, as a way of helping them target investments in a more strategic way – i.e., projects should be drawn from a comprehensive county development strategy. 186. Four types of projects are envisaged within this framework. The four types were designed to allow a balance between impact and absorption – i.e., allow some smaller projects to move faster through the system, while also trying to select a number of larger, more strategic projects. The individual types are described below. 187. Type 1: Large Regional Projects. This type is specifically designed to enable the funding of truly regional projects of over 100 million RON. Due to a number of limitations (e.g., lack of an administrative tier at the regional level), there have been only a few truly regional projects that implemented under the ROP 2007-2013. For the ROP 2014-2020, it would be important to consider targeting a number of strategic regional projects within each region. Such investments can be drawn from the Regional Development Plans and would be strategically screened by the Managing Authority of the ROP. It is also important to consider that the newly passed Government Emergency Ordinance 88/2013 indicates that the Ministry of Finance will do an additional vetting for projects of more than 100 million RON, leaving open the possibility that this additional filter would apply to EU-funded interventions. The criteria for the prioritization of large projects (over 100 million RON) that receive state budget funding are included in Annex 13. 188. Type 2: Integrated Development Strategies. This type targets growth poles, county councils, and county capitals, primarily aiming for large projects and upper-middle-sized projects. The idea is to have pre-allocated funding for all major sub-regional administrative units, but with an important twist: given that in the 2007-2013 programming period absorption has been rather sluggish under Axis 1 (as funds were guaranteed for growth poles and urban development poles), this scenario proposes a two- staged approach. Thus, each county, each growth pole, and each county capital would receive dedicated funding, but it would have to contract allocated sums within a set timeframe (e.g., no longer than 2-3 years). All funds that are not contracted go into a competitive pool and can then be accessed on a First- In-First-Out basis by other similar administrative units (counties can access un-accessed funds from other counties, cities can access un-accessed funds from other cities, all within the same region). In addition to pre-allocated funds, growth poles, counties, and county capitals, will be able to access an open pool of funds (the way it is done currently) on a First-In First-Out basis, after contracting pre- allocated funds. 189. Type 3: FIFO Selection of Medium-Sized Projects. Type 3 follows the current selection model, as it is applied to Axes 2, 3, 4, and 5, with the improvements that were mentioned before the scenario discussion. All types of beneficiaries can apply for funding under Type 3: public authorities, private companies, NGOs, and Local Action Groups (LAGs) (which would implement CLLD projects). 91 Overview of Scenario 2 – “Semi-competitive” Project Project Fund Allocation Selection Applicant Selection Model Payment Authority Type Size Authority Authority Pre-allocation Central MA Associations of  Central MA Central MA  Dedicated budget with limited flexibility for re- > 100 Counties/ Cities allocates funds allocation to other project types Type 1 million or future to each region  Projects must be listed in Regional Development RON regional Plans administrative  Central MA sets a  Explicit Strategic Screening in accordance with entities range of fund OUG 88/2013 Counties (41) allocation for RDA or a future Pre-allocation and FIFO Central MA or a future each type of regional Phase 1. Pre-allocation regional administrative Growth Poles project per administrative  Dedicated allocation for each applicant entity (7-8) region entity within a set deadline  Projects must be in accordance with OUG Urban  Regional Council 88/2013 and abide by the prioritization Development or a future criteria Poles regional entity  Projects must be listed in Development (34 – rest of decides fund Strategy or equivalents county allocation for  Restricted call of proposals embedding residence each type of strategic criteria municipalities) project within  Technical and financial evaluation using a >30 the range threshold Type 2 million RON  Funds allocated Phase 2. First-In-First-Out Selection among types of  Pre-allocated funds for Phase 2 and non- project are committed dedicated funds from Phase 1 for flexible within counties are available in Phase 2-FIFO for the range set by competitively selected projects among all the MA and counties within a region within the total  Pre-allocated funds for Phase 2 and non- budget allocated committed dedicated funds from Phase 1 for for each region growth poles and urban development poles set by the MA are available in Phase 2 for competitively selected projects among all growth poles and urban development poles within a region 92  Restricted call of proposals (restrict the domains to be funded)  Technical and financial evaluation using a threshold Public RDA or a future FIFO: First-In-First-Out Selection Central MA or a future authorities regional  Dedicated allocation funds for this Type of regional administrative Private sector administrative projects within a deadline entity < 100 NGOs entity  Restricted call of proposals (restrict the million LAGs domains to be funded) RON; Type 3  Medium Projects must be in accordance > 1,5 with OUG 88_2013 and to respect the million prioritization criteria RON  Technical and financial evaluation using a threshold Public RDA or a future FIFO: First-In-First-Out Selection Central MA or a future authorities regional  Restricted call of proposals (restrict the regional administrative Private sector administrative domains to be funded) entity < 1.5 NGOs entity  Simplified submission forms and procedures Type 4 million LAGs  Technical and financial evaluation using a RON threshold  To be analyzed involvement of banks in the selection process 93 190. Type 4: Small Projects. This type is specifically designed for small projects, with simplified application procedures to enable a quick move through the system. All types of beneficiaries can apply for such projects. 191. The table above provides a full overview of Scenario 2. 192. The advantages and disadvantages of this scenario are discussed below, looking at four key criteria: absorption; impact; legitimacy; and feasibility: CRITERIA ADVANTAGES DISADVANTAGES  Pre-allocating funds for a number of large projects  Project preparation for large projects and upper- may significantly reduce processing times and middle-sized projects may take a long time, as these bureaucracy. projects tend to be complex.  Limiting the time that growth poles, county  The FIFO approach may continue to encourage the councils, and county capitals have to access pre- submission of poor quality projects (with reduced allocated funds will help prohibit unnecessary preparation times), which may lead to delays in the delays. implementation phase.  The FIFO approach will enable relatively quick  Large regional projects may require that Inter- ABSORPTION absorption, particularly for small and medium-sized Community Development Associations be put in projects. place (if the regionalization process will not be  The introduction of FIDIC contract forms may help completed), and such associations are from smooth the implementation phase and reduce experience hard to establish and manage. This may delays. make project implementation cumbersome.  There is a risk that sub-national entities, particularly county councils, will not have the financial capacity to take on additional EU projects.  The concrete focus on large projects, at regional,  The FIFO approach for some of the projects may not county, and city levels will help achieve a bigger necessarily enable the selection of the most impact. strategic projects.  FIFO approach generally reserved for smaller, less  Regional and local political interests may drive the strategic projects. selection process of strategic projects, which in the  Requiring that an integrated development plan be end might mean that not the most impactful IMPACT projects will be selected. the basis for financing such projects will encourage not only more strategic projects, but also a better  There is a risk of receiving unsustainable projects coordination between strategies at different levels with a high price tag and a small actual impact. (i.e., regions, counties, and cities will have to have integrated development strategies in place when they apply for pre-allocated funds).  Pre-allocation and introducing strategic pre-  Potential to have regional and local interests drive selection criteria can reduce the pressure on the project selection, rather than clear strategic criteria. MA, IBs and assessors and increase accountability with regional and local decision makers.  Projects proposed for funding will be established at LEGITIMACY regional or local level through broad consultation with all stakeholders.  Efforts in the assessment phase will be focused toward proposing complete projects, ready for implementation.  There already is a lot of experience with the  The preparation of integrated development FEASIBILITY implementation based on a FIFO approach and with strategies (at the growth pole, county, and county pre-allocated funding for growth poles and urban capital level) will require a lot of capacity at the 94 CRITERIA ADVANTAGES DISADVANTAGES development poles. local, regional, and national level – both for the actual elaboration of these plans and for their strategic screening.  There may not be enough specialists to enable both the elaboration and the proper screening of these strategies. Some specialists may not be experienced enough in their respective fields. 193. A general scheme of Scenario 2, as well as a discussion around the PIM framework of each type of project, is provided in Annex 12. Scenario 3 - “Competitive” 194. Like Scenario 2, Scenario 3 proposes that projects be selected differently depending on their size. Projects of smaller size follow simplified selection procedures and less demanding selection criteria. There are four types of projects. Type 1 projects, “regional-impact projects”, are large investments, with values of at least 100 million RON. Applicants are associations of counties or potential future regional administrative entities. Type 2 projects, “county/growth pole/urban development pole projects,” are upper-medium-sized projects of more than 30 million RON, proposed by counties, growth poles, or county capitals. Type 3 projects are medium-sized projects of more than 1.5 million RON and less than 100 million RON, put forth by local administrations, private sector entities, NGOs, or Local Action Groups (LAGs) (who would implement Community Led Local Development – CLLD – projects). Type 4 projects are small, under 1.5 million RON, submitted by local administration, private sector firms, NGOs, and LAGs. The table below provides a summary of the four project types proposed. 195. The budget allocation for the next ROP will be more flexible within and among project types. The central Managing Authority (MA) will allocate funds to each region. Within this regional allocation, the MA will set a certain percentage for each type of project and a percentage range within which each region can decide how much funding it will allocate for each type of project. For example, the MA decides that 25% of the total funds allocated for each region will be used for Type 1 projects and that each region can allocate between 20% and 30% of the total funds based on the region’s priorities and availability of project proposals for Type 2 projects. Within this range set by the MA for each type of project and requirements set by the EU, each region can decide on the fund allocation among project types. This flexibility in budget allocation would ensure that it still meets the main objectives of the ROP and, at the same time, it remains tailored to the regional priorities and capacity in ROP implementation of each region, for enhanced absorption of ROP funds. 196. Pre-determined funds allocation to applicants through Integrated Development Plans may encourage planning and integration of investments, but also create incentives for delays in project submission and low quality projects. In order to promote the strengths and reduce the weaknesses of the current pre-allocation model, this report proposes a “semi pre-allocation” model. Thus, total funds allocated for Type 2 projects would be divided into two phases: the majority of the total funds would be pre-allocated to applicants in Phase 1; and a smaller share would be entered into a competitive pool in Phase 2. For example, 50% of funds could be available for contracting within the first three years, and 50% for the last four years. The funds for Phase 2 will comprise the competitive funds allocated at the beginning of the period, as well as un-contracted pre-allocated funds from Phase 1. The competition will be within priority axes and between similar types of applicants. Specifically, competitive funds for Phase 95 2 and non-committed dedicated funds from Phase 1 for counties are available in Phase 2 for competitively selected projects among all counties within a region. Competitive funds for Phase 2 and non-committed dedicated funds from Phase 1 for growth poles and urban development poles are available in Phase 2 for competitively selected projects among all growth poles and urban development poles within a region. This semi-competitive model, i.e., non-competitive among applicants in Phase 1 and competitive among applicants in Phase 2 will ensure that no applicants are left behind and it will also reward more engaged and capable applicants. 197. Type 3 projects would be selected competitively based on a thorough technical and financial evaluation or by using a more qualitative assessment method. There would be a restricted call for proposals embedding strategic criteria, after which evaluators will rank projects against each other within each priority axis. It will be important to have sufficient capacity available to properly process all applications at this stage, particularly during peak times. 198. Finally, type 4 projects will maintain the current FIFO rule. In this case (i.e., smaller investments with reduced strategic impact), it makes sense to preserve the status quo system and its inherent benefits, including an accelerated absorption rate. In addition, there should be simplified forms and procedures to reflect the smaller value of these types of projects. 199. Applicants will have opportunities for improving projects at the feasibility study stage. Independent evaluators, upon conducting the technical and financial evaluation/desk review, and after the site visits, will not only give scores but also recommendations for project improvements. Applicants would then enhance projects according to the recommendations and evaluators would verify the adjustments made and give final evaluation scores. Projects will be selected based on these final scores. 200. Strategic screening is one of the key determinants for the success of the proposed selection models. Therefore, strategic screening criteria, procedures, and committees require very careful design. Strategic screening can be done by a RCSEC-like strategic screening committee comprising both central and regional representatives and based on objective measurable criteria and clear procedures (proposed by the MA and approved by the Monitoring Committee of the ROP). The new strategic committees must have the following features: clear criteria and procedures for member selection; a balanced structure for membership composition; and clear criteria and detailed procedures for strategic prioritization. Annex 4 includes a more complete proposal for a new strategic committee. Screening criteria will be based on strategic documents, among which Regional Development Plans currently drafted for the 2014-2020 period are some of the most important guidelines. The draft RDPs put an emphasis on comparative and competitive advantages specific to each region, ensuring a better link to strategic program-level objectives. Annex 6 includes an analysis of the 2014 – 2020 RDPs. 96 201. The table below provides a full overview of Scenario 3. Overview of Scenario 3 – “Competitive” Project Project Fund Allocation Selection Payment Applicant Selection Model Type Size Authority Authority Authority Associations of  Central MA Central MA Pre-allocation Central MA Counties/Cities allocates funds  Dedicated budget with limited flexibility for re-allocation to or future to each region other project types > 100 regional  Restricted and permanent call of proposals Type 1 million administrative  Central MA sets  Projects must be listed in Regional Development Plans RON entities a range of fund  Explicit Strategic Screening for RDPs and specific projects, allocation for including the requirements and criteria set by GEO 88/2013, each type of by MA/MoF/CRESC type committee Counties (41) project per RDA or a Pre-allocation and Competitive Selection Central MA or a region future regional Strategic Screening: Explicit for IDPs (and for specific projects future regional Growth Poles administrative only under the requirements of GEO 88/2013 – i.e., over 100 administrative (7-8)  Regional Council entity million RON) by RDA/MoF entity or a future Phase 1. Pre-Allocation Urban regional entity  Dedicated allocation for each applicant within a Development decides fund deadline Poles allocation for  Projects must be listed in the Development Strategy or (34 – rest of each type of equivalents county project within  Restricted and temporary call of proposals embedding residence the range strategic criteria >30 municipalities)  Technical and financial evaluation using a threshold for Type 2 million  Funds allocated every category of criteria RON among types of  Evaluation criteria will directly reflect the objectives of projects are the ROP specific priority axis and the requirements of flexible within GEO 88/2013 the range set by the MA and Phase 2. Competitive Selection within the total  Competitive funds for Phase 2 and non-committed budget dedicated funds from Phase 1 for counties are allocated for available in Phase 2 for competitively selected projects each region set among all counties within a region by the MA  Competitive funds for Phase 2 and non-committed dedicated funds from Phase 1 for growth poles and 97 urban development poles are available in Phase 2 for competitively selected projects (selection can be done using quantitative or qualitative selection methods) among all growth poles and urban development poles within a region  Projects are selected twice a year  Restricted and temporary call of proposals embedding strategic criteria, which reflect the objectives of ROP specific priority axis and the requirements of GEO 88/2013  Technical and financial evaluation using ranking within specific priority axis < 100 Other cities RDA or a Competitive Selection Central MA or a million Private sector future regional  Restricted and temporary call of proposals embedding future regional RON; NGOs administrative strategic criteria, reflecting the objectives of ROP specific administrative Type 3 > 1,5 entity priority axis and the requirements of GEO 88/2013 entity million  Technical and financial evaluation or qualitative assessment RON using ranking within specific priority axes Other cities RDA or a First-In-First-Out Selection Central MA or a Private sector future regional  Restricted and permanent call of proposals (restrict the future regional < 1.5 NGOs administrative domains to be funded) administrative Type 4 million entity  Simplified submission forms and procedures entity RON  Technical and financial evaluation using a threshold for every criteria category 98 202. Lastly, it needs to be mentioned here that, while the suggested move toward more competitive selection models is generally positive, there are certain downsides of competitive approaches (e.g., related to capacity needs when preparing and processing calls, narrowing the range of beneficiaries, longer waiting times on decisions). Obviously, as with all systems, trade- offs need to be made, but it is important to keep in mind potential capacity-building needs. Capacity building should be an important consideration in response to problems, which are likely to arise due to the introduction of new and complex systems. 203. The advantages and disadvantages of this scenario are discussed below, looking at four key criteria: absorption; impact; legitimacy; and feasibility: CRITERIA ADVANTAGES DISADVANTAGES  Pre-allocating funds for a number of large projects  Project preparation for large projects and may significantly reduce processing times and upper-middle-sized projects may take a long bureaucracy. time, as these projects tend to be complex.  Limiting the time that growth poles, county  Competitive selection may require cumbersome councils, and county capitals have to access pre- and lengthy procedures and negatively affect allocated funds will help prohibit unnecessary absorption. Delays may also be caused by the delays. need to have staff learn to implement and get  The introduction of FIDIC contract forms may help accustomed to the new selection procedures. smooth the implementation phase and reduce  Large regional projects may require Inter- ABSORPTION delays. Community Development Associations to be put  This scenario may encourage absorption by using in place (if the regionalization process will not the competition principle together with pre- be complete) and such associations are from allocation. experience hard to establish and to manage. This may make project implementation cumbersome.  There is a risk that sub-national entities, particularly county councils, will not have the financial capacity to take on additional investment projects.  The concrete focus on large projects at regional,  Regional and local political interests may drive county, and city level will help achieve a bigger the selection process of strategic projects, impact. which in the end might mean that not the most  Competitive selection (either quantitative or impactful projects will be selected. qualitative) imposed for all medium-sized projects,  There is a risk of receiving unsustainable meaning an increased focus on impact. projects with a high price tag and a small actual IMPACT  Requiring that an integrated development plan be impact. the basis for financing will encourage not only more strategic projects, but also a better coordination between different level strategies (i.e., regions, counties, and cities will have to have integrated development strategies in place when they apply for pre-allocated funds).  Pre-allocation and introducing strategic pre-  Potential to have regional and local interests selection criteria can reduce the pressure on the drive project selection, rather than clear MA, IBs, and evaluators, and may also increase the strategic criteria. accountability of regional and local decision-  Competitive selection criteria (particularly LEGITIMACY makers. qualitative selection criteria) may be questioned  Projects proposed for funding will be established by beneficiaries whose projects have not been at regional or local level through broad selected. consultation with all stakeholders.  Efforts in the assessment phase will be focused 99 toward proposing complete projects, ready for implementation.  Adding the competitive selection principle to pre- allocated projects may create positive incentives and by extension also increase legitimacy, which could also be helped by a well-designed, transparent, and functional strategic screening committee.  There already is experience in the system with  The preparation of integrated development pre-allocated funding for growth poles and urban strategies (at the growth pole, county, and development poles. county capital level) will require a lot of capacity at the local, regional, and national level – both for the actual elaboration of these plans and for their strategic screening. FEASIBILITY  There may not be enough specialists to enable both the elaboration and the proper screening of these strategies. Some specialists may not be experienced enough in their respective fields.  Competitive selection methods may require some adjusting from people working in the system. 204. A general scheme of Scenario 3, as well as a discussion around the PIM framework of each type of project, is provided in Annex 12. 100 VIII. Improving the Current Evaluation and Selection System 205. The purpose of this section is to propose a series of improvements to the ROP’s current evaluation and selection model – these recommendations would apply to all three scenarios described above. The discussion will be split into two sections: the first part of the discussion will cover general aspects of the evaluation and selection system; the second part will focus more on the mechanics and proposed marginal changes to the current evaluation and selection system, particularly the Technical and Financial evaluation grids. This will be followed by a number of other chapters, which will discuss in more detail distinct proposals made for individual scenarios, with a particular focus on the “Semi-Competitive” and “Competitive” options. Annex 25 discusses some of the new requirements of the European Commission when it comes to project selection, with a stronger emphasis on project outcomes and impact. Recommended adjustments for all proposed selection models 206. Regardless of the scenario ultimately chosen, there is a set of changes that promise to deliver improved outcomes relative to one or several of the key objectives – absorption, impact, legitimacy, and capacity. Recommendations span multiple stages in the PIM framework, from project preparation through actual evaluation and contracting, and they are meant to apply all possible scenarios under consideration. In that sense, these are minimal changes that the status quo requires in order to ensure better project selection under the ROP 2014-2020. Strategic assessment of Integrated Development Plans 207. Integrated development plans have been the main tool used by the ROP 2007-2013 to fund more strategic and impactful projects, by focusing investments on the largest regional urban centers. To achieve their objective, it is important to have high-quality Integrated Development Plans. This means that proper application guides are needed, requiring critical information, along with proper screening procedures to ensure that IDPs meet quality requirements.23 208. The screening procedures used currently focus primarily on administrative issues and do not really permit a strategic assessment of these documents. The figures in the Annex 14 indicate how IDPs are screened for Growth Poles, Urban Development Poles, and Urban Centers. The interesting thing is that screening procedures vary for these different types of cities, and screening criteria do not necessarily encourage the elaboration of good documents. 209. The table below provides an overview of actual evaluation criteria for growth poles and urban development poles, and ways in which these criteria could be improved: 23 It is important to note that this discussion on IDPs is done considering that IDPs will continue to be the main tool used to assess pre-selected projects. Later on, this report discusses the option of doing pre-allocation based on existing development strategies and a Capital Investment Plan. 101 Table 18. Improving IDP evaluation criteria Actual Evaluation (Admissibility) New Evaluation Criteria Criteria for Growth Poles and Urban Development Poles 1. IDP has as objective the social and A. Are IDP objectives consistent with the Regional economic development and the urban Development Plan (RDP)? regeneration of the city/municipality growth pole/urban development pole 2. The development needs of the urban B. Is the IDP consistent with ROP or ROP axis objectives? development pole are correlated with the results of the social and economic analysis and with development priorities. 3. IDP targets the simultaneous solving C. Is the analysis of problems/context coherent and adapted of a series of interdependent issues to the purpose of these strategic documents (IDP)? 4. (Only for urban development poles) D. Are there interdependency relations among the main IDP is accompanied by at least 2 identified problems? individual projects:  The projects fit into a minimum of two different categories of eligible operations  At least one of the projects fits within the eligible operation category regarding the rehabilitation of urban infrastructure and improving urban services, including urban transport 5. Individual project will be E. The identified development needs result from the analysis implemented in the area of the urban of problems/context and are reflected by development development pole, as resulting from the priorities/objectives? localization project map of the city/municipality 6. Individual projects are aligned with F. Are the IDP objectives SMART (Specific, Measurable, the development needs identified and Achievable, Realistic, Time-bound) and with assigned the development priorities of the performance indicators? city/municipality. 7. (Only for growth poles) Management G. Measures/Projects contained by IDP are logical consistent structure of IDP is clearly identified and with the objectives and directly contribute to their can assure an adequate achievement? implementation? H. The co-financing is available? I. The commitment to assure the sustainability (especially regarding the operation/maintenance costs) is credible? Will be sufficient fiscal/budgetary space in the future for these expenditures? How this budgetary space will be created/covered? J. Management arrangements are clear and credible to assure an efficient implementation of the IDP? 102 210. Such criteria should not be used only for IDPs developed for cities, but also for IDPs developed for other administrative units (e.g., counties). Similar evaluation criteria may be used by the MA-ROP to vet Regional Development Plans and ensure that these meet quality standards. 211. Below we have provided a number of key strategic categories and evaluation criteria/questions, which could form the basis of the assessment of a development strategy:  STRATEGIC ALIGNMENT : o Does the project align with ROP objectives? (If not, remove from pipeline / redirect to other) o Is the project needed by the community? o Does it align with existing strategies at various levels (e.g., regional, county, sectoral, etc.)? o If not already captured in an existing strategy, is it aligned with the people's needs and expectations? What proof do we have of that (e.g., surveys, petitions, consultations, etc.)?  CAPACITY o Two critical sub-components: Human and Financial Resources o Will this project be carried through by a qualified team? o Does it have sufficient resources for its timely completion, from project preparation through implementation and post-implementation? o Bonus points if project is already partly implemented, if they already have a technical project, etc.  QUALITY o Technical and financial o Does the project put forth the most effective technical solution for achieving its objectives? o Does the project propose the most efficient financial solution? o Or, put differently, in balancing efficiency and effectiveness, is the solution proposed optimal?  SUSTAINABILITY: o Is the project self-sustainable in the long term? If yes, how? o If not, are there resources allocated to cover the net deficit post-completion? Introduction of Applications for Pre-Qualification (APQ) 212. Particularly for programs where many submissions are likely, the introduction of an application for pre-qualification (APQ) phase would help improve the system’s efficiency and reduce duplication for both beneficiaries and MA/IB staff. Essentially, the goal of the APQ is to provide a first filter of applicants who intend to submit an application for ROP financing. For applicants, the APQ provides the opportunity to validate their project ideas and required documentation before actually investing all the required resources into preparing an entire application. This would avoid instances of potential beneficiaries who find out late in the process that the fact that they are missing a certain property document means that their project is not eligible for ROP funding. Moreover, an effective APQ would not simply turn applicants away from the program, but rather it would try to guide them into the right direction, either toward additional recommended steps before an actual application can be submitted or toward another Operational Programme, as needed. In this sense, the APQ is part of the broader 103 support and guidance mechanisms available to interested applicants, who can leverage the IBs’ points of contact to exchange preliminary ideas on various possible interventions and their likelihood to succeed. At the same time, it is important to consider that there may be some restrictions for using APQs for Competitive Calls. The Germany case study in Annex 10 indicates that a balance has to be found between support and preferential treatment of applicants. 213. An APQ can also be a relatively easy and cost-effective way to reduce the burden on MA and particularly IB staff, who have to go through all applications submitted for the administrative and eligibility check. It would mean that critical resources actually focus more of their time on those viable project ideas that actually have a shot at receiving ROP funding, provided they receive a favorable evaluation. This would help reduce the pressure, particularly during peak times (e.g., before a submission deadline), and promises to also lower the average processing time for all applications that end up entering the pipeline and passing the initial APQ. At the same time, it is important to keep in mind that adding this new filter will require some additional resources, especially at the IB level, where specialists should be available to discuss with interested applicants and provide both formal and informal feedback throughout the APQ process. 214. A February 2012 report on “Comparative Study of the Project Selection Process Applied in Cohesion Policy Programmes 2007-2013” recommends the introduction of APQ under certain conditions. In particular, the assessment praises the Czech Republic and Germany for instituting such a process with relatively good results. The authors conclude: “APQ procedures effectively filter out projects that are not eligible and do not fit into the funding scheme, at an early stage of the process. Hence, early feedback is provided to applicants on their chances of success, which reduces uncertainty and the administrative burden […] at the first stage, which is less extensive and time consuming than the second (formal application).”24 For example, in both the Czech Republic and Germany, the APQ form is brief and simple, and it is also available online. In fact, a staggering 90% of applications in Germany are submitted electronically, which is another recommendation of this report and of the concurrent World Bank work on MA-IB collaboration and ROP beneficiary support. The same report cited earlier introduces the idea of an online self-check tool that would allow interested applicants to receive instantaneous feedback on whether their idea is eligible under a particular program. This solution would not only create an easy way to guide applicants at a very early stage in the project preparation process, but it would require very little resources on the part of the IBs/MA, and could also be scaled up relatively easy. 215. Importantly, the current help desks established at the RDA level effectively fulfill a similar role, although their intervention is limited – i.e., demand-driven. In other words, helpdesks intervene only when an interested applicant comes forward with a question related to the procedures/criteria/etc. It is required that all projects go through this initial check, which is why some applicants make it further into the process than necessary. An expansion of the helpdesk functions to cover the full set of applicants, in a more formal pre-qualification phase, remains a recommendation worth considering for the future ROP – certainly, it would be preferable to build on the already existing infrastructure (i.e., helpdesk specialists, where available and where properly trained). The figure below synthesizes the arguments presented in this sub-section. 24 European Commission, DG REGIO, “Comparative Study of the Project Selection Process Applied in Cohesion Policy Programmes 2007-2013,” Final Report, February 2012. 104 Figure 26. The introduction of an APQ phase appears to be a win-win for all sides involved Prequalification Evaluation Outcome Eligible Eligible Eligible Eligible Noneligible Eligible Successful Noneligible Eligible Noneligible Eligible Eligible Successful Noneligible • The introduction of an application for prequalification (APQ) phase would redirect projects that are unlikely to succeed under the ROP toward more suitable opportunities • This would enable beneficiaries to save time and look for eligible opportunities • IB/MA would face more manageable workloads by putting only eligible projects through a formal evaluation Elimination of Cost-Benefit Analysis for Small and Medium-Sized Projects 216. The requirement to include a Cost Benefit Analysis (CBA) for ROP projects submitted during the 2007-2013 cycle has generated a range of negative unintended consequences. In effect, the initial push toward using this instrument derived from a desire to ensure that ROP- financed interventions are truly impactful, with benefits outweighing costs. In theory, the CBA is indeed a powerful tool used around the world to determine whether a particular project is worth undertaking and, ceteris paribus, it provides more complete information than a simple Cost Effectiveness Analysis (CEA). In practice, however, the introduction of the CBA did not align well with the current capacity constraints specific to the Romanian context of structural funds management and implementation. 217. As noted earlier in this report, the reasons for why the CBA did not deliver on its intended goals are multiple-fold. First, the data needed to substantiate various assumptions about anticipated costs and benefits are often hard to access and, in some cases, do not exist at all. Ironically, the costs incurred with running a proper CBA (e.g., collecting, processing, and interpreting data) often exceed the anticipated benefits – put differently, the CBA of a CBA, for a typical ROP project, would likely result in a negative net present value. Second, even if this endeavor was worth it, most beneficiaries do not have the internal capacity to implement a comprehensive CBA, while external consultants with adequate experience in deploying this tool are too few and in high demand. 218. In addition, evaluators who are supposed to determine whether a CBA is accurate have not always taken the time to validate the assumptions behind calculations presented within submitted proposals. As part of the field research for this project, a range of interviews 105 with RDA representatives revealed, once again, the lack of accountability mechanisms for ensuring quality evaluations. In other words, there are no strong incentives for evaluators to ask clarifying questions to unveil how applicants determined a project’s costs and benefits. This, coupled with the lowest-cost selection criterion for evaluators, opened the door to cutting corners and simply “checking the box” if the internal rate of return was above the agreed threshold, even in instances where the calculations were based on unrealistic assumptions (e.g., increase in customers by an order of magnitude, underestimated operations and maintenance costs for the long term, etc.). For such reasons, the CBA is a good example of a powerful and useful concept (in theory) that has generated significant challenges in practice, sometimes failing to promote better impact projects, while often slowing down absorption, further burdening the system’s capacity, and reducing the credibility of evaluation results. 219. In addition to its practical shortcomings, the CBA is not legally mandatory for ROP projects. On the national level, Government Decision 28/2008 and the more recent Government Emergency Ordinance 88/2013 note that CBAs should be included for major projects, i.e., over 100 million RON (22.2 million EUR). As noted earlier in this report, the current caps on ROP project sizes exclude de facto the requirement for a CBA. At the European level, the EU Guide to Cost Benefit Analysis of Investment Projects notes: “Cohesion Policy regulations require a cost- benefit analysis of all major investment projects applying for assistance from the Funds. The legal threshold for the definition of the ‘major’ investment is 50 million EUR in general, but for environmental projects it is 25 million EUR and for IPA assisted projects, 10 million EUR.” None of these criteria apply to ROP projects for the current budgeting cycle and it is likely that this will not change during the next programming period, i.e., no ROP project would exceed 50 million EUR. 220. In sum, a full economic CBA is not recommended as a selection criterion for small and medium-sized projects given the lack of detailed guidelines and lack of a proper quality control process to audit the CBA analyses. CBA should be required where it is mandatory under EU guidelines based on project size. Instead of CBA, project selection could instead rely on: (a) the participatory political planning process undertaken by local authorities to identify local priorities, develop strategies and plans, and select/vote on projects; and (b) results-based indicators of project outputs or services according to the sector/project type. Where feasible, these indicators should focus on the primary beneficiaries or users and degree of benefit (for example, traffic flow on a road and the time saving per trip.) Indicators of outputs or service delivery from a project can be used as a selection criterion. A full CBA includes consideration and valuation of all the externalities generated by a project. Many externalities can be assumed to be achieved if the project is consistent with national standards for safety, pollution, etc., rather than attempting to estimate their value for each project. For example, a road should be technically designed in terms of national standards that reflect safety standards in terms of lane width, camber, drainage, signage, etc. The selection criteria should be limited to the determination that the design standards have been achieved. 221. One alternative to the full Cost-Benefit Analysis is the Cost Effectiveness Analysis (CEA), which can maintain a rigorous focus on achieving the desired outcome through the best possible means, while entailing simpler procedures and a lower burden on the ROP system’s capacity. Cost-effectiveness focuses on measuring the unit cost of producing key outputs (services, results or “effects”) or possibly final outcomes in the economy. Generally, it is expected that projects with lower unit costs of production should be selected as promoting 106 technical efficiency. Alternatively, when comparing two projects of similar type and scale, for example, two county rural roads, the project with the higher expected “effects” should be selected, assuming they have similar underlying designs and costs per kilometer. In the case of a road, the effects could be measured at minimum as the highest daily traffic flow or better as the highest total time saved per day. The formal calculation of unit costs of an “effect” is covered in Annex 21. 222. The table below compares the CBA to the CEA, noting the comparative advantages of the latter, as well as some instances where the former may still be appropriate to use . All in all, the CEA is easier to implement by ROP applicants and also easier to evaluate by experts contracted at the MA/IB level, thereby the expectation is that the quality and timing of both project preparation and evaluation will improve. At the same time, the CBA does not need to be completely excluded from the selection model. In some cases, it can still provide useful inputs, particularly for more complex, larger projects that involve a set of different integrated interventions (e.g., Type 1 projects, Integrated Territorial Investments – ITI, etc.). Table 19. A comparison of CBA and CEA highlights the latter’s superior fit with the current ROP Cost Effectiveness Features Cost Benefit Analysis Comments Analysis A dynamic A static technique to CBA uses the technique to measure measure efficiency by discounting technique efficiency (in Pareto comparing, for different for the time value of sense) by comparing all alternatives, the total money, along with Definition costs and benefits (both costs (expressed in many complicated expressed in monetary monetary terms) per assumptions regarding terms) one main outcome (not the monetary value of in monetary terms) qualitative benefits and costs Assess all benefits, Only one benefit is CEA avoids monetary direct and indirect, by considered – the main calculations for all expressing them in outcome – and benefits, but presumes monetary terms expressed in its physical a pre-defined resource Benefits form (e.g., number of allocation (this is well saved lives, number of aligned with the profile vaccinated children, of ROP projects) etc.) Assess all direct, Assess only direct and CEA does not assess or indirect, and indirect costs, expressed try to quantify opportunity costs, in monetary form opportunity costs, Costs expressed in monetary which can be a terms tremendously complex exercise  Net Present Value  Cost-Effectiveness It will be much easier to (NPV) Ratio: CEi = Ci/Bi verify a CEA, but the  Internal Rate of  Ci = Cost per beneficiary manuals Return (IRR) project may become more Main indicators  Bi = Benefit per complex and would project (for only need to contain one main outcome thresholds for the CEi, indicator) for different axes/key intervention areas 107 Cost Effectiveness Features Cost Benefit Analysis Comments Analysis Technical efficiency and Technical efficiency only For a “pre-designed” efficiency in funding ROP with allocations per What does it allocation axes, technical measure? efficiency is what truly matters and CEA is easier to implement High Medium – Low CEA easier to be evaluated, promising Complexity more objective outcomes and improved process credibility Does it allow a cross- Yes No sectoral analysis? Multi-sector projects Single-sector project CEA may be difficult to implement for integrated interventions Type of projects currently financed under ROP Axis 1.1 (i.e., Integrated Urban Development Plans) 223. For all its benefits, the CEA is not a foolproof method either. Some of its limitations include: a. A project can deliver multiple services or affect a range of outcomes in an economy either directly or indirectly. The CBA aims to put a value on each of these effects or outcomes and aggregate the benefits and costs of these into a total net benefit – the CEA does not. In some cases, a weighted average index of effects can be used to measure the project effect – for example, a grade point average could be used to measure the overall improvement caused by the upgrading of an educational institution providing training in a number of disciplines. b. The effect of a project typically has at least two dimensions –an indicator of the number of users and the change in value per user (basically, the impact of a project on quantity and price of a service). For example, a road improvement can benefit existing users as well as induce an increase in road users. At the same time, the most of the gain to the road users arises from time saved per trip. Other gains also come from savings in fuel efficiency, wear and tear on vehicles, decreased accidents, etc. The measurement of the impact of a project will be limited only if the traffic flow is known (but this is clearly better than not knowing the traffic flow.) The effects are better measured if the time saving can also be estimated from improved average speeds. These two measures can be combined into a better measure as the total amount of time saved by users of the road. In a CBA, this gets converted into a value based on the marginal values of labor market time of the types of road users. c. In some sectors, the immediate output effects are not good indicators of the ultimate economic impact; outcome measures are preferred measures of project effects. While in the case of road improvements the effects on traffic flows and 108 speeds captures most of the economic gains,25 in other sectors such as health the outcomes are more important indicators of benefits. Given the impact of medical treatment can have a wide range of impacts on different populations in various environments and sometimes with delayed effects or over long periods of time (a vaccination today may protect someone over a long period, but also have no impact if the person is never exposed to the infection), the preferred approach is to focus on outcome measures. While the unit cost of medical service delivery is important for budgeting or contracting purposes, for economic decision-making purposes, the unit cost of the outcome of a medical service is more important. Hence, estimates of the unit cost of saving a life (preventing a death) or adding a year to the life of a patient is the preferred approach. This preferred approach demands extensive epidemiological data to be able to estimate the impact of different treatments on different target populations over time. 224. While the CEA is a simpler and more straightforward method compared to the CBA, it is not always a good replacement. Of the eight Member Countries that were analyzed in the Case Study section, none used a full CEA approach, and only a few used the CBA for larger projects. For example, in Germany, a CBA analysis has to be carried out for strategically important projects (even if their value is below 50 million Euro) and for infrastructure investments of over 10 million Euro. Moreover, the CBA can be used to show the overall strength of an individual project, but it cannot be used to compare projects (i.e., in a competitive approach). Annex 22 provides an elegant mathematical explanation for why the CBA cannot be used in comparative selection systems. Introduce, promote, and build capacity for the adoption of the FIDIC contract forms 225. In particular, the FIDIC standard contract forms would help beneficiaries manage complex projects better, while protecting the interests of all stakeholders involved. FIDIC, which stands for the International Federation of Consulting Engineers, is an international private organization that has developed since the 1950s various contract forms to facilitate the implementation of infrastructure and construction projects. Gradually, the superior quality of these proposed forms has translated into more and more people adopting them around the world. The main benefits of the FIDIC contracts revolve around a balanced sharing of risks between the beneficiary and the contractor and a greater degree of familiarity of international investors with the specific conditions and procedures mandated by these contracts. Indeed, the FIDIC contracts have already demonstrated their usefulness by covering those situations that may arise during the project’s implementation and having clear mechanisms for resolving them in a fair and transparent manner. Since both beneficiaries and service providers benefit from adopting the FIDIC contract (compared to other formats that may expose one of the parties to greater risks), this practice has taken deep roots in the vast majority of developed countries, as well as for projects implemented by multilateral development banks. 226. Recognizing the benefits of the FIDIC forms, the Romanian Government has made a previous attempt at institutionalizing this practice. A 2007-2008 PHARE project was meant to translate the FIDIC framework into Romanian and adapt it to the local context, which further led 25 Alternative approaches to measuring the economic impacts of road improvements are through the measurement of the present value of the profits of businesses (including farms, mines, hotels, etc) affected by the road improvement or by measuring the changes in land values that capitalize the gains to owners of residential and commercial properties affected by a road improvement. 109 to a common order by the Ministers of Finance, Transportation, and Regional Development in June 2008. This act made the FIDIC form required for beneficiaries of EU-funded projects, in one of three versions: red, for executing construction works based on a technical project provided by the beneficiary; yellow, for design-and-build contracts; and green, appropriate for projects under EUR 500,000 and subcontracting work. By May 2009, it became clear that the application of FIDIC was not optimal and another order by the three aforementioned Ministries eliminated the requirement of using FIDIC. The following key reasons were responsible for the ultimate failure of this first attempt at expanding the use of the standard contract forms:  The initial order making FIDIC required was issued in June 2008 and began applying in July of the same year, leaving virtually no room for proper training of beneficiaries on how to use this complex instrument.  The order made the application of FIDIC extremely rigid, stipulating both “general” and “specific” conditions. Normally, the latter type should be customized by beneficiaries for each project under implementation, but this was not possible given the order’s format at the time, which included an actual contract template. As JASPERS noted in a recent study, “it is recommended not to impose special conditions by law but to provide guidance and leave contracting authorities adapt special conditions to each project particulars.”26  The order was insufficiently aligned to applicable legislation and, moreover, it was superseded by legislative acts with superior power – including Government Decisions. Some of the terminology was poorly translated from the common law system, without a proper equivalent in under Romanian law. This created significant confusion among beneficiaries, who did not always know which rules to follow. Figure 27. Selecting the right type of contract (i.e., FIDIC color) is relatively simple Source: http://fidic.org/node/149 26 “Support for the Preparation of Romanian Standardized Tender Documents for the Transport Sector 2010 132 RO ROD Ref. PO42217/CC. 4195,” JASPERS 110 227. This brief analysis suggests that the reintroduction of FIDIC for ROP-financed contracts should be much more carefully planned and delivered. First, there should be ample time for public consultations and trainings regarding how to apply and enforce different types of FIDIC contracts. Second, the process should be officially launched only after fully aligning the FIDIC contract form to the current Romanian legislation and, moreover, there should be a Government Decision mandating the use of FIDIC (as opposed to a Minister order, which is superseded by any government decisions that are at odds with the standard contract form). This was the solution adopted by the Ministry of Transportation in order to facilitate the use of FIDIC for the Transport Operational Programme. Last but not least, only the general FIDIC conditions should be specified by law, leaving the specific conditions flexible enough to be customized to the needs of each project. Undoubtedly, aligning FIDIC fully with Romanian legislation is a precondition of any efforts to promote beneficiary use of this contract form, and indeed this will not be an easy task. As noted by interviewed experts in this field, the FIDIC system was developed based on the UK’s legal system (common law), which is markedly different than the Romanian one (civil law). In the past, there have been challenges in translating certain terms (e.g., “reasonable profit”) and in making sure that Romanian courts can understand and uphold FIDIC. As noted earlier, extensive preparation will be needed in some areas (legal, capacity development, etc.). Evaluation Recommendations 228. Three additional recommendations concern the technical and financial evaluation: making the technical project submission and assessment stage optional, depending on the type of contract chosen by the beneficiary; adding a field visit to the evaluation process; and limiting the number of situations that require clarifications from the IBs to the MA. On the first suggestion, many beneficiaries prefer the design and build form of contract, which allows them to nominate a construction company that takes full ownership over the technical project (TP). This would largely solve the problem of unpredictable expenses; if any discrepancies arise between the TP and the execution stage, it is the contracted company’s responsibility to cover them. This could be backed up by the adoption of a stronger, standardized contract like the yellow FIDIC. Of course, design and build is not possible under the current ROP structure, which requires technical projects to be submitted for evaluation. In practice, requesting this documentation during evaluation may push beneficiaries to move on a faster timeline, but the technical project is not really evaluated on content; rather, it is more of a check-list-type of verification to ensure all required documents are included. For this reason, for certain types of projects, the ROP should allow beneficiaries to resort to “design and build” yellow FIDIC (as OP Transport and OP Environment, axis 1, currently do). 229. Second, a field visit included in the evaluation stage would help correct major technical errors early on for all ROP projects that require some construction components and corresponding technical documentation. It could take place in the presence of the evaluator, the beneficiary, an RDA representative, and possibly an MA expert (if there is capacity and the sum of the project is particularly large). During the visit, the evaluator would compare the feasibility study (required for evaluation, even when “design and build” is chosen) against the situation in the field. The evaluator would be able to recommend corrections, and beneficiaries would have a defined period of time to make the adjustments and resubmit the feasibility study. This practice is currently implemented for axis 1 (with the exception of the urban centers intervention area), but it would benefit the ROP system to expand it to all projects that rely on technical documentation (feasibility study, technical project). 111 230. Also, the MA should ask for clarifications on the technical and financial evaluation only if there are solid grounds to believe that the evaluators’ decision needs to be overturned. If independent evaluators and the evaluation commission are entrusted with the responsibility to recommend projects for approval, the MA should consider requesting clarifications only under special circumstances – i.e., glaring errors in evaluation, contradictory reports, etc. Otherwise, delays can result from repeated clarification requests exchanged between the MA and the IBs. The 2013 World Bank report on “ROP 2.0: MA-IB Collaboration and Communication” provides an in-depth assessment of ways to increase the efficiency of exchanges between the main components of the ROP system. Selection and Contracting 231. In general, selection should not be done simply based on an average score, which may “hide” very weak areas in an application (e.g., technical aspects), and the introduction of weights for individual criteria should be considered. The next section elaborates on these recommendations. In any case, changes to the selection framework should be carefully considered and weighed against currently available data on how the pipeline of projects is likely to evolve. Put differently, a system that is too strict runs the high risk of slowing down absorption due to the fact that too many projects will not make it with a high-enough score. The good news is that the ROP can now draw from its previous experience and – by looking at the data on applicants and beneficiaries for different axes – it would be possible to determine those areas that require stricter selection criteria. 232. As for contracting, full transparency and accountability are required to ensure the system’s legitimacy, and such measures would also help reduce delays. Based on the Procedures Manual, contracting should take a maximum of 27 working days, which are required for collecting all necessary approvals at the MRDPA level. An issue that affects the efficiency of both evaluation and contracting stages relates to the fact that the MA maintains the responsibility of approving reports at every step, but only sees the complete project right before the contract is due to be signed. As such, the MA routinely has a range of questions at the contracting phase that have been addressed, in one way or another, by the RDA and the evaluation commission during the evaluation phase. This procedure creates delays and should be optimized by either having an MA representative look over the complete file from the completion of the administrative and eligibility check through the contracting phase, or trusting the IBs to have properly handled each request. While having the MA approve every step in the process may make sense for high-value, strategic projects, or for some other projects flagged as particularly risky, better triage is recommended for less complex financing proposals, with the delegation of as much authority as possible to the regional level. 233. Beneficiaries understand procedural steps and overall selection models and criteria, but they have less visibility into the timeline and progress of their application throughout the process and particularly once their documentation is sent to the MA-ROP. A solution with benefits on all sides would be to increase the transparency of the flow of documents, developing a simple online tracking system. Applicants would be able to access the platform to determine where their application is at any one point in time, how long the process is expected to take, and who the main point of contact is at each step in the process. This could increase the timeline’s predictability and transparency, mitigate concerns among beneficiaries, and free up capacity in the MA and RDAs helpdesks, which could quickly refer applicants to the online 112 system. It would also help improve accountability up and down the process chain within the MA and the IBs, ensuring visibility around bottlenecks and capacity constraints. In addition, special care should be devoted to increasing the transparency of the contracting process. Some interviewed beneficiaries have noted that it is not clear how applications get prioritized for contracting once they pass the evaluation phase. Again, the “ROP 2.0: Beneficiary Support Mechanisms” provides further details pertaining to this recommendation. Electronization: A Functional e-ROP 234. There are several important advantages of an e-based system: speed, cost savings, and accuracy. First, everything is faster because information can move almost instantaneously between beneficiaries, IBs, and the MA, which reduces delays and contributes to the broader goal of efficient absorption of EU funds.27 Second, overall costs are lower because there is less need for expansive storage and physical transfer of documents. Third, an electronic system allows for more automated verifications and filters to ensure that information sent by applicants and beneficiaries is accurate and complete. This would eliminate some of the back-and-forth between the IB, the MA, and applicants/beneficiaries, and generate additional time and cost savings. At the same time, some managing authorities argue that e-cohesion may create additional work and cause the loss of direct, personal contact with beneficiaries. 235. Recognizing the potential of “e-cohesion,” the EC has pushed strongly in this direction in recent years, as part of the effort to reduce the administrative burden on beneficiaries and streamline the implementation of the ERDF, the ESF, and the CF. The most prominent example is in Article 112 from the EC proposal on common provisions regulation for cohesion funds. This covers Member States responsibilities in the area of management and control systems, and mandates under Section 3: “Member States shall ensure that no later than 31 December 2014, all exchanges of information between beneficiaries and managing authorities, certifying authorities, audit authorities and intermediate bodies can be carried out solely by means of electronic data exchange systems. The systems shall facilitate interoperability with national and Union frameworks and allow for the beneficiaries to submit all information only once.”28 In April 2013, the EC published an implementing act on the rules concerning “e-cohesion,” including the definition and minimum requirements of electronic data exchange, the “only once” encoding principle (the same data shall be submitted only once by a beneficiary of a particular OP), and other details on how these e-systems are expected to work.29 236. Importantly, this proposal would require Member States to enable beneficiaries the option of electronic exchange – this is not an obligatory provision for beneficiaries, but 27 It is important to ensure that the beneficiaries have the proper infrastructure (e.g., broadband internet) at their disposal to submit large documents (e.g., detailed technical projects). Some more information on these issues can be obtained from these two sources: http://www.rejournal.eu/Portals/0/JE%2049/Y%20Ciocoiu,%20Cristina.pdf and http://ec.europa.eu/regional_policy/sources/docgener/studies/pdf/measuring/measuring_impact_report.pdf 28 European Commission, “REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL laying down common provisions on the European Regional Development Fund, the European Social Fund, the Cohesion Fund, the European Agricultural Fund for Rural Development and the European Maritime and Fisheries Fund covered by the Common Strategic Framework and laying down general provisions on the European Regional Development Fund, the European Social Fund and the Cohesion Fund and repealing Council Regulation (EC) No 1083/2006, http://ec.europa.eu/regional_policy/sources/docoffic/official/regulation/pdf/2014/proposals/regulation/general/gen eral_proposal_en.pdf 29 See “Fiche no. 6, Implementing act on the rules concerning electronic information exchange with beneficiaries (“e - cohesion”), Version 1 – 10 April 2013, http://ec.europa.eu/regional_policy/what/future/pdf/preparation/1_fiche6_ia_e-cohesion_2013_04_09.pdf 113 countries are specifically allowed to make it a firm requirement. This is in fact the path that this report recommends for the Romanian ROP; otherwise, if the “paper” option remains a possibility, changing the mentality of some beneficiaries and MA/IB staff will become extremely difficult and the new system may not gain ground. The EC proposal further clarifies the term “all exchanges of information” to refer to information requirements applicable to the beneficiary “once a grant has been awarded, [which] includes reporting on progress, declaration of expenditure and exchange of information related to management, verifications and audits.”30 Moreover, it notes that documents available through the computer system should suffice and beneficiaries shall be requested to provide original copies only through on-the-spot verifications and ad-hoc controls (for exceptional, risk-based situations). Again, this report goes further and recommends the implementation of e-documentation throughout the project cycle, i.e., covering the application phase as well (as the UEFISCDI Direct Manager example provided above). 237. For its part, Romania has launched a process for developing a “mySMIS” applic ation, which would serve as an interface with beneficiaries, complementing the current SMIS backend database. MySMIS has already been contracted by the Ministry of European Funds using technical assistance funding, although the exact details on when it will roll out and what functionalities it will actually include are still lacking, despite the fact that the project is reportedly complete as of December 2013. The quality of the request for services used in the public procurement procedure was very good and included a comprehensive list of requirements. What remains to be determined is the quality of the actual product and its implementation, which would benefit from ample time for consultations, beta testing, and piloting. Learning from the experience of developing SMIS, it is recommended to maintain service agreements with the software developer for a longer period of time, including assistance on demand for any issues that need to be addressed. Since the finalized form of mySMIS is not known at this point in time, the observations below remain relevant and summarize the main benefits of an e-ROP at various stages of the project cycle. What is more, if the mySMIS launch will be delayed further, the MA within the MRDPA should strongly consider developing its own components of a future electronic system. 238. As a first step, the introduction of electronic forms for the application period would reduce bureaucracy and associated costs, and help prevent beneficiaries from making mistakes, ensuring smoother completion of projects further down the line. A relatively simple system can be implemented, whereby the applicants would fill out an electronic form and attach supporting documentation. All forms should be foolproof, meaning that they should have predefined sets of acceptable values and required information. For example, beneficiaries would not be able to enter project indicators that fall beyond the acceptable program-level set and they also should not be able to submit the final form until all required documents are attached. Not only would this process reduce the amount of paperwork generated, but also it would significantly reduce the time needed for eligibility and verification checks, which could be partly automated. Smart software could analyze applications received and flag potential areas of concern for RDA staff, which would need to spend even less effort on this first verification. Depending on pilot results, the principle of four eyes may also be eliminated if the system proves to be sufficiently rigorous with an automated computer check doubled by one RDA expert verifying all submitted documents. 30 Ibid. 114 239. The evaluation process should also take place on an easily accessible electronic platform. Under this new setup, there is no need to pay the extra costs of transporting evaluators to the RDA headquarters for the desk evaluation phase. An e-based ROP would mean that evaluators could analyze submitted documents remotely, so a larger proportion of the budget dedicated to covering evaluation-related costs could go toward actual fees, supporting the goal of higher-quality evaluations. All interactions between evaluators, beneficiaries, and IB staff could take place through electronic channels during the desk evaluation phase, and the evaluation software could connect all these parties quickly and securely. It would also be easy to track when documents were submitted, including initial and updated versions, reasons for any changes, as well as how long a process is taking to identify bottlenecks and take corrective measures. A fully integrated system would also mean that evaluation results can be quickly organized and communicated to applicants, and procedures for contracting those qualifying proposals could also begin in a timely manner. 240. During the implementation phase, all required exchanges of information, including reimbursement requests and supporting documentation, would be submitted and verified electronically. This would avoid duplication of tasks for beneficiaries. For instance, by submitting the public procurement documents one time, they would no longer need to upload them again with every reimbursement request. If further verifications are needed, the documents could be easily found in the system by all verifying officers who are granted access to a particular project. As in earlier stages, formal communications between beneficiaries and MA/IB staff could be e-based to save time and ensure proper records of all exchanges. 241. In addition to adequate capacity, a change in mindset is needed to ensure the success of switching to an e-based system. There is still a strong bias toward relying on stamped, hard copy documents. A common understanding is required at all levels (MA, IBs, and especially beneficiaries) that an electronic document endorsed with an e-signature is as valid as a hard- copy version. Given that both EU and Romanian legislation allow for the use of electronic documentation and e-signatures, there are no obvious legal barriers against the switch to an e- ROP. Habits and mentalities can be tough to change, but the alternative of maintaining the status quo is a lot worse for stakeholders struggling through overly bureaucratic procedures. If the switch to an electronic platform is decisive, meaning that the hard-copy option is only allowed in exceptional circumstances, people within the system will gradually accept the advantages of electronic exchanges and rely increasingly on this type of communication. Proposed Improvements to the ROP 2007-2013 Evaluation and Selection System 242. The following discussion proposes a number of changes to the current evaluation and selection system. The discussion starts from the premise that the MA-ROP may chose not to stray too much away from the current evaluation and selection system, which has proven its efficacy within the Romanian context, and one should not necessarily change something that works well. In practice, this means that for the 2014-2020 programming period the MA-ROP may chose to continue with a selection system largely based on the FIFO system, and pre- allocation of funds for urban development (i.e., the Status Quo+ scenario). This analysis goes through all the key steps of the evaluation and selection process (administrative evaluation, eligibility evaluation, and technical and financial evaluation), but discusses the technical and financial evaluation in more detail, as this is the part of the process that carries the most weight in the evaluation and selection process. 115 Administrative evaluation grid 243. The administrative evaluation is a simple and primarily bureaucratic process, and there is not a lot of scope for innovation in this area. Nonetheless, this stage can be simplified by introducing an electronic application system, which would also cut down on overall processing times and encourage higher absorption rates. Some key issues to be considered for this stage include:  The key question should be: Does the application for funding contain all the required documents, in the required format, with the required information?  Many of the current criteria can be verified automatically under an electronic system (e.g., whether a project summary is attached). In fact, an e-ROP would mean that an applicant cannot send a file through if required sections are missing. This would facilitate communication, document handling, and storage. A more in-depth discussion on this issue is included in the “ROP 2.0: MA-IB Collaboration and Beneficiary Support” reports.  All documents can be in soft copy, within a secure system (encrypted PDF files with digital signatures and authorized users). Budgets should be in the form of spreadsheets. Hard copies should be required only in exceptional circumstances and for verification purposes.  Also, some criteria become irrelevant / can be eliminated under an e-ROP (e.g., whether the envelope was sealed, whether there is a label on the envelope, etc.).  Essentially, every line in this grid can be included in an automated verifications system. Eligibility evaluation grid 244. Several of the key issues to be considered for this part of the process are discussed below:  Key question should be: Does the project meet all legal requirements related to the nature of the applicant and the project, the implementation process, and the expected impact?  Under an e-ROP, many of the eligibility verifications can be automated / simplified/ ensured at the time of the submission (e.g., checking a box regarding the type of beneficiary organization, submitting electronic copies of ownership documents, etc.) Main categories of current eligibility grid Comments Type of applicant & basic check for red flags Automated through application form (box checking) (e.g., insolvency, criminal activity etc.) Ownership rights Electronic submission with desk review and verification in the field (based on sampling for simple projects and with full check for very complex projects – e.g., road going over railroad track, etc.) Financial capacity to manage the project (e.g., This should be renamed “financial commitment” (a local council decision) local council decision does not necessarily mean that they also have the capacity to implement the project). One idea is to remove the 2% requirement from this stage, as it is already in the financial evaluation phase later on. Administrative capacity Here the most important differentiating factor is 116 Main categories of current eligibility grid Comments “previous experience” in managing a similar project. This should be maintained for complex projects (e.g., roads) Other eligibility checks (project value, This can also be automated through the electronic implementation timeline, etc.) verification of the application form Technical and Financial Evaluation Grids 245. This is the most important part of the evaluation and selection system and the section where there is the greatest scope for improvement. Some of the key issues to be addressed are discussed below.  Key questions should be: Are there sufficient economic, social, and environmental benefits from the project? Is the applicant capable of successfully implementing and operating the project? A. General observations:  The grids do not clearly separate between technical and economic criteria, even though the evaluation itself is based on two evaluators, one for technical, one for economic aspects. While it is clear that the two evaluators should collaborate in the process, the technical engineer should not score economic aspects and the economic analyst should not assess technical issues.  One recommendation is to split the grids into: o Technical and financial evaluation:  Question: Is it feasible for the applicant to implement and operate the project?  Categories: technical feasibility and features; adequacy of implementation plan; administrative and financial capacity to operate and sustain the project  Scope of criteria: Feasibility and operation  Scoring: Minimum threshold (“passing grade”) o Economic benefits (impact) assessment:  Question: Are the economic benefits of the project’s operation worth the costs?  Categories: Benefits and costs  Scope of criteria: Expected outputs  Scoring: Absolute (competitive selection)  Criteria that refer to basic fulfillment of current regulations should be moved to the “eligibility check” phase. Project features that add to these minimum requirements and deliver enhanced impact (beyond what the law mandates) should be evaluated under the economic benefits assessment.  Project financial sustainability during operational phase should be assessed. Aside from the applicant being able to show its current financial capacity to meet its share of the project implementation budget where certain costs are not covered by ROP funds, 117 under this criterion it should be required to show that (a) it can fund the operation and maintenance of its existing investments and (b) is either running fiscal surpluses or is expected to have revenue growth or expenditure savings over the future years sufficient to cover the added costs required to sustain the operations of the new project. To simplify this exercise, some basic indicative O&M costs per unit of capital invested can be developed for different types of project based on sector experience in Romania or countries in the region with similar cost structures. For example, the annual expected costs of O&M for different types of road could be in the range of 5% to 15% of the initial capital outlay, whereas that for an educational facility could be in the range of 30% to 70% (or even higher) of the capital cost. Cost information can be gathered from different sector ministries on the typical range of annual O&M per unit of capital assets of different types in different sectors. Subsequent chapters go into more technical details on how to assess an applicant’s financial capacity to undertake a particular investment, proposing specific thresholds to watch for.  Weights for individual criteria should be introduced to reflect the fact that some factors are more important than others. For example, for axis 2, the fact that a road project connects to the TEN-T network (criterion 1.1) should weigh more heavily than “improved road safety” (criterion 1.2). Similarly, the technical solution (criterion 2.4) should weigh more heavily than environmental safety and energy efficiency (criterion 2.7), and in any case the two are somewhat related/overlapping. By adjusting the weights, one can prioritize certain aspects like absorption (e.g., project maturity or the stage of the technical documentation’s preparation). Most countries analyzed in the case study section use weights for the individual selection criteria.  Separate horizontal criteria could be introduced. Currently, social and environmental criteria are clustered under the “Project Maturity and Sustainability” section. The EU recommends the use of horizontal criteria such as social equity and environmental sustainability, encouraging beneficiaries to indicate how their projects will bring benefits to society and the environment. In Germany, these horizontal criteria are placed in a separate category from economic, financial, and technical criteria. Moreover, when two projects receive the same score under the Nordrhein-Westfalen ROP, the project that received the higher scores on the horizontal criteria moves forward.  The idea of describing 2-3 “situations” for each category (corresponding to 1, 3, or 6 points) is helpful as a best practice for both evaluators and applicants. In some instances, though, there should be less room for bias in assigning a score (e.g., “solid” vs. “relative” monitoring capacity). Estonia provides a good example of a relatively simple evaluation and selection system, but with clear “situations” outlined for individual scores under each selection criterion.  A score of zero for any category means that a project is disqualified, which makes sense and should be maintained going forward. However, it is important to identify selection criteria that are clearly defined and clearly important. For example, it may not always be easy to show how a project contributes to social equity.  In some cases (e.g., emergency equipment) the current criteria overlap. At a minimum, criteria for a particular intervention should follow the “MECE principle”: mutually exclusive, collectively exhaustive. 118  Evaluators should be required to test assumptions behind CBA numbers and not merely check the box that the value is over/under the required threshold (if the CBA is kept as part of the evaluation requirements).  Given that the CBA technically includes a myriad of economic impact indicators, it is important to discuss if other evaluation indicators should be used if the CBA is required. Also, if only the CBA is required, then for smaller projects the CBA could be replaced with the Cost-Effectiveness Analysis (which would be particularly useful for competitive project selection).  The MA-ROP could also consider replacing the use of CBA (or expected NPV >0) with measurable key performance indicators that would be inputs into a full CBA o Generally the indicators would be scored and weighted approximately as follows: indicators of direct benefits to users, 60%; indicators of indirect benefits, 10% and external economic benefits, 30%. It is critical for the project to score at least 50% on the benefits to the direct users to qualify for selection. Indirect and external benefits only boost the scores. o Types of key indicators therefore include: existing/induced demand for a public service (number and growth rate of users/beneficiaries); direct gains to users resulting from the project (better service, reduction in costs of use, lower access prices, etc.); indirect effects on users (substitute and complementary effects on user demand); positive/negative economic externalities. o Critical need: assess the availability of key indicators in the Romanian context, existing gaps, and potential of applicants to fill gaps through their own data collection efforts. o Example of how to apply the key performance indicators approach to a roads project:  The indicators of the direct economic benefits gained by the project users can be assessed in a two-way grid that includes the key sources of benefit: the number of users and the gain per user.  For example, for a road project, the grid could look like the two-way table below, where the estimates are made of the daily traffic flow on average over the project life and the time savings per trip-kilometer. Two-way table giving points weighting based on average daily traffic flow and time saved per 1000 vehicle-km Average time saving (hours per 1000 trip-km) Under 3 3-6 hours 6-9 hours Over 9 hours hours Average daily Below 600 0% 0% 0% 50% traffic flow 600-1200 0% 50% 50% 100% (std vehicles 1200-2400 50% 100% 100% 100% per day) Above 2400 100% 100% 100% 100%  The time savings could come about because of rough surface improvements and lane widening in a road rehabilitation project, or because of additional lanes provided, or because of improved traffic signaling, or because of a new connecting road shortening the distance or because of congestion relief, etc., or some combination of these. The 119 traffic flow would be the current traffic flow from the Traffic Study adjusted for growth and the induced flows as the cost of a trip declines with time saving.  This selection grid would reject projects with low traffic flows unless a major saving in trip times could be achieved by the project. The percentage score in the grid would be multiplied by the direct user benefit weight of 60%, say, in determining its contribution to the economic benefit score. This type of grid needs to be calibrated based on the costs of construction per kilometer of different types of road project in Romania.  Consideration could also be given to requiring the applicant to submit sector development plans that show how the proposed project ranks in the sector priorities in the plan. For example, a region should have a regional road development, rehabilitation and maintenance plan or plans that show the prioritization and scheduling of projects within a region. This plan should ideally have some selection criteria for prioritizing the projects. The nature of the ROP selection methods will influence the regional plans and the regional plans should also educate the ROP allocations and selection methods. Another option is to use a Capital Investment Plan to prioritize investments over a set period of time – e.g., 9 years. This option will be discussed in more detail later in the report.  The criteria and methodology for evaluating projects should mirror the framework used for assessing regional/county/local development plans. This will ensure a coherent, standardized approach and will also enable the ROP to better track and eventually reach its program-level targets.  There is an insufficient focus on evaluating indicators and ensuring post- implementation monitoring of performance. Indicators should be SMART and there should also be clear mechanisms for monitoring if targets are reached, in line with the EC’s increasing focus on impact / value for money. Ideally, some of the evaluation and selection criteria should be tied to the program performance indicators, clearly showing how the project will contribute to helping the ROP achieve its objectives. B. A better understanding and use of quantitative technical and financial criteria 246. Under the current ROP, the analysis of the expected cash flows of a project appears in the Technical and Financial Assessment Grid under three criteria: a. Under section I covering the “Project contribution to the achievement of ROP goals,” typically a cost benefit analysis (CBA) criterion is included to assess the economic impact or “importance of the project for the region.” This criterion is variously stated as ERR >= 5.5% and/or B/C >=1 and/or ENPV>0, where ERR = economic rate of return, B/C = benefit-cost ratio, and ENPV= economic net present value at a discount rate of 5.5% (real).31 These criteria follow EU guidelines.32 31 Acronyms and technical terms are explained in more detail in Annex 21 and Annex 22. 120 b. Under section 2 covering the “Project quality, maturity and sustainability,” two criteria are included, covering: i. “The project’s need for grant co-financing” or the verification that the project is not self-financing, and hence, needs a grant in order to be financially viable. This is typically stated as FRR(C) < 5% and/or FNPV(C) < 0, where FRR = financial rate of return to total investment capital, and FNPV(C) = Financial Net Present Value to Total Investment Capital at 5% (real). This is based on EU guidelines.33 ii. “Project sustainability after grant cessation,” which always includes a criterion for financial sustainability over the lifecycle of the project, but may include criteria that the beneficiary needs to demonstrate the support structure or human resources and technical capacity (or equipment) to sustain the project. In some cases a separate criteria is included specifically requiring “financial sustainability of the investment” be demonstrated by showing either that the cash flows of the project are positive in every year or the cumulative cash flows are positive. In the grids for some priority axes, the project sustainability criterion is expressed the same way as the financial sustainability criterion. 247. These three criteria all depend upon the financial and economic feasibility studies . A review of the Technical and Financial Assessment Grids across the various Priority Axes, however, shows that these criteria are expressed in often inconsistent and unclear fashions. This section aims to clarify the meaning, use and specification of these criteria. Basics of project appraisal: financial and economic analysis 248. Any investment project supported from the government budget needs to be subjected to two basic appraisals or analyses: financial and economic appraisals . Each can be further refined to reveal the financial interests of different parties or stakeholders involved in, or affected by, the project. 249. Financial appraisal is an investigation of the cash flows of a project from the perspective of the sponsors (or owners or equity holders) of a project, as well as other financial investors in a project (typically, the debt holders). The appraisal can be conducted both from the perspective of the combined financiers or total investment perspective and from the perspective of the sponsors or owners of the project or the equity capital perspective. The sponsors receive loan disbursements from the debt financiers and then service this debt. Financial appraisal of projects with public involvement is conducted for two purposes. a. If the project is expected to earn revenues from its services and be self-financing (as may be the case with a price regulated utility or public private partnership such as toll road), then the main objective is to assess whether the financial attractiveness of the project to sponsors (given the debt financing they can raise) in order to assess whether the sponsor will proceed with the project. In this case the basic criterion is FNPV > 0 where the cash flows from the sponsor’s perspective are 32 EUROPEAN COMMISSION, Directorate General Regional Policy, “Guide to Cost-Benefit Analysis of investment projects: Structural Funds, Cohesion Fund and Instrument for Pre-Accession” 2008, section 2.5, pp. 47-58. 33 Op cit 1, section 2.4, pp 34-46. 121 discounted by the supply price or the minimum rate of return required by the sponsors to invest equity capital in the project. b. If the project is not expected to earn revenues or to earn insufficient revenues to cover the project costs, then the main objective of the financial appraisal is to determine the amount of public revenues required to make the project financially viable. The public revenues could be some combination of the own revenues of a public agency plus a grant from an external agency to fill in remaining financial gap. Clearly, this role of financial appraisal addresses both criteria in the current grids noted above, namely the need for a capital grant and the financial sustainability of the operational phase of the project. These issues are addressed further below. 250. Economic appraisal of a project aggregates the benefits and costs of all persons or stakeholders affected by an investment project. It includes the already internalized benefits and costs of the sponsors and debt financiers of the project plus all the external benefits or costs of other affected persons. These other affected persons include importantly the direct users of a project that are receiving services without full payment such as users of a public road. They also include all other persons in their roles as consumers, suppliers, workers, property owners, etc. that experience changes in the prices they pay or receive for their services or products as a result of the project. For example, a new road may reduce the congestion on alternative routes to the same destination as the new road, but the road may have resulted in some property owners having their properties expropriated and others suffering added noise and air pollution from the traffic on the new road. At the same time, the government experience changes in tax revenues or subsidies from the changes in market supply or demand induced by the project. All these external costs and benefits are aggregated into an economic appraisal and discounted by the social discount rate to calculate the economic net present value (ENPV) of the project. At the same time, all the net gains and losses to all the affected parties or stakeholders can be assessed to identify the distributional impacts on the net winners and losers. 251. The conduct of economic appraisal or CBA is core to assessing the true nature of net benefits derived from a project, but it is also extremely demanding in terms of the information required and the efforts required by the appraisers. These costs are relatively fixed such that the conduct of CBA is typically cost-effective if the scale of the project is large, such that the net benefits (or net costs avoided) are large relative to the costs of conducting a full CBA. In the absence of conducting a full CBA on a small project, the project should nevertheless have some estimate of the number of beneficiaries and expected gain per beneficiary to know that a positive result would be expected to arise if a full CBA had been conducted. For example, that a rehabilitated road would result in high enough value of time savings to a large enough volume of traffic to justify the capital expenditure. Investment criteria 252. The basic or primary investment criterion is the net present value (NPV) criterion . The NPV includes the discounted net cash flows from the perspective of a group of stakeholders in the investment where the discount rate reflects the opportunity cost of funds of those stakeholders.34 For example, the NPV could be calculated for sponsors of a project or for the 34 Discount rates can be adjusted for risk premiums or transaction costs specific to the type of investment being under taken. In such cases the term supply price of capital or minimum required rate of return is typically applied. 122 aggregate economy. The NPV of a project represents the change in wealth (or capitalized net cash flows) that the stakeholders expect as a result of the project. Hence, the requirement for NPV >0 or choosing investments with the highest NPV forms the basic criterion for investment selection. All other criteria are derived from the NPV criterion, but have shortcomings in different situations. The two major complementary measures are the internal rate of return (IRR) and the benefit-cost ratio (B/C). 253. The Internal Rate of Return (IRR) is the discount rate at which the NPV equals zero. Typically, the investment criterion becomes IRR > discount rate of stakeholders, which implies NPV >0. The IRR has a number of shortcomings such as (a) it does not exist for some cash flows or gives multiple results and (b) it cannot be used to compare projects unless they are have similar scales, timing, lengths of life, and hence cannot be used to optimize an investment project. The IRR does provide some useful information when compared to discount rate in the sense that gap between the two shows the margin for fluctuations in performance of the project without the NPV going negative. For limitations in the use of the IRR as a project selection criterion also see Annex 22. 254. Benefit-Cost Ratios measure the ratio of the present value of the benefits to the present value of the costs. NPV measures the present value of benefits minus the present value of costs. Hence, the criterion of B/C > 1 is exactly the same as NPV > 0. While B/C >1 is a valid investment criterion, making comparisons or choices between projects based on B/C suffers from similar shorting comings as IRR as well as being open to manipulation. For example, reclassifying a cost as a negative benefit (or a benefit as a negative cost) does not change the NPV but does change the B/C value. Accordingly B/C provides no added value to NPV. Annex 22 a simple mathematical explanation of some of the limitation of the Cost-Benefit Analysis. 255. Annex 21 includes a synthetic discussion on economic and financial appraisal and selection criteria. C. Common criteria (applicable to most grids) Common categories of Comments/Suggestions evaluation grids Project maturity / preparation  Critical for quick absorption phase:  A higher weight for this criterion would accelerate absorption  Public procurement early on for ROP 2.0. This could address some of the concerns procedures related to making the technical project optional, although for  Authorizations investments where “design and build” is recommended, this  Ownership over land (as criterion could be dropped (applicants will not have the needed) detailed design at this stage)  Need to assess whether public procurement procedures can be launched before the contract is approved/signed (“licitatie cu clauza suspensiva”), which would address some of the current delays in completing public procurement procedures. This is already happening for OP Enviroment and even the ROP, and could be further incentivized by awarding higher scores to applicants who have started public procurement procedures  Authorizations and ownership rights can/should be verified during the pre-qualification phase or the admin & eligibility 123 Common categories of Comments/Suggestions evaluation grids verification phase. There is no need to include them here as part of the scoring Methodology for  A clear progression from objectives to activities and results implementation:  In some instances (e.g., axis 3.3 – purchases of emergency  Objectives: clear, equipment) the criteria on clear and reachable objectives can reachable be superfluous / self-evident from the type of intervention  Activities: clear, planned. Under a simplified grid, superfluous criteria should be correlated to dropped. The key test is: does asking for some particular implementation information add anything to the application itself? timeline, team  Need to assess whether risks and risk management systems can members’ attributions, be detailed better. For example, environmental risks are and public procurement covered by a different category of the grid and should not be timeline duplicated here  Results and indicators:  Do we need to also consider the public procurement timeline, correlated to activities, separate from the implementation timeline? This has caused feasible targets long delays and addenda to the contract when the public  Risks and risk procurement schedule needed to be amended, often for no management systems fault of the applicant (e.g., challenges in court, etc.)  Project results and performance indicators have to be correlated to goals/activities, but there should also be some kind of requirement to select SMART indicators. This will avoid current situations where a beneficiary cannot prove that he/she reached the targets because of lack of data, inability to measure targets, etc. Evaluators should penalize/eliminate indicators that are not SMART (specific, measurable, attainable, relevant, and time-bound). This is in line with the broader EC trend to focus on performance indicators for 2014-2020. Coherence of documentation  Operations and maintenance costs should both be factored in (Feasibility Study) (not just operating costs)  Data: sufficient, correct,  Decide on proper economic analysis (CBA vs. CEA, etc.) and justified  Maintain current criterion related to feasibility study; exclude  Operating costs: technical project from evaluation (current system is only a realistic, correlated to conformity check), but may require its timely submission hypotheses through the financing contract (6-12 months after signing the  Corrections contract) (externalities) correctly  The application should be correlated to the feasibility study but, applied in economic within reasonable circumstances, changes to the feasibility analysis study should be allowed  Bill of quantities  Consider separating the technical analysis from the economic (“devize”): clear, analysis. The two evaluators looking at these financing requests complete, realistic, and are generally only specialized in 1 area (technical or economic), tightly correlated to so it does not make sense for the economist to score technical designs aspects or for the engineer to score economic aspects  Designs: complete, correlated to written sections Technical solution  Some duplication with the previous criterion (the technical  Fulfills project goals part). Recommend merging the technical analysis of the data,  Corresponds to latest bill of quantities, etc. from the previous criterion with the 124 Common categories of Comments/Suggestions evaluation grids standards in the field analysis of the technical solution (innovation)  A new approach is needed for assessing this criterion: instead of verifying mostly the correlation among various technical documents, evaluators should focus on the technical solution itself (level of innovation, sustainability, etc.)  This criterion does not seem to look at whether the chosen technical solution is the most efficient in terms of costs. In other words, many solutions could fulfill the project goals and even the latest standards in the field, but there is probably one solution that is most efficient and straightforward (technically)  Need to better define what the “latest standards” refer to: national, international, EU? If national, the risk is that the formal/de jure standards are outdated Need for grant funding  CBA, including this criterion, could be dropped for all projects  IRR-financial <5%, NPV- where it is not mandatory (at least those that are not expected financial <0 (all based on to charge fees or earn user charges for service) the feasibility study)  Essentially, a public-sector body should always pass this test because it does not typically get revenues for such investments (for most projects under the ROP)  Evaluators should be required to look behind the numbers and evaluate the assumptions made by applicants (not simply check the box on whether the values “fit” the criteria) Budget of the project  Sound and necessary criterion. Applicants should clearly justify  Complete every budget line and its relationship to the project’s  Correlated with foreseen objectives. There should be full correlation between the activities, project’s budget and the General Estimate from the Feasibility allocated/estimated Study resources, and the bill of  Potential improvements needed: (1) eliminate/broaden the quantities eligible expenditures order to cover all possible costs related to  Costs are realistic and an investment; (2) explicitly require evaluators to eliminate necessary for budget items that are not related to the investment completing the project  Bill of quantities should be required in the form of “articole comasate” (bundled items), not “indicatori de norme de deviz” (individual items) to make the implementation phase easier. Environmental protection and  Necessary criterion sustainability  Need to better define which are the measures needed for a  Measures for improving specific type of investment. For example, these will vary widely the environment (based in scope from a project under 4.3 (SMEs/e.g., purchasing a on the feasibility study printer) vs. under 4.2 (brownfield redevelopment), and may and environmental include: collection and management of water; collection of impact study) waste; landscape rehabilitation; etc.  Assessment should be based on the Feasibility Study (not the detailed design, which would become optional under these proposals).  Could add a sub-criterion for energy efficiency improvements (e.g., fuel efficiency gains for roads investments).  Could be placed in a separate Horizontal Criteria section. 125 Common categories of Comments/Suggestions evaluation grids Equality of opportunity and non-  This criterion is necessary, but needs to be better defined. discrimination  In practice, beneficiaries have often misunderstood this  A non-discrimination criterion. They have noted in interviews that they argued in the policy exists case of a road that it would help women and children walk  Mechanisms to ensure around villages. This is just an example, but a highly relevant equality of opportunity one (see also report on “ROP 2.0: Beneficiary Support (including for public Mechanisms”). procurement contracts)  A set of typical/expected risks should be defined for each type  Potential discrimination of investment. risks identified and  Some aspects should be absolutely required: for example, addressed access for disabled people is a must. Without it, a project  Facilities for disabled should be disqualified. people  Could be placed in a separate Horizontal Criteria section. Information society and new  This criterion is generally useful, but should be better defined. technologies (some axes) What qualifies as a (useful) innovative technology? How is the  Increase in the use of potential increase in use measured? new technologies and IT solutions in service provision Monitoring capacity  Critical criterion that warrants a higher weight  Clear strategy for  This refers only to the implementation period and can assess monitoring the whether enough internal or external resources are allocated for implementation of the monitoring activities. project  The applicant should also demonstrate the capacity to monitor  Clear division of tasks the post-implementation phase, including the performance  Monitoring procedures indicators selected for the project. Again, this will be critical to & calendar avoid corrections for 2014-20  May add this “post-implementation” component to this criterion or to the next one (on project sustainability). Project sustainability  Critical criterion that warrants a higher weight.  Capacity to ensure  Need to define O&M costs better (with a methodology for maintenance and calculating them and potential benchmarks for various types of functioning of investment). infrastructure  Multiannual budgets may be worth assessing/recommending  Some axes require positive net cash flows every year. This is impossible for certain types of investment (e.g., social center, hospital, etc.). Positive cash flows should not be required for investments that are expected to be subsidized in some way or another. 126 D. Specific criteria by priority axis and key intervention area PA/ Particular categories for Comments/Suggestions KAI specific intervention areas (contribution to ROP goals) Note: This is typically the first part of all grids, in some form or another. The general hea ding, “contribution to achieving the ROP goals,” is not entirely exact. The Applicant Guides mention general and specific goals. The former include the “sustainable economic, social, and balanced territorial growth of Romanian regions” – this can be considered too broad. It is recommended to change the heading to “contribution to achieving specific ROP goals corresponding to each intervention area.” Axis 1.1 Category A1: Urban infrastructure  Need to better define what kind of data/indicators can Growth rehab substantiate the “need for investment” (1.1) Poles  1.1: Need for investment  See comments under “common categories” (table (need, project features, above) accessibility)  Other criteria (same as “common criteria” – see table above) Category A2: Transport  Baseline data for both traffic/travel times and public  1.1: Improved traffic, transport ridership are often nonexistent or hard to faster travel times (by 15% obtain. No methodology is provided for how to for max score) estimate these indicators  1.2: Increase in public  Only poor estimates exist regarding the number of transport ridership (by 10% people who drive (personal vehicle use) or use non- for max score) motorized transportation (biking, walking)  Other criteria (same as  See comments under “common categories” (table “common criteria” – see above) table above) Categories A3 and A4: Public  No project-specific criteria gardens & other green spaces;  Possible indicators: target population; evidence of need public information equipment (e.g., public surveys, petitions, etc.)  General criteria (same as  See comments under “common categories” (table “common criteria” – see above) table above) Categories A4 and A5: Historical  No differentiation based on the number of jobs (just monuments; tourism based on type, i.e., permanent vs. temporary) infrastructure  Creation of jobs in the public sector should not  1.1: Rehabilitation of necessarily be encouraged (unless clearly warranted by cultural patrimony a particular project) and is often hard to accomplish (improved architectural (e.g. GEO 34/2009 imposes strict limits) aspect, increase in number  Need to better define how the marketing strategy is of tourists – 5% for evaluated – i.e., its “adequacy.” This can include specific maximum score) components to be assessed, e.g., segmented audience;  1.2: New jobs (permanent targeted messages; “competitors,” etc. for 6 points, temporary for  See comments under “common categories” (table 3 points) above)  1.9 for A4 and 2.7 for A5: Marketing strategy 127 PA/ Particular categories for Comments/Suggestions KAI specific intervention areas (contribution to ROP goals)  Other criteria (same as “common criteria” – see table above) Category B: Business  Some Member Countries (e.g., Estonia) give a higher infrastructure weight based on the wages of the newly created jobs  1.1: Job creation (number (i.e., a higher score for a job paying above the average of jobs, proportion of wage and a lower weight for jobs paying lower-than- SMEs) average wages).  1.2: Need assessment  Some Member Countries (e.g., Germany) consider (need in target area; whether the project increases productivity rather than project features; focusing on more jobs. It is true that more jobs do not accessibility to main necessarily lead to higher economic development (e.g., transport routes) low paying basic service jobs, such as retail jobs, will not  1.10: Marketing strategy necessarily give local/regional economies a boost).  Other criteria (same as  Need to better define the types of indicators that can “common criteria” – see be used to justify “need” (e.g., population, types of table above) firms in target area, etc.).  The sub-criterion on accessibility is a good practice and should weigh more given the importance of connectivity  The criterion on “marketing strategy” (1.10) covers well the range of critical elements: market demand; competitors & comparative advantage; marketing strategy; management strategy; capacity; risk management. This should be renamed “business plan assessment” and should also include criteria on project sustainability (1.16 and 1.17), which are integral parts of the value proposition  Criterion 1.17 requires positive cash flows for the entire period of investment. This is unrealistic; most new businesses may lose money in the first couple of years, but can become highly profitable in years 3-5. Category C1: Social infrastructure  Insufficient specificity and weak quantification of  1.1: Improved criterion 1.1. Also, increasing the quality of existing infrastructure (6 points for activities may matter more than simply adding new new activities, 3 points for (and possibly irrelevant) services. existing ones)  More is not necessarily better. See UN Directive on  1.2: Number of reducing the number of beneficiaries per center (this is beneficiaries the new best practice).  Other criteria (same as  IRR must be higher than 5.5%. It is unclear how the IRR “common criteria” – see can apply to a social center, which is a public service table above) (profitability per se should not be a primary concern). Also, a 5.5% discount rate is not really realistic for Romania.  Potential/additional indicators: quality of services provided. 128 PA/ Particular categories for Comments/Suggestions KAI specific intervention areas (contribution to ROP goals) Category C2: Public safety  No specific indicators for this type of intervention interventions  Potential criteria: crime rates and trends; target  General criteria (same as population; innovative technologies, etc. “common criteria” – see table above) Axis 1.1 Note: Categories of interventions  It is reasonable to maintain the same standards for Urban and their corresponding grids are Growth Poles and Urban Development Poles when it Developme largely the same as for Growth comes to implementing similar projects. This nt Poles Poles (see above) standardization across the sets of grids should be maintained going forward. Axis 1.1 Note: Categories of interventions  It is reasonable to maintain the same standards for Urban and their corresponding grids are Growth Poles, Urban Development Poles, and Urban Centers largely the same as for Growth Centers when it comes to implementing similar Poles (see above), with one projects. This standardization across the sets of grids addition: Criterion 1.1 refers to should be maintained going forward. the “Project’s relevance for the  It is a good practice to evaluate a project’s “level of Integrated Plan’s objectives” integration” with other proposed investments in the IUDP. More weight should be given to this criterion. Axis 1.2 Note: This is a pilot intervention  It is recommended to develop specific criteria for Energy area. It omits the “technical and prioritizing investments in the rehabilitation of efficiency financial evaluation” phase, as it is residential buildings. of understood under the other axes  A financial analysis would help compare estimates of residential energy saved per resident. buildings  A social analysis would ensure that such grants are primarily targeted at households that could not afford to complete required work with own funds.  A technical analysis would enable choosing the best (most efficient and effective) proposed solution, unless there is a standard project (with recommended lists of materials) for all such interventions. Axis 2  Project relevance for  Relevant criterion that should be maintained and (roads) PA/KAI objectives (e.g., weighed more heavily connectivity to TEN-T,  See discussion above (part A, “General Observations”) beltway traffic, major for additional ideas on how to estimate economic urban roads) benefits for roads (number of users; gain per user)  County roads should: have traffic levels above a certain threshold; connect places within the region that have high demographic and/or economic mass; enable large economic engines to access a larger labor pool; enable people in the region easier access to the opportunities (jobs, education, culture) that large cities offer; connect large population areas to important transportation hubs (e.g., airports, ports, railway links).  Will the road rehabilitation/development create high enough additional traffic flows? You may invest in roads 129 PA/ Particular categories for Comments/Suggestions KAI specific intervention areas (contribution to ROP goals) rehabilitation, but traffic flows may stay the same.  Is the investment strategically aligned to the ROP 2014- 2020? If strategic alignment is to be enhanced, this criterion should come first.  Improved road safety  This should not be part of the evaluation. Legal requirements beyond the ROP imply that road investments have to follow certain road safety standards.  Reduced travel times  This needs to be better defined. In principle, any project will score a “6” here because any upgraded infrastructure can be shown to reduce travel times. That said, an express road with an extra lane will have a greater impact on travel times compared to single-lane road rehabilitation. Different scores could be assigned to, say, 10% reduction, 20% reduction, etc.  Importance of project for  Regional roads (i.e., connecting 2+ counties) should be the region (IRR, cost- encouraged by awarding these applications a bonus benefit ratio) (e.g., 10%)  Projects should be included in the Regional Development Plans  IRR and cost-benefit ratio only reflect whether an investment is worth it in of itself. These measures do not reflect whether a project is important for the region  Undertaking a full CBA is not required for projects under 50 million EUR. Axis 3 3.1.1 Hospitals  Entirely lacks the category for “project relevance for (social ROP.” Practically, only criteria are the general ones that infra.) are common for all axes (see table above). This may have to do with the fact that hospitals that received ROP funding were pre-selected by the Ministry of Health.  Potential/alternative indicators if hospitals will not be pre-selected for 2014-2020: o Number of hospital beds o Number of patients treated every year o Disease profile of a certain area o Demographics (population of target area, age groups, etc.) o Accessibility (e.g. travel time to the nearest hospital)  Note: There is a need to ensure that smaller hospitals, critical for a particular community, are not left behind. This is a key risk if the main indicators focus solely on number of patients or population of the target area. 3.1.2 Ambulatories  Potential (additional) indicators: number of inhabitants 130 PA/ Particular categories for Comments/Suggestions KAI specific intervention areas (contribution to ROP goals)  New services offered; (per locality/area served) fewer patients  Criterion 1.2: IRR must be higher than 5.5%. It is unclear redirected; faster how the IRR can apply to an ambulatory, which is for diagnostic/treatment providing a public service (profitability per se should not time be a primary concern)  Number of localities served 3.2 Social infrastructure  It is unclear why new services are scored higher than  New social better current services services/activities (6 pts.);  More is not necessarily better. See UN Directive on improved current services reducing the number of beneficiaries per center (this is (3 pts.) the new best practice).  Increase in number of  IRR must be higher than 5.5%. It is unclear how the IRR beneficiaries (6 pts.); can apply to a social center, which is a public service same number of (profitability per se should not be a primary concern) beneficiaries (3 pts.)  Potential/additional indicators: quality of services provided 3.3 Emergency equipment  There is an overlap between the two criteria (i.e., better  Improved access for access for the population goes hand in hand with an population to public improved capacity to intervene). safety and emergency  Need to better define “improved access for services population.”  Better capacity to o Specifically, which population? intervene o And how “improved” is the access after the intervention? o Access isochrones of 20, 40, and/or 60-minutes could be used o Degree of improvement could be measured as % reduction in intervention time  Alternative measures of impact: average response/intervention time; number of people reached/treated; number of potential casualties  Note: This is a simplified grid, which could serve as a best practice for other grids. For example, it does not require IRR, NPV; has a specific criterion for operations and maintenance after investment is complete. 3.4 Educational infrastructure  IRR must be higher than 5.5%. It is unclear how the IRR (some variation across different can apply to educational infrastructure, which is for types – i.e., campus, vocational providing a public service (profitability per se should not training center, schools) be a primary concern).  Improvement of the  Additional potential indicators: enrollment educational process (e.g., numbers/trends; % of students successfully integrated lower dropout rates) on the local labor market; graduation rates; dropout  Regional/local need rates. (demographic and  Indicators should be different based on the type of economic evolution, educational infrastructure. For example, universities 131 PA/ Particular categories for Comments/Suggestions KAI specific intervention areas (contribution to ROP goals) average usage rate, target a much larger area than schools (regional vs. impact on rural/ local coverage). disadvantaged  Note: “Complementarity with other projects submitted communities) for financing” (for annex 5d – professional centers)   Integration of students on this could be a best practice for all axes to encourage the labor market integrated projects.  Importance of project for the region (IRR, NPV) Axis 4 4.1 Business support structures  First two criteria are bundled, but should be separated (business)  New permanent jobs (new jobs, % of SMEs) within companies in the  Some Member Countries (e.g., Estonia) give a higher structure, within 2 years weight based on the wages of the newly created jobs of completion; % of SMEs (i.e., a higher score for a job paying above the average within the structure wage and a lower weight for jobs paying lower-than-  Local need; features for average wages). companies; accessibility  Some Member Countries (e.g., Germany) consider  Importance of project for whether the project increases productivity rather than the region (IRR>=5.5%) focusing on more jobs. It is true that more jobs do not necessarily lead to higher economic development (e.g. low paying basic service jobs, such as retail jobs, will not necessarily give local/regional economies a boost).  Need to distinguish between direct, indirect, and induced impact of a particular investment (can require estimates of direct/indirect employment generated).  IRR only reflects whether an investment is worth in and of itself, not whether a project is important for a region.  Assumptions behind numbers (e.g., IRR) should be tested heavily (e.g., number of expected firms, new jobs, etc.).  Other potential indicators: number/revenue of firms in a particular field in the region; trends (e.g., Shift-Share analysis); population of locality and corresponding area (within 1 hour drive/rail access).  Criteria on monitoring implementation and sustainability are critical and should receive higher weights. 4.2 Brownfield redevelopment  Same observations as for the previous item  New permanent jobs; size  Data on the average cost to rehabilitate 1 hectare of of rehabilitated surface polluted former industrial sites would be useful to  Local need; features for benchmark the various proposals companies; accessibility  Importance of project for the region (IRR>=5.5%) 4.3 SMEs  Need to better explain when the number of jobs is  New permanent jobs measured (i.e., the time horizon).  Local resources use  Need to better define mostly/partially for use of local 132 PA/ Particular categories for Comments/Suggestions KAI specific intervention areas (contribution to ROP goals)  Equal opportunities and resources. non-discrimination  Need to distinguish between direct, indirect, and  Environment protection induced impact of a particular investment (can require and energy efficiency estimates of direct/indirect employment generated).  New technologies and  Business plan quality (criterion 2.4) should be moved to information society the first category because it relates to whether the project is feasible and can help achieve ROP goals. This is a comprehensive analysis (e.g., from market demand to competitors), but lacks clear thresholds. Axis 5 5.1 Cultural patrimony  The first two criteria should be unbundled; the (tourism) investments extension of the tourist season can be dropped (e.g., it  Increase in number of is impossible to argue that the renovation of, say, the tourists Suceava Fortress, can extend the tourist season by  Extension in the duration itself). of the tourist season  Not all investments in tourism will necessitate hiring  New more people. Moreover, hiring public servants is permanent/temp/existing incredibly difficult under current legislation. jobs  IRR only reflects whether an investment is worth it in of  Importance of project for itself, not whether a project is important for a region. the region (IRR) 5.2 Tourism infrastructure  Annual increases (15%, 10%, 5%) in the number of  Increase in number of tourists appear randomly selected. These will also tourists (for 5 years after depend on the baseline and are notoriously hard to investment completion) measure/monitor.  Strategic integration and  Best practice of using strategic integration as a added value criterion. This could be assigned a heavier weight to  Promotion of natural promote integration of investments (not just under this resources axis). (optimal/average)  Criterion on promotion of natural resources is heavily  Location (spa/hot springs subjective. tourist resorts, etc.)  Location criterion does not appear to be justified and it  Importance of the project seems redundant (specifically, why are hot spring for the region resorts prioritized for financing?).  Number of jobs created  IRR only reflects whether an investment is worth in and (criterion 2.14) of itself, not whether a project is important for a region. 5.3 Tourism promotional  Impossible to isolate the effect of such measures on the activities number of tourists or new jobs created.  Extent of impact  Extent of impact is also hard to prove/substantiate. (national/regional/local)  Unclear why marketing per se would create additional  Increase in the number of jobs and hard to prove direct effect. tourists  “Contribution” to creation of new jobs 133 Proposals for new evaluation and selection grids 256. Annex 26 includes proposals for two types of evaluation and selection grids: 1) StatusQuo+; and 2) Semi-Competitive / Competitive. The StatusQuo+ grids follow the recommendations made above and propose a set of marginal changes on the grids that are currently used by the ROP. The Semi-Competitive and Competitive proposals bring a number of new elements and put a stronger focus on impact. Some of the key changes include:  “Financial Sustainability” is a separate criterion to underscore the need for beneficiaries to prove that they can cover both investment costs for planned projects and the subsequent operation and maintenance costs;  The evaluation of the quality, maturity, and overall sustainability of the project keeps to a large extent the criteria used in the current evaluation and selection grids, as these were found to be generally sound. Only a number of changes were made to make the individual criteria clearer;  Impact indicators were separated into three categories (Economic Impact, Social Impact, and Environmental Impact) to assess the potential gains enabled by an individual project in these three categories. Different weights were allocated to impact indicators depending on the type of project assessed (e.g., a higher economic impact score for a SME project and a higher social impact score for a health project).  To the extent possible, impact indicators tried to look both at desired outcomes and to also assess the extent to which an individual project helped achieve the output indicators proposed by the European Commission. 257. The Semi-Competitive and Competitive proposals can be used both for the Semi- Competitive Scenario and the Competitive Scenario. In the first case, the grids can be used to select the projects that have scored above a certain threshold (e.g. 5 points), while in the second case they can be used to triage projects that are received with competitive calls for proposals. 134 IX. Identifying Strategic Regional Projects (Type 1) 258. Only a few regional projects were implemented through the 2007-2013 ROP. For example, within Axis 3 and the Key Area of Intervention 3.3, emergency equipment was acquired to service each individual region, although these were obviously projects with relatively low budgets. There have also been attempts to do cross-jurisdictional projects, such as the Sibiu-Alba inter-county road project. Additionally, the Growth Poles policy and the project implemented through Axis 1 and Major Intervention Area 1.1 can be considered to be regional in nature. 259. Overall, however, despite the elaboration of Regional Development Plans, there has been little thinking around defining regional priorities. Without a regional administrative tier that could implement regional projects, the focus has been predominantly on county-level and local-level projects. And indeed, it is hard to draw regional priorities when the implementation of these strategies hinges on national decisions – e.g., the rehabilitation of a national road of critical importance for the region, administered by the National Company for Highways and National Roads. 260. For the most part, Regional Development Agencies have tried to define regional priorities in the Regional Development Plans for 2014-2020 and they have started compiling priority projects lists.35 Some of the RDAs (e.g., Bucharest-Ilfov and the West Region) are quite advanced in identifying a limited number of regional strategic projects and several regions (Center, West, and South) have drafted criteria for selecting project of regional importance. Other regions, however, have not moved significantly beyond compiling a general list of projects submitted by different public authorities. Why Regional Strategic Projects are important 261. Before setting out to implement regional strategic projects, it is important to first clarify why such projects are important. Doing large projects just for the sake of doing large projects can be counter-productive and even harmful. Large projects take longer to prepare and implement and can slow down absorption rates; they require larger budgets and can lead to a greater wastage of resources if they fail to achieve set targets; and large projects require stronger institutional frameworks (e.g., for inter-jurisdictional cooperation). 262. Large projects also come with many benefits. For one, as discussed earlier, large projects can achieve a higher impact, and bring greater benefits for each Euro invested. While a larger size does not necessarily translate into a higher impact (one can have a very high impact with very small projects), it is usually the large public investments that bring the strongest development benefits. These investments usually address challenges faced by a higher number of people and cover a wider territory. For the ROP 2014-2020, one of the key ways of achieving a higher impact is to focus more and allocate more funding to large, Type-1 projects. For the 2007-2013 programming period, only around 3.75% of the ROP’s allocated funding went to large projects. As such, it is not clear how the projects financed through the 2007-2013 ROP contributed to regional-level development in Romania. 35 A more detailed analysis of the Regional Development Plans is included in Annex 6. 135 263. Second, from a conceptual point of view, it does not make sense to have a Regional Operational Programme without really having regional projects. The purpose of the ROP should be to enable regions in Romania to come closer to and surpass average EU productivity levels. What Bucharest-Ilfov has achieved in the past years can also be achieved by other regions. One of the most important ways of achieving higher development rates is through the identification of regional priorities and securing funds to achieve those priorities. As discussed in the Competitive Cities report though, it is important to carefully define regional priorities. Hoping to achieve balanced economic development with smaller discrepancies between individual regions is a strategy that is doomed from the start. As the history of developed countries clearly shows, development means that in an initial stage some regions (the economic engines) will take off faster than other regions (the lagging areas). Over time, however, differences in living standards tend to disappear though. The 2013 World Bank report on “Competitive Cities” presents the full reasoning behind this argument. 264. Third, thinking about regional projects now can help set the foundation for the future regionalization (in 2014 and beyond). The process has commenced in 2013 and from the start it became clear that it would not be an easy task. Among the challenges faced was the question of what regions will be actually be responsible for. In this respect, developing strong regional development plans and trying to identify regional needs and priorities could go a long way in identifying what should fall within the purview of regional administrations, helping bolster the case of stakeholders who push forward the regionalization agenda. How to define regional priorities 265. Without the regionalization process complete, it is hard to define what is truly “regional.” However, rather than seeing this as a weakness of the process and a hindrance to defining regional projects, RDAs, local authorities, county authorities, the MRDPA, and other vested stakeholders should see this as an opportunity to lay the foundation for the regionalization process. More specifically, in trying to define regional priorities for the 2014- 2020 programming period, RDAs and public authorities can identify early on what some of the regional-level competencies should be. 266. The current Growth Poles policy is a good foundation upon which other regional initiatives could be developed. This has helped define the economic capitals of each region. While the future administrative capital for each region may not necessarily correspond to the economic capital, each region has a city that serves as the growth engine for the respective region. The Growth Poles report has shown that Growth Poles tend to be responsible for at least 20% of a region’s economic output, and bring the largest absolute contribution to a region’s economic growth. 267. Using the Growth Poles as the starting point for defining regional priorities, RDAs will be able to identify a number of projects physically located in the Growth Pole, but with impact on the region as a whole. Some critical areas to consider include:  Higher education. Growth poles usually house the largest universities in the region and they provide higher education opportunities for people in the region. Strengthening these university centers (e.g., by creating a regional network of campuses, as was done in Spain, or by creating academic curricula that follow regional economic strengths) would benefit not only the Growth Poles, but the region as a whole. 136  Health infrastructure. Growth poles tend to have a higher density of medical facilities and provide a larger variety of treatments. As such, the growth pole is where people in the region go for treatment they cannot find in their own locality. Regional hospitals could be placed in these hubs, attracting patients not only from across the region, but possibly also from international segments.  Economic infrastructure (e.g., business/industrial parks). As the most important regional economic centers, Growth Poles tend to be the main employers in the region. Developing economic infrastructure in the growth pole, in accordance with clear dynamics, would ultimately bring benefits not just for the growth poles themselves, but for the region as a whole.  Airports. Each growth pole should have an airport that serves people in the region36. Airports are now managed by county councils, but given the fact that they are of regional importance, they may be passed over to regions once the regionalization process is complete. The development of airport infrastructure can go beyond the airport itself (e.g., landing strips, hangars, terminals) and also focus on improving accessibility to these airports (e.g., through intermodal transport centers) and improving their functionality (e.g., by developing cargo handling capacity to better service the regional economy). Ideally, each region should have easy accessibility to the outside world, and airports are critical in this respect.  Cultural infrastructure. Growth poles often amass the largest number of cultural sites in the region and include several heritage sites that are exemplary for the culture of the region. Consequently, the conservation and rehabilitation of these sites may be of critical importance for the region. Of course, cultural patrimony may also be strewn throughout the region, such as the painted monasteries in the North-East Region, or the fortified churches in the Center Region, which – albeit harder to coordinate at the institutional level – are good candidates for projects with a regional impact.  Environmental infrastructure. Water and wastewater companies, which now often extend their service at the regional level, are usually housed within the growth poles. Similarly, large industrial rehabilitation and brownfields redevelopment projects are most needed in the growth poles themselves, but may have an impact way beyond the urban center itself. 268. In addition to growth poles and their immediate surroundings, there are a number of sectors of regional importance:  Regional connective infrastructure. The large majority of connective infrastructure projects that are of regional importance (e.g., roads and rail links) are now the purview of national authorities. As discussed in the Competitive Cities report and the Growth Poles report, some of the economically most critical investments are in connective infrastructure, which allows dynamic economic centers (e.g., the growth poles) to enlarge their economic and demographic mass. Economic mass and urbanization are critical for the development of a region, and regions should have the power to define and implement connective infrastructure projects of regional importance. Currently, the ROP can do nothing more than finance county roads, while the majority of large infrastructure projects are financed through OP Transport. Even so, investments in county roads can be done in an integrated and strategic fashion to enable more people 36 With the exception of Brașov and Ploiești (which are close to Bucharest), each of the other Growth Poles has a functioning airport. 137 in the region easier access to opportunities (e.g., jobs, healthcare, education, culture) and to enable dynamic economic centers access to a larger labor pool.  Public services infrastructure. For several regions (e.g., North-East and South), the development of public services infrastructure is one of the most important priorities. In these regions, a large share of the population continues to have poor access to running water, sewage, gas, heating, or even electricity. The large majority of people with poor access to public services live in small urban areas and in rural areas. Most of the time these localities lack the funding to finance/co-finance much needed public utility investments. These costs are now covered by county councils, but even they, as the next chapters will show, lack the financial strength to finance such projects. It may pay off therefore to think about whether it would not be more appropriate to prioritize these investments at the regional level, based on: available funds, well-defined needs, and actual viability and sustainability of such investments.  Environmental investments. The environment has different scales (local, regional, and national) and tackling environmental issues (e.g., forestation of an area, improving access to a natural park, etc.) is often a responsibility of regional governments. Within the current administrative set-up, one or more county councils could take on environmental projects of regional importance.  Cultural infrastructure. Several cultural sites of regional, national, and international importance require integrated regional approaches. For example, the painted monasteries in the North East, the fortified churches in the Center, the Dacian sites in the West, and the wooden churches in the North-West would all benefit from integrated approaches, which would help rehabilitate and preserve these sites, make them more accessible, market them better, and create regional tours for tourists.  Tourism infrastructure. Tourism is another area with a regional dimension and could include, for example, integrated tours on the sea-side (e.g., sea resorts and the Danube Delta) or integrated circuits of mountain resorts. 269. The gamut of potential regional priorities can go well beyond what was presented above. It should ideally be the responsibility of regional authorities and vested stakeholders to define regional priorities, not only within the framework presented by EU programmes, but based on a clear analysis of regional needs, strengths, and opportunities. There are often regional needs that cannot be financed from EU funds and they should not be disregarded because of this. 270. RDAs have already started developing frameworks for identifying and prioritizing strategic regional projects. This is a good approach and indicates a push from the regional level for more impactful projects. Annex 7 provides an overview of some of the selection (or rather pre-selection) criteria that were developed by several RDAs. These do need to be refined further, but it is important that a start in this direction has taken place. Addressing some of the shortcomings of large projects 271. While large strategic projects can certainly generate higher regional impact, there are also a number of shortcomings that need to be kept in mind. Some of these dangers are well known, some could be uncovered only when such projects are implemented. In what follows, this report discusses some of the key challenges of large projects in more detail. 138 Risk of slowing down absorption 272. First and foremost, large projects can be responsible for slower absorption rates. Almost by definition, they take longer to prepare and longer to implement. Given that Romania has had the worst performance in terms of absorption rates in the EU, and given that some of the worst performing OPs are those that deal with large infrastructure projects (i.e., OP Transport and OP Environment), it is normal for the Managing Authority of the ROP to shy away from large projects, just as it did for the 2007-2013 ROP. Figure 28. ROP 2007-2013 Financing Requests and Commitments by Axis, by then end of 2012 273. And indeed, when looking at the ROP 2007-2013, the worst performance on absorption was registered by Growth Poles and Urban Development Poles, which received pre-allocated funding and focused on larger integrated projects. The figure above shows that, by the end of 2012, Growth Poles and Urban Development Poles (under Axis 1.1) had the worst performance in terms of financing requests and commitments. The relatively poor performance can be explained by the fact that Axis 1 was the last one to be added to the ROP 2007-2013 and it took some time before the Integrated Development Plans were finalized. Overall, the lower- value projects (e.g., Microenterprises, Educational Infrastructure, or Social Infrastructure) had a better performance in terms of commitments and absorption. However, this is not necessarily the rule. Some of the largest projects implemented through the ROP (e.g., Business Infrastructure or Road Infrastructure) were also some of the best performing ones. 274. However, when comparing the share of financing requests to actual commitments, one can see that everything that was submitted for financing under Axis 1 was also contracted. This is based on the pre-approval process of the Integrated Development Plan, which extended the overall project selection for Axis 1 projects, but also ensured that these projects would eventually receive financing. More importantly though, the projects selected under Axis 1 are the projects considered to be truly strategic by the respective cities, rather than proposals opportunistically submitted to access funds quickly. 275. Moreover, when it comes to commitments, the ROP 2007-2013 performs better than many similar operational programmes in Member Countries with a much higher overall absorption rate. As the table below indicates, the Romanian ROP 2007-2013 made 139 commitments that were higher than those of the OPs in other Member Countries. This in effect indicates that the MA-ROP and the RDAs have done have done a good job in the pre-contracting stage, but the problems with absorption actually happen in the implementation stage – a stage of the process over which the MA and the RDAs have relatively little control. In fact, the MA-IB Collaboration reports have shown that several of the most significant absorption problems in the current programming period were system-wide issues that did not depend on the ROP itself (e.g., the public procurement framework, drawn out legal battles and challenges from companies that lose bids for EU funded projects, or delays in the construction phase). Table 20. Performance of different OPs at the end of 2012 Commitments Payments by Operational Programme by end of end of 2012 2012 ROMANIA – Regional Operational Programme 96% 34.5% GERMANY – Nordrhein-Westfalen ROP 95.1% 47.3% ESTONIA – OP for the Development of Living Environment 87.7% 54.3% SLOVENIA – Strengthening Regional Development Potential 86.6% 69.5% UK – West Wales and the Valleys Convergence Programme 103% 34% POLAND – Slaskie Regional Operational Programme 88.4% 48.5% 276. Overall, it seems that the MA-ROP could process larger investments without affecting absorption rates too much – at least in the pre-contracting phase, which it has the most control over. Even if the commitments for Growth Poles and Urban Development Poles were relatively low at the end of 2012, the overall performance of the ROP was quite good in the pre- contracting phase. Commitment rates were higher than those of other OPs in the EU, and it is likely that the MA-ROP could handle larger projects in the 2014-2020 Programming Period without jeopardizing absorption too much. At the same time, it is important to be aware of institutional and capacity limitations. As the Italy case study shows, the implementation of complex integrated projects can lead to substantial delays in the process and even to the suspension/cancelation of such projects. Risk of wasting resources 277. Large projects also imply large sums of money and significant allocations of limited funds for a limited number of interventions. As such, if these projects do not succeed, or if the strategic projects chosen turn out to be the wrong ones, there is a higher risk of wasting resources. Basically, if 25% of regional funds are allocated for 2-4 large regional projects, and if these projects are not chosen properly, than the 25% of the funds will have limited/no impact (and worse even, they will come with significant operations and maintenance costs). 278. Allocating a large share of the funds for the wrong projects usually happens when there is a poor strategy/plan in place. If the needs/challenges of the region are not properly identified, then the funds could be spent for projects that would not necessarily help the development of the region and may come with large O&M costs. In this respect, it is critical that national/regional/county/local strategies are not only well crafted, but also well-coordinated with each other. 279. At the same time, a poor strategy will lead to unwanted consequences regardless of the size of the project. More specifically, if the strategy calls for the wrong type of investments, 140 it does not matter if they are big or small. It is true that a small project bears a smaller risk, but several projects compounded together are a different story. Risk of lacking the proper institutional frameworks and adequate capacity 280. It is also very important to clearly identify who will implement large projects and if they have sufficient capacity to do successfully. While regional strategic projects do not necessarily have to cross jurisdictional boundaries, they often do. In the absence of regional administrations, cross-jurisdictional projects will have to be implemented by inter-community development associations (e.g., several counties, several localities, or several counties and localities).37 281. Inter-community development associations in Romania have a number of limitations. For one, such agreements require that the member jurisdictions work together over an extended period of time (e.g., up to 9 or 10 years in the case of EU-funded projects). There is often the risk that political shifts in one or more of the jurisdictions will lead to changing priorities or even the dissolution of the inter-jurisdictional agreement. Even if the agreement holds over the entire duration of the project’s implementation, there are other risks involved. For example, individual jurisdictions would have to coordinate expenditures and budgets – i.e., given annual budget cycles, each jurisdiction would have to allocate the necessary funds in tandem. This is quite a difficult task in a context where systemic challenges (e.g., drawn out legal battles) can make it tough to do proper budget planning even for one jurisdiction given the lack of cash flows’ predictability. 282. Should the regionalization process be completed, then regional projects would naturally be implemented by regional governments. In this case, the problem is that of capacity. Projects would most likely have to be implemented by administrative bodies with no experience of operating at the regional level, and potentially with little administrative experience generally. This could then lead to prolonged preparation and implementation periods and long delays, involving higher risks in terms of financial corrections. 283. While Inter-Community Development Associations do have their limitations, there is also evidence that they can in fact be functional. One of the best examples in this respect comes from the Local Action Groups that had to be constituted under the LEADER Program to attract funds from the National Rural Development Programme. The LEADER Program did indeed focus on projects of smaller value (that are often easier to implement), but the institutional frameworks that implemented these projects proved functional. This suggests that Inter-Community Development Associations could work at a larger scale too. 284. If regional governments will implement regional projects, their potential lack of capacity and experience could be outweighed by the RDAs. As the MA-IB Collaboration reports have recommended, RDAs should keep their status as intermediary bodies and public utility NGOs, because they have proven their effectiveness in this capacity. As such, they could continue with their current role and attributions, but work with a regional MA instead of a national MA. There is also scope to recruit staff with experience in Structural Funds 37 According to Law 215/2001, inter-community development associations are cooperative structures with legal status, established by territorial administrative units to jointly implement development projects of zonal or regional importance. 141 management from other levels, as tends to happen in other Member States when administrative changes take place. Until regional governments gain experience, the RDAs could take on more responsibility and provide technical assistance to the newly created authorities. 142 X. Pre-allocating Funds for County Residences and County Councils (Type 2) 285. The pre-allocation of funds for county residences and county councils is one of the key ways for implementing strategic and impactful projects at the sub-national level. County residences and county councils are the largest sub-national public authorities, and they are also the public authorities that are most likely to implement projects with a significant impact. 286. Type 2 projects build on the experience of Axis 1 (Integrated Urban Development) under the ROP 2007-2013. Through Axis 1, seven Growth Poles and thirteen Urban Development Poles had access to dedicated funding for integrated urban development projects, provided that they presented an Integrated Development Plan (IDP) to justify the choice of projects submitted for financing. In addition, a pool of funds was made available under Axis 1 for Urban Centers – cities with a population of over 10,000 people that were neither Growth Poles, nor Urban Development Poles. Urban Centers had to compete for funding and funds were received by those cities that submitted Integrated Development Plans first (and, of course, provided that the IDPs responded to the guidelines from the Applicant’s Guide). 287. This report recommends building on the experience of Axis 1 and expanding pre- allocations to all county residences and all county councils. Some of the reasons for this expansion were discussed in the Growth Poles report. In short, it was relatively clear why the Growth Poles were chosen (they are the key economic engines in their respective region), but it was less clear why the 13 Urban Development Poles were chosen. The team indicated that county residences play a key polarizing role within their respective counties (e.g., they are administrative centers and most often the key hubs of opportunity – in terms of jobs, education, healthcare, etc. – in the county). For their part, county councils can provide a set of complementary investments, which could help strengthen the polarizing role of county residences (e.g., they could build connective infrastructure, which would allow county residences to expand their economic mass). Counties could also complete a number of investments that are critical for their development. Why pre-allocation for county residences and county councils is important 288. The pre-allocation of ROP 2014-2020 funds for key sub-national projects requires a big financial commitment and comes with several risks. It is therefore important to have a clear reasoning for why this approach should be taken. This section discusses some of the advantages of pre-allocating funds for county residences and county councils, while later we will assess some options for addressing the risks that come with pre-allocation for strategic sub-national projects. 289. First and foremost, pre-allocation ensures that a number of major sub-national projects and priorities will be undertaken. This also helps overcome the dilemma between competitiveness and cohesion, and may increase the legitimacy of Structural Funds based on a fairer distribution of funds. In essence, this is a key way of ensuring that more impactful projects are implemented at the sub-national level. So far, with the exception of Growth Poles and Urban Development Poles, all other public entities had to compete for limited funds on a project-by- project basis. Given that projects were selected using the First-In First-Out method, public authorities had an incentive to submit simpler, quicker projects for which the technical 143 documentation could be completed quickly. This meant that they usually focused on projects they knew how to do well and on projects with a lower complexity level. This does not necessarily mean that these were not impactful (most were usually part of a development strategy), but it is likely that public authorities paid more attention to bringing up the projects quickly for financing rather than focusing on the projects that were needed the most. This seems to be corroborated by the data, with the highest commitments and the quickest absorption registered for road improvements and school rehabilitation work – investments that public authorities were already quite familiar with. 290. Pre-allocated funding means that the most important sub-national public authorities in Romania have the opportunity to take the time and prepare for financing what they consider to be key priorities. Basically, these public authorities will be given the opportunity to think about what they would want financed above all, and have sufficient time to prepare these strategic projects for financing. This would also enable a more integrated approach, allowing public authorities to submit complementary packages of projects, rather than trying to access funds one project at a time, before the total allocation is depleted. 291. Another reason for pre-allocating funds is to ensure that all major public authorities have an opportunity to finance projects that they consider to be strategic. As the map below indicates, the counties where most funds were attracted by public authorities in Romania are not always the counties that needed these funds the most. There is no clear pattern between richer and poorer counties, between counties with higher and lower infrastructure endowments, or between counties with a larger or smaller population. The same pattern holds when looking at individual axes (see Annex 5), with no obvious trends behind the location of investments for road infrastructure, social infrastructure, business infrastructure, or tourism infrastructure. The only axis where there is a clear, discernable pattern is Axis 1, where funds have been purposefully pre-allocated for the largest and most dynamic cities to help them fuel strategic investments. Figure 29. Total ROP 2007-2013 funds accessed by public authorities by August 2013 Data Source: MRDPA 144 292. Using the project-by-project FIFO approach can leave behind many unmet public needs. This can happen because some public authorities cannot submit viable projects quickly enough, because they lack the capacity to design and implement projects, or because they have other priorities. What this means in practice is that some people in Romania will be shortchanged because the public authorities representing them did not move fast enough on absorbing EU funds. Allocating funds to all major public institutions can ensure that sub-national priority needs are addressed and that EU grants (instead of limited local and national budget funds) are used for this purpose. The allocation of funds to different sub-national entities can follow the same principles used to allocate funds to individual regions, to growth poles, and to urban development poles. 293. Moreover, tying pre-allocation to the need for having multi-year operational plans can help improve public authorities’ capacity at the sub-national level. That is, EU funds should not only be used to help meet key sub-national needs, but they can also act as an incentive to improve public administration. For example, public authorities may get better at: long-term planning, organizing around multi-year budgets, prioritizing projects, and spending limited funds in a more efficient way. How to pre-allocate funds at the sub-national level 294. The implementation of Axis 1 under the ROP 2007-2013 already provides plenty of evidence on how pre-allocation could be handled in the 2014-2020 programming period. To access pre-allocated funds, Growth Poles, Urban Development Poles, and Urban Centers had to submit Integrated Development Plans (IDPs), which were drafted according to a pre-defined structure and had to justify the choice of projects submitted for ROP funding. This approach could be continued for the next ROP, with a slight change. 295. To access pre-allocated funds for the 2014-2020 programming period, public authorities should be allowed to make use of the strategies/plans they already have in place. More specifically, if county residences and county councils have development strategies that meet a number of criteria, they should be allowed to use those development strategies rather than be requested to develop new strategies just for accessing EU funds. As the 2013 World Bank report on “Enhanced Spatial Planning” argues in depth, strategies should not simply serve the purpose of attracting one type of financing, but should include projects that are vital for the community, regardless of the anticipated funding source. Strategy and planning framework in Romania for cities and counties 296. The planning system in Romania contains statutory plans and non-statutory plans. Both types usually include a list of proposed projects, which are organized by sector (e.g., social, economic, environment). For cities, the General Urban Plan (GUP) is statutory – a territorial development plan, combined with zoning regulations. The GUP is valid for 10 years. Counties use the County Spatial Plan (Planul de Amenajare a Teritoriului Județean – PATJ). Both the General Urban Plan and the County Spatial Plan have a methodological framework and an approval procedure. 297. Cities and counties also use non-statutory plans. The most frequent non-statutory plans are the development strategies, which are usually crafted for 15-20 years, and cover 145 sectors that support the development of the community. While there is no clear methodological framework guiding the elaboration of these development strategies,38 they usually encompass a number of standard sections: vision and strategic development objectives, general analysis, multi-sectoral analysis, policies, programs, and the list of projects that respond to identified problems, as well as potential sources of funding. 298. Since development strategies do not follow any guideline/methodological framework, they are approved by the city/county council. Most often, because of lack of capacity at the local/county level, statutory and non-statutory plans are subcontracted out to private or public entities. In the end, the quality of the strategy is influenced by the experience of the selected entities. 299. For the most part, individual projects submitted for ROP 2007-2013 financing were drawn from such development strategies. Integrated Development Plans were used under Axis 1 of the ROP 2007-2013 to enable cities to access pre-allocated funds for integrated projects. If the IDPs did not cover a new territory (e.g., a metropolitan area), they often duplicated development strategies that were already in place. Even when the IDPs covered a new territory, they were not always correlated with the development strategies of the constituent jurisdictions – lack of correlation and integrated thinking is a broader issue in Romania – across both territorial (at various levels) and sectoral strategies. 300. If a pre-allocation approach will be used for the ROP 2014-2020, it would be more efficient for the MA-ROP to ask participating public beneficiaries to draw on existing strategies, rather than request that new ones be developed from scratch. The only situation when new development frameworks should be requested for accessing pre-allocated funds is when these strategies are missing completely, or when addressing a territory that is not limited to one jurisdiction (e.g., metropolitan areas). Even in the case of metropolitan areas, most cities in Romania that have a metropolitan dynamic now have Integrated Development Plans in place, which were used to attract 2007-2013 ROP funds. For the most part, and to the extent that the territory of the metropolitan area did not change dramatically, these strategies could simply be updated. The vision and objectives would normally remain unchanged, but there would be a need to update parts of the socio-economic analysis and the list of priority projects. 301. The large majority of county residences and county councils already have a development strategy in place, which they could draw on to access EU funds. Annex 16 gives a detailed picture of the situation among county residences and county councils, looking at some key issues that a development strategy should have, and the degree to which these are included in individual strategies. The good news is that out of 40 county residences, only 3 lacked a development strategy (and it is possible that one was pending), and 9 had development strategies with one or more key components missing. The other 29 county residences had development strategies in place with key information available. The situation was generally positive for the county councils too, with only 5 counties missing a development strategy altogether, and with 12 counties having a development strategy that missed one or more key elements. This indicates that the large majority of county residences and county councils could 38 There are of course exceptions in this respect. For example, the Integrated Development Plans crafted under Axis 1 of the ROP 2007-2013 need to be developed a standard format. The same is valid for standard strategies such as the Local Agenda 21, or the Covenant of Mayors Sustainable Energy Action Plans (SEAP). 146 take on integrated development projects without a substantial additional upfront effort (i.e., the elaboration of an integrated development strategy). 302. The MA-ROP could use existing development strategies as a given, but require that some basic elements be taken into consideration. When these elements are missing, the MA- ROP could withhold the release of EU grants until they are incorporated in an updated version of the strategy in question. Some recommendations for how the implementation of development strategies could be improved include: 1. Establishment of an inter-department steering group at the beneficiary level, led by one department/project implementation unit to coordinate the process and ensure ownership of the development strategy. Political back-up of the strategy is also crucial in order to facilitate success. 2. A stronger focus on investment-type plans, as a base for future project proposals in line with available funds. A priority setting process and corresponding instruments need to be identified in this respect. To make this instrument effective, it is important to efficiently assess the beneficiary’s capacity to co-finance the project/projects. 3. The strategy should have an integrated approach. This will assure the concentration and coordination of resources to solve a mix of problems for a specific area, fulfilling the principle that “the whole is greater than the sum of the parts.” 4. The strategy should focus on key problems and trends rather than concentrate on statistical data. In this context, it is crucial to deploy mechanisms for measuring user demand and needs. The strategy should focus on the coordination between identified community problems/needs and the proposed projects which aim to respond to them. 5. The strategy should demonstrate/provide information regarding the involvement of the community and interested stakeholders in the design process. It should be mandatory to have public consultations – at a minimum: one for the existing situation analysis; one for the SWOT analysis, vision and development objectives; and two meetings for programs and project formulation/prioritization. 303. From the list above, one area that needs strengthening the most relates to investment-type plans. More specifically, there is a need for a mechanism that helps public authorities prioritize investments over a set period of time – e.g., 5-9 years. While public authorities may have well-defined objectives and comprehensive lists of projects, they should also clearly indicate the priority projects they will focus on, given fiscal and financial constraints. Only a limited numbers of projects can be financed within one programming period and public authorities should have an instrument at their disposal to help them properly prioritize projects that can be realistically undertaken given available resources. 304. Capital Investment Planning (CIP) is a tool that is often used in developed countries to prepare multi-year investments plans. Given that many capital investments span several years and given that available budgets can usually cover only some of the priorities included in a development strategy, the CIP comes to help public authorities prioritize investments over a set period of time. Romania also has some experience with CIP and a comprehensive toolkit was 147 developed with USAID assistance in 2004. This could be used as such by public authorities in Romania to plan their capital investments over a programming period (e.g., 2014-2022). The next sections will discuss the CIP in more detail. Capital Investment Planning as a tool to help formulate and implement projects 305. As discussed above, development strategies are not always enough for proper investment planning for multiple years – e.g., 5-9 years. Most often, they do not provide reliable information about the financial capacity of the beneficiary, the project costs, and particularly operational and maintenance (O&M) costs. 306. Capital Investment Planning (CIP) can help fill this planning gap . The Guidebook on Capital Investment Planning (CIP) for Local Governments, issued by World Bank in October 2011, places the CIP in the following context: Figure 30. Putting the CIP within the local context Source: World Bank. 2011. “Guidebook on Capital Investment Planning for Local Governments.” Urban Development Series 307. USAID has funded several CIP toolkits and guidebooks throughout the world and one was done for Romania too. The Romanian “Capital Investment Planning and Budgeting Toolkit” 148 was developed in 2004, with technical support from the Governance Reform and Sustainable Partnership (GRASP) Program. The methodology proposed for the implementation of the CIP could be used by both cities and counties, following 12 straightforward steps (see image below). The toolkit also comes with a number of methodological annexes and case studies, which can guide public authorities through the process. Figure 31. A 12-step process to implementing the CIP Source: USAID. 2004. “Capital Investment Planning and Budgeting Toolkit” [done with technical support from GRASP] 308. Ideally, the CIP should be seen as a complement to development strategies, and to the extent possible it should be kept simple. More than anything, the CIP should not be seen as a burden, but as an aid for better planning and improved decision making. Some of the key elements that could be included in the CIP are: 1. Introductory Comments - Summary comments about the project category /sectors. 2. Funded CIP Projects - CIP Plan numbers, project names, project status, and project costs for the 2014-2020 CIP Plan. 3. Project Map - Physical location for each project in the category/sector. 4. Cost and Resource Summary - Project costs and resources by year for the project category/sector. 5. Project Description Pages - Detailed project information, including the budget and schedule for each project in the category/sector. 6. Completed Projects - List of projects that were in included in the 2007-2013 programming period and are completed or anticipated to be completed by the end of 2015. 7. Operating Costs - New operating costs funded by the CIP. 149 309. There are distinct benefits to using the CIP. In brief, it is an ideal tool for linking physical development to a local government’s urban development and strategic plans. It has the following advantages:39  Coordinates physical and financial planning and relates public facilities to other public and private development plans;  Prioritizes current and future capital needs and helps balance needs with ability to pay;  Coordinates the activities of other public and private agencies in project planning, scheduling, financing, and construction, thereby reducing duplication, conflict, and costs;  Facilitates coordination with the operating budget;  Enhances credit worthiness and ensures financial stability by helping control tax rates, avoiding unanticipated borrowing, and maximizing alternative financing options;  Identifies the most efficient and economical means of financing capital projects;  Increases opportunities for obtaining funding from EU funds, private partners, donors, etc.;  Works well as a communication tool, keeping the public informed about capital needs and projects, and assists in building public support. 310. The CIP could become a strategic tool for the MA-ROP in the next programming period. It can provide critical information within a reasonable timeframe – after the CIP’s formulation (in maximum 12 months) and evaluation (in maximum 2 months) – on the number of projects and their values, in all regions. In addition, there is no possibility to change projects within CIP after the evaluation process. The USAID CIP Toolkit developed for Romania includes a detailed example of the implementation of the tool in Giurgiu. Annex 17 provides another summary example from the Toolkit from Nis, Serbia. 311. At the same time, it is important to be aware of some of the risks that the CIP tool brings with it. While it is usually easy to comprehend and implement, it often requires technical and financial data for individual projects (which may not be readily available), and demands a commitment from local authorities to see the process through. Also, in evaluating and selecting projects, it is important to engage a wide stakeholder base rather than leaving this process to be completed a small group of local/county staff. Proposed framework for evaluating development strategies 312. A development strategy should be the cornerstone for pre-allocating funds to distinct sub-national governments. It should clearly spell out the key priorities and needs of the respective administrative territorial unit. Moreover, a tool such as Capital Investment Planning could be used to prioritize projects and focus on the most stringent needs within the available fiscal space. 313. Of course, it will also be important for the ROP 2014-2020 to have an evaluation system in place that enables the proper vetting of development strategies. There already is some experience with evaluating integrated development plans for the 2007-2013 Programming 39 See for example USAID. 2004. “Capital Investment Planning and Budgeting Toolkit” [done with t echnical support from GRASP] 150 Period and this could be built upon going forward. Within the framework presented above, there are three major approaches for evaluating development strategies:  Option 1: Evaluate only the CIP;  Option 2: Evaluate only the development strategy and individual projects; and  Option 3: Evaluate both the development strategy and the CIP. Figure 32. Option 1 for evaluating city/county development strategies 151 314. Option 1 starts from the premise that the development strategies prepared by county residences and county councils are good as they are (reflecting the priorities and needs of those communities), and there is only a need to evaluate the CIP. In essence, in line with the subsidiarity principle, it is assumed that local/county authorities know best what their own priorities/needs are, and there is only a need to evaluate the CIP to see if the respective public authority has the fiscal space to complete the proposed investments. The figure above gives a snapshot of how this process could look. 315. Basically, pre-allocated funds (i.e., for Type 1 and Type 2 projects) will be made available only if the targeted public authorities show that the proposed projects are part of a comprehensive Capital Investment Plan. The Plan should include not only the projects submitted for ROP financing, but all the projects that the respective public authority plans to complete over a specific time period – e.g., 5-9 years. Individual projects would be evaluated using existing evaluation and selection grids. 316. Option 2 follows to a significant degree the current evaluation approach, with an assessment of the development strategy and individual projects. The basic change to the existent approach is to not request a new integrated development strategy unless: a) there is no strategy in place; or b) the strategy addresses a new territory for which no integrated development strategy was crafted. Where development strategies do not meet basic quality criteria, a request should be made to incorporate updates/improvements. Individual projects would be evaluated using existent evaluation and selection grids. Option 2 – Evaluation of development strategies and individual projects Phase 1a - Evaluation as part of the administrative assessment phase: The evaluation process covers projects in coordination with the content of strategy documents. At this stage the beneficiaries have to submit projects listed in the following document: 1. City or County Development Strategy (Request: The strategies have to be updated if out of date) (Evaluation Yes/No or PASS/FAIL) The beneficiaries should present the City/County decisions that prove that the community strategies have been approved by city/county councils (Request: Decision of the City/County council). Phase 1.b – Project evaluation within the strategy context The project performance is rated, linked with strategies, as follows: 1. Project responds to local needs/demand. The project is rated highly if it has a large number of users and negatively if the number of users is insignificant (from -1 to +1). 2. The project supports the community to achieve development objectives. These can be rated from a very low level to a very high level (from -1 to +1). 3. Project complements other public facility. It can be rated negatively if duplicating an existing facility, and positively if it complements an existing one (from -1 to +1). 152 Phase 2 – Project evaluation After phase 1b, the project is evaluated from a technical point of view (feasibility study, etc.), using the appropriate evaluation and selection grids.40 317. Option 3 is the most prudent one, but also the most time consuming. It involves both an evaluation of the content of the development strategy, and a technical and procedural evaluation of the CIP. The key steps of this evaluation option are listed in the box below. Option 3 – Evaluation of development strategies and CIP Phase 1a - Evaluation as part of the administrative assessment: The evaluation process covers the projects, in coordination with the content of strategy documents. At this stage, the beneficiaries have to submit projects that are listed in the following: 1. City or County Development Strategy (Request: The strategies have to be updated as of 2013-2014) (Evaluation Yes/No or PASS/FAIL) Beneficiaries should present the City/County decisions proving that the community strategies have been approved by city/county councils (Request: Decision of the City/County council). Phase 2 - Evaluation of the Capital Investment Plan41 The evaluation criteria encompass aspects presented in the 12 steps described before.42 Example for evaluators: A. Evaluate technical aspects: Cost recovery of the project from taxes and user fees may be used to rate projects. Full, partial or not at all – this simple scale can be used as a good criterion in the project selection process. (Evaluate from -1 to +1) Project impact on operation and maintenance costs can be rated as well, considering various levels of “a decrease” and “an increase” in costs. (Evaluate from -1 to +1). b. Evaluate procedural aspects: Criteria to establish priorities for projects (Evaluation from -1 to +1) 40 This option risks promoting projects without a clear image about the impact on the local budget, the estimation of the operational costs of the proposed interventions, and of the projects already implemented during 2007-2013. 41 This plan should be presented as a model within the applicant guide. The CIP takes from 6 to 9 months to complete. The evaluation is accomplished by a team of 2 – 3 professionals (public finance specialist, urban planner, infrastructure engineer). The evaluation normally takes 7-10 days. 42 A model of detailed rating of the CIP may be developed if the MA-ROP decides to use the tool in the next programming period. 153 Available funds for proposed projects (Evaluation from -1 to +1) Phase 3 – Project evaluation After phase 2, the project is evaluated from technical point of view (feasibility study, etc.), using specific grids. What should be the pre-allocation for each county residence and county council 318. Doing pre-allocation for sub-national entities will naturally bring up the thorny question of how much money each of these entities should receive. This will be, in part, a question of available funds and of MA-ROP policy. For the ROP 2007-2013, the allocation per region was decided upon according to population and development level (with more funds going to the less developed regions). Similarly, the allocation for Growth Poles and Urban Development Poles was determined primarily based on their development level, with more funds going to cities from relatively poorer regions. A similar approach may be taken for the financial exercise and it is ultimately the decision of the MA how it will split the money, should it decide to do pre-allocations. 319. This section attempts to provide a potential estimate of allocations, based on the fiscal space for each of the sub-national entities under consideration. In essence, we look at the financial performance and strength of all the county residences and county councils, for three fiscal years – 2012, 2011, and 2010. The data are available in full in Annex 20. In what follows, we will go step by step through the process of trying to estimate pre-allocations for county residences and county councils. 320. A prudent approach for public authorities is to not spend more than 30% of their non- earmarked revenues for capital expenses. Earmarked revenues are those revenues that have to be spent with a particular purpose. EU funds for example, are earmarked funds, as they can only be spent on certain issues. Capital expenses generally represent non-recurrent expenses, such as investments in infrastructure. Non-earmarked revenues are revenues that can be spent by public authorities at will, within the boundaries of the law. For our purposes, non-earmarked revenues were calculated using the following formula (Annex 20 includes all the detailed data): Non-Earmarked Revenues = (Own Revenues) + (Shared Personal Income Tax) + (Non-Earmarked Grants) + (Other Revenues) 321. Next, we estimated the non-earmarked revenues that would be earned during 2014- 2022 (the entire financial exercise). We used three estimates in this respect. We considered that the economy would grow at different compound annual growth rates: 1%, 3%, and 5%. For each of these compound annual growth rates, we estimated how many non-earmarked revenues would be earned by each subnational entity. As a starting base, we used the average non-earmarked revenues earned in 2012, 2011, and 2010. This offers a better base than just taking the data for a single year. However, to get a more reliable base, it would help if the average would be calculated for 5 consecutive years. 154 322. Once the sum of non-earmarked revenues was calculated for the 2014-2022 period, we took 30% of that. Again, it is considered prudent to not spend more than 30% of non- earmarked grants on capital expenses. Each capital expense, such as the development of a road, also entails operation and maintenance costs once the investment it is completed. If capital expenses amount to more than 30% of non-earmarked revenues, sub-national entities run the risk of not having enough revenues to pay for the operation and maintenance of the investment, once it is finished. Of course, different capital expenses will come with corresponding operations and maintenance costs once they are completed. For example, the planting of a forest will require lower recurrent costs than the development of a road (i.e., a forest does not need a lot of maintenance once it is planted). The 30% threshold is, all things considered, a threshold that would ensure fiscal prudence for public authorities. 323. The table below indicates the Prudent Fiscal Space for Capital Expenses over 2014- 2022 for all 40 county residences and all 41 county councils. This is basically the upper threshold that these sub-national entities should have in mind when they plan their expenditures for 2014-2022. In this calculation these beneficiaries should include not only the expenses they expect to make from EU funds, but also the expenses from other sources (e.g., local budget, national transfers, other donors, etc.). Table 21. Prudent Fiscal Space for Capital Expenditures over 2014-2020, for County Residences and County Councils Prudent Fiscal Space for Capital Expenditures over 2014-2022 Estimated Compound Annual Growth Rates (various scenarios) County Residences 1% 3% 5% Alba Iulia 55,754,130 61,656,245 68,220,337 Arad 165,397,057 182,905,937 202,378,603 Pitesti 123,618,908 136,705,166 151,259,172 Bacau 145,271,135 160,649,491 177,752,675 Oradea 155,641,306 172,117,444 190,441,538 Bistrita 59,206,021 65,473,552 72,444,045 Botosani 55,058,715 60,887,213 67,369,432 Brasov 235,197,600 260,095,542 287,786,025 Braila 108,243,824 119,702,481 132,446,334 Buzau 95,797,857 105,938,987 117,217,542 Resita 43,549,976 48,160,162 53,287,425 Calarasi 40,471,772 44,756,101 49,520,958 Cluj-Napoca 281,284,761 311,061,475 344,177,931 Constanta 232,476,934 257,086,867 284,457,038 Sfantu Gheorghe 36,374,064 40,224,611 44,507,033 Targoviste 67,589,270 74,744,248 82,701,725 Craiova 192,385,505 212,751,372 235,401,465 Galati 186,735,569 206,503,336 228,488,246 Giurgiu 35,023,333 38,730,891 42,854,288 Targu Jiu 56,279,199 62,236,897 68,862,807 Miercurea Ciuc 34,043,364 37,647,184 41,655,206 Deva 57,788,759 63,906,258 70,709,893 Slobozia 32,562,470 36,009,523 39,843,195 Iasi 200,965,345 222,239,472 245,899,694 Baia Mare 87,721,130 97,007,260 107,334,918 Turnu Severin 67,043,758 74,140,988 82,034,241 Targu Mures 130,871,300 144,725,293 160,133,144 155 Piatra Neamt 71,777,109 79,375,411 87,825,934 Slatina 61,800,636 68,342,831 75,618,796 Ploiesti 182,658,081 201,994,207 223,499,063 Satu Mare 74,227,741 82,085,466 90,824,509 Zalau 47,951,983 53,028,164 58,673,688 Sibiu 150,080,982 165,968,506 183,637,967 Suceava 95,482,372 105,590,105 116,831,517 Alexandria 32,050,163 35,442,983 39,216,340 Timisoara 293,095,024 324,121,969 358,628,881 Tulcea 56,192,722 62,141,266 68,756,995 Vaslui 33,418,555 36,956,232 40,890,694 Ramnicu Valcea 84,066,125 92,965,338 102,862,682 Focsani 56,271,984 62,228,918 68,853,979 County Councils Alba 38,258,067 42,308,054 46,812,285 Arad 70,465,105 77,924,519 86,220,576 Arges 69,926,060 77,328,411 85,561,005 Bacau 61,009,940 67,468,432 74,651,306 Bihor 56,632,053 62,627,104 69,294,557 Bistrita-Nasaud 44,999,440 49,763,066 55,060,978 Botosani 34,337,273 37,972,205 42,014,830 Brasov 77,826,692 86,065,401 95,228,159 Braila 35,098,898 38,814,457 42,946,750 Buzau 39,693,630 43,895,585 48,568,829 Caras Severin 29,427,170 32,542,321 36,006,865 Calarasi 27,888,691 30,840,979 34,124,394 Cluj 130,082,070 143,852,516 159,167,449 Constanta 107,286,673 118,644,005 131,275,171 Covasna 23,063,413 25,504,899 28,220,220 Dambovita 46,010,448 50,881,099 56,298,040 Dolj 75,278,849 83,247,843 92,110,636 Galati 53,220,977 58,854,933 65,120,790 Giurgiu 33,594,070 37,150,328 41,105,453 Gorj 39,979,690 44,211,927 48,918,850 Harghita 33,063,610 36,563,713 40,456,386 Hunedoara 54,121,188 59,850,440 66,222,282 Ialomita 24,025,936 26,569,314 29,397,956 Iasi 86,671,914 95,846,975 106,051,106 Ilfov 86,016,448 95,122,121 105,249,083 Maramures 50,195,630 55,509,323 61,418,998 Mehedinti 35,540,212 39,302,487 43,486,737 Mures 55,546,031 61,426,116 67,965,709 Neamt 52,886,223 58,484,741 64,711,187 Olt 35,823,733 39,616,021 43,833,651 Prahova 87,326,382 96,570,724 106,851,908 Satu Mare 36,483,848 40,346,017 44,641,364 Salaj 26,580,339 29,394,125 32,523,504 Sibiu 64,130,033 70,918,817 78,469,028 Suceava 62,590,720 69,216,554 76,585,538 Teleorman 41,596,561 45,999,960 50,897,242 Timis 99,336,035 109,851,716 121,546,830 Tulcea 40,211,156 44,467,896 49,202,070 Vaslui 34,996,167 38,700,850 42,821,049 Valcea 34,786,953 38,469,489 42,565,056 Vrancea 42,186,933 46,652,828 51,619,616 156 324. To take the analysis a step further, we also looked at the percentage of capital expenses that went to EU investments. These are averages that can provide a better understanding of the dependence on EU funds for county residences and county councils. As can be seen in the table below, for county residences around 30% of capital expenses are represented by EU investments, whereas for county councils a staggering 60% of capital expenses are done with EU grants. As such, county councils are likely to be more dependent on EU funds to finalize needed investments within their territory, whereas county residences and municipalities in general seem to have more varied funding sources. This could also be a useful criterion to use if funds will be pre-allocated to county residences and county councils. Table 22. Share of Capital Expenses that go to EU investments Sub-national Entity 2012 2011 2010 Average for 2010-2012 County Residence 39% 32% 17% 29% County Council 64% 59% 57% 60% Figure 33. Total budget figures for county councils and county residences in 2012 Data Source: Ministry of Regional Development and Public Administration– www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html Figure 34. Degree of dependence on earmarked grants for county councils and county residences, in 2012 Data Source: Ministry of Regional Development and Public Administration – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html 157 325. Lastly, it helps to look at the overall revenues of county residences and county councils, and at their degree of dependence on earmarked grants to have a better idea of their capacity to absorb EU funds in the future. The maps above show cities and counties that have larger overall revenues, and others that remain more dependent on earmarked grants. Overall, it is safe to assume that the sub-national entities with larger budgets will also be able to handle larger pre-allocations than the entities with smaller budgets. Similarly, the sub-national entities that have a high dependence on earmarked grants may have a more limited fiscal space during the next programming period. What kind of projects should receive pre-allocations under Type 2 326. Pre-allocating funds for county residences and county councils offers the MA-ROP an opportunity to guide sub-national investments in a particular direction. More specifically, the programme can use pre-allocations to help county residences and county councils focus on some key development priorities, defined at three distinct levels:  At the EU level, it is the Europe 2020 strategy that draws the major areas of intervention.  At the national level, it is the Partnership Agreement and the ROP 2014-2020 Programmatic Document that will provide major guidelines.  At the local/county level, it is local and regional priorities that matter the most. 327. Looking at these three different tiers, one can come closer to identifying a number of local/county priority areas for the 2014-2020 Programming Period. The Europe 2020 Strategy is in place and was discussed in more detail in an earlier chapter; most county residences and county councils have development strategies in place, with outlined objectives and lists of projects (see Annex 16); the Partnership Agreement is still under negotiation as of January 2014, but a first draft of proposed axes for the ROP 2014-2020 is used for planning purposes by the MA-ROP. 328. Annex 18 lists the draft proposed axes for the ROP 2014-2020 and indicates areas that could be financed through pre-allocation for cities and counties (Type 2 projects). The proposed axes respond to the demands of the Europe 2020 Strategy and include a number of additional priorities in addition to the areas covered during the 2007-2013 programming period. Some of the key areas that could be financed under Type 2 interventions (i.e., projects of over 30 million RON) include:  Connective infrastructure;  Health and social infrastructure;  Energy efficiency in public buildings;  Public transport;  Brownfields redevelopment;  Natural and cultural heritage; and  Business incubators. 329. The next step in identifying potential Type 2 projects is to look at the ROP 2007-2013. For one, we can look at what was financed under Axis 1. Annex 11 provides a detailed overview of the main areas and key types of projects financed. Thus, we have urban infrastructure (e.g., rehabilitation of buildings, public transport, cultural heritage), business infrastructure, and social 158 infrastructure. But Axis 1 provides only a partial picture, as cities and counties have also applied for funding under Axes 2, 3, 4, and 5. 330. Annex 19 highlights some of the key investments undertaken under all ROP 2007-2013 axes by cities and counties in Romania. They cover a wide gamut of investments and, based on the funds allocated for such interventions, some of the key areas include:  Integrated urban development: o Urban infrastructure; o Business infrastructure; and o Social infrastructure.  Road infrastructure;  Education infrastructure;  Cultural heritage;  Health infrastructure;  Tourism infrastructure;  Business infrastructure;  Social infrastructure;  Tourism promotion;  Emergency equipment;  Brownfields redevelopment; and  Energy efficiency in buildings (pilot call, launched in December 2012). 331. More than providing a glimpse of what types of investments were pursued by cities and counties under the ROP 2007-2013, the data collected for the current programming period also provide an idea of the preferences of cities and counties. Thus, the share of financing requests out of the total allocation by Key Area of Intervention indicates what cities and counties cared about the most. Of course, these numbers are an imperfect measure, as the FIFO approach often worked as an incentive to focus on things that public authorities could do well (e.g., roads rehabilitation). Nonetheless, it is interesting to see which Key Areas of Intervention were most sought after by cities and councils. Consequently, we have, in order of preference (in parentheses we have included the allocation for that particular key area of intervention to add another dimension to the analysis – i.e. an area with a high share of financing requests may have had a low allocation):  Business infrastructure (247 mil Euro)  Educational infrastructure (242 mil Euro)  Road infrastructure (748 mil Euro)  Tourism infrastructure (294 mil Euro)  Social infrastructure (85 mil Euro)  Cultural heritage (242 mil Euro)  Health infrastructure (174 mil Euro)  Integrated Urban Development Plans for Urban centers (~301 mil Euro)  Brownfields redevelopment (27 mil Euro)  Tourism promotion (128 mil Euro)  Emergency equipment (85 mil Euro)  Integrated Urban Development Plans for Urban Development Poles (~195 mil Euro)  Integrated Urban Development Plans for Growth Poles (~621 mil Euro) 159 332. If the actual allocation for each key area of intervention is multiplied by the actual financing request rate, one gets a better sense of the key priorities for cities and counties under the ROP 2007-2013. This simple Priority Index (PI=Allocation*Financing Request Rate) gives a more rounded picture than just the commitment rate. The top priorities are:  PRIORITY 1: Road infrastructure (PI=1960)  PRIORITY 2: Business infrastructure (PI=805)  PRIORITY 3: Tourism infrastructure (PI=714)  PRIORITY 4: Educational infrastructure (PI=711)  PRIORITY 5: Cultural heritage (PI=462)  PRIORITY 6: Health infrastructure (PI=289) 333. Of course, the numbers above present just one side of the coin. The projects that cities and counties preferred under the ROP 2007-2013 are limited to the gamut of project types financed by the ROP. To get a fuller picture, it would help to also see what cities and counties prefer to finance from other sources (e.g., local/county budget, national transfers, PPPs). Unfortunately, there is no centralized database that would allow such an analysis. 334. Nonetheless, scanning a number of local and county development strategies does provide a glimpse of other preferences, apart from those expressed under the ROP. Project types were assigned different priorities depending on the frequency with which they appeared in those development strategies and the estimated financial investment. Thus, for county residences we have the following priority list, in order of listing: Table 23. Investment priorities for county residences and county councils Covered Covered Not by ROP Rank Type of Investment by ROP covered 2014- 2007-2013 by ROP* 2020 1 Urban roads x x 2 Tourism (branding, business tourism, cultural heritage) x x x 3 Modernization of Urban Heating Systems x 4 Social infrastructure x x 5 Business infrastructure (technological/industrial parks, x x x non-conventional industries, incubators) 6 Energy efficiency measures (not limited to residential x x x buildings) 7 Rehabilitation of building facades x x x 8 IT&C x 9 Public utilities infrastructure x 10 Public transport x x x 11 Educational infrastructure x x 12 Housing x *Note: Some sub-sectors (e.g., acquisition of public transport vehicles) were not covered by the ROP 2007-2013, although the general sector was (e.g., public transport) 160 For county councils, the priority list is as follows: Covered Covered by by ROP Not covered Rank Type of Investment ROP 2007- 43 2014- by ROP 2013 2020 1 County road infrastructure x x 2 Highways x 3 Tourism x ? 4 Business support infrastructure x x 5 Social infrastructure x x 6 Education infrastructure x x 7 Waste Management x 8 Water & sewage management systems x 44 9 Health infrastructure x x 10 Culture – promotion/infrastructure x x 11 Rural development x 12 Regeneration of industrial sites x x 13 Connective transport infrastructure to EU x networks 14 Airport infrastructure (local, regional and x international) 15 Water transportation infrastructure (shipyards) x 16 Railways Transport x 17 Border transport infrastructure x 18 Alternative energies (including local industries x producing equipment) 19 Fisheries industry x 335. The picture painted above is not that much different, from the priorities listed under the ROP. As such, one could propose a number of priorities to be covered under Type 2 pre- allocations. For county residences, the priorities could be kept largely the same as they are now under Axis 1 Integrated Development Plans, which would also make things easier for the MA- ROP and RDA staff – i.e., the approach used in 2007-2013 could be continued for 2014-2020. The one priority that could be added is energy efficiency, which is a large pillar under the Europe 2020 Strategy. 336. For county residences, priority investment areas under Type 2 pre-allocations could include:  URBAN INFRASTRUCTURE o Connective infrastructure – roads rehabilitation; new metropolitan access ways; under-ground or over-ground passage ways. o Public transport – rehabilitation and extension of tram and bus lines; acquisition of energy efficient rolling stock; metropolitan transit lines; multi-modal transit stops. o Energy efficiency in public buildings. o Rehabilitation of educational and health infrastructure. 43 Some of the investments were partially covered by other OPs, but in most cases LPAs were not eligible for funding (e.g. highways). 44 Very limited coverage 161 o Development of new education and health infrastructure where a distinct needs exists.  BUSINESS INFRASTRUCTURE  SOCIAL INFRASTRUCTURE 337. For county councils, priority investment areas under Type 2 pre-allocations could include:  CONNECTIVE INFRASTRUCTURE o Rehabilitation/development of county roads, with a focus on improving access to opportunities in the dynamic centers of activity in the county. o Development of ring roads for important centers of activity or for busy road section.  HEALTH, EDUCATIONAL, and SOCIAL INFRASTRUCTURE  BUSINESS INFRASTRUCTURE  CULTURAL HERITAGE 338. To a large extent, the priorities above correspond to the list discussed in the Competitive Cities report: connective infrastructure; public services infrastructure; quality of life investments; and targeted measures for marginalized communities. Some of these areas, however, particularly measures for marginalized communities, should have dedicated funding lines (e.g., through a CLLD approach), ensuring that public authorities also pay attention to issues that are often overlooked. The way pre-allocations will be designed could also encourage a focus on more forward-thinking measures (e.g., climate change). 339. Ultimately and most importantly, however, local and county priorities under the ROP should be discussed with local and county authorities, as well as with communities and vested stakeholders. It is important to understand directly from the grassroots level what they consider to be of priority for the 2014-2020 Programming Period. Closely listening to the needs of local and county authorities could also ensure faster absorption rates, as beneficiaries will apply more quickly for projects that they truly need. Addressing some of the shortcomings of pre-allocating funds to sub-national entities 340. While the pre-allocation of funds for sub-national projects can ultimately lead to a more efficient and impactful distribution, it does not come without risks. Therefore, this section will address some of the most important risks identified by the team in the discussion with the MA-ROP, the IBs, beneficiaries, and other stakeholders. For each identified risk, the team will discuss ways to address it. Risk of slowing down absorption 341. As with the regional interventions (Type 1), local and county strategic projects (Type 2) may help make the ROP 2014-2020 more impactful, but they can also negatively affect absorption. This is particularly poignant if we consider that Type 2 pre-allocations would require that 82 development strategies be in place, and that these strategies and/or the corresponding Capital Investment Plans would have to be evaluated under the ROP. There is also the risk of having major sub-national public authorities know that they have pre-allocated funds they can count on (unlike a competitive approach, where they would have to compete with other authorities to access funds). Particularly relevant in this respect is the relatively poor 162 performance of the Growth Poles and the Urban Development Poles under axis 1 of the ROP 2007-2013, which have the lowest commitment rates and the lowest absorption rates of all the axes, while also having the largest financial allocation. 342. Fears with respect to how absorption will be affected by such a large pre-allocation are absolutely valid and it is ultimately the MA’s decision on how it will design the ROP 2014- 2020. Given that Romania has the worst performance on absorption for the current programming period makes these fears that much more poignant. It is important, however, to also look at the bigger picture and at available evidence, as well as at new approaches, to see how a balance between absorption and impact could be struck. As Annex 15 shows, it is only under Axis 1 (the one with the slowest absorption now) that the ROP shows more strategic outcomes – i.e., allocating funds where they are most needed from a strategic point of view. The performance of Axes 2, 3, 4, and 5 indicate a haphazard pattern, with funds going to those beneficiaries that moved the quickest, without a clear indication that this was the best use of limited EU funds and that actual pressing needs on the ground were targeted first. 343. To avoid a situation where county residences and county councils “sit” on the funds pre-allocated, the approach for Type 2 is to impose a realistic and credible deadline by which these authorities have to contract their projects. If a county residence or a county council does not move quickly enough on contracting out those projects, then the funds go to a competitive pool and the more active local/county authorities can access those funds. This also ties in with one of the strongest principles to come out of behavioral economics, the Loss Aversion Principle. According to this, people are more averse to losing something they already have than they are happy about gaining something new. Moreover, county residences and county councils that will not be capable of accessing allocated funds in time will face the opprobrium of their electorate. It is bad from a political point of view to lose pre-allocated EU grants, particularly to a “rival” city. Thus, rather than slowing down absorption, this measure may in fact speed-up absorption, while also assuring a higher impact of used funds. It is important still to build from the start mechanisms that ward off against political interference (e.g., sub-national authorities that come and plead with the MA-ROP to get an extension for contracting out pre-allocated funds). The moment an extension is granted to a county residence or a county council, the whole system risks falling apart, as everybody else will feel entitled to receive at least the same extension. 344. The slow absorption of the Growth Poles and the Urban Development Poles can be explained by the slow start they had in the beginning of the 2007-2013 programming period. Not only was Axis 1 the latest one to be added to the roster, but it also took some time before Growth Poles and Urban Development Poles managed to finalize their integrated urban development plans. Moreover, as the discussion on regional strategic projects has shown, the MA-ROP, when compared with other (and similar) Operational Programmes in the EU, has a very high commitment rate – more than double the commitment rates registered by other OPs. This indicates, on the one hand, that absorption issues can largely be attributed to systemic issues, such as the inefficient national public procurement framework. On the other hand, it suggests that the MA has performed quite well and, provided that systemic issues are resolved, it could achieve absorption rates similar to better performing countries, while at the same time also ensuring a higher impact. 163 345. It will also be important to look at the experience of the OP Environment 2007-2013, which pre-allocated funds to all county councils for water, wastewater, and solid waste management projects. While absorption under the OP Environment is quite low (22.31% in December 2013), the commitment rate is well above 100%. This indicates that the large majority of projects have been contracted, but it is during the implementation stage where things move slower. 346. As for the capacity of county residences and county councils to produce good strategies and/or capital investment plans, the experience with Axis 1 provides some solid evidence. The most important thing to note is that in addition to the 7 IDPs submitted by Growth Poles, and the 13 IDPs submitted by Urban Development Poles, there were over 70 IDPs submitted by Urban Centers. Moreover, there are relatively small cities like Alba Iulia, Slatina, or Bistrița, which have managed to attract a more significant share of available funds than larger cities and Urban Development Poles, such as Oradea, Satu Mare, Baia Mare, or Deva. Moreover, in terms of the amount of funds attracted by August 2013, they were on par with much larger cities, such as Timișoara, Constanța, Galați, Brăila, or Bacău. 347. The fact that relatively small cities in Romania have been capable of generating good Integrated Development Plans, and the fact that many have managed to perform quite well on absorption, is an indication that the same can be achieved in 2014-2020. Overall absorption and commitment rates for urban centers are above the performance of some individual axes, although the start was much delayed and they had to complete integrated development plans. The relative good performance of urban centers can be explained by the competition they had to face for limited funding. The same competitive spirit can be instilled among county residences and county councils for 2014-2020, but the competition should be primarily with themselves, not with other sub-national entities. Only when they lose the competition with themselves (i.e. they fail to contract projects in the set timeframe) should they enter in a competition with other public authorities. 348. Of course, the MA may choose to keep the current Urban Centers approach for the 2014-2020 Programming Period – i.e., have a dedicated funding pool for smaller cities (the cities that are not a county residence). The experience of the ROP 2007-2013 shows that even smaller cities have the capacity to generate and implement projects. It will be important however to open up the competition for all cities, not just the ones with a population over 10,000. Many of the smaller towns have complained that they were cut off from EU funding because they were not eligible for either ROP funding or for Rural Development Funding. Moreover, given that smaller cities and towns would not be able to finance more than a few projects under an Urban Centers approach (i.e., dedicated pool for smaller cities, which can be accessed by the cities in a competitive fashion) and because they typically have a limited administrative capacity, they should not be required to submit an integrated development plan. A simpler approach, such as a Capital Investment Plan, could be encouraged instead. 164 Figure 35. Number of projects submitted under Axis 1 by individual cities Data Source: Ministry of Regional Development and Public Administration – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html Figure 36. Value of projects submitted for funding under Axis 1 by individual cities Data Source: Ministry of Regional Development and Public Administration– www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html 165 349. To avoid a situation where all Type 2 projects come in for evaluation at the same time, these could be staggered depending on the size and importance of the respective county residence and county council. The map below gives an overview of the budget situation for these sub-national entities. An easy way of staggering the calls for Development Strategies and/or Capital Investment Plans could be done following the allocations for Axis 1 (under ROP 2007-2013) and would look as follows:  Growth Poles and counties where Growth Poles are located – 3 years from the start of the corresponding axis;  Urban Development Poles and counties where UPDs are located – 2.5 years from the start of the corresponding axis;  Other county residences and respective counties – 2 years from the start of the corresponding axis. Figure 37. Total budgets for sub-national entities, in 2012 Data Source: Ministry of Regional Development and Public Administration – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html 350. An important thing to keep in mind is the collaboration and coordination between county residences and county councils. Ideally, county residences and county councils should coordinate the elaboration of their development strategies and capital investment plans, and they should submit their proposals together. This will allow independent evaluators to gage the extent to which priorities are aligned within the same county, and to award bonus points when there is clear evidence of joint planning and common strategic thinking. Again, pre-allocated 166 funds could be used as incentives for encouraging smart planning and efficient spending of public resources. 351. Another argument against a potential slowing down of absorption under the pre- allocation model is offered by the current performance under Axis 1. As the figure below indicates, Growth Pole and Urban Development Pole projects went much faster to evaluation and eventual contracting than the projects under the other axes. Thus, although the preparation of the integrated development strategies took longer, the individual projects submitted moved through the system much quicker than on the other axes. At the same, urban centers had one of the worst performances on this measure, for causes discussed elsewhere (the report on “ROP 2.0: MA-IB Collaboration”) and primarily related to the system’s capacity to process a large batch of applications received at the same time. Figure 38 How fast (in days) ROP projects move through the system Data Source: Ministry of Regional Development and Public Administration 352. Indeed, the poor performance of urban centers can be explained by the high number of cities (around 70) that have submitted integrated development plans under this Key Area of Intervention. The evaluation of these IDPs took considerably more time than the evaluation of a simple project. At the same time, it is important to remember that at the end of 2012, Urban Centers had the highest commitment rate (143%) of any other Key Area of Intervention (KAI). This means that Urban Centers were among the best performers on absorption. 353. Projects submitted under Axis 1 also received from independent evaluators the highest scores on the Technical and Financial grids. Growth Poles received the highest average scores than any other KAI, while Urban Development Poles and Urban Centers received scores above the ROP 2007-2013 average. Some of the poorer performing KAIs included Microenterprises, Road Infrastructure, and Touristic Infrastructure – two areas where a significant number of applications were received. Brownfields Redevelopment, which scored poorly on almost every criteria related to absorption, also scored poorly on the technical and financial evaluation – primarily because such projects tend to be complicated and there is not enough experience in Romania in having the public sector seal with such investments. 167 Figure 39. Technical and Financial Evaluation Scores for ROP 2007-2013 projects Data Source: Ministry of Regional Development and Public Administration 354. The discussion above brings a number of important points to the foreground. One: although Type 2 projects take a lot of time to prepare, there are significant time savings in the subsequent phases of the process. Two: when a competitive element is introduced (e.g., cities competing with each other for funds, or cities competing to finalize submission of strategies on time), absorption rates are increased, while at the same time ensuring that funds are allocated to strategic priorities. Risk of not having enough funds for 40 county residences and 41 county councils 355. Another issue that comes up is the lack of sufficient pre-allocated funds for all 81 entities. Moreover, it is rightfully argued that the ROP 2014-2020 may become too one- dimensional and may jeopardize absorption by putting all eggs in one basket. Some of these fears can easily be addressed if we simply look at the ROP 2007-2013 experience and if we consider the future planned funds for the ROP. 356. The allocations for the ROP in the next programming period are slated to more than double (to around 8 billion Euros). This means that the ROP will have more than 4 billion Euros extra for investments at the sub-national level. Simple math shows that even this extra allocation will be more than enough to cover the pre-allocation for county residences and county councils. Thus, if we consider that for the ROP 2007-2013 Growth Poles received around 90 million Euros each, and if we consider that Urban Development Poles received around 15-20 million Euros each, we can have a starting basis for estimating potential allocations for Growth Poles and County Residences. We can go with the upper limits of the above figures and consider that Growth Poles will continue to be allocated around 90 million each, and that each county residence would get around 20 million. This means 630 million Euros for Growth Poles and 720 million Euros for the other county residences, for a total of 1,350 million Euros for all county residences. If we assume that each county council will receive on average of 30 million Euros (for around 2-4 major projects), then we have a total pre-allocation of 1,230 million Euros for 168 county councils. In total, this means that around 2,500 million Euros will be allocated for Type 2 projects – around 30% of the total estimated allocation for the ROP 2014-2020. 357. To take the argument a bit further, we looked at the Prudent Fiscal Space for Capital Expenditures for county residences and county councils. If we take these numbers as the upper threshold for prudent capital spending over 2014-2022, then county residences as a whole should not spend more than 4-5 billion Euros over 2014-2022 on capital expenditures. If we consider that between 2010-2012 they spent around 30% of their capital expenditures on EU investments, than we can assume that they will not spend more than around 1.5 billion on EU investments (close to what we have estimated in the paragraph above). If we look at the Prudent Fiscal Space for Capital Expenditures over 2014-2022 for county councils, than we will see that it will be prudent for them to not spend more than 2-2.5 billion Euros on capital expenditures in this time period. If we consider that over 2010-2012 around 60% of capital expenditures made by counties represented EU investments, then we can assume that counties will not be able to spend more than around 1.6 billion Euros on EU investments over 2014-2020. 358. In essence, county residences and county councils will likely not be able to spend more than around 3 billion together on EU investments, and not more than around 7.5 billion on capital expenditures in general. They simply do not have the prudent fiscal space for more. As such, it pays for the MA to think about as many beneficiaries as possible (including beyond the public sector), as there is likely to be limited fiscal space among these key sub-national entities. Thus, the ROP with pre-allocations for Type-2 projects may not create a problem of too little money, but likely one of too much money. Risk of lack of fiscal space for further capital expenditures 359. A look at the financial data in Annex 20 indicates that many sub-national entities, particularly county councils, have a relatively tight fiscal space. More concretely, several counties have spent more on capital investments than they normally should have, given their current and expected revenues. As such, they should be very careful assuming even higher expenditures in the following years. Also, it is interesting to see that the evolution in real terms (when you adjust for inflation) of sub-national revenues has been negative, almost across the board. This has significant implications for how prepared sub-national entities will be to continue to absorb EU funds in the future. To give a better picture of which counties are facing fiscal constraints, we will go through a step-by-step analysis. 360. First, we introduce two financial indicators that were developed by members of the team – Graham Glenday and Victor Giosan. These financial indicators are dubbed, for simplicity and easy reference, the Glenday Indicator and the Giosan Indicator. They provide thresholds of fiscal prudence, which should be used by beneficiaries, the MA, and the RDAs, when EU funds are contracted. As we will discuss later, these indicators could also be used to assess financial sustainability when evaluating ROP projects. 361. The GLENDAY INDICATOR calculates the ratio of Capital Expenditures on Total Non- Earmarked Revenues. For this purpose, using data from the MRDPA, non-earmarked revenues include Own Revenues, Shared Personal Income Tax, Non-earmarked Grants, and Other Revenues. The Glenday Indicator was explained in more detail earlier. It basically measures the 169 extent to which public authorities will be able to pay in the future the operation and maintenance costs for the capital expenditures they are making today. 362. The GIOSAN INDICATOR calculates the ratio of Non-Earmarked Revenues for Current Expenses on Total Revenues. The indicator tries to assess how much budgetary flexibility exists at sub-national government level to cover the co-financing and the cash flows of future investments. Non-earmarked revenues for current expenses include Own Revenues, Shared Personal Income Tax, Non-earmarked grants, Other Revenues, Reimbursements from EU funds, minus Capital Expenses. The GIOSAN INDICATOR assumes that a public authority is fiscally prudent when the Ratio of Non-earmarked Revenues for Current Expenses on Total Revenues is larger than 30% - basically, when a large enough share of the revenues a public authority makes go to cover recurrent costs. 363. An outlook on the Glenday and the Giosan indicators by year, region, and county residence/county council is presented in the table below. Particularly worrisome is that many sub-national entities seem to face fiscal constraints, particularly in the poorer regions of Romania. Table 24. An Outlook of the Glenday and the Giosan Indicators, by Region For the GLENDAY INDICATOR, a YELLOW marker is used when indicators is between 30%-40%, an ORANGE marker when it is between 40%-50%, and a RED marker when it is above 50% For the GIOSAN INDICATOR, a YELLOW marker is used when indicators is between 20%-30%, an ORANGE marker when it is between 15%-20%, and a RED marker when it is below 15% Ratio non-earmarked revenues for Ratio of capital expenses to non- current expenses out of total earmarked revenues revenues (GLENDAY INDICATOR) (GIOSAN INDICATOR) Average Average 2010 2011 2012 2010-2012 2010 2011 2012 2010-2012 North-East Region Suceava County 36.2% 52.2% 34.3% 40.9% 26.9% 17.6% 17.9% 20.8% Suceava City Hall 24.6% 30.8% 49.2% 34.9% 46.5% 41.5% 18.9% 35.6% - Botosani County 40.2% 121.5% 121.5% 94.4% 19.2% -9.3% 16.1% -2.1% Botosani City Hall 22.5% 53.1% 82.4% 52.7% 38.1% 27.4% 8.4% 24.6% Neamt County 46.3% 71.2% 67.3% 61.6% 22.1% 12.8% 11.0% 15.3% Piatra Neamt City Hall 55.8% 93.2% 65.3% 71.4% 27.0% 4.1% 20.7% 17.3% Iasi County 23.9% 54.4% 54.8% 44.4% 31.9% 22.5% 16.3% 23.6% Iasi City Hall 18.7% 23.1% 33.6% 25.1% 52.2% 50.9% 42.7% 48.6% Bacau County 34.3% 63.6% 67.9% 55.3% 29.9% 18.1% 13.2% 20.4% Bacau City Hall 21.7% 43.1% 51.9% 38.9% 47.8% 39.7% 31.8% 39.8% Vaslui County 76.5% 44.9% 48.5% 56.6% 8.2% 19.6% 14.6% 14.1% Vaslui City Hall 28.4% 50.3% 67.0% 48.6% 38.8% 30.0% 17.5% 28.8% Total County Councils 40.0% 66.4% 68.0% 58.1% 24.2% 14.7% 11.8% 16.9% Total County Residences 26.0% 43.7% 51.3% 40.3% 44.7% 36.1% 30.7% 37.2% South-East Region Vrancea County 51.3% 125.5% 85.8% 87.5% 21.5% -6.4% 2.7% 5.9% Focsani City Hall 17.3% 24.3% 47.7% 29.8% 44.7% 48.6% 31.0% 41.5% Galati County 45.8% 63.2% 38.5% 49.2% 25.7% 17.5% 25.7% 23.0% Galati City Hall 27.9% 21.9% 41.7% 30.5% 46.7% 53.7% 27.2% 42.5% Buzau County 56.5% 73.2% 50.4% 60.0% 16.7% 8.7% 15.5% 13.6% 170 Buzau City Hall 10.6% 22.1% 8.6% 13.8% 49.8% 53.2% 60.3% 54.4% Braila County 35.5% 81.7% 46.8% 54.7% 29.7% 8.3% 21.0% 19.7% Braila City Hall 20.5% 49.0% 66.7% 45.4% 44.7% 33.1% 2.5% 26.8% - Tulcea County 95.2% 130.8% 45.0% 90.4% 2.3% 13.1% 19.8% 3.0% Tulcea City Hall 24.7% 33.0% 17.4% 25.0% 46.6% 41.3% 52.2% 46.7% Constanta County 63.3% 102.3% 45.7% 70.4% 17.2% -1.0% 16.8% 11.0% Constanta City Hall 2.5% 29.2% 32.5% 21.4% 69.0% 49.2% 49.9% 56.0% Total County Councils 59.6% 95.1% 48.6% 67.8% 18.3% 1.9% 22.7% 14.3% Total County Residences 15.2% 29.3% 38.7% 27.8% 53.3% 47.6% 42.5% 47.8% South Region Prahova County 34.1% 30.0% 46.1% 36.7% 26.5% 26.8% 20.7% 24.7% Ploiesti City Hall 25.2% 16.1% 15.4% 18.9% 49.8% 59.6% 59.1% 56.2% Ialomita County 49.7% 55.5% 29.0% 44.7% 20.3% 17.9% 28.3% 22.2% Slobozia City Hall 42.7% 41.2% 45.9% 43.3% 36.0% 39.7% 33.5% 36.4% Calarasi County 21.0% 94.4% 109.1% 74.8% 35.0% 2.3% -4.5% 10.9% Calarasi City Hall 22.8% 29.6% 29.6% 27.3% 45.4% 45.5% 42.4% 44.4% Giurgiu County 15.4% 93.1% 85.9% 64.8% 44.6% 3.9% 6.2% 18.2% Giurgiu City Hall 22.5% 37.1% 18.5% 26.0% 42.2% 39.2% 47.2% 42.9% Dambovita County 37.2% 92.1% 55.9% 61.7% 27.8% 3.0% 18.1% 16.3% Targoviste City Hall 22.6% 26.7% 31.9% 27.1% 47.1% 48.9% 42.8% 46.3% Teleorman County 80.8% 75.2% 69.4% 75.1% 10.9% 12.1% 10.7% 11.2% Alexandria City Hall 15.4% 25.2% 54.4% 31.6% 47.6% 48.3% 28.4% 41.5% Arges County 27.5% 52.8% 26.0% 35.4% 30.9% 20.4% 33.5% 28.3% Pitesti City Hall 21.3% 21.3% 50.2% 30.9% 49.6% 49.6% 19.2% 39.5% Total County Councils 39.1% 66.6% 57.0% 54.2% 27.5% 14.3% 17.9% 19.9% Total County Residences 24.1% 23.6% 32.6% 26.8% 47.4% 50.9% 44.7% 47.7% South-West Region Valcea County 8.9% 49.0% 48.3% 35.4% 22.9% 15.6% 16.6% 18.4% Ramnicu Valcea City Hall 22.8% 26.8% 40.9% 30.2% 47.7% 41.2% 28.5% 39.1% Olt County 27.2% 38.1% 73.2% 46.2% 30.9% 25.4% 11.6% 22.6% Slatina City Hall 23.8% 36.9% 47.5% 36.1% 44.2% 42.3% 33.6% 40.0% Gorj County 28.1% 37.9% 34.3% 33.4% 32.4% 26.0% 27.2% 28.5% Targu Jiu City Hall 14.6% 41.3% 30.8% 28.9% 50.1% 36.7% 38.3% 41.7% Dolj County 59.5% 39.3% 45.3% 48.1% 23.0% 29.3% 27.6% 26.7% Craiova City Hall 37.0% 21.6% 49.0% 35.8% 39.6% 52.1% 21.9% 37.9% Mehedinti County 63.5% 121.4% 111.3% 98.7% 17.5% -7.7% -3.7% 2.0% Turnu Severin City Hall 20.4% 60.4% 57.7% 46.2% 45.6% 23.5% 14.3% 27.8% Total County Councils 44.6% 55.0% 58.2% 52.6% 24.6% 18.0% 17.2% 19.9% Total County Residences 27.3% 32.7% 47.2% 35.7% 44.0% 42.4% 35.2% 40.6% West Region Arad County 16.0% 74.5% 45.1% 45.2% 50.1% 14.0% 29.2% 31.1% Arad City Hall 28.7% 47.3% 45.7% 40.5% 44.1% 36.0% 41.0% 40.4% Timis County 8.5% 54.3% 39.0% 33.9% 44.5% 22.9% 29.4% 32.3% Timisoara City Hall 9.8% 16.8% 27.8% 18.1% 61.2% 63.7% 50.3% 58.4% Caras Severin County 48.5% 62.7% 66.3% 59.2% 17.3% 13.4% 12.3% 14.3% Resita City Hall 10.8% 29.6% 11.0% 17.1% 53.5% 42.2% 57.3% 51.0% Hunedoara County 32.7% 66.6% 51.1% 50.1% 33.5% 13.1% 18.3% 21.7% Deva City Hall 37.9% 31.3% 42.8% 37.4% 42.3% 49.5% 27.8% 39.9% Total County Councils 21.1% 62.7% 46.6% 43.5% 38.7% 17.4% 24.6% 26.9% Total County Residences 19.1% 28.4% 32.8% 26.8% 52.7% 51.6% 50.6% 51.6% 171 North-West Region Bihor County 10.4% 40.4% 38.9% 29.9% 33.2% 22.8% 23.4% 26.5% Oradea City Hall 13.6% 37.1% 28.2% 26.3% 49.0% 36.4% 44.1% 43.2% Satu Mare County 58.3% 49.4% 13.9% 40.5% 15.2% 18.8% 32.5% 22.2% Satu Mare City Hall 8.4% 34.2% 14.1% 18.9% 53.3% 44.3% 49.6% 49.1% Maramures County 24.9% 46.1% 21.2% 30.7% 38.4% 20.5% 28.8% 29.2% Baia Mare City Hall 24.2% 24.1% 33.4% 27.3% 44.1% 48.1% 27.6% 39.9% Cluj County 10.0% 76.7% 28.6% 38.5% 53.8% 13.5% 37.9% 35.1% Cluj-Napoca City Hall 11.0% 19.8% 32.3% 21.0% 62.0% 61.5% 51.2% 58.2% - Bistrita Nasaud County 41.5% 173.4% 88.0% 101.0% 28.4% 28.9% -6.6% -2.4% Bistrita City Hall 23.3% 45.4% 38.8% 35.8% 50.8% 35.8% 39.0% 41.9% - Salaj County 10.0% 143.5% 115.1% 89.5% 40.0% 19.3% -5.6% 5.0% Zalau City Hall 8.4% 11.6% 27.3% 15.8% 53.3% 54.1% 41.7% 49.7% Total County Councils 21.4% 86.2% 46.7% 51.5% 37.8% 6.1% 23.6% 22.5% Total County Residences 13.7% 27.2% 30.1% 23.7% 53.9% 49.1% 47.6% 50.2% Centre Region Alba County 46.3% 21.9% 26.6% 31.6% 19.4% 27.8% 24.9% 24.0% Alba Iulia City Hall 30.4% 68.6% 67.9% 55.6% 47.8% 22.6% 19.8% 30.1% Mures County 1.8% 49.5% 56.0% 35.8% 37.3% 20.7% 19.2% 25.7% Targu Mures City Hall 18.0% 28.5% 25.6% 24.0% 51.9% 48.6% 45.8% 48.8% Harghita County 70.1% 97.0% 50.7% 72.6% 13.7% 1.4% 21.2% 12.1% Miercurea Ciuc City Hall 26.5% 63.2% 83.4% 57.7% 42.0% 21.0% 9.5% 24.2% Covasna County 5.1% 108.0% 47.8% 53.6% 45.5% -3.8% 26.8% 22.8% Sfantu Gheorghe City Hall 51.5% 51.7% 41.2% 48.1% 25.2% 27.9% 35.5% 29.5% Brasov County 22.1% 32.8% 8.4% 21.1% 40.0% 38.3% 49.4% 42.6% Brasov City Hall 21.1% 54.2% 40.7% 38.7% 51.3% 29.1% 43.6% 41.4% Sibiu County 12.2% 65.0% 52.4% 43.2% 40.9% 15.4% 23.0% 26.4% Sibiu City Hall 28.8% 33.5% 22.5% 28.3% 51.1% 53.9% 58.3% 54.4% Total County Councils 24.7% 55.7% 39.8% 40.1% 33.1% 20.0% 27.9% 27.0% Total County Residences 25.2% 46.1% 38.4% 36.6% 48.5% 36.7% 43.4% 42.9% Bucharest-Ilfov Region Bucharest 18.3% 25.5% 29.2% 24.4% 68.9% 64.0% 60.3% 64.4% Ilfov County 48.1% 12.4% 8.4% 23.0% 37.5% 57.4% 59.1% 51.3% Total Region 19.2% 25.2% 28.8% 24.4% 67.9% 63.8% 60.3% 64.0% Source: Calculations based on Ministry of Regional Development and Public Administration data – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html 364. The two figures below indicate a clear deterioration of budgetary flexibility over 2010 – 2012, especially at the county level. There is an obvious move from the bottom-right of the graph area to the top-left one, which means a simultaneous increase of the capital expenses ratio, with reduced fiscal space for covering the current expenses, expressed by the absolute and relative decrease of the Giosan Indicators. However, an important difference exists between the situation of county residences and the situation of county councils, with the former in a much better fiscal position than the latter. Some county councils have entered in the negative (red) area when it comes to the Giosan Indicators. This phenomenon is confirmed also by the Glenday Indicator. In fact, there is a negative correlation between the Glenday and the Giosan indicators. 172 Figure 40. An outlook on how County Residences perform over time on the Glenday-Giosan indicators Source: Calculations based on Ministry of Regional Development and Public Administration data – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html Figure 41. An outlook of how County Councils perform over time on the Glenday-Giosan indicators Source: Calculations based on Ministry of Regional Development and Public Administration data – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html 365. Many county councils seem to be facing a tight fiscal space, with less and less room for further capital investments. This should be something the MA should keep in mind when designing the ROP 2014-2020. Should it decide to implement a Type 2 approach with pre- allocations for county councils, then the analysis above provides some good estimates on how much individual counties can handle. In the same vein, going for the pre-allocation route allows the MA and the RDAs to control the process much better, deciding from the get-go how much funding individual counties can receive. If pre-allocation for county councils will not be considered an option, then it is important to carefully vet financial sustainability in the project evaluation and selection phase. 173 366. It is important to note however, that the analysis above only looked at data for three consecutive years: 2010, 20111, and 2012. To fully analyze the full budgetary impact of the ROP 2007 – 2013 on sub-national government budgets, it would be necessary to complete an overall assessment for the whole period. Given actual data availability, this may not be possible at this point in time, and one would also have to consider the effect of the global crisis on local and county budgets, as the crisis made its presence felt in Romania particularly in 2009 and 2010. 367. Another point to mention here refers to the nature of capital expenditures from EU funds. For the most part, these expenditures focused on the rehabilitation of existing infrastructure rather on the development of new one. As such, these investments may have helped reduce future recurrent costs because O&M costs are lower for upgraded infrastructure than they are for old infrastructure. Most of the ROP expenditures fell under the category of infrastructure rehabilitation, but there were other Operational Programmes that focused more on the development of new infrastructure – e.g., OP Transport and OP Environment. Particularly expenditures made through the OP Environment may be a burden for county councils, because they focused to a large extent on the development of new solid waste management, water, and wastewater infrastructure. 368. It is also very important to assess the impact of the last changes to the local government fiscal framework effective 2014. These changes have a direct impact on the revenues generated by sub-national entities, and by consequence on the Glenday and Giosan indicators. Some of the more dramatic changes include the extension of the fiscal base for the property tax levied on companies and the elimination of the limit for sub-national governments’ discretion to increase/decrease all forms of property tax. The first change will have an inherent and significant effect on own revenues of first-tier local governments, and the second a potential but limited one, only if local authorities decide to increase the property tax ratio very significantly. Both changes will not have any impact on county councils’ budgets. 369. In conclusion, we can say that most sub-national governments face a tightening fiscal space for funding the operational and maintenance expenses of actual, large investments, and to cover the co-financing and the cash flow for new investments. The situation got worse in past years when many of the investments that were contracted in earlier years have come to fruition. Nonetheless, the situation in not homogeneous among different types of sub-national governments. County councils seem to particularly be in a tight spot, many of them being at the limit in terms of their overall capacity to support new investments. County residences are in a better position, with more space to continue investing, as long as they do so in a moderate manner. 370. Going forward, if this pattern of spending will continue, with a lot of capital expenses accompanied by a narrowing budgetary capacity to finance operation and maintenance costs, it is possible to witness a number of perverse and unwanted side-effects. Some of these could include: a) Open arrears under the form of non-paid invoices for investments, services, or goods at the sub-national level; b) Hidden arrears under the form of provided works, services or goods, but without any invoices issued; 174 c) The lack of maintenance and abandon of new infrastructure, assets, or services developed by the investment projects. There are already some instances of PHARE and SAPARD investments abandoned after the end of monitoring period. 371. There are of course solutions for this tightening fiscal space, both on the revenue side and on the expenditure side. First of all, it is important to allow the fiscal base at sub-national level to expand, both with regard to Own Revenues and with regard to the Shared Personal Income Tax. Second, it is important to increase non-earmarked transfers from the central level to enable a larger fiscal space for sub-national entities. On the capital expenditure side, some helpful measures could include: a) Improve the selection of EU-funded projects (as well as projects funded from local/county budgets), focusing on those that promise to have strong economic impact, and which in turn could help the fiscal base grow. Estonia, for example, uses a criterion in the selection of EU projects to assess the degree to which a particular project would have a beneficial effect on the fiscal base. b) Prioritize EU investments over investments made from the local/county budget to ensure that, whenever possible, capital expenditures use EU grants. This will significantly improve the fiscal space for sub-national authorities. Basically, a higher priority should be given to EU-funded investments and to the financing of operational and maintenance expenses generated by these investments, and only when sufficient fiscal space has been created for new investments from the local budget, should these be undertaken. This can be done as a best practice exercise or using some rules attached to the allocation of non-earmarked grants. 372. Last but not least, it should be noted again that a large share of EU investments is used for rehabilitation of existing infrastructure, not for the development of new infrastructure. As such, many of these investments incur, in essence, lower maintenance costs. This means that many EU-funded investments will in fact help reduce recurrent costs rather than increase them. 175 XI. Monitoring and Evaluation 373. Upon reviewing the different scenarios for project selection models, the current report turns to the question of ensuring adequate monitoring and evaluation (M&E), at multiple levels. For one, M&E has important implications for how an evaluation system is designed in the first place. There are also a range of key criteria that should define how indicators are chosen, both in the application preparation phase and in subsequent stages of implementation and post-implementation. Second, it is useful to review the highlights of the ROP’s program and project-level M&E, drawing from ongoing World Bank work on the MA-IB- beneficiary relationships. The last section under this heading turns to the need for devising a dedicated M&E component specifically for assessing the performance of project selection models and proposing adjustments, as needed and when possible. 374. As an introduction, it is useful to set the current report into the right overall context of a strong turn toward more rigorous monitoring and evaluation. Interestingly, this appears to be a global trend in the international aid community, with donors increasingly preoccupied with selecting interventions that deliver the most “bang for the buck” or, in technical jargon, maximize value for money. This may be a consequence of less flexible budgets in the aftermath of the recent economic crisis, but also a reflection of a general perception that the aid community should know, after decades of mixed results, which investments actually deliver the most positive, tangible impact relative to the needs on the ground. 375. The EU makes no exception to this global discourse that has brought new force behind the idea of responsible, efficient spending. In fact, after years of placing only a marginal emphasis on tracking impact, the Commission has significantly bolstered the case for rigorous, evidence-based monitoring and evaluation, particularly in light of preparations for the 2014- 2020 budgeting period (Annex 25 includes a more detailed discussion on this issue). Romania’s ROP, along with the other Operational Programmes that pump EU funds into the national economy (although the important nuance of negative net gains post-accession should be kept in mind, per the earlier discussion), have to adapt to these key trends and ensure functional M&E mechanisms. The aim should not be to simply persuade the EC that these programs are serious about delivering impact, but they should genuinely seek to understand which types of interventions generate more positive change per EUR or RON spent. The Performance of the ROP 2007-2013 376. As all Operational Programmes, the ROP had to put a Monitoring and Evaluation system in place. This system is meant to assess at fixed intervals (usually mid-way through and after the completion of the program) the degree to which the ROP has actually achieved its objectives. The system is primarily designed for the staff of the ROP, but it is made public and others can assess its effectiveness too. 377. The Romanian Institute for Public Policy has done just such an exercise, and has assessed the performance of the different Operational Programmes in Romania in October, 2013. At a rate of absorption of 39% in August 2013 and at a rate of contracting of 110%, the ROP managed to realize 23% of its stated indicators. While this is not a stellar performance, it was higher than that of other Operational Programmes. The only OPs that realized a higher share of their performance indicators were the OP for the Development of Administrative Capacity (with 71% of project objectives realized) and the OP for the Development of Human Resources (26%). 177 Financial Signed allocation Results contracts as Basic Proposed 2007 - 2013 obtained as Key area of intervention of 31 August Indicator values targets (EU of 31 August 2013 (mill. 2005 2015 45 contribution 2013 euro) euro) DMI 1.1: Integrated urban 1,117,806,529 Accepted urban development integrated plans (number) 30 0 development plans Projects ensuring improvement of the urban and urban service infrastructure, including urban transportation 60 53 (number) Projects promoting sustainable development of the 15 1 business environment (number) Projects ensuring rehabilitation of the social infrastructure, including social housing and improvement of social services 25 38 (number) Inhabitants who benefit from the implementation of projects from urban development integrated plans 400,000 4,693,591 (number) Companies established within the regional and local growth 400 7 poles (number) 4,108.1 Jobs creates/maintained (number) 1500 9 DMI 1.2: Support of investments in Including to DMI Apartments rehabilitated for an improved energetic 1.1 efficiency (number) 46,920 0 the energetic efficiency of blocks of flats Total energy savings for rehabilitated apartments (GWh/year) 256 0 DMI 2.1: Rehabilitation and 758.355.021 Length of rehabilitated/ modernized county roads (outside 36,009.7 877 1074 TEN – T) (km) modernization of the county road Length of rehabilitated/ modernized urban streets (outside network, and of urban streets – TEN – T) (km) 25,696 411 115 including construction/ Length of rehabilitated/ built beltways (outside TEN – T) 219 22 rehabilitation of beltways (km) Increase in the traffic of passengers46 (%) 10 0 45 Values calculated for projects completed financially 46 To be measured cumulated at the priority axis level: county roads + beltways 178 Financial Signed allocation Results contracts as Basic Proposed 2007 - 2013 obtained as Key area of intervention of 31 August Indicator values targets (EU of 31 August 2013 (mill. 2005 2015 45 contribution 2013 euro) euro) Increase in the traffic of carried or transited goods (%) 10 0 DMI 3.1: 174,199,525 Rehabilitated/ modernized/equipped medical units (number) 53 25 Rehabilitation/modernization / equipment of the health care Persons who benefit from service infrastructure rehabilitated/modernized/equipped healthcare 30,000 274,625 infrastructure (number/day) DMI 3.2: 84,580,694 Rehabilitated/ modernized/extended/ equipped social centers (number) 270 83 Rehabilitation/modernization / development and equipment of the Persons who benefit from rehabilitated/ modernized/ social service infrastructure extended/ equipped social service infrastructure(number) 10.000 21,262 DMI 3.3: Improvement of 84,580,694 Mobile units equipped for emergency interventions (number) 510 499 equipment of operational bases for interventions in emergency Average response time of mobile units in rural areas Up to 30’ situations (communes) – emergency situation infrastructure (minutes) – 45’ in Up to 12’ in 21 rural rural areas areas Average response time of mobile units in urban areas Up to 20’ Up to (cities/towns) – emergency situation infrastructure in urban 8’ in urban 12 (minutes) areas areas DMI 3.4: 242,191,411 Rehabilitated/ modernized/ equipped education units – pre-university education infrastructure (number) 47 130 111 Rehabilitation/modernization/ development and equipment of the Rehabilitated / modernized/extended/ equipped campuses - pre-university education infrastructure (number) 30 4 pre-university and university Rehabilitated/ modernized/ extended/equipped education infrastructure and of the continuous professional training centers(number) 35 0 continuous professional training Rehabilitated / modernized/extended/ equipped campuses infrastructure - pre-university education infrastructure (number) 15 2 47 This indicator also includes extension activities 179 Financial Signed allocation Results contracts as Basic Proposed 2007 - 2013 obtained as Key area of intervention of 31 August Indicator values targets (EU of 31 August 2013 (mill. 2005 2015 45 contribution 2013 euro) euro) Pupils who benefit from rehabilitated/ modernized/extended/ equipped pre-university education 40,000 40,383 infrastructure (number) Children belonging to unflavored groups of population who benefit from rehabilitated/ modernized/ extended / 5000 0 equipped education units- pre-university education infrastructure (number) Persons who benefit from rehabilitated/ modernized/ extended/ equipped continuous professional training 3000 0 infrastructure (number) Students who benefit from rehabilitated/ modernized/ 2000 450 extended/ university campuses (number) DMI 4.1: Sustainable development 246,551,000 Assixcted business support structures (number) 17 18 of structures supporting businesses of regional and local importance Occupation rate of business support structures (two years after the project completion) (%) 50 0 Jobs newly created in business support structures (number) 3000 1,504 DMI 4.2: Rehabilitation of polluted and 26,710,000 Polluted and unused industrial sites rehabilitated and unused industrial sites and their prepared for new economic activities (ha.) 75 0 preparation for new activities Jobs newly created in business support structures (number) 1000 0 DMI 4.3: Support for the development 229,089,325 Supported microundertakings (number) 1500 1,282 of micro undertakings Jobs newly created in supported micro undertakings 3000 5,780 (number) DMI 5.1: Sustainable restoration and 241.894.026 Projects in tourism (number) 100 16 valorization of the cultural inheritance 180 Financial Signed allocation Results contracts as Basic Proposed 2007 - 2013 obtained as Key area of intervention of 31 August Indicator values targets (EU of 31 August 2013 (mill. 2005 2015 45 contribution 2013 euro) euro) and creation/modernization of ancillary Jobs newly created / maintained (number) infrastructures 200 34 DMI 5.2: Creation, development, and 293,631,003 Tourism projects (number) 300 46 modernization of tourism infrastructure Supported SMEs (number) 350 41 for the valorization of natural resources and for an improved quality of tourism Touirists arrived in rehabilitated/ modernized/ equipped accommodation facilities (number) 400,000 136,319 services Overnight stays in the rehabilitated/ modernized/ equipped accommodation infrastructure (number) 800,000 0 Jobs newly created / maintained (number) 800 580 DMI 5.3: Promotion of the touristic 127,802,546 Touristic brand promotion campaigns (number) 0 10 0 potential and creation of the necessary infrastructure for the purpose of National Touristic Information and Promotion increasing Romania’s attractiveness as Centers(number) 0 100 2 a touristic destination Visitors in National Touristic Information and Promotion Centers (number) 0 2.5 mill. 42,759 Website visitors (number) 0 1.5 mill. 299,837 DMI 6.1: Support of implementation, 90,393,948 Studies, analyses, reports, strategies (number) 40 344 management and assessment of the Regional Operational Program Participants in training courses (OI/AM personnel, 2,000 4,397 beneficiaries and potential beneficiaries) (number) Training days per participant (number) 10,000 10,342 DMI 6.2: Support of activities of 8,236,040 Communication and promotion events (number) 900 1,778 advertising and information on ROP Public awareness level on ROP (%) 20 39 181 378. As the table above shows, the ROP 2007-2013 performance indicators are primarily output- based. As such, they provide only a facet of the extent to which the ROP has actually managed to encourage regional development. Some of the indicators have not been properly filled in. For example, the first indicator measures the number of accepted Integrated Urban Development Plans. The proposed target was 30 IDPS. The ROP’s M&E system indicates that 0 IDPs have been accepted, whereas in reality over 90 IDPs were accepted for financing. There are of course indicators where performance was actually weak (e.g., the number of rehabilitated polluted sites) and there are also indicators where the baseline has been surpassed (e.g., kilometers of rehabilitated road or number of pupils who benefit from rehabilitated schools). 379. Overall, it is clear that the ROP still has some way to go before it has a functional and efficient M&E system. There still is need for a more realistic development of baseline indicators; more efficient monitoring of performance indicators is also required; and it would be helpful to also introduce a system that will help assess the actual impact of ROP investments (more specifically, a system that would help determine if the ROP actually made a development impact). The next sections discuss some of these issues in more detail. Best-Practice Elements of a Functional M&E Framework 380. To begin with, a functional M&E system requires proper alignment with the project selection model of choice. In other words, the same principles that underlie the methods for selecting the projects to be financed should apply to how the same interventions are subsequently monitored and evaluated. A mismatch between the selection model and the M&E mechanisms would lead to suboptimal situations where a project receives a high score in the evaluation stage but performs poorly thereafter because the tracked indicators do not measure the relevant performance components. As such, the link between the two frameworks – project selection and M&E – should be strong and explicit. 381. Equally important, indicators should be easily accessible to all key stakeholders . This is ultimately the main aim of any M&E mechanism, namely to provide continuous updates on the performance of a particular program and the intervention it finances. A simple, but powerful metaphor compares the M&E system to a “dashboard.” On a plane, very few people can understand what every gauge and light mean, and even fewer know what to do in case something goes wrong. By contrast, on a car, dashboards are simple enough for everyone to be able to read how the vehicle’s performance is evolving and whether any special interventions by the driver may be needed. In addition, the indicators used – both in evaluation and later on for M&E activities – should follow the “SMART” set of criteria: specific, measurable, attainable, relevant, and time-oriented. Figure 42. An effective M&E system is like a good dashboard: simple and easy to read Source: http://www.biltronix.se/ Program and Project-Level M&E 382. The ROP’s monitoring mechanisms are based on specific articles and provisions of EC Regulation 1083/2006.48 Chapter II under Title VI (Management, Monitoring, and Controls) is entirely dedicated to monitoring requirements for structural funds, in broad terms, mandating the establishment of a Monitoring Committee governed by its own rules of procedure and tasked with ensuring the effectiveness and quality of the program’s implementation. For Romania’s ROP, as noted elsewhere, the MC-ROP is presided by the Minister of Regional Development and Public Administration, and its members come in equal proportions from among: presidents of RDCs; central ministries with other MAs; and socio-economic partners (labor unions, private sector associations, academia, etc.). Importantly, an EC representative participates in a consultative capacity to all MC-ROP meetings. The main task of the MC-ROP is to monitor and report on progress toward set targets for each priority axis. 383. MC-ROP meetings are highly valuable for fine-tuning the program. The MA has provided timely, complete, and very professional information on the program’s status for each axis, which has allowed the Committee to reallocate funding and correct certain emerging problems. For example, when it became clear that the money allocated for intervention area 4.2 – brownfield redevelopment – would not be spent (primarily due to property rights issues concerning old industrial platforms), MC-ROP promptly moved part of the financing to other areas, where necessary. Importantly, in line with EC 1083/2006, MC-ROP focuses a lot on tracking program-level targets, but has also dedicated important time to discussing specific implementation challenges (e.g., public procurement legislation, measure to accelerate absorption, etc.).49 RDA representatives routinely voice their concerns and describe actual obstacles faced by beneficiaries on the ground, albeit at a technical level that is appropriate for the format and time constraints of these meetings. Because more constant institutionalized mechanisms for 48 Council Regulation (EC) No. 1083/2006 of 11 July 2006. 49 See minutes of specific MC-ROP meetings. For instance, minutes of the May 24-25, 2012, MC-ROP meeting available at http://www.fonduri-ue.ro/res/filepicker_users/cd25a597fd- 62/Documente_Suport/Rapoarte/1_Rapoarte_POR/27.08.2012/3_PV%2024-25%20mai%202012.pdf 183 feedback and program adaptation are not always present, MC-ROP has served the ROP system very well by bringing key stakeholders at the same table, twice every year. This has also become the key instrument for ensuring a direct communication channel from regions to the EC (and vice-versa) and should be maintained going forward, regardless of the program’s future structure in the context of the regionalization proposal. 384. As for the ROP’s evaluations, these are highly centralized and mostly handled by MA-ROP. Chapter I under Title IV (Effectiveness) includes article 48, which clearly mandates Member States to “provide the resources necessary to carry out evaluations, organize the production and gathering of necessary data, and use the various types of information provided by the monitoring system.”50 Several types of evaluations include ex-ante (performed by an external consultant for the ROP), continuous, and ex-post (performed by the Commission). Per the ROP 2007-2013 (June 2007 version), an Evaluation Unit functions within MA-ROP from the beginning of the program and is responsible for drafting the Evaluation Plan and for leading the continuous evaluations (permanent, intermediary, and ad-hoc). There is also a Central Evaluation Unit (CEU) under the Authority for the Coordination of Structural Instruments (ACSI). The CEU ensures the development of administrative capacity for the AM-ROP’s evaluation unit, as well as specific horizontal evaluations that cover several or all OPs in Romania. Finally, MC-ROP approves the program’s evaluation plan for each year and reports on its progress and key findings. 385. All in all, the ROP has derived significant value from having institutionalized M&E mechanisms from the very beginning. There are many investments and foreign aid programs that still get launched without proper consideration for the importance of M&E in understanding, at all times, what is going well, where adjustments are needed, and what impact is generated. Going forward, building on the accumulated experience in the system, the ROP has the opportunity to further strengthen M&E mechanisms along with moving from a predominant focus on output-type indicators (e.g., absorption targets) to a more refined outcome-based indicators that makes refined judgments on the level of impact generated by ROP-financed projects on the ground. 386. At the project level, the ex-post monitoring for project durability entails two annual components: a beneficiary report and a field visit. The ROP Procedures Manual on Monitoring provides complete details around both processes, which are meant to verify that projects have not suffered major changes and indicators defined during the application stage are still fulfilled. RDA/IB staff is responsible for annual monitoring visits, along with representatives of the Directorate for Coordinating Field Activities (DCFA) within the MA, who follow the same procedure as the IB. The templates for beneficiary reports and for the visit reports by monitoring officers are brief but clear, allowing beneficiaries to explain any special circumstances, if needed, and should be maintained going forward. The final decision on whether to impose any penalties belongs to the MA, based on the reports received. 387. Improvements to the status quo are needed particularly when it comes to the types of acceptable project-level indicators and the methodology and standards used in assessing them after the completion of an ROP investment. There is a lack of clarity regarding how monitoring officers are evaluating whether projects have delivered on their indicators and how potential financial corrections may be decided for beneficiaries who fall short of the targets. The first problem is that there were no adequate controls on the set of indicators defined by beneficiaries. The applicant guides and the 50 Council Regulation (EC) No. 1083/2006 of 11 July 2006. 184 financing request template instruct applicants to carefully select indicators correlated with program or axis-level ones. In practice, however, overzealous or inexperienced beneficiaries (and/or their consultants) often included a wide set of targets, believing that this increased their chances to receive financing. Some of the indicators chosen were either unrealistic or impossible to measure and attribute, particularly with respect to the indirect impact of particular investments. 388. For their part, evaluators did not call attention to this problem and a number of projects ended up receiving financing and committing their beneficiaries to achieving the indicators they had chosen. Further down the line, particularly in the post-implementation phase, these beneficiaries have a hard time (a) reaching particular targets; and/or (b) demonstrating that those targets were reached. This is not always the direct fault of those who put together the application. In a few instances, there are objective reasons for failing to get to a particular number, as in the case of public-sector beneficiaries who promised to create new jobs but cannot fill them because of hiring restrictions imposed through GEO 34/2009. Such cases should have a clear, objective, and fair resolution. 389. On the choice of indicators, applicants should be encouraged to select from a pre-defined list aligned with program- and axis-level targets, which would be easy to implement particularly through an e-application form. An electronic application system would not allow users to submit a financing proposal with indicators beyond the pre-defined set. As an alternative, applicants could be free to choose their own targets, but evaluators should be allowed and instructed to correct these based on the principles enunciated above. These solutions would avoid instances of beneficiaries over-promising on the impact of their projects and would also save time when preparing the monitoring reports. Currently, beneficiaries have trouble proving that they are reaching set targets and either need to invest additional funds into this, or hope that they do not get penalized. 390. There is also need for a detailed methodology on how project-level indicators are monitored and evaluated throughout the implementation and post-implementation phases. For instance, beneficiaries do not know what point in time counts for which indicator; if they promised to hire, say, five additional people, it is unclear whether they are required to keep these positions filled for the entire monitoring period or under what conditions there could be exceptions to this rule. While generally simpler is better when it comes to regulation, in this case there need to be more details provided around what is evaluated and how, along with potential applicable sanctions in those cases that fail to reach set objectives. Whatever is decided, it should be fully aligned with EU-level and national legislation to make sure that if any corrections are imposed they are upheld by courts in potential legal actions. For projects that fail to reach their targets, corrections should be applied only in relation to those program and axis-related indicators and remain proportional to the calculated gaps between target and actual values. An M&E Dashboard for ROP 2014-2020 391. Despite some of the shortcomings mentioned earlier, program and project-level M&E is generally functional and appreciated as valuable by parties involved, but the same is not true of the monitoring and evaluation of project selection models in particular. It is beyond doubt that selection models play a most critical role in the success of any EU-funded program, including the ROP, because they fundamentally shape which interventions receive financing, through what process, and to what effect. This is the reason behind the proposal below – i.e., a simple dashboard for monitoring and evaluating the ROP’s project selection models based on the four key objectives described earlier: absorption – is the model delivering a satisfactory absorption rate?); impact – are the selected 185 interventions generating positive impact?; legitimacy – is the model (still) credible among applicants and beneficiaries?; and capacity – are all key components of the system (MA, IB, beneficiaries, and evaluators) able to perform well and deliver on the tasks required by the selection model? Figure 43. A possible dashboard for the ROP's M&E system ROP PROJECT SELECTION MODEL DASHBOARD • Is the ROP’s project selection model performing well on all four critical dimensions? If not, where are adjustments possible/ needed? ABSORPTION PERSPECTIVE IMPACT PERSPECTIVE Are absorption targets on track? If not, is the selection model to blame? Are the projects selected impactful? Do they reach intended targets? • Are ROP projects selected based on a quick and efficient process, • Are performance indicators properly chosen and evaluated? Are with simple and straightforward procedures? poor indicators screened out during the evaluation phase? KEY TESTS KEY TESTS • Are there any delays in evaluation, selection, or contracting? • Are there clear, comprehensive, and easily accessible data on project and program-level indicators? • Does the selection model itself inadvertently limit the number of projects coming through the pipeline? • Are projects delivering impact and reaching their intended milestones and indicators? • Are there undetected problems in evaluation that become evident during the implementation phase, delaying the project’s completion? LEGITIMACY PERSPECTIVE CAPACITY PERSPECTIVE Is the selection model credible among applicants and beneficiaries? Is the system (still) capable to implement the selection model? • Do applicants and beneficiaries believe in the fairness of the • Are there any delays along the chain for which MA/IB staff or system? independent evaluators are responsible? KEY TESTS KEY TESTS • Are they satisfied with its transparency and do they have full • Are there any capacity issues during peak times of visibility into the evaluation, selection, and contracting process, receiving/processing/evaluating applications? including vis-à-vis accountable parties in the MA/IB? • Are there any further simplifications to the selection models • Are public perceptions among the target group positive relative chosen that would free up capacity at the MA or IB level? to the legitimacy/credibility of the selection model? • Are there any capacity issues, caused by the chosen selection model, at the level of applicants/beneficiaries? 392. Of course, to provide answers to such questions there is a need to collect, easily access, process, and assess an extensive database with information related to project and program-level performance. If SMIS is not the answer to this need, the MA-ROP could consider developing its own internal system for tracking essential indicators such as: evaluation scores and the “quality” of project implementation; rejected projects, including a typology of reasons (particularly to fine-tune the proposed pre-qualification filter and “catch” ideas that are unlikely to succeed before they enter the pipeline as full applications); evaluation scores and success in reaching set targets; etc. Better Performance Management for the ROP 393. As indicated throughout the report, performance management, or the concept of successful operational delivery for the Regional Operating Programme (ROP), must focus on balancing the dual objectives of absorption and impact. While a high absorption rate is a characteristic of a strong programme, absorption rates by themselves do not ensure success for the ROP. Absorption is necessary, but the ultimate objective of the ROP is to create high-impact development. Furthermore, successfully measuring programme impact is not something that can be left only to ex-post evaluations, which may be useful for future planning, but will not improve the current performance of the ROP 186 programme. With these objectives in mind, successful performance management for the Regional Operating Programme will require three core elements: 1. A clear understanding of the broader European Commission performance framework that the ROP operates under; 2. A strategic decomposition of the top-level goals of the ROP, building toward the development of the right objectives and indicators for success; and 3. A performance management structure for ongoing ROP operations across regions, with a clear understanding of sub-level objectives and indicators by region and a plan to engage Regional Development Agencies (RDAs) to achieve these targets. 394. This section addresses these three elements and then goes a bit further with a final section suggesting long-term improvements to the operational structure of the ROP that builds on these elements and could further improve ROP operations in the future. The Performance Framework for the Regional Operating Programme 395. At the highest level, the performance review of ROP activities falls under the performance framework for European Structural and Investment Funds (ESI Funds).51 The performance framework, which consists of selected financial, output, and result indicators as well as key implementation steps for each priority, is intended to ensure that programmes such as the ROP deliver what is under their control. This means that priorities are implemented as planned and the programme is kept on course to achieve its objectives. In particular, a successful programme operating under this framework should anticipate any expected shortfalls to overall programme performance and take mitigation actions as soon as possible. Milestones and Targets for the ROP Performance Framework 396. The performance framework includes two key phases through 2025, with set deadlines at which top-line programme performance will be assessed: 1) Milestones and intermediate targets achieved as of 31 December, 2018 – Assessed in 2019; 2) Final performance against targets as of 31 December, 2023 –assessed at the closure of the programming period in 2025. Figure 44. Two Phases for ROP Performance Framework 397. According to Annex II of the Common Provisions Regulation (CPR), milestones and targets shall be:  Realistic, achievable, relevant, capturing essential information on the progress of a priority;  Consistent with the nature and character of the specific objectives of the priority;  Transparent, with objectively verifiable targets and the source data identified and, where possible, publicly available;  Verifiable, without imposing a disproportionate administrative burden; and  Consistent across the programmes, where appropriate. 51 Currently in draft form: http://ec.europa.eu/fisheries/reform/emff/guidance-performance-framework-review_en.pdf 187 Performance Framework Indicators 398. There are four categories of indicators reviewed under the performance framework, as described below: Indicator Recommended Usage Financial There should be one financial indicator per priority. The financial indicator should relate to Indicator the total amount of eligible expenditure incurred by beneficiaries and paid through the ROP. This is the actual absorption rate. Output The output indicators are selected from among the indicators already chosen for the Indicator programme. The aim is to relate the selected indicator(s) to operations representing the majority of the resources allocated to each priority axis (e.g. kilometers of road built). The number of indicators selected per axis should be kept as low as possible, however, as further indicators will raise the risk of failing the milestone set in performance review. Result Result indicators describe specific and measurable outcomes which are delivered at the Indicator review points and are closely linked to the supported policy interventions. All result indicators relate to priority axes which are aligned with thematic objectives. These are, in turn, aligned with the Europe 2020 objectives. Result indicators are critical aspects of performance management, but may not be appropriate for inclusion within the performance framework as reviewed because of the timing when results can be achieved in the ROP and the need for evaluation to disentangle the effects of the policy from those of factors external to the program. Result indicators are primarily used for internal review purposes. In the current EC framework, you need to primarily meet financial indicators (absorption) and output indicators (e.g. km of built roads). Key Key implementation steps can be an important stage in the delivery of a priority, whose Implemen completion is verifiable. Their achievement could be expressed either in the form "yes" or tation "no" (for example, "All major projects for building waste water treatment plants submitted Steps to the European Commission") or by a number or percentage (for example, "Works have started on 80% of selected locations to eliminate level crossings"). In a situation where no measurable output is expected by the end of 2018 (for example, due to the lengthy implementation of infrastructure projects, kilometers of new railroad will not be completed), a key implementation step should be used to set a milestone (for example, award of contracts for laying rail tracks). Since by 2023 outputs must be delivered and can be demonstrated by an appropriate output indicator, key implementation steps should not be used as end of programme targets. The Performance Framework and the Partnership Agreement 399. Each Member State outlines in the Partnership Agreement the methodology and mechanisms to ensure consistency in the functioning of the performance framework across programmes and ESI funds, while the actual performance framework (indicators and their respective milestones and targets) are included in the programme for each priority. The guidance provided in the draft template of the Partnership Agreement calls upon the Member States to give an overview of how consistency in the selection of indicators and the setting of milestones and targets is ensured across programmes and priorities. Monitoring mechanisms designed to ensure early detection of potential performance issues and the system for the follow-up of detected issues should be included. 188 Selecting Indicators for the ROP 400. Financial indicators for the ROP will simply be the total funding expenditure to date compared to the overall target rate. Selecting the correct output indicators and/or key implementation steps, however, requires an analysis of the overall goals and thematic objectives relating to the programme. In addition to determining the correct output indicators, considerable care must be put into setting milestones and targets for these outputs that will put the ROP in a reasonable position to be successful in meeting the targets. All performance indicators and milestones will be presented in the following a framework similar to the example presented in the table below. Note that this example includes one financial indicator, one output indicator, and one key implementation step, respectively. Table 25. EXAMPLE Performance Framework Indicator by Axis Indicator and measurement unit, (where Milestone for Target for Explanation Priority axis appropriate) 2018 2023 Source data of relevance Developing Expenditure (EUR 100 250 Comprehensive high million) quality and Length of rail 50 interoperable completed with railway speed capacity of Optional Fields infrastructure 120 km/hour (km) Public Yes procurement procedures completed Monitoring the Performance Framework 401. The European Commission (EC) and the Monitoring Committee for the ROP will monitor performance regularly, and additionally conduct formal reviews following the two phases of the performance framework, scheduled for 2019 and the closure of the programing period in 2025, respectively. The MA will include information on progress in achieving the milestones and targets in the Annual Implementation Reports (AIR), beginning from the reports submitted in 2017. The data transmitted will relate to values for indicators for fully implemented operations. Following this examination, the EC may make observations to the MA concerning issues that significantly affect the implementation of the programme. In such a case, the MA should provide all necessary information with regard to those observations and, where appropriate, inform the EC within three months of the measures taken. 402. The monitoring committee reviews the implementation of the programme and progress toward achieving its objectives, taking into account the milestones defined in the performance framework. The monitoring committee may also make observations to the MA regarding the implementation of the programme and its evaluation, and monitors actions taken as a result of its observations. Romania may propose the revision of milestones and targets in duly justified cases, such as a significant change in the economic, environmental, and labor market conditions in the country or a specific region, or in the case of changes in allocations for a given priority.52 52 The procedure foreseen in Article 26 of the CPR for amending programmes applies. 189 Figure 45. Timeline for Performance Framework Setting ROP Goals 403. Setting correct goals and indicators for the ROP must be done in line with the top level development goals as set in the Europe 2020 Romania development targets, as well as the investment priorities for the European Regional Development Fund. Figure 46. Strategic Decomposition of ROP Goals Top-down Goals vs. Bottom-up Project Selection 404. These top-level goals set the strategic alignment for ROP priorities, but the reality of development under the ROP is that project selection will be largely developed at the regional level. Therefore, it is important to develop a common understanding across all ROP stakeholders of these top- level goals, while allowing for local action to self-align with the overall goal areas. Top-level goal development must also take into account the realities of development in regional areas, where differing budget and operational realities will affect the level of absorption possible by various strategic projects, and the level of demand for transportation, education, and other projects in each of these regions. Finally, the specific needs of people in these regions (to include at risk populations and sub-populations) will greatly shape the impact that can be derived by seemingly similar projects in different regions. These local considerations are the purview of the local cities and county councils, and the RDA’s must be able to inform goal selection and ongoing measurement of goal performance with these considerations. 405. The top-line development targets relating to the ROP come from the Europe 2020 Romania- specific targets, as indicated in the table below. 190 Table 26. Europe 2020 Romania Development Targets Reference period Indicators Unit Target 2005 2008 2009 2010 2011 2012 75% of the population aged 20-64 should be employed Employment rate - % of population age group 20-64 aged 20-64 63.6 64.4 63.5 63.3 62.8 63.8 70 Male 70.4 71.6 70.7 70.8 69.9 71.4 (:) Female 56.9 57.3 56.3 55.9 55.7 56.3 (:) 3% of the EU's GDP should be invested in R&D Gross domestic % of GDP expenditure on R&D 0.41 0.58 0.47 0.46 0.5(b) 0.42(dp) 2 Greenhouse gas emissions should be reduced by 20% compared to 1990 The share of renewable energy sources in final energy consumption should be increased to 20% Energy efficiency should improve by 20% Greenhouse gas Index 1990 = 57.89 57.48 49.24 47.76 50.46 (:) (:) emissions 100 Share of renewable % energy in gross final 17.6 20.3 22.3 23.4 21.4 (:) (:) energy consumption Primary energy million tons of consumption oil equivalent 36.8 37.5 33.1 33.3 33.9 (:) (:) (TOE) Index 2005 = 100 102.2 90.1 90.7 92.4 (:) (:) 100 % of savings (:) (:) (:) (:) (:) (:) (:) Final energy million tons of consumption oil equivalent 25.1 25 22.4 22.5 22.6 (:) (:) (TOE) Index 2005 = 100 99.8 89.4 89.7 90 (:) (:) 100 % of savings (:) (:) (:) (:) (:) (:) (:) The share of early school leavers should be under 10% and at least 40% of 30-34 years old should have completed a tertiary or equivalent education Early leavers from % of population education and aged 18-24 19.6 15.9 16.6 18.4 17.5 17.4 11.3 training Male 20.1 15.9 16.1 18.6 18.5 18 (:) Female 19.1 16 17.2 18.2 16.6 16.7 (:) Tertiary educational % of population attainment aged 30-34 11.4 16 16.8 18.1 20.4 21.8 26.7 Male 10.8 14.9 15.2 16.7 19.7 20.5 (:) Female 12.1 17.1 18.5 19.6 21 23.2 (:) Poverty should be reduced by lifting at least 20 million people out of the risk of poverty or social exclusion People at risk of Thousand poverty or social (:) 9418 9112 8890 8630 8907 (:) exclusion(1) % of total (:) 44.2 43.1 41.4 40.3 41.7 (:) population 191 People living in Thousand households with very (:) 1413 1299 1176 1135 1215 (:) low work intensity % of total (:) 8.2 7.7 6.8 6.7 7.4 (:) population People at risk of Thousand poverty after social (:) 4988 4745 4522 4748 4824 (:) transfers % of total (:) 23.4 22.4 21.1 22.2 22.6 (:) population People severely Thousand materially deprived (:) 7023 6817 6643 6286 6391 (:) % of total (:) 32.9 32.2 31 29.4 29.9 (:) population Last update: 08/12/2013 11:05:02 (1) People represented at least in one of the 3 sub-indicators (i) See the pdf document http://portal.eurostat.test.ec.europa.eu/portal/page/portal/europe_2020_indicators/documents/Europe2020_i-icon.pdf Flags and Special values for data: please visit the Information page: http://epp.eurostat.ec.europa.eu/NavTree_prod/htdocs/explanation/explanation_en_auth.html Investment Priorities for the European Regional Development Fund 406. The starting point of programming is the identification of development needs or challenges to be addressed, which should help to identify the funding priorities . This is followed by: 1) selection of thematic objectives and investment priorities as set out in the CPR and Fund-specific rules; 2) definition of at least one specific objective per investment priority to target the latter appropriately in the specific national or regional context; 3) definition of result indicators linked to specific objectives Figure 47. The Programming stages the ROP has to go through 407. Only after the development needs, objectives and the result sought have been clarified, should one consider the types of actions to be supported, choosing (a mix of actions) which best achieves the objectives defined. Output indicators should be logically linked to and reflect the types of actions planned. Outputs generated by actions should also contribute logically toward the results that 192 one aims to achieve.53 The table in Annex 23 decomposes the investment priorities for the European Regional Development Fund (ERDF), and the ROP priorities for funding as set in the current Partnership Agreement. The Annex also shows how the US has set performance indicators for similar programs. Managing for Performance across the Regional Operational Programme 408. Understanding that the MA-IB structure depends on both strong coordination at the MA level and mature operations at the RDA level, the MA and the RDAs should cultivate an ongoing understanding of the level at which each RDA is operating. This maturity model framework would define the organizational features (both current and future) of a successful RDA. With this model, MA operations can better take into account what is capable at present across RDA operations as well as identify key intervention areas to support developing new RDA capabilities. Figure 48. Maturity Model View of RDA Operations 409. Through this maturity model approach, the MA can understand how effective its new strategies for regional development may be by region. The MA should also use this framework to understand what capabilities it should work to encourage at each RDA within the ROP. Setting Sub-goal Targets and Milestones by RDA 410. In working toward the overall goal objectives of the ROP as set in the EC performance framework, the MA should be able to clearly break out sub-goal targets and milestones for each RDA. These sub-goal targets and milestones should collectively contribute toward success for the overall ROP objectives. In setting appropriate sub-goal targets and/or milestones, the MA should work with each RDA to set specific, time bound targets across each investment axis that: a) Reflect the local needs within the region, and b) Are believed to collectively deliver the outcomes on each axis which the MA is responsible for as part of the performance framework. 53 http://ec.europa.eu/regional_policy/what/future/pdf/preparation/2709_gui_presentation_guidance.ppt 193 Data-Driven Performance Reviews with RDAs 411. On at least a quarterly basis, the MA should conduct data-driven performance reviews with the RDAs to monitor and improve the efficiency and effectiveness of the ROP programme across each region and as a whole. These data-driven reviews, also referred to as PerformanceStat – or just STAT programs, bear a close resemblance to other types of program reviews that government officials traditionally hold with members of their staff to identify emerging trends and discuss key program issues and problems. However, the process described here is distinguished by the frequency and regularity of its meetings, the focus on the latest performance indicators, and the somewhat structured format.54 What is PerformanceStat? “PerformanceStat programs employ regular meetings between the chief executive (chief, principal, mayor, or governor) and agency directors where performance data is used to analyze each agency’s performance, establish performance objectives and accountability for those objectives, and schedule follow-up to ensure these objectives are met. Each of these elements is important to the integrity of the program. Stat programs without regular meetings, accurate and timely data, clear direction, and rigorous follow-up will rarely accomplish much.” —Thornburgh et al., 2010, p. 1 “A jurisdiction or agency is employing a PerformanceStat leadership strategy if, in an effort to achieve specific public purposes, its leadership team persists in holding an ongoing series of regular, frequent, integrated meetings during which the chief executive and/or the principal members of the chief executive’s leadership team plus the director (and the top managers) of different subunits use current data to analyze specific, previously defined aspects of each unit’s past performance; to provide feedback on recent progress compared with targets; to follow-up on previous decisions and commitments to produce results; to examine and learn from each unit’s efforts to improve performance; to identify and solve performance-deficit problems; and to set and achieve the next performance targets.” —Robert D. Behn, The PerformanceStat Potential: A Leadership Strategy for Producing Results (forthcoming 2014). 412. The use of regularly scheduled, structured, data-driven meetings to review project data with MA and RDA staff will:  Encourage attention to the need for continuous improvement  Help identify policies and practices that are working well and ones in need of improvement  Increase awareness of unforeseen 2nd and 3rd order impacts of regional ROP activities (such as broader environmental and energy efficiency trends or raising local government operations and maintenance expenditure loads due to increased investment)  Promote cross-regional collaboration and the dissemination of best practices  Improve ROP effectiveness (in terms of absorption) and efficiency (in terms of impact)  Provide ongoing assessment of success under the performance framework and a more convincing case to Romania and the EC that the ROP is using its funds wisely and that its budget requests are justified  Increase accountability of ROP projects under review. 54 Hatry, Harry, and Elizabeth Davies. "A Guide to Data-Driven Performance Reviews. www.businessofgovernment.org IBM Center for the Business of Government, n.d. Web. 9 Dec. 2013. 194 A Data Dashboard for Reviewing RDA Performance 413. A data driven performance review of RDAs must begin with a standardized dashboard to facilitate comparison across all RDA operations. However, regions across Romania have vastly different needs, and as such the different RDAs should have a variety of focuses. With this in mind any successful data-driven review across these regions must be capable of diving deeper into more specific activities to expose the unique successes, challenges, and needs of each RDA. Long Term Maturation of the ROP Capabilities 414. This section addresses long-term considerations for improving ROP operations that are not necessarily feasible in the short-term, but which should be considered in planning and developing current and future ROP operations. Priority Goal Setting across Cross-Cutting Thematic Challenges 415. There are a number of thematic challenges faced by all regions of the ROP, as exhibited in the table below. Social Challenges Environmental Challenges  Marginalized communities  Energy efficiency  Equal opportunity and non-discrimination  Environmental impact and rehabilitation 416. In working toward achieving objectives across the priority axes, the MA should consider designating a small number of related cross-cutting goal targets along these thematic challenges, which would prioritize specific outcomes in these areas across all national, regional, and local projects. These outcomes would be aggregated nationwide as an additional output of the data-driven performance review process across the regions, and would be managed by designated goal leaders for these areas operating at the MA, and potentially the ROP level. Measuring the overall performance of the ROP 417. In addition to internal ROP performance indicators, it is important to also look at how the ROP performs using external indicators. Most importantly, it is important to see if ROP has met its overall objective –generating development at the regional level. Overall development performance is computed through GDP per Capita, which is now calculated at the country, regional, and county (județ) level. There is however no standardized measure assessing development performance at the local level. Given that a large majority of ROP investments are done by localities, or directly and indirectly affect localities, it is that much more important to also gage the efficiency of the ROP by measuring the economic performance of localities. 418. One of the best measures for measuring developmental performance was elaborated by Dumitru Sandu, a leading sociologist in Romania. Professor Sandu has adapted the UNDP Human Development Index methodology and has computed a Local Human Development Index (LHDI) for localities in Romania. Given restrictions pertaining to data availability, the LHDI was computed only at Census moments (as some of the data required for its computation only gets released with the Census). Annex 24 presents, in detail, the LHDI methodology and how it could be adapted so that the index can be calculated every 2-3 years. This will allow an assessment of whether the ROP, and public investments in general, manage to have a discernable development effect at the local level. Using the LHDI at regular time intervals can help give a better picture of where public money was spent wisely, and where it was wasted. 195 XII. Conclusions 419. Based on this analysis and many others – both by the World Bank and other stakeholders – Romania faces a major challenge in managing and implementing the Cohesion Policy in the 2014-2020 programming period. In order to improve the quality of spending, the new regulatory framework at the EU level has created more demanding requirements for managing authorities and intermediate bodies with respect to strategic direction, performance, and results-based implementation. This added layer of complexity – i.e., looking at the impact of investments beyond just output measures – is as needed as it is challenging to adopt, particularly for a country that has struggled to deliver on simpler absorption targets. Now Romania is rightfully asked to adopt a more nuanced, balanced approach that takes into account both the absorption and the expected impact of EU funds. This new direction comes against a background still dominated in Romania by important system-wide issues that affect the implementation of EU-funded interventions (and public investments more generally): cumbersome public procurement procedures; poor quality technical documentation; and insufficient beneficiary capacity, both internal (own human and financial resources) and external (unreliable service providers, inadequate contract forms, etc.). 420. The good news is that the Regional Operational Programme has been a constant top performer in terms of absorption and commitment rates, in the Romanian context and even beyond, as this report has shown. If there is one OP in Romania that can handle an enhanced, dual focus on absorption and impact and can be expected to deliver on set targets, it is likely the ROP, with its strong, hard-tested MA-IB relationships and beneficiary support mechanisms. This optimistic expectation will likely be fulfilled even in the context of a financial allocation that is expected to double for the ROP, putting additional strain on the system’s capacity to deliver. One thing is certain: the selection model and criteria will remain key components of the ROP’s performance during the 2014-2020 programming period. 421. The ROP has hitherto relied on two main mechanisms: pre-allocated funds and FIFO. Both have their advantages, but they also suffer from important weaknesses in relation to the quality of projects coming through the system’s pipeline. In part, weaknesses are due to the current strategic approach to the ROP’s management and Romania’s institutional framework (lacking a regional tier), which do not provide sufficient flexibility for interventions to be targeted to the development needs and challenges of regions and places. There are also operational constraints, notably inadequate administrative capacity in applicant organizations, along with shortcomings in the appraisal and evaluation of projects. The current analysis is motivated by an honest and essential admission that the current system is improvable, although expectations of radical changes are likely unreasonable, not in the least because a whole range of core issues do not even depend on the ROP and its key players, let alone on how individual projects are evaluated and selected for financing. That said, maintaining the scope of the current analysis within reasonable parameters, the aim has been to identify ways of improving the status quo system. 422. A primary conclusion of this extensive analysis is that there is no silver bullet in the endeavor to craft the perfect selection model. Each of the options explored has both advantages and drawbacks, and the MA-ROP is ultimately the main actor accountable for its choices. The figure below is a simplified attempt at assessing the various scenarios discussed based on: corresponding presumptions that would need to hold to justify a particular choice; and, perhaps more importantly, the four key dimensions that help characterize a project selection model – absorption (how quick?), impact (how effective?), 197 legitimacy (how credible?), and capacity (how feasible?). As this snapshot shows, there is no perfect option – in simple terms, none of the scenarios presented feature four fully black circles. All in all, scenario 2 seems to have a slight edge over the alternatives, as a “middle of the road” solution that combines a focus on large, strategic projects with pre-allocated funding with the need to introduce some competition among all beneficiary types, even in the form of competing against tighter deadlines and facing the risk of “losing” money set aside for them. Scenario 2 is also consistent with the principle that less risky, smaller projects should benefit from simplified evaluation procedures and criteria, and the FIFO rule would help accelerate absorption without reducing the quality of proposals received (since at this level of funding applications are expected to be relatively standard). By preserving some of the elements of the framework applicable during the 2007-2013 ROP, scenario 2 is also able to fare relatively well on the capacity dimension, in the sense that it does not completely revolutionize a largely functioning system. Figure 49. Simplified comparative assessment of proposed Project Selection Models Note: A larger black-fill in the pie-charts indicates the degree to which each individual scenario fulfils a specific criterion 423. The purpose of the current analysis, however, is not to promote one option ahead of the others, particularly given that tradeoffs between the four essential desiderata – absorption, impact, legitimacy, and capacity – cannot be avoided. Instead, this report has provided the following, in logical order: 1) Adaptation of the PIM framework to the ROP, making the distinction between public works and service-based projects; 2) Analysis of the 2007-2013 project selection models, including areas for improvement; 198 3) Collection of best practices from other EU Member States, both to inspire ROP decision- makers and to make the point that there is no perfect system to be adopted by Romania; 4) Definition of several core principles for an optimal project selection framework: balancing absorption and impact; facilitating high-quality development strategies that respond to regional needs; and adjusting selection models based on project size/complexity; 5) Proposal of three different scenarios for the 2014-2020 ROP project selection models, going from marginal changes (“Status Quo+”) to more significant adjustments of current systems (“Competitive”), while emphasizing that the greater the changes, the harder it will be for the system to adapt to them in a timely and effective manner; 6) Horizontal recommendations applicable to all project selection scenarios (e.g., strong development strategies, FIDIC contracts, ROP electronization, etc.); 7) Step-by-step and grid-by-grid suggestions for improvement, at the level of: the entire process (from a potential pre-qualification phase through evaluation and selection); overall selection grids (e.g., the order of criteria, corresponding weights, etc.); and individual criteria, suggesting very specific ways of enhancing the definition and measurement of various project characteristics; 8) Arguments in favor of promoting strategic projects at the level of regions, based on clearly identified priorities and new/innovative institutional set-ups for the management of such interventions; 9) Arguments in favor of pre-allocating funds (in an initial phase of 2-3 years) to counties and county residences (including Growth Poles), with pre enhanced opportunities to better prepare projects, while recognizing and properly responding to the risks of pre-allocation – especially the potential lower absorption of funds; and 10) Framework for optimal monitoring and evaluation of the ROP, with a special focus on the actual project selection model and feedback loops for ensuring constant improvement. 424. Beyond doubt, the assessment of ROP project selection models does not and should not stop with the completion of the current analysis. The first year of the new programming period (2014) is fundamental for setting the right foundations for the new ROP, particularly when it comes to how projects are evaluated and selected. While generally it is not advisable to change the rules of the game after the first “whistle,” this principle should hold even stronger with respect to project selection procedures and criteria. Keeping in mind the need to ensure a high degree of legitimacy for the system, applicants and beneficiaries should remain confident that they are judged based on the same rules, ensuring fair and equal treatment. Regardless of subsequent political pressures for tweaking the applicable framework, the chosen selection model should remain relatively consistent across time, while also ensuring functional M&E mechanisms to make critical adjustments (if needed and if possible) and capture learnings for the coming programming periods. 425. Regardless of the MA’s ultimate choices in terms of the 2014-2020 ROP’s selection model and criteria, the fundamental aim of the programme should remain front and center – to promote Romania’s sustainable and inclusive development. In that endeavor, the importance of the ROP, both as an indispensable source of funds and as a unique catalyzer of administrative capacity building, can hardly be overstated. Some of this impact will continue to be very visible across the country, changing people’s lives in very concrete terms: a new road; a better equipped, modern hospital; a proper school 199 with comfortable temperatures year-round; a new business center employing hundreds or thousands of people; a renovated museum, etc. Other types of impact, including the motivation of countless public servants, the shift in mentalities, and the improved capacity to deliver key services to citizens, may be harder to quantify, but no less important. Ultimately, this is the power of EU convergence at its best, and Romania’s ROP will continue to play a key role in this historic effort. 200 Bibliography Babbie, E.R. (2010). The practice of social research (Twelfth ed.): Wadsworth Pub Co. Bachtler, J., Begg, I., Polverari, L., and Charles, D. (2013) Evaluation of the Main Achievements of Cohesion Policy Programmes and Projects over the Longer Term in 15 Selected Regions (from 1989-1993 Programme Period to the Present (2011.CE.16.B.AT.015), Final Report to the European Commission (DG Regio), European Policies Research Centre, University of Strathclyde (Glasgow) and London School of Economics. Byrne, B. M. (2012). Structural equation modeling with MP.L.U.S. : basic concepts, applications, and programming . New York: Routledge Academic. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling - a Multidisciplinary Journal, 14(3), 464-504. Davidov, E. (2008). A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey. Survey Research Methods. doi: papers2://publication/uuid/4410D2C5-B81F-42D1-B5E3-CFF3096AB4D7 Davidov, E., Datler, G., Schmidt, P., & Schwartz, S. H. (2010). Testing the invariance of values in the Benelux countries with the European Social Survey: Accounting for ordinality. In E. Davidov, P. Schmidt & J. Billiet (Eds.), Cross-Cultural Analysis: Methods and Applications (pp. 149-168). London/New York: Routledge. Deloitte & Touche GmbH (2012): Evaluierung von wettbewerblichen Auswahlverfahren des Ziel 2-Programms 2007- 2013, Ministry for Business, Energy, Industry, SMEs & the Craft Sector of the Land Nordrhein-Westfalen, Final Report (http://www.ziel2.nrw.de/3_Ergebnisse/Evaluierung_Wettbewerbsverfahren/Evaluierung_wettbewerblic he_Auswahlverfahren_Ziel_2_Endbericht.pdf, accessed 19.11.2013). Deloitte & Touche GmbH (2013). Evaluierung von wettbewerblichen Auswahlverfahren des Ziel 2-Programms 2007- 2013, Presentation of Results, PowerPoint Presentation, 19.03.2013. Economic Growth Operating Program, Lithuania. Enterprise Estonia (2013). Measure Strengthening of the competitiveness of regions ( Piirkondade konkurentsivõime tugevdamise meede) (http://www.eas.ee/et/avalikule-ja-mittetulundussektorile/avalike-teenuste-arendamine/piirkondade- konkurentsivoime-tugevdamise-programm-sf-2007-2013/ueldjutt, accessed 13.01.2014). Ernst and Young, Praxis & Institute of Baltic Studies (2010). Selection criteria for the assessment of structural instruments (Struktuurivahendite valikukriteeriumide hindamine), Research Report (http://www.struktuurifondid.ee/public/Struktuurivahendite_valikukriteeriumide_hindamine.pdf, accessed 13.01.2014). Estonian Minister for Regional Affairs (2007). Internal decree on the measure “Strengthening of the competitiveness of regions” (Meetme «Piirkondade konkurentsivõime tugevdamine» tingimused ). Estonian Minister for Regional Affairs (2012). Internal decree on the measure “Strengthening of the competitiveness of regions” (Regionaalministri 22. oktoobri 2007. a määruse nr 7 „Meetme „Piirkondade konkurentsivõime tugevdamine” tingimused” muutmine ). 201 Estonian Ministry of Interior (2013). Strengthening of the competitiveness of regions (Piirkondade konkurentsivõime tugevdamine) (2007-2013) (https://www.siseministeerium.ee/35929/, accessed 13.01.2014). European Union Structural Assistance to Estonia, 2013 (www.struktuurifondid.ee, accessed 13.01.2014). Ferry M, Gross F, Bachtler J and McMaster I (2007) ‘Turning Strategies into Projects: The Implementation of 2007 - 13 Structural Funds Programmes’, IQ-Net Thematic Paper, 20(2), European Policies Research Centre, University of Strathclyde, Glasgow (http://www.eprc.strath.ac.uk/iqnet/downloads/IQ- Net_Reports%28Public%29/ThematicPaper20%282%29Final.pdf, accessed 19.11.2013 Government of Catalunya: Criterios De Selección De Operaciones Del Programa Operativo FEDER Catalunya 2007- 2013 Government of Estonia: Annual Implementation Report of the OP for Development of Living Environment 2012. Government Office for Local Self-Government and Regional Policy (2007) Operational Programme for Strengthening Regional Development Potentials for Period 2007-2013, 26.07.2007. Guildford G (2013) Investing in Growth and Jobs: An Independent Review of Arrangements for Implementation of European Structural Funds programmes 2014-20 http://wales.gov.uk/docs/wefo/publications/130314guilfordreporten.pdf Kaldmäe, M (2007). Priority for Integral and balanced development of regions, General Selection Criteria (Piirkondade terviklik ja tasakaalustatud arengu prioriteedi üldised valikukriteeriumid ). Presentation in the meeting of a monitoring committee for the OP for development of the Living Environment, 27.09.2007. Klaudijus Maniokas, Agne Miseliuniene: Expert evaluation network delivering policy analysis on the performance of Cohesion policy 2007– 2013 Year 2 – 2012 Task 2: Country Report on Achievements of Cohesion policy Lithuania. Ministry for Economy, SMEs and Energy oft he Land Nordrhein-Westfalen (2009 and 2010) ElektroMobil.NRW, Elektromobilität in Nordrhein-Westfalen – Gesucht: Die besten Ideen für eine mobile Zukunft, Wettbewerbsaufrufe 1 und 2 (http://www.ziel2- nrw.de/2_Wettbewerbe_und_weitere_Foerdermoeglichkeiten/1_Wettbewerbe_2009/ElektroMobil_NR W/ElektroMobil_Wettbewerbsaufruf.pdf and http://www.ziel2.nrw.de/2_Wettbewerbe_und_weitere_Foerdermoeglichkeiten/1_Wettbewerbe_2010/ ElektroMobil_NRW_2/ElektroMobil_2_Wettbewerbsaufruf_2010.pdf accessed 10.12.2013) Ministry for Business, Energy, Industry, SMEs & the Craft Sector of the Land Nordrhein-Westfalen (2012): Operationelles Programm (EFRE) für das Ziel "Regionale Wettbewerbsfähigkeit und Beschäftigung" für Nordrhein-Westfalen, Version of 05.07.2012 (http://www.ziel2.nrw.de/1_Ziel2- Programm/3_1_Programmtexte/OP_Gesamt_Endversion_20120507_-_Online__2_.pdf, accessed 19.11.2013). Ministry for Business, Energy, Industry, SMEs & the Craft Sector of the Land Nordrhein-Westfalen (2012): Operationelles Programm (EFRE) für das Ziel "Regionale Wettbewerbsfähigkeit und Beschäftigung" für Nordrhein-Westfalen, Annual Implementation Report 2012, 26.08.2013 (http://www.ziel2.nrw.de/1_Ziel2-Programm/5_Berichte/Durchfuehrungsbericht_2012.pdf, accessed 12.12.2013). 202 Ministry of Public Administration, Government of Spain: La Gestión De Proyectos Ofinanciados Por El Fondo Social Europeo, Guí; a Para Beneficiarios Ministry of Regional Development, Poland: National Development Strategy 2007-2015 Ministry of Regional Development, Poland: National Strategy of Regional Development 2010-2020. Regions, cities, rural areas Ministrstvo za gospodarski razvoj in tehnologijo (2013) Letno poročilo 2012 o izvajanju Operativnega programa krepitve regionalnih razvojnih potencialov za obdobje 2007-2013, September 2013. Ministrstvo za gospodarski razvoj in tehnologijo (2013) Letno poročilo 2012 o izvajanju Operativnega programa krepitve regionalnih razvojnih potencialov za obdobje 2007 – 2013, Ljubljana, September 2013. National Strategic Reference Frameworks for Ireland, Lithuania, Poland, Spain, UK OECD (2012): Public Investment across levels of government; the case of Galicia, Spain Operational Programme “Regional Competitiveness and Employment 2007 -2013” (ERDF) of Nordrhein-Westfalen: Project Selection Criteria (http://www.ziel2.nrw.de/1_Ziel2- Programm/4_1_Auswahlkriterien/Auswahlkriterien.pdf, accessed 19.11.2013). Operational Programme “Regional Competitiveness and Employment 2007 -2013” (ERDF) of Nordrhein-Westfalen: Monitoring Sheet, Measure 2.2 Business-related technology and research infrastructure (http://www.ziel2-nrw.de/1_Ziel2- Programm/6_Formulare/6_2_Monitoringboegen/6_2_1_Monitoringboegen/index.php, accessed 11.12.2013). Pentz, M. A., & Chou, C.-P. (1994). Measurement invariance in longitudinal clinical research assuming change from development and intervention. Journal of Consulting and Clinical Psychology, 62(3), 450. PRS Centralny (2011) Programu Rozwoju Subregionu Centralnego 2007-13 (http://rpo.slaskie.pl/zalaczniki/2011/03/30/1301487854.pdf, accessed 1.12.2013). PSDB (2012) Wstępna ocena realizacji i efektów Regionalnego Programu Operacyjnego Województwa Śląskiego na lata 2007-2013. (http://www.ewaluacja.gov.pl/Wyniki/Documents/Wstepna_ocena_realizacji_i_efektow_RPOWS_140220 13.pdf, accessed 1.12.2013). PSDB (2010) Pozakonkursowe ścieżki wyboru projektów w ramach RPO WSL: Indykatywny Wykaz Projektów Kluczowych oraz Programy Rozwoju Subregionów (http://rpo.slaskie.pl/zalaczniki/2010/09/02/1283415977.pdf, accessed 1.12.2013). Rambøll Management Consulting (2012) Comparative Study of the Project Selection Process Applied in Cohesion Policy Programmes 20017-2013 in a Number of Member States. Final Report (http://ec.europa.eu/regional_policy/sources/docgener/studies/pdf/selection/selection_process.pdf, accessed 12.12.2013). Regione Lazio, Dipartimento Programmazione Economica e Sociale, Direzione Regionale Programmazione Economica, Ricerca e Innovazione (2011) Programma Operativo Regionale cofinanziato dal FESR 2007- 2013: motivi ed effetti della revisione, a cura del Nucleo di valutazione e verifica degli investimenti pubblici, 7 febbraio 2011, pp. 1-28. 203 Regione Lazio (2011), Obiettivo Competitività Regionale e Occupazione, Programma Operativo cofinanziato dal F.E.S.R. – Fondo Europeo di Sviluppo Regionale 2007-2013, Regione Lazio (Revisione draft 18/03/2011), CCI 2007IT162PO004, pp. 1-164. Regione Lazio (2012) Programma Operativo Regionale F.E.S.R. Lazio 2007-2013, Asse V Sviluppo Urbano e Locale, Attività V.1, Rigenerazione delle funzioni economiche, sociali e ambientali delle aree urbane, Disciplinare regionale per la realizzazione di interventi integrati di sviluppo urbano (undated), in BUR Lazio N. 3, parte terza, 21/01/2012. Regione Lazio, Direzione Programmazione Economica, Ricerca e Innovazione (2012), Determinazione del Direttore 13 febbraio 2012, no. B00870, Determinazione n. B00027 del 9 gennaio 2012. Proroga scadenza avviso pubblico, “Invito a presentare candidature per la prequalificazione dei progetti a valere sull’Attività V.1 – Rigenerazione delle funzioni economiche sociali ed ambientali delle aree urbane in virtù della proposta di revisione del POR FESR Lazio 2007-13”, approvato con DGR n. 349 del 29 luglio 2011 e DGR n. 359 dell’8 agosto 2011, in BUR Lazio n. 7, parte terza, 21/02/2012. Regione Lazio (2012) Descrizione del Sistema di Gestione e Controllo, versione aprile 2012 http://porfesr.lazio.it/PORFESR/galleria_allegati/documentazione/SIGECO/DESCRIZIONE_SISTEMA_GESTIONE_E_C ONTROLLO.pdf Regione Lazio (2012) Determinazione del Direttore 24 maggio 2012 n. B03110, POR FESR Lazio 2007-2013 – Determinazione del Direttore della Direzione Regionale Programmazione Economica, Ricerca e Innovazione, n. B00027/2012 e DGR 192/12 Approvazione della graduatoria dei P.L.U.S. approvazione degli interventi da ammettere a finanziamento a valere sulle Attività V.1 e Attività II.1; approvazione degli interventi finanziabili a valere sull’attività I.7 e degli interventi finanziabili con le economie dell’Attività V.1, in BUR Lazio n. 21, parte terza, 7/06/2012, pp. 405-427. Regione Lazio (2012) Programma Operativo Regione Lazio Fondi Europei, Avviso pubblico: Invito a presentare candidature per la prequalificazione dei progetti a valere sull’Attività V.1 “Rigenerazione delle funzioni economiche, sociali e ambientali delle aree urbane” in virtù della proposta ti revisione del POR FESR Lazio 2007-2013, Allegato alla Delibera n. 348, in BUR Lazio N. 30, 13/08/2012, pp. 70-90 (contains pro forma for submitting P.L.U.S. applications). Regione Lazio, Dipartimento Programmazione Economica e Sociale, Direzione Regionale Programmazione Economica, Ricerca e Innovazione, Area Programmazione Negoziata per lo Sviluppo Locale (2012) Determinazione del 24 ottobre 2012 N. B08056, Proposta n. 22518 del 23/10/2012, Oggetto: Fondo di garanzia a favore delle PMI interessate dai P.L.U.S. nell’ambito dell’Attività I.5 del POR FESR Lazio 2007- 2013. Approvazione dello schema di atto aggiuntivo alla convenzione tra la Regione Lazio e Unionfidi Lazio S.p.A., Registro cronologico n. 15252 del 24 maggio 2012 (richiesta di pubblicazione sul BUR - sì). Regione Lazio, Direzione Programmazione Economica, Ricerca e Innovazione (2012) Atti dirigenziali di gestione, Determinazione 24 ottobre 2012, n. B08055 ‘Integrazioni al cap. 5.3.1 del “Disciplinare regionale per la realizzazione di interventi integrati di sviluppo urbano”, Allegato 3 della Determinazione dirigenziale del 9 gennaio 2012, n. B00027, in BUR Lazio n. 71, 11/12/2012. Regione Lazio, Dipartimento Programmazione Economica e Sociale, Direzione Regionale Programmazione Economica, Ricerca e Innovazione, Area Programmazione Negoziata per lo Sviluppo Locale (2012), Determinazione N. B09388 del 06/12/2012, Proposta n. 25947 del 06/12/2012, oggetto: POR FESR Lazio 2007-2013 – Approvazione elenco “Progetti Retrospettivi” coerenti con le strategie dei Piani Locali e Urbani di Sviluppo (P.L.U.S.) (richiesta di pubblicazione sul BUR - sì). 204 Regione Lazio (2012) ‘Integrazioni al documento Descrizione del Sistema di Gestione e di Controllo’ POR FESR LAZIO 2007/2013, CCI 2007IT162PO004, Novembre 2012. Regione Lazio, Direzione Regionale Sviluppo Economico e Attività Produttive, Area Infrastrutture per lo Sviluppo Economico (2013) Determinazione – proposta n. 18679 del 21/11/2013, oggetto: modifiche e integrazioni al “Disciplinare Regionaole per la realizzazione di interventi integrati di sviluppo urbano” approvato con determinazione dirigenziale n. B00027 del 09/01/2012, modificato dalle determinazioni dirigenziali n. B08055 del 24/10/2012 e n. B00891 del 13/03/2013 (disciplina rimodulazione progetti e utilizzo economie derivanti da ribassi d’asta). Regione Lazio (2013) Programma Operativo F.E.S.R. Lazio 2007-2013, Asse V, Sviluppo Urbano e Locale, Attività V.1 Rigenerazione delle funzioni economiche, sociali e ambientali delle aree urbane, Disciplinare regionale per la realizzazione di interventi integrati di sviluppo urbano approvato con Determinazione dirigenziale N. B00027 DEL 9/1/2012, modificato con Determinazioni Dirigenziali n.B08055 del 24/10/2012, n. B00891 del 13/3/2013 e n. G02878 del 22/11/2013, pp. 1-39. Regional Operating Programs for BMW Region, Ireland; Świętokrzyskie Region, Poland; Comunidad de Valenciana, Spain; West Wales and the Valleys, UK and the Operating Program for the Development of Eastern Poland. Republic of Estonia (2010). Operational Programme for the Development of the Living Environment. 2007EE161PO002, May 2010. Sandu, D. (coordinator). (1998). Community poverty in Romania rural areas .Foundation for poverty alliviation by Romania Social Development Fund Sandu, D., (1999), Community and Regional Poverty in Rural Romania, in Romanian Journal of Sociology, vol. X. Sandu, D. ( 2011). Social disparities in the regional development and policies of Romania. International Review of Social Research, I, 1-30. Sandu, D. (Coord.), Disparităţi şi fluxuri în fundamentarea social-economică a regionalizării administrative a României, CONREG, 2013 Seier, Katri (2007). Selection Criteria (Valikukriteeriumid), Presentation in the Monitoring Committee meeting for the OP for Development of the Living Environment, 27.09.2007. Steenkamp, J.-B. E., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78-107. The Implementation Plans for the Operating Programs. UNDP. (2005). National Human Development Report (NHDR) for Romania . UNDP. (2013). Human Development Report 2013.The Rise of the South Human Progress in a Diverse World . NY. Urząd Marszałkowski Województwa Śląskiego (2013) Sprawozdanie okresowe z realizacji Regionalnego Programu Operacyjnego Województwa Śląskiego na lata 2007-2013 w I półroczu 2013 roku (http://rpo.slaskie.pl/zalaczniki/2013/10/11/1381497902.pdf, accessed 2.12.2013). Urząd Marszałkowski Województwa Śląskiego (2013) Podręcznik Procedur Wdrażania Regionalnego Programu Operacyjnego Województwa Śląskiego na lata 2007-2013 wersja dwudziesta druga (http://rpo.slaskie.pl/?grupa=1&art=1235722902&id_m=148, accessed 2.12.2013). 205 Urzad Marszalkowski (2013) Załącznik nr 6 do Szczegółowego opisu priorytetów RPO WSL na lata 2007 -2013 - Kryteria wyboru projektów (http://rpo.slaskie.pl/zalaczniki/2013/10/29/1383053871.pdf, accessed 2.12.2013). Urzad Marszalkowski (2010) Regionalny Program Operacyjny Województwa Śląskiego na lata 2007 – 2013 - Kryteria oceny Programów Rozwoju Subregionów (http://rpo.slaskie.pl/zalaczniki/2011/09/05/1315205330.pdf, accessed 2.12.2013) Urząd Marszałkowski Województwa Śląskiego (2010) Regionalny Program Operacyjny Województwa Śląskiego na lata 2007-2013 (RPO) (http://rpo.slaskie.pl/zalaczniki/2010/10/07/1286452333.pdf, accessed 1.12.2013). Vilnius University: Effectiveness of implementing the EU cohesion policy in Lithuania Paper for the Fourth Pan- European Conference on EU Politics of the ECPR Widaman, K. F., Ferrer, E., & Conger, R. D. (2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child development perspectives, 4(1), 10-18. World Bank Working Paper No 161: Public Investment Management in the New EU Member States. Strengthening Planning and Implementation of Transport Infrastructure Investments World Bank: May 2013 Report: Romania - Improving the National Framework for Preparing and Implementing Public Investment Projects World Bank. (2009). World Development Report 2009 (World Bank Ed.): World Bank Washington DC. WEFO (2010) West Wales and the Valleys Convergence Programme Operational Programme for the European Regional Development Fund 2007-2013, Version of 24.09.2010. http://wales.gov.uk/docs/wefo/publications/convergence/erdfoperational/110601erdfconvopen.pdf WEFO (2011) The Effectiveness of Implementation in the 2007-2013 Structural Funds Programming Period.Undertaken by a consortium led by Old Bell 3 Ltd. http://wales.gov.uk/docs/wefo/report/110331processevaluationen.pdf WEFO (2010) Making your proposals to the Welsh European Funding Office. How does it work? http://wales.gov.uk/docs/wefo/publications/applyingforfundingguidance/100924makingyourproposalsto wefoen.pdf WEFO (2009) European Structural Funds 2007-2013 Strategic Framework Sustainable Transport ERDF Convergence: P3T1 http://wales.gov.uk/docs/wefo/publications/strategicframeworks/100202sustainabletransportframeworken.pdf WEFO (undated) Selection and Prioritisation Criteria to be applied to project proposals under the 2007-2013 Programmes http://wales.gov.uk/docs/wefo/publications/applyingforfundingguidance/070926projectselectionandprioritisation criteriaen.pdf WEFO (undated) Applying for EU project funding http://wales.gov.uk/docs/wefo/publications/applyingforfundingguidance/071116applyingforeufundingen.pdf 206 Annexes 207 208 Annex 1. The 2007-2013 ROP Allocation Table 27. ROP Planned Allocation by PAs and Years (in EUR) National Contribution Priorit Private Year Total EU Contribution y Axis Contribution Other State Budget Local Budget Public Total Sources PA 1.1. 2007 127,359,275 102,281,115 22,550,437 2,527,723 0 25,078,160 2008 149,867,265 120,395,009 26,483,635 2,988,621 0 29,472,256 2009 159,872,128 128,448,505 28,229,451 3,194,172 0 31,423,623 2010 193,757,249 155,698,831 34,177,710 3,880,708 0 38,058,418 2011 205,313,836 164,988,660 36,211,799 4,113,377 0 40,325,176 2012 249,316,898 200,361,161 43,956,303 4,999,434 0 48,955,737 2013 305,685,134 245,633,248 53,932,485 6,119,401 60,051,886 Total 1,391,171,785 1,117,806,529 245,541,820 27,823,436 0 273,365,256 PA 1.2. Total Savings/Redistribution PA 2.1. 2007 77,686,668 67,198,968 8,933,967 1,553,733 0 10,487,700 2008 95,088,482 82,251,537 10,935,175 1,901,770 12,836,945 2009 103,796,585 89,784,046 11,936,607 2,075,932 14,012,539 2010 123,228,666 106,592,796 14,171,297 2,464,573 16,635,870 2011 131,004,222 113,318,652 15,065,486 2,620,084 17,685,570 2012 156,195,982 135,109,525 17,962,537 3,123,920 21,086,457 2013 189,710,401 164,099,497 21,816,696 3,794,208 25,610,904 Total 876,711,006 758,355,021 100,821,765 17,534,220 0 118,355,985 PA 3.1. 2007 18,159,603 15,436,079 2,360,332 363,192 0 2,723,524 2008 22,227,355 18,893,761 2,889,047 444,547 3,333,594 2009 24,262,911 20,624,032 3,153,622 485,257 3,638,879 209 2010 28,805,247 24,485,121 3,744,021 576,105 4,320,126 2011 30,622,820 26,030,099 3,980,264 612,457 4,592,721 2012 36,511,503 31,035,615 4,745,658 730,230 5,475,888 2013 44,345,648 37,694,818 5,763,917 886,913 6,650,830 Total 204,935,087 174,199,525 26,636,861 4,098,701 0 30,735,562 PA 3.2. 2007 8,817,437 7,494,821 1,146,267 176,349 1,322,616 2008 10,792,543 9,173,661 1,403,031 215,851 1,618,882 2009 11,780,913 10,013,776 1,531,519 235,618 1,767,137 2010 13,986,454 11,888,486 1,818,239 279,729 2,097,968 2011 14,868,979 12,638,632 1,932,967 297,380 2,230,347 2012 17,728,244 15,069,007 2,304,672 354,565 2,659,237 2013 21,532,131 18,302,311 2,799,177 430,643 3,229,820 Total 99,506,701 84,580,694 12,935,872 1,990,135 0 14,926,007 PA 3.3. 2007 8,817,437 7,494,821 1,146,267 176,349 1,322,616 2008 10,792,543 9,173,661 1,403,031 215,851 1,618,882 2009 11,780,913 10,013,776 1,531,519 235,618 1,767,137 2010 13,986,454 11,888,486 1,818,239 279,729 2,097,968 2011 14,868,979 12,638,632 1,932,967 297,380 2,230,347 2012 17,728,244 15,069,007 2,304,672 354,565 2,659,237 2013 21,532,131 18,302,311 2,799,177 430,643 3,229,820 Total 99,506,701 84,580,694 12,935,872 1,990,135 0 14,926,007 PA 3.4. 2007 25,248,167 21,460,942 3,282,262 504,963 3,787,225 2008 30,903,756 26,268,193 4,017,488 618,075 4,635,563 2009 33,733,888 28,673,804 4,385,406 674,678 5,060,084 2010 40,049,316 34,041,919 5,206,411 800,986 6,007,397 2011 42,576,371 36,189,916 5,534,928 851,527 6,386,455 2012 50,763,693 43,149,139 6,599,280 1,015,274 7,614,554 2013 61,655,880 52,407,498 8,015,264 1,233,118 9,248,382 210 Total 284,931,071 242,191,411 37,041,039 5,698,621 0 42,739,660 PA 4.1. 2007 24,427,421 21,847,251 2,580,170 2,580,170 2008 29,899,163 26,741,037 3,158,126 3,158,126 2009 32,637,296 29,189,951 3,447,345 3,447,345 2010 38,747,425 34,654,694 4,092,731 4,092,731 2011 41,192,334 36,841,356 4,350,978 4,350,978 2012 49,113,508 43,925,848 5,187,660 5,187,660 2013 59,651,621 53,350,863 6,300,758 6,300,758 Total 275,668,768 246,551,000 29,117,768 0 0 29,117,768 PA 4.2. 2007 2,698,662 2,413,600 285,062 285,062 2008 3,303,162 2,954,246 348,916 348,916 2009 3,605,663 3,224,793 380,870 380,870 2010 4,280,690 3,828,516 452,174 452,174 2011 4,550,794 4,070,090 480,704 480,704 2012 5,425,901 4,852,758 573,143 573,143 2013 6,590,118 5,893,997 696,121 696,121 Total 30,454,990 27,238,000 3,216,990 0 0 3,216,990 PA 4.3. 2007 30,244,263 20,253,160 2,392,515 2,392,515 7,598,588 2008 37,018,978 24,789,867 2,928,439 2,928,439 9,300,672 2009 40,409,138 27,060,098 3,196,622 3,196,622 10,152,418 2010 35,921,175 32,126,102 3,795,073 3,795,073 2011 38,187,749 34,153,214 4,034,535 4,034,535 2012 45,531,150 40,720,784 4,810,366 4,810,366 2013 55,300,609 49,458,100 5,842,509 5,842,509 Total 282,613,062 228,561,325 27,000,059 0 0 27,000,059 27,051,678 PA 5.1. 2007 23,514,121 21,260,996 1,795,712 457,413 2,253,125 2008 28,781,285 26,023,461 2,197,951 559,873 2,757,824 2009 31,417,046 28,406,663 2,399,237 611,146 3,010,383 211 2010 37,298,728 33,724,762 2,848,406 725,560 3,573,966 2011 39,652,227 35,852,748 3,028,137 771,342 3,799,479 2012 47,277,241 42,747,134 3,610,439 919,668 4,530,107 2013 57,421,351 51,919,235 4,385,117 1,116,999 5,502,116 Total 265,361,999 239,934,999 20,264,999 5,162,001 0 25,427,000 0 PA 5.2. 2007 41,913,478 26,192,675 2,211,933 563,962 2,775,895 12,944,908 2008 51,302,098 32,059,832 2,707,408 690,289 3,397,697 15,844,569 2009 56,000,290 34,995,837 2,955,350 753,506 3,708,856 17,295,597 2010 66,484,280 41,547,517 3,508,631 894,572 4,403,203 20,533,560 2011 70,679,344 44,169,106 3,730,020 951,017 4,681,037 21,829,201 2012 84,270,791 52,662,706 4,447,293 1,133,896 5,581,189 26,026,896 2013 102,352,476 63,962,357 5,401,533 1,377,192 6,778,725 31,611,394 Total 473,002,757 295,590,030 24,962,168 6,364,434 0 31,326,602 146,086,125 PA 5.3. 2007 12,525,316 11,324,774 956,796 243,746 1,200,542 2008 15,330,986 13,861,523 1,171,118 298,345 1,469,463 2009 16,734,983 15,130,947 1,278,368 325,668 1,604,036 2010 19,867,990 17,963,659 1,517,695 386,636 1,904,331 2011 21,121,633 19,097,140 1,613,460 411,033 2,024,493 2012 25,183,265 22,769,469 1,923,723 490,073 2,413,796 2013 30,586,750 27,655,034 2,336,490 595,226 2,931,716 Total 141,350,923 127,802,546 10,797,650 2,750,727 0 13,548,377 0 PA 6.1. 2007 6,665,516 4,999,137 1,666,379 1,666,379 2008 11,922,812 8,942,109 2,980,703 2,980,703 2009 17,271,909 12,953,932 4,317,977 4,317,977 2010 16,956,060 12,717,045 4,239,015 4,239,015 2011 18,998,430 14,248,823 4,749,607 4,749,607 2012 18,359,748 13,769,811 4,589,937 4,589,937 2013 19,980,443 14,985,332 4,995,111 4,995,111 212 Total 110,154,918 82,616,189 27,538,729 0 0 27,538,729 0 PA 6.2. 2007 680,000 510,000 170,000 170,000 2008 3,464,200 2,598,150 866,050 866,050 2009 3,487,100 2,615,325 871,775 871,775 2010 3,418,532 2,563,899 854,633 854,633 2011 3,374,500 2,530,875 843,625 843,625 2012 3,454,600 2,590,950 863,650 863,650 2013 3,472,800 2,604,600 868,200 868,200 Total 21,351,732 16,013,799 5,337,933 0 0 5,337,933 0 TOTAL ROP 4,556,721,500 3,726,021,762 584,149,525 73,412,410 0 657,561,935 173,137,803 Source: ROP - Framework Document for Implementation of ROP 2007 – 2013 Romanian Version, October 2012 213 ROP Planned Allocation by Priority Axis/Key Area of Intervention and Development Region - thousands EUR PA 1.1 PA PA 2.1 PA 3.1 PA 3.2 PA 3.3 PA 3.4 PA 4.1 PA 4.2 PA 4.3 PA 5.1 PA 5.2 PA 5.3 1.2 TOTAL 1,391,172 235,400 285,860 235,400 330,010 150,360 876,711 173,580 99,520 99,520 284,910 274,390 EU 1,117,807 210,534 231,187 212,844 206,231 135,948 Contributi 758,355 147,547 84,592 84,592 242,174 245,407 on Nord- Total 38,420 46,650 38,420 53,860 24,530 East 227,040 143,080 28,330 16,240 16,240 46,500 44,780 Region EU 34,361 37,727 34,738 33,658 22,178 Contrib 182,427 123,764 24,081 13,804 13,804 39,525 40,050 ution DR 16.32% 16.32% 16.32% 16.32% 16.32% 16.32% 16.32% 16.32% 16.32% 16.32% 16.32% 16.31% Ratio South- Total 31,190 37,880 31,190 43,730 19,920 East 184,330 116,160 23,000 13,190 13,190 37,740 36,360 Region EU 27,895 30,635 28,201 27,327 18,010 Contrib 148,109 100,478 19,550 11,211 11,211 32,079 32,519 ution DR 13.25% 13.25% 13.25% 13.25% 13.25% 13.25% 13.25% 13.25% 13.25% 13.25% 13.25% 13.25% Ratio South Total 33,500 40,670 33,500 46,960 21,400 Region 197,960 124,760 24,700 14,160 14,160 40,540 39,050 EU 29,961 32,892 30,290 29,346 19,349 Contrib 159,061 107,917 20,996 12,036 12,036 34,459 34,925 ution DR 14.23% 14.23% 14.23% 14.23% 14.23% 14.23% 14.23% 14.23% 14.23% 14.23% 14.23% 14.23% Ratio South Total 32,980 40,050 32,980 46,230 21,070 West 194,900 122,830 24,320 13,940 13,940 39,920 38,440 Region EU 29,496 32,390 29,820 28,890 19,050 Contrib 156,602 106,248 20,673 11,849 11,849 33,932 34,380 ution DR 14.01% 14.01% 14.01% 14.01% 14.01% 14.01% 14.01% 14.01% 14.01% 14.01% 14.01% 14.01% Ratio West Total 24,340 29,560 24,340 34,120 15,550 Region 143,850 90,650 17,950 10,290 10,290 29,460 28,370 214 EU 21,769 23,906 22,007 21,322 14,060 Contrib 115,583 78,412 15,258 8,746 8,746 25,041 25,373 ution DR 10.34% 10.34% 10.34% 10.34% 10.34% 10.34% 10.34% 10.34% 10.34% 10.34% 10.34% 10.34% Ratio Nord- Total 28,460 34,560 28,460 39,900 18,180 West 168,190 105,990 20,990 12,030 12,030 34,450 33,170 Region EU 25,454 27,950 25,733 24,934 16,437 Contrib 135,141 91,681 17,842 10,225 10,225 29,282 29,666 ution DR 12.09% 12.09% 12.09% 12.09% 12.09% 12.09% 12.09% 12.09% 12.09% 12.09% 12.09% 12.09% Ratio Centre Total 25,660 31,160 25,660 35,970 16,390 Region 151,640 95,560 18,910 10,850 10,850 31,060 29,910 EU 22,950 25,200 23,201 22,478 14,819 Contrib 121,843 82,659 16,074 9,222 9,222 26,401 26,751 ution DR 10.90% 10.90% 10.89% 10.90% 10.90% 10.90% 10.90% 10.90% 10.90% 10.90% 10.90% 10.90% Ratio Buchare Total 20,850 25,330 20,850 29,240 13,320 st - Ilfov 123,260 77,680 15,380 8,820 8,820 25,240 24,310 Region EU 18,648 20,485 18,852 18,273 12,043 Contrib 99,039 67,193 13,073 7,497 7,497 21,454 21,742 ution DR 8.86% 8.86% 8.86% 8.86% 8.86% 8.86% 8.86% 8.86% 8.86% 8.86% 8.86% 8.86% Ratio 215 Contracting Ratio by Priority Axis and Development Region (% = Total Contracted Value/ Total Allocated Resources) as of 2011 PA 1.1. PA 2.1. PA 3.1. PA 3.2. PA 3.3 PA 3.4. PA 4.1. PA 4.2. PA 4.3. PA 5.1. PA 5.2. TOTAL RD Growth Urban Urban Poles Development Centers Poles RD North 61% 51% 146% 133% 107% 98% 91% 103% 29% na 81% 71% 90% 93% East RD South 3% 82% 72% 121% 106% 102% 72% 91% 31% na 76% 62% 64% 72% East RD South 25% 18% 122% 136% 94% 69% 68% 97% 71% na 63% 95% 69% 83% RD South 35% 31% 121% 128% 81% 82% 68% 97% 76% 0% 62% 90% 54% 82% West RD West 10% 69% 137% 125% 77% 67% 92% 102% 52% 57% 82% 89% 76% 82% RD 48% 30% 103% 118% 106% 90% 80% 93% 46% na 54% 60% 95% 80% North West RD Center 19% 56% 147% 108% 94% 82% 76% 98% 52% 0% 90% 92% 55% 79% RD Na Na 37% 78% 94% 28% 95% 53% 83% na 118% 28% 33% 58% Bucharest – Ilfov ROP 28% 43% 121% 121% 95% 80% 79% 93% 54% 43% 74% 74% 68% 80% 216 ROP Submission/contracting and Payment Situation at the end of 2012 Million EUR Reporting EU Submitted Projects Project Projects in Approved Projects Period Allocation Rejected Evaluation 2007-2013 Process Number Total Value EU % EU Number Number Number EU Total % EU Contribution to EU Allocation Contribution Contribution Contribution Value to EU Projects Allocation Approved 1 2 3 4 5 6=5/2 7 8 9 10 11 12=11/2 31-Dec-12 3,726 8221 12,659 7,681 206.14% 2,826 426 3,690 3,824 6,162 102.64% 30-Nov-12 3,726 8211 12,548 7,610 204.23% 2,808 472 3,652 3,710 5,975 99.56% Difference 0 10 111 71 1.91% 18 -46 38 115 187 3.08% Million EUR Contracts/Financing Decisions Payments to Beneficiaries Payment Intermediary Payments Applications sent to from EC EC Number Eligible Value Non- Projects % EU Pre- Reimbursements Total % EU EU % EU Reimbursed % Eligible Total Contributi financing Payments Contribution Contrib. Contribution Value Reimbursed Grant Value Benef. Expenses Value on to EU to EU to EU Value to EU Contrib. Allocation Allocation Allocation Allocation EU Nat’l EU Contrib. Nat’l Contrib. Budget Budget 18=14+15 23=20+21 24=(21+2 13 14 15 16 17 19=14/2 20 21 22 25 26=25/2 27 28=27/2 +16+17 +22 0)/2 31- Dec- 3319 3,455 504 382 120 4,460 92.72% 524 763 153 1,439 34.53% 954 25.60% 920 24.70% 12 30- Nov- 3316 3,397 493 381 118 4,389 91.17% 520 712 143 1,374 33.06% 786 21.10% 786 21.10% 12 Differ 3 58 11 0 1 70 1.55% 3 51 10 65 1.46% 168 4.50% 134 3.60% ence Source: Ministry for EU Funds – Situation at 31st December 2012 0f Submission, Approval, Signing and Payments for Operational Program 217 Annex 2. 2007 – 2013 ROP Structure Priority Axis 1: Support to sustainable development of urban growth poles PA 1.1. Integrated urban development plans The integrated urban development plans should be implemented by projects addressing the following issues – main activities: a) Rehabilitation of the urban infrastructure and improvement of urban services, including urban transport; b) Development of sustainable business environment; c) Rehabilitation of social infrastructure, including social housing and improvement of social services. Eligible Beneficiaries: 1) Local government units – counties or urban municipalities; 2) Inter-Community Development Associations (IDAs) defined according with the actual legislation; 3) Partnerships among local government units. Target groups/Target Areas: 1) Target Areas: o Growth Poles (according with the Government Decision 998/2008): Iaşi, Constanţa, Ploieşti, Craiova, Timişoara, Cluj-Napoca,and Braşov and the their influence areas; o Urban Development Poles (according with the Government Decision 998/2008): Arad, Baia-Mare, Bacău, Brăila, Galaţi, Deva, Oradea, Piteşti, Râmnicu-Vâlcea, Satu Mare, Sibiu, Suceava, Târgu- Mureş; o Urban centers – cities over 10.000 de inhabitants, others those above mentioned cities, including Bucharest and its districts (sectors). 2) Target Groups: Marginalized/Deprived population groups, including Roma population, beneficiaries of integrated projects for social inclusions Period for submission of project application forms: 2008 – 2012 Submission Types for Project Application Forms: a) Open Application Form with continue submission b) Open Application Form with deadline submission Technical and Financial Evaluation Criteria: a) Quality, maturity and sustainability of the project (the following details are described by Evaluation Procedure Manual): - Investment Opportunity - Economic Efficiency: value of economic IRR ( if economic IRR>= 5,5% the project obtains 6 points, otherwise the project obtains 0 points) - Applicant capacity to implement the project - Applicant capacity to monitor the project implementation - Implementation project methodology - Coherence of the technical and economic/financial documentation, including the technical project - Technical solution relevance - Management plan of investment - Project needs co-financing from the grant funds – the financial IRR calculated on investment cost (if the financial IRR calculated on investment cost <=5% the project obtain 6 points, otherwise the project obtain 0 points) - Project budget - Lasting development and energy efficiency - Society information and new technologies - Equal opportunities and non-discrimination - Sustainability project after the end of grant financing – the sustainability is defined as total cost for operation during the full life of the asset - Financial sustainability of the project (if the Cumulative Net Cash Flow is permanently positive during the investment period the project obtain 6 points, otherwise 0 points) 218 PA 1.2 Support for the investments in the energy efficiency of the buildings Objective: generating/maintaining jobs and social cohesion promotion by supporting the improvement of the energy efficiency of block of flats in Romania, according with the Strategy Europe 2020, which will generate the increase of employment rate and savings in energy consumption. Main general activities: 1) Construction operations to the exterior of block of flats 2) Construction operations to interior commune water, sewage and heating installation of block of flats Eligible Beneficiaries: local government units, residence of counties, including the 6 districts of Bucharest. Target Groups/Target Areas:  Municipalities county residence and the 6 districts of Bucharest  Block of flats where leave families with small per capita revenues and from marginalized groups Period for submission of project application forms: 2012 - 2013 Submission Types for Project Application Forms: Open Application Form with continue submission Priority Axis 2: Improvement of regional and local transport infrastructure PA 2.1.: Rehabilitation and modernization of the county roads and urban streets networking including construction/rehabilitation of ring roads Main Activities: 1) Rehabilitation and modernization of the county road network; 2) Rehabilitation and modernization of the urban street network; 3) Building/rehabilitation/modernization of the ring roads. Eligible Beneficiaries: 1) Local government units – counties or urban municipalities; 2) Inter-Community Development Associations (IDAs) defined according with the present legislation; 3) Partnerships among local government units. Target Groups/Target Areas: Does not apply Period for submission of project application forms: 2007 – 2013 Submission Types for Project Application Forms: Open Application Form with continual submission Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project (the following details are described by Evaluation Procedure Manual): - The degree of project preparation and maturity - Implementation methodology - Coherence of the documentation and solution - Technical solution - Project needs co-financing from the grant funds (if the IRR<=5% and NPV<0 the project obtains 6 points, otherwise the project obtains 0 points) - Project budget - Environment protection and energy efficiency - Equal opportunities and non-discrimination - Applicant capacity to implement the project - Sustainability project after the end of grant financing – the sustainability is defined as total cost for operation during the full life of the asset b) Project contribution to the achievement of ROP objectives (the following details are described by Evaluation Procedure Manual): - Project relevance for priority axis and key area on intervention - Project contribution to the safety road traffic - Project contribution to the reduction of the traffic travel time – traffic fluidization Priority Axis 3: Improvement of social infrastructure PA 3.1.: Rehabilitation, modernization and equipping of the health services’ infrastructure 219 The specific objective of this field of intervention is the improvement of the quality of medical care assistance and a balanced regional-territorial distribution in order to ensure equal access of the population to health services Main Activities: 1) Rehabilitation, modernization, and supplying specialized equipment for hospitals; 2) Rehabilitation, modernization, developing, and supplying specialized equipment for ambulatories (in hospitals and specialized). Eligible Beneficiaries: 1) Local government units – counties or municipalities; 2) Inter-Community Development Associations (IDAs) defined according with the present legislation; 3) Partnerships among local government units. Target Groups/Target Areas: Areas/localities with insufficient and low quality hospital and ambulatory infrastructure, identified by Ministry of Health studies Period for submission of project application forms: 2008 – 2012 Submission Types for Project Application Forms: a) Project Application Forms based on the list of eligible projects - for hospitals and ambulatories identified by Ministry of Health; b) Open Application Form with continue submission – for other ambulatories that those identified by Ministry of Health. Technical and financial evaluation criteria: Quality, maturity and sustainability of the project. PA 3.2.: Rehabilitation, modernization, development, and equipping of social services infrastructure The specific objective of this area of intervention is the improvement, throughout the country, of the quality and capacity of social services infrastructure, in order to ensure equal access for all citizens Main Activities: 1) Rehabilitation, modernization and supplying equipment for the multi-functional social centers; 2) Rehabilitation, modernization and supplying equipment for the residential social centers. Eligible Beneficiaries: 1) Local Government Units – direct or through subordinated social services providers; 2) Partnerships among LGUs and/or social service providers (public or private), accredited according with the present legislation; 3) Public or private social service providers accredited according to current legislation. Target Groups/Target Areas: Areas/localities with insufficient social service infrastructure and/or that do not provide social services according with the actual standards Period for submission of project application forms: 2008 – 2012 Submission Types for Project Application Forms: Open Application Form with continue submission Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project; b) Project contribution to the achievement of ROP objectives. PA 3.3.: Improving the equipment of the operational units for public safety interventions in emergency situations The specific objective of this field of intervention is the improvement of the response capacity for emergency situations at the level of each development region through the reduction of intervention time for the qualified first aid and for emergency intervention. Main Activities: supplying vehicles and specific equipment for regional and county operational bases for emergency situation interventions Eligible Beneficiaries: Inter-Community Development Association organized according with the present legislation Target Groups/Target Areas: Does not apply Period for submission of project application forms: 2007 - 2011 Submission Types for Project Application Forms: Open Application Form with continue submission Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project; b) Project contribution to the achievement of ROP objectives. 220 PA 3.4.: Rehabilitation, modernization, development, and equipping of pre–university, university education and continuous vocational training infrastructure. The specific objective of this key area of intervention is to improve education infrastructure, school equipment, accommodation structures for students and the continuous vocational training centers in order to ensure initial and continuous educational process at European standards and the increased participation of the school population and of the adults in the educational process. Increased participation, resulting in improved skill levels related to the local job market, will also help to address the problem of youth unemployment Main Activities: 1) Rehabilitation/modernization/supplying equipment for the pre-university and university education infrastructure 2) Building and developing the pre-university education campus 3) Rehabilitation/modernization/supplying equipment of Centers for Continue Professional Training Eligible Beneficiaries: 1) Local Government Units 2) State Universities 3) Public institutions providing continue professional training services Target Groups/Target Areas: 1) Areas/localities with weak school infrastructure (primary, secondary and vocational education) 2) State Universities with weak infrastructure 3) Centers for Continue Professional Training with weak infrastructure Period for submission of project application forms: 2008 - 2012 Submission Types for Project Application Forms: Open Application From with continue submission Technical and financial evaluation criteria: b) Quality, maturity and sustainability of the project; c) Project contribution to the achievement of ROP objectives. Priority Axis 4: Strengthening the regional and local business environment PA 4.1.: Development of sustainable business support structures of regional and local importance The specific objective of this key area of intervention is to attract investment to valorize local resources. ROP activities will focus on providing support to local authorities and companies in order to set up and develop their own regional/local importance business support structures, to attract enterprises, mainly SMEs. Main Activities: Setting and developing different types of regional/local structure to support the business. Eligible Beneficiaries: 1) Local Government Units; 2) SMEs according with the present legislation; 3) Chambers of Trade and Industries organized according with the present legislation; 4) Business Associations organized according with the present legislation 5) Partnerships among Local Government Units Target Groups/Target Areas: Does not apply Period for submission of project application forms: 2008 – 2012 Submission Types for Project Application Forms: Open Application Form with continue submission Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project; b) Project contribution to the achievement of ROP objectives. PA 4.2.: Rehabilitation of unused polluted industrial sites and preparation for new activities The specific objective of this key area of intervention is to revitalize areas, in order to include them in the economic flow, more specifically making them available for companies interested in such locations, adequate for business development, including the creation of business support structures. Main Activities: Rehabilitation of polluted and abandoned industrial sites and preparing them for new economic activities Eligible Beneficiaries: 221 1) Local Government Units 2) Partnerships among Local Government Units Target Groups/Target Areas: Does not apply Period for submission of project application forms: 2008 – 2012 Submission Types for Project Application Forms: Open Application Form with continue submission Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project; b) Project contribution to the achievement of ROP objectives. PA 4.3.: Support the development of micro-enterprises The specific objective of this key area of intervention is to support the micro-enterprises from the urban area to increase the competitiveness, utilization of local resources and labor force Main Activities: Support for the development of micro-enterprises Eligible Beneficiaries: Micro-enterprises from urban area with activities in the fields of production, services and construction and respect the criteria of the EC Regulation 1998/2006 regarding the minimis aid with some exceptions Target Groups/Target Areas: Does not apply Period for submission of project application forms: 2008 – 2012 Submission Types for Project Application Forms: Open Application Form with continue submission Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project; b) Project contribution to the achievement of ROP objectives. Priority Axis 5: Sustainable development and promotion of tourism PA 5.1.: Restoration and sustainable valorization of cultural heritage and setting up/ modernization of related infrastructure Specific Objectives: 1) Increasing the importance of cultural tourism as stimulating factor for economic growth at region level, respecting the principles of lasting development and environment protection; 2) Extension of the tourism season; 3) Increasing the number of tourists by using the local and regional potential of the cultural tourism on national and international market. Main Activities: 1) Restoration, protection and conservation of the world cultural patrimony and modernization of the connected infrastructure; 2) Restoration, protection and conservation of the national cultural patrimony with touristic potential and modernization of the connected infrastructure with the aim to introduce this in the tourism circuit; 3) Restoration, protection and conservation of the cultural patrimony from the urban area Eligible Beneficiaries: 1) Local Government Units 2) Central Government Authorities 3) Cult units defined according with the present legislation 4) NGOs 5) Partnerships among Local Government Units 6) Partnerships among Local Government Units and Central Government Authorities 7) Partnerships among Local Government Units and NGOs Target Groups/Target Areas: Areas with objects from world, national or local (urban area) cultural patrimony, with tourism potential. Period for submission of project application forms: 2008 - 2012 Submission Types for Project Application Forms: Open Application Form with continue submission Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project; b) Project contribution to the achievement of ROP objectives. 222 PA 5.2.: Creation, development, and modernization of the tourism infrastructure for sustainable use of natural resources and for increasing the quality of tourism services Specific Objectives for this key area of intervention are: 1) Utilization of natural resources for tourism 2) Diversification of tourism services 3) Developing/extension of tourism infrastructure with the aim to increase the number of tourists and the sojourn period Main Activities: 1) Arranging the natural objects with tourism potential; 2) Using the mountain tourism potential by building the necessary infrastructure 3) Developing the balneal (spa resorts) tourism 4) Building, rehabilitation and extension of the recreation infrastructure, including the connected utilities Eligible Beneficiaries: 1) For activities regarding the public utility tourism infrastructure: a) Local Government Units; b) Inter-Community Development Associations defined according with the present legislation; c) Partnerships among Local Government Units; 2) For activities regarding the public/private tourism infrastructure functioning under the state aid rules: a) Local Government Units; b) Inter-Community Development Associations defined according with the present legislation; c) Partnerships among Local Government Units; d) Partnerships among Local Government Units and NGOs; e) SMEs from the tourism domain and/or connected activities registered according with the present legislation; f) Partnerships among Local Government Units and cult units organized according with the present legislation; Target Groups/Target Areas: 1) Localities from the urban area 2) Localities from the rural area, where are implemented projects over 6.4 million RON; 3) Tourist, spa/balneal resorts from both areas: urban and rural Period for submission of project application forms: 2008 - 2012 Submission Types for Project Application Forms: Open Application Form with continue submission Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project; b) Project contribution to the achievement of ROP objectives. PA 5.3.: Promoting the tourism potential and setting-up the needed infrastructure in order to increase Romania’s attractiveness as tourism destination Specific Objectives for this key area of intervention are: 1) Promoting the Romanian tourism potential by improving country image, with the aim to promote Romania abroad and to increase the its attractiveness for tourism and business; 2) Establishing the National Centers for Tourism Information and Promotion (NCTIP) with the aim to increase the number of tourists; 3) Designing and implementing an integrated IT system for the Romanian touristic supply. Main Activities: 1) Creating a positive image for Romania as touristic destination by defining and promoting the national tourism brand; 2) Developing and consolidating the internal tourism by promoting the tourism services and specific marketing activities; 3) Establishing and equipping the National Centers for Tourism Information and Promotion (NCTIP). Eligible Beneficiaries: 223 Activity 1) Activity 2) Activity 3)  Ministry of Regional  Local Government Units  Local Government Units from Development and Tourism  NGOs organized according with the the Annex 8 of the ROP present legislation and with  Ministry of Regional minimum 6 months experience Development and Tourism  Partnerships  LGUs from the urban area with  Professional Association in Tourism important natural touristic organized according with the actual resources and touristic resorts legislation and with minimum 6 from the urban area months experience Source: ROP - Framework Document for Implementation of ROP 2007 – 2013 Romanian Version, October 2012 Target Groups/Target Areas: Does not apply Period for submission of project application forms: 2008 - 2012 Submission Types for Project Application Forms: 1) Open Application Form with continue submission 2) Open Application Form with deadline submission 3) Open Application Form with continue submission and deadline Technical and financial evaluation criteria: a) Quality, maturity and sustainability of the project; b) Project contribution to the achievement of ROP objectives. Priority Axis 6: Technical assistance PA 6.1.: Support for the implementation, overall management and evaluation of the ROP Main Activities: 1) Support for Managing Authority (MA) and Intermediate Bodies (IBs) to implement the ROP, including preparation, selection, monitoring, evaluation, control and audit; 2) Procurement and installing the IT equipment necessary for the management and implementation of ROP; 3) Management and logistic support for the Monitoring Committee of ROP and other committees involved in ROP implementation; 4) ROP evaluation, including the assessment of implemented projects; 5) Wage expenses with the contractual staff (non-civil servants) involved in preparation selection, evaluation, monitoring and control of the ROP 6) Organizing workshops and trainings to improve the knowledge of the MA and IBs staff 7) Support to prepare the ROP for the following programing period (2014 – 2020) Eligible Beneficiaries: 1) MA ROP 2) IBs ROP Target Groups/Target Areas: 1) MA ROP 2) IBs ROP 3) ROP Beneficiaries Period for submission of project application forms: 2007 - 2013 Submission Types for Project Application Forms: Open Application Form with continue submission PA 6.2.: Support for the publicity and information activities of the ROP Main Activities: 1) Designing an IT system for the information on ROP content; 2) Realizing and distributing information materials: official documents, beneficiary guides, information bulletins, leaflets, posters, logos; 3) Organization of conferences, forums, presentations, information actions, trainings for beneficiaries. Eligible Beneficiaries: 224 1) MA ROP 2) IBs ROP Target Groups/Target Areas: 1) MA ROP 2) IBs ROP 3) ROP Beneficiaries, NGOs, economic, social and professional organizations, public Period for submission of project application forms: 2007 - 2013 Submission Types for Project Application Forms: Open Application Form with continue submission 225 Annex 3. Scheme of ROP Project Selection Process 226 Annex 4. Analysis of Regional Committees for Strategic Evaluation and Correlation (RCSECs) Government Decision 764/2007 initially assigned the strategic screening role to the Regional Committees for Strategic Evaluation and Correlation (RCSEC), organized at the level of the eight development regions, under the supervision of Regional Development Councils. Each RCSEC comprises two representatives of Local Government Units from every county in the Development Region and an equal number (with LG representatives) of representatives from professional associations, business association, trade unions, and other NGOs. The Regional Development Council approves the appointment or replacement of the RCSEC members. The Regional Development Agency acts as the secretariat for the RCSEC. Initial main functions of the RCSEC, according to the Government Decision 764/2007, were as follows: 1) Strategically evaluate projects proposed to be financed under the ROP in the RCSEC’s corresponding region – the RCSEC made a prioritized list of projects after these individual interventions had been evaluated in the technical and financial evaluation phase; 2) Provide opinions and recommendations regarding the regional correlation of projects financed by public resources; 3) Make recommendations to the MA-ROP on potential needed changes in evaluation and selection criteria and funds reallocation among priority axes and key area of interventions within ROP; and 4) Make recommendations to the MA or IBs on improving the coordination of structural instruments at the regional level. Under this regulation, function (1) was essential. However, the strategic evaluation was conducted only after the technical and financial evaluation, as noted above. International good practice suggests that strategic screening is conducted before project appraisal and selection evaluation. A project that fails to meet the strategic consistency test should be rejected, making it unnecessary to subject it to further evaluation. An appropriate institutional arrangement to ensure that all major project proposals are screened is a critical must-have feature so that resources are not wasted in more detailed project appraisal. In 2008, Government Decision 1383/2008 stipulated that the RCSECs would henceforth have only a consultative role and function (1) was eliminated. The reasons for this significant change are presented by the 2008 Annual Implementation Report of the ROP, prepared by the MA, and are summarizes below: a) In the comments of the EU Commission on the audit report prepared by the National Audit Authority: - The role of RCSECs is unclear and duplicates some of the evaluation criteria (used for the technical and financial evaluation). - The strategic prioritization by the RCSEC is not properly justified. Some members of RCSECs (e.g., in the West Development Region) did not provide objective reasons for their choices; - The actual structure of RCSECs generates delays and even restricts some beneficiaries ’ access to financing; - The actual structure of RCSECs reduces the transparency of the whole project selection process. b) There were many difficulties in organizing RCSECs, which obstructed the selection process after the technical and financial evaluation (e.g., in the Bucharest-Ilfov development region); c) Monthly meetings required by GD 764/2007 did not always take place, which created delays in the selection process in some regions (e.g., in the South-East and North-East development regions). During field interviews conducted by the team in June 2013, participants emphasized that having an RCSEC was a good idea, but it was not well-designed to achieve the desired functions. The main reasons for the RCSEC failure are as follows: a) The strategic screening was organized after the technical and financial evaluation and not before; b) Many RCSEC members, essentially the representatives of Regional Development Councils, were in conflict of interest because they were also applicants under the ROP; c) The RCSEC activity was not well defined: there were no clear criteria and procedures for strategic prioritization; 227 d) RCSEC failed to coordinate projects financed through different Operational Programs (OPs) due to the lack of clear procedures regarding data collection. There are no clear requirements and enforcement measures on who should send what information to the RCSEC, when, and how. In the future ROP, a RCSEC-like strategic committee can be reorganized with the following essential conditions: 1. Clear criteria and procedures for member selection; 2. Balanced structure between elected officials and representatives of business and economic environment, civil society, and academic institutions; 3. Clear rules and procedures for the coordination of projects financed through different OPs, especially on data collection and reporting; 4. Clear criteria and detailed procedures for strategic prioritization: pre-defined grids that will emphasize the relevance and opportunity of submitted projects; 5. The strategic committee will assess the project proposals based on a limited technical documentation: project fiche, ownership documents, application form, and prefeasibility study; 6. Limited number of members to maintain efficiency of procedures; 7. Clear code of conduct in dealing with potential conflicts of interests; 8. Regarding the sustainability of O&M, any detailed assessment of O&M needs and the ability of the government to support these will generally be prepared at the feasibility study stage. At the point of strategic screening, the focus should be on the potential outputs or services of the project relative to regional needs. 228 Annex 5. Analysis of Integrated Urban Development Plans (PIDUs)55 Integrated Urban Development Plans (PIDUs) are strategic documents specifically required to be produced by growth poles (GPs), urban development poles (UDPs) and urban centers seeking ROP financing under priority axis 1.1. For GPs and UDPs, there is a list of projects attached to each PIDU based on a pre-determined ROP fund allocation. Therefore, the quality of the PIDUs is critical for the identification of projects and their impact. These strategic documents were seen as integrated approaches that focus on a few key areas for the long lasting development of cities: infrastructure improvement, economic development, environmental protection, and social inclusion. It was for the first time that Romanian local governments had designed such complex strategic documents dealing in an integrated way with cross-sector issues. In this context, the experience of PIDUs can be seen as a positive one, supporting a multi-area integrated approach, completely different from the narrow and isolated sector approach, usually used by local governments in Romania. This exercise was very complex and challenging for the local government institutional capacity, even for important cities that fall under the GPs and UDPs classification. Due to the importance of these strategic documents, this short analysis will underline the main issues that require special attention and improvement in the future. It is also important to note that the World Bank conducted in 2013, as part of a technical assistance project focused on “Growth Poles,” a full assessment of PIDUs, including recommendations for their improvement. In our analysis, we looked at eight PIDUs from various development regions (except for Bucharest-Ilfov, since it does not have a GP or UDP). The GPs/UDPs in the sample are: Brasov (DR-Center), Cluj-Napoca (DR-North West), Iasi and Bacau (DR-North East), Ploiesti (DR-South), Arad (DR-West), Constanta (DR-South East) and Ramnicu Valcea (DR-South West). Our sample covers 40% of all GPs and UDPs. Our analysis focuses essentially on strategic planning issues: the relevance of the actual situation analysis; the quality of SWOT analysis; the logic of the strategic chain: main goal leading logically to specific objectives/priority domains for intervention, which in turn lead to measures/actions/projects; performance measurement, budgetary sustainability, especially on operation and maintenance costs for projects with almost free capital costs. The following findings apply to all the PIDUs in the sample, unless otherwise indicated. Section on General Features of the Growth Pole/Urban Development Pole This section in the PIDUs is an exhaustive actual description of the cities from many perspectives: geographic and demographic, economic and social, education and health, urban and infrastructure. This part is the longest in the PIDU, always taking up more than a half of the entire document. There are a lot of descriptive data, maps, and graphs on all subjects. There are a number of issues in this section that affect the following sections of the PIDU: a) The scope of the section is too large (at least 100 pages), exceeding the needs of a PIDU. There are many interesting but unnecessary data; b) There is no clear connection between the descriptive data in this section and the list of problems underlined. The data generally do not help define and provide evidence for the problems listed. The problems are not defined based on evidence or performance indicators; c) There is no prioritization of the main problems to make a direct link with the objectives and actions detailed in the PIDU. The analysis in this section does not focus on a limited number of areas. The analysis should have been not only documented with gross data but also with impact indicators to help develop subsequent action plans in later sections of the PIDU; d) Other higher-level strategic documents are only enumerated and the connections in terms of objectives and actions are not detailed and not always followed. 56 Section on Strengths-Weaknesses-Opportunities-Threats (SWOT) Analysis 55 The Strategic IDPs Assessment report provides a more in-depth analysis of the 2007-2013 Integrated Development Plans. 56 This analysis is based on the Manual of methods used in policy planning and impact assessment realized by Latvian experts under the Twining PHARE Project “Consolidation of the institutional capacity of the Romanian Government to manage and 229 SWOT analysis is one of the most used strategic planning tools in Romanian public administration. However, the quality of the SWOT analysis is subject to improvement, as presented below: a) The SWOT analysis is reduced to a table divided in 4 sections, containing a large number of items (in some PIDUs the table extends on 14 pages), many of them very detailed and fragmented. It is not clear how the consultative process to develop the SWOT analysis was organized – i.e., the methodology for this activity: number and structure of focus groups, objectives of the analysis, list of questions, methods for prioritization; b) The SWOT analysis is not integrated, but a specific table with 4 sections is completed for every sector: economic development (or even more detailed: agriculture, industry, small and medium-sized enterprises, foreign investments, tourism, etc.), social sector, infrastructure, environment, education, health – in some cases, the number of sectors is high – e.g., 18. In this way, the capacity to integrate the analysis is significantly reduced by the impossibility to prioritize Strengths, Weaknesses, Opportunities, and Threats among sectors, not only within a specific sector. c) In most of the PIDUs in the sample, there is confusion between Internal Environment (S-W dials) and External Environment (O-T dials): items that are from the internal environment are enumerated among Opportunities/Threats or vice-versa. For example, EU funds were listed in the Strengths instead of in the Opportunities; economic crisis was listed in the Weaknesses instead of the Threats. d) There is no prioritization process based on items under each of dials: Strengths, Weaknesses, Opportunities, and Threats, finally obtaining a limited number of “the most important” S, W, O, T. e) There is no strategic analysis trying to set specific approaches: S-O, S-T, W-O, W-T, based on what to set as the vision and the main/strategic goals/objectives. f) In brief, the SWOT analysis has a limited role in setting the strategic vision and objectives. It remains a tool that must be used as a mandatory section of the strategic document. From all the PIDUs analyzed, only one PIDU (from Brasov city) develops the linkage between the problems identified as Weaknesses and the needs understood as future key area of interventions. Unfortunately, even in this case, the linkage is not very clear and the rest of SWOT dials (Strengths, Opportunities, and Threats) remained unused in the strategy section. The Strategy Section The Strategy section in PIDUs describes the visions, strategic objectives, specific objectives, main areas of intervention, and a set of actions/measures. There are a number of critical issues that should be improved in the future: a) The structure of the strategic chain is too cumbersome: General Goal  Strategic Objectives Specific Objectives Priority Domains  Policies  Actions/Measures/Projects. This complication of the logic chain often leads to incoherence between the goals and actions. The logic chain should be more simple and straightforward: Strategic Goals  Specific Objectives  Actions/Measures/Projects; b) There are too many strategic objectives and specific objectives. In some PIDUs, there are 20-30 strategic objectives and over 100 specific objectives. A good PIDU should have 5-6 strategic objectives and 25-30 specific or operational objectives. In this case, the level of detail is too high and there is a clear lack of prioritization and integration of what should be achieved eventually. PIDUs do not highlight potential trade-offs among different objectives, which would confront the decision-makers at the city level; c) The specific objectives are designed more as actions/measures or in some cases as simple wishes – no SMART (specific, measurable, attainable, relevant, and time-bound) definitions are used in any of the PIDUs in the sample. This an essential weakness because the implications are very important: the lack of SMART definition generates the absence of performance indicators assigned to specific objectives, which leads to the impossibility to monitor and evaluate the strategic document; d) There are too many actions/measures without prioritization and clear justification for their selection. For example, it is not clear why the modernization of a street or the rehabilitation of a bridge is selected, and coordinate the policy and decision making processes,” Bucharest 2006, GSG web site: http://www.sgg.ro/docs/File/UPP/doc/manual_metode_impact.pdf. 230 not some other street or another bridge. There is unclear justification for actions on how they contribute to specific objectives. e) There is no budget analysis on sustainability of all these projects, from two perspectives: the co-financing of investment costs and, essentially, the budgetary envelope for the operating and maintenance costs. There is a lack of analysis and forecast of local budgets, using performance indicators (as ratio of own revenues in total revenues, budget dependency on state transfers, rigidity of expenditures, capital investment ratio in total expenditures, annual debt service ratio in current revenues, or gross/net current surplus/deficit). The lack of this analysis may create the illusion that projects whose capital costs are largely funded by the EU will have little impact on local budgets over the medium and long term. f) There is no performance measurement, monitoring system, or reporting system on the results of these PIDUs. This is the result of missing the SMART definition for specific objectives. In conclusion, IDPs are generally focusing more on technical and detailed descriptions of the actual situation and less on strategic issues such as: vision; priorities and the trade-offs facing the decision-makers; the logic of the chain (strategic objectives – specific objectives – actions); results and impact measurement; budgetary sustainability, especially in terms of operations and maintenance costs in the context of relatively low capital costs. The strategic section of PIDUs is strongly driven by the eligibility criteria for the Priority Axis 1.1 rather than the specific priorities and needs of the GPs/UDPs. 231 Annex 6. Analysis of Regional Development Plans for the period 2014 - 2020 Regional Development Plans have been drafted for the 2014 – 2020 period. These are expected to provide investment guidance to projects in each region, mainly for the 2014 – 2020 ROP and other OPs. The RDPs drafts continue to be driven by what is at this point considered the “most probable” potential structure of the 2014 -2020 programming period – it is not entirely clear as of January 2014 if Romania will have only one ROP, like in the 2007- 2013 budgeting cycle, two ROPs (one for the Bucharest-Ilfov region and one for the rest of regions), or eight ROPs (one for each region). In this context, the major challenge will be how to fit the regional priorities within the national framework and the EU 2020 strategic objectives. For the 2014 – 2020 period, the Bucharest-Ilfov region will have a new status as a developed region because its 57 GDP per capita has risen to 106% of the EU average GDP per capita. Therefore, in the 2014 - 2020 ROP, the set of eligible projects for this region will change compared to the current period. At the same time, the real needs of many local governments from the region remain basic and they continue to ask for much simpler projects. The actual project pipeline is realized starting from basic needs and from the requirements of the actual programming period, and not based on specific requirements for the next planning period (2014-2020). According to the opinion of some participants to the interviews conducted as part of this study, the EU 2020 strategy objectives, targets, and tools are too sophisticated or even not appropriate for Romania at this point in the country’s development. All RDAs have started collecting project ideas and project fiches for the new RDPs. The correlation of strategic projects with the needs and objectives from the RDP is done based on some set criteria and through the project fiche. For example, RDA North West uses the following criteria to collect project proposals:  integrated (multi-dimensional and inter-sectoral);  with a significant territorial impact (preferably at the regional level);  contributes to the achievement of regional objectives;  are the result of cooperation between relevant key actors and are developed in partnership;  are sustainable and aligned with previous, current, and/or future initiatives; and  have a budget of at least 25 million EUR. Unfortunately, many of the project proposals received by late 2013 do not achieve these criteria. Local governments have mainly proposed non-strategic, small, and local projects. The main reasons for this result are as follows:  Lack of capacity for strategic thinking: local governments tend to think within local boundaries;  The current ROP does not have projects of regional impact, which makes it difficult for local governments to “imagine” such projects for the future;  Lack of new ways of thinking such as Integrated Territorial Investment (ITI) and Community-Led Local Development (CLLD) at the local level;  Lack of administrative capacity and professional expertise at local government level for programming.  Lack of regional administrative tier, with clear responsibilities and financial resources: it is difficult for local governments to have an interest in, and make efforts toward, the identification and implementation of regional-impact projects. It is also difficult to get local governments to work together on regional-level projects. In the absence of the regional administrative tier, strategic projects will be implemented by associations of local governments, essentially county councils and/or important cities from the region. To be successful these associations should be based on clear rules:  Strong incentives, including financial incentives;  Effective measures against local governments’ non-cooperative behavior;  Legal changes in the status of these associations. 57 According to the EU rules, any region that exceeds 90% of the EU average GDP per capita is considered as a developed region. 232 The drafts RDPs for the 2014 – 2020 period, as of December 2013, have some improvements compared to the RDPs in the previous period. For example: 1. The draft RDPs follow a standardized methodology prepared by the MA for RDP design. This methodology requires that RDPs have performance indicators and monitoring systems. However, it does not give any further instructions on how to build out the indicators and systems. 2. There is an exhaustive analysis of the current situation, including the regional social and economic profile. Many data are presented to identify the trends and other qualitative aspects of the developments during the last seven-year period. In the North-East Region, in order to support the design of the new RDP, the RDA Programming Unit even made a map of all partnerships at local governments. Among them, Local Action Groups can be transformed into CLLD, while Inter-Community Development Associations (ADIs) among counties and metropolitan areas are appropriate for the ITI tool. In this way, it is possible to better plan future complex interventions using ITI or CLLD instruments; 3. There is a need for a clearer emphasis on comparative and competitive advantages, specific to the regions; 4. Better linkage between comparative and competitive advantages and the strategic objectives set by the RDPs – i.e., a much more coherent approach to use the specific situation of any region; 5. Clear and coherent visions and strategic objectives. However, despite some of these major improvements, there are still a number of problems in drafting RDPs, as follows: 1. The strategic planning process misses the necessary qualified expertise – the new Regional Development Plans should reflect the EU 2020 objectives, targets, indicators, and implementation tools, which is challenging in the context of the actual administrative/institutional capacity of the Romanian public administration. At the same time, they should also strive to go beyond EU objectives and try to focus on the actual needs of the regions (regardless of whether these needs could be financed with the help of EU funds). 2. There is a lack of clear connection between the descriptive data and the list of problems underlined. The problems are not defined based on evidence or performance indicators. 3. The SWOT analysis has a limited role in setting the strategic vision and objectives. It remains a tool that must be used as a mandatory section of the strategic document. 4. There is a lack of coherence among strategic objectives and between specific objectives and actions. 5. The specific objectives continue to not be SMART (specific, measurable, attainable, relevant and time- bound) and this undermines the performance measurement and monitoring. 6. The specific actions are not prioritized according to comparative and competitive advantages. For example, all SMEs eligible for EU funding are equally supported, regardless of whether they are in the sectors identified as particularly strong in a region. 7. Budgetary and financial impact analysis of RDPs remains limited: there is no assessment of the total investment costs or budgetary sustainability for the public administration. This type of analysis is difficult to complete in the absence of a regional administrative tier. 8. There is still an unclear strategy between the competitiveness and cohesion objectives – not always is it possible to find the optimal compromise between them and it is not clear if the decision-makers are fully aware of the trade-offs and the pros and cons of every main strategic choice. 9. A regional administrative tier to implement effectively the RDPs continues to be lacking. 233 Annex 7. Criteria used by RDAs to identify regional strategic projects Guiding principles for selecting strategic projects in the West Region in 2014-2020  For 2014-2020, the Regional Development Plan for the West Region will include a list of strategic projects (with major impact), which will have to be approved by the Regional Development Council.  Regional strategic projects should cover an area as wide as possible in terms of priorities and intervention sectors.  Lists of projects will be collected at the local and the county level, and technical assistance will be sought to mature strategic projects.  Three types of regional projects will be identified: 1) regional strategic projects; 2) individual infrastructure projects; and 3) projects that address systemic/thematic issues.  Regional Strategic Projects will:  solve some major challenges at the regional level;  have a territorial impact, addressing challenges in more than one county and for areas with more than 50,000 inhabitants;  benefit from the support of a consortium of regional stakeholders;  have economic impact (jobs, business development, productivity increases, utilizing local resources, time saving, energy);  have a social impact (e.g., improve quality of life for people);  fall within the development priorities defined in the Regional Development Plan; and  not have property/ownership issues.  Individual infrastructure projects will:  solve major challenges at the county/local level;  have a territorial impact, addressing challenges within a single county/locality (with at least 5,000 inhabitants);  benefit from the support of a consortium of county/local stakeholders;  have economic impact (jobs, business development, productivity increases, utilizing local resources, time saving, energy);  have a social impact (e.g., improve quality of life for people);  fall within the development priorities defined in the Regional Development Plan; and  not have property/ownership issues.  Systemic/Thematic Projects will:  look to engender systemic changes within a distinct thematic area;  benefit from the support of a consortium of stakeholders;  have a social impact (e.g., improve the quality of life for people);  fall within the development priorities defined in the Regional Development Plan. Selection criteria for identifying strategic projects in the South Region in 2014-2020 1) Projects follow the objectives and priorities included in the 2014-2020 Regional Development Plan. 2) Projects contribute to achieving national targets within the Europe 2020 Strategy – national targets assumed by Romania through the 2011-2013 National Reform Program. 3) Projects with a significant regional impact: a. Projects that help address a number of key regional, county, and local issues; b. Projects with a significant territorial impact – affecting communities in several regions, counties, territorial administrative units, several small cities, city neighborhoods, metropolitan areas, or peri-urban areas. 4) Projects for which there is a formal administrative engagement – e.g., a written agreement from regional actors (Regional Development Council, county councils, local authorities, regional bodies) to finance and implement them. 234 5) Well-founded and mature projects: projects that already have in place analyses/studies that support their implementation (pre-feasibility studies, feasibility studies, traffic plans, business plans, etc.). 6) Complex projects, which require a common approach with respect to planning, financing, and resource use – i.e., projects that are based on a partnership throughout the entire project cycle. 7) Projects without legal problems or with legal problems that can be resolved within less than 6 months from selection. 8) There is capacity to generate value-added: projects that will bring socio-economic benefits. 9) Projects aligned with horizontal themes in the Regional Development Plans and that propose innovative solutions and approaches. Criteria for collecting strategic project proposals in the North-West Region in 2014-2020 RDA North West uses the following criteria to collect project proposals:  integrated (multi-dimensional and inter-sectorial);  significant territorial impact (preferably regional);  contribute to the achievement of regional objectives;  are the result of cooperation between relevant key actors and are developed in partnership;  are sustainable and aligned with previous, current, and/or future initiatives; and  have a budget of at least 25 million EUR. Criteria for prioritizing projects in the Center Region No. Criteria Yes No Score Observations STRATEGIC CRITERIA 1 The Project follows the thematic Eliminatory Has to address at least one objectives included in the 2014-2020 thematic objective Common Strategic Framework 2 The Project contributes to attaining Eliminatory Has to address at least one national goals within the Europe 2020 goal Strategy 3 The Project falls within the priorities set Eliminatory Has to address at least one in the Regional Development Plan 2014- priority 2020 (applies when the RDP will be finalized) 4 The Project is aligned with the county Eliminatory Has to respond to at least strategy (addresses at least one county one objective set in the level objective) and/or is aligned with a county and/or sectoral national sectoral strategy (addresses at strategy least one measure from the sectoral strategy) 5 The Project is part of an integrated Eliminatory For example, within the development strategy for a well-defined strategy, investments in area, or is of itself an integrated project. physical capital could be The Strategy draws on the priorities set complemented by by the Regional Development Plan and investments in human contributes to attaining national goals capital set in the Europe 2020 Strategy 6 The Project has a significant regional Eliminatory Covers the communities of impact – looks to address a number of several counties, territorial important problems identified as a administrative units, several result of in-depth socio-economic small cities, neighborhoods, analyses, and has a significant territorial larger communities, city 235 impact (the territorial dimension) sectors, including metropolitan areas or peri- urban areas. 7 The Project is based on sound studies Eliminatory Can be buttressed by justifying its implementation, or these studies/analyses completed studies will be completed within 3 at the regional, county, local months of the project’s selection or national level, and which refer to the respective area 8 There is local co-financing for the Eliminatory Estimated budget/Council project’s financial and institutional decision/Written Agreement sustainability for the financing and implementation of the project GENERAL PRIORITIZATION CRITERIA 1 Mature project: Individual Scores - property documents - property documents over the are in place – 2p land/building are in place or will - technical and economic be obtained within less than 3 documentation is in months from the project selection place- 2p date; - property documents - the technical and economic will be obtained within documentation are in place or will 3 months and be finalized within at least 6 technical-economic months from the project selection documents will be date. finalized within 6 months – 1p - will be obtained later – 0p 2 The Project is realized in partnership Individual Scores - local/county – 1p - regional - 2p A higher score will be given to projects - national/transnational that are done in local, regional, – 3p national, transnational partnership (with a higher impact) 3 The Project is correlated with other Individual Scores - correlated at the projects at the local/county level – 1p local/county/regional/national level - correlated at the regional level - 2p - correlated with 2 or more projects – 3p 4 The Project contributes to achieving: Individual Scores - 1 target – 1p - targets set in the Europe 2020 - 1 indicator - 1p Strategy; - ERDF program indicators. 5 Number of people who benefit from Area - Several counties – 3p project (territorial coverage) covered/population - Several served localities/development areas (areas with integrated strategy) – 2p 236 - One locality – 1p 6 Importance/relevance of project at the Individual Scores - Level 1 – 5p county level - Level 2 - 4p - Level 3 – 3p - Level 4 – 2p - Level 5 – 1p 237 Annex 8. Content of Feasibility and Detailed Technical Design in Romania According to Government Decision 28/2008, the content of prefeasibility study, feasibility study, and detailed technical design of projects are as follows: Content of the prefeasibility study A. Written pieces General data: 1. Name of the investment objective; 2. Location (county, city, street, number); 3. Owner of the investment; 4. Beneficiary of the investment; 5. Designer of the study. The need and opportunity of the investment 1. The need for the investment: a) Short presentation regarding the existing situation, of which the need for the investment results; b) Tables, maps, graphics, drawn boards, photographs etc., which explain the current situation and the need for the investment; c) Major deficiencies of the current situation regarding the need for area development; D) medium- and long-term prognoses; 2. The opportunity of the investment: a) The observance of the objective of the general, sector or regional investment policies; b) Legal acts that regulate the field of investment, by case; c) International state agreements that obligate the Romanian party to perform the investment, by case. The technical economic scenarios through which the objectives of the investment project can be reached: 1. Proposed scenarios (at least two); 2. Scenario recommended by the designer; 3. Advantages of recommended scenario. Data regarding the location and land on which the investment objective is to be located Information about the land on the location: 1. Legal situation regarding the property on the land to be occupied – definitively and/or temporarily – by the investment objective; 2. estimated surface of the land; 3. Geophysical characteristics of the land in the location determined based on the geo-technical study performed especially for the investment objective regarding: a) The seismic calculation area and the corner period; b) The preliminary data on the nature of the foundation land and the conventional pressure; c) The maximum level of underground waters; 4. Preliminary topographic studies; 5. Climate data of the area in which the site is situated. The estimated cost of the investment 1. Expenses for elaborating the technical economic documentation: a) Expenses for elaborating the design documentations (prefeasibility study, feasibility study, technical expertise, technical project and execution details), by case; b) Expenses for the technical consultancy and assistance activity; c) Expenses for obtaining the approvals and principle agreements necessary to elaborate the prefeasibility study; d) expenses for preparing documents regarding the application of the procedure for attributing the work contract and service contract for design, urbanism, engineering, other technical services, according to legal provisions (instructions for bidders, advertising, fees and transport expenses etc.). 2. The total estimated cost of the investment 238 Approvals and principle agreements, by case B. Drawn pieces: 1. Plan placed in the area (1:25.000 - 1:5.000); 2. General plan (1:2.000 - 1:500). Content of the feasibility study A. Written pieces General data: 1. Name of the investment objective; 2. Site (county, city, street, number); 3. Owner of the investment; 4. Beneficiary of the investment; 5. Designer of the study. General information regarding the project 1. Current situation and information about the entity responsible with the implementation of the project; 2. Description of the investment: a) The conclusions of the prefeasibility study or of the detailed long-term investment plan (in case they were previously elaborated) regarding the current situation, the need and opportunity of promoting the investment, as well as the selected technical economic scenario; b) The technical economic scenarios through which the objective of the investment project can be reached (in case, before the feasibility study, no prefeasibility study or a detailed long-term investment plan was elaborated): - Proposed scenarios (at least two); - Scenario recommended by the designer; - Advantages of the recommended scenario; c) Constructive, functional and technological description, by case; 3. Technical investment data: a) Area and site; b) Legal status of the land that is to be occupied; c) Situation of the definite land occupations: total surface, representing lands within/outside the incorporated area; d) Land studies: - Topographic studies including topographic plans with locations of marks, lists of marks in a national reference system; - geotechnical study including plans with location of the drills, complex sheets with results of laboratory determinations, analysis of underground water, geotechnical report with recommendations for foundation and consolidations; - Other necessary specialty studies, by case; e) Main characteristics of the constructions within the investment objective, specific to the activity field, and the constructive variants of performing the investment, with the recommendation of the optimum variant for approval; f) Current situation of utilities and consumption analysis: - The need of utilities for the variant proposed for promotion; - Technical solutions of ensuring with utilities; g) The conclusions of evaluating the impact on the environment; 4. The duration of performance and main phases; graphic for performing the investment. The estimated costs of the investment 1. Total value with detailing the structure of the general estimate; 2. Grading of costs together with the investment performance graphic. The cost-benefit analysis: 1. Identification of the investment and definition of objectives, including the specification of the reference period; 239 58 2. Analysis of options); 3. Financial analysis, including the calculation of financial performance indicators: cumulated flow, net actual value, and internal feasibility rate and cost-benefit ratio; 59 4. Economic analysis ), including the calculation of economic performance indicators: net actual value, internal feasibility rate and cost-benefit ratio; 5. Sensitivity analysis; 6. Risk analysis. Investment funding sources: The investment funding sources are constituted according to the applicable legislation and consist of its own funds, bank loans, funds from the state budget /local budget, external credits guaranteed or contracted by the state, external non-refundable funds and other legally constituted sources. Estimate regarding the employment provided by the investment 1. Number of workplaces created in the execution phase; 2. Number of workplaces created in the operating phase. Main technical and economic indicators of the investment 1. Total value (INV), including VAT (thousands lei) (In prices – month, year, 1 euro = ..... lei), Of which: - constructions-mounting (C+M); 2. Grading of investment (INV/C+M): - Year I; - Year II .................; 3. Duration of performance (months); 4. Capacities (in physical and value units); 5. Other indicators specific to the field of activity in which the investment is made, by case. Approvals and principle agreements 1. Approval of the investment beneficiary regarding the need and opportunity of the investment; 2. Urbanism certificate; 3. Principle agreements regarding the insurance of utilities (thermal and electric energy, methane gas, water-channel, telecommunications etc.); 4. Environment agreement; 5. Other approvals and other specific principle agreements. B. Drawn pieces: 1. Plan of location in the area (1:25000 - 1:5000); 2. General plan (1: 2000 - 1:500); 3. General architecture, resistance, installations plans and sections, including plans to coordinate all specialties that help perform the project; 4. Special plans, longitudinal profiles, transversal profiles, by case. Content of the approval documentation of intervention works - DTD A. Written pieces General data: 1. Name of the investment objective; 2. Site (county, city, street, number); 3. Owner of the investment; 4. Beneficiary of the investment; 5. Designer of the documentation. Description of investment: 1. The current situation of the investment objective: 58 Zero variant (variant without investment), maximum variant (variant with maximum investment), average variant (variant with average investment); the selected variant will be specified. 59 It is mandatory only in cases of major public investments 240 - Technical state, from the point of view of insuring the essential quality requirements in constructions, according to the law; - Inventory value of the construction; - Proof act of force majeure, by case; 2. The conclusions of the technical expertise /energy audit report: - Presentation of at least two options; - Recommendation of the expert /energy auditor on the optimal solution from a technical and economic point of view, of development within the approval documentation of intervention works. Technical data of the investment: 1. Description of basic works and of the ones resulted as necessary to perform after performing the basic works; 2. Description, by case, of modernization works performed in the consolidated/rehabilitated/repaired spaces; 3. Utilities consumptions: a) Resulted utilities demand, by case when executing modernization works; b) Estimates regarding the exceeding of initial utilities consumptions. Duration of performance and main phases: - Investment performance graphic: Estimate costs of the investment: 1. Total value with details of the general estimate structure; 2. Grading the costs together with the investment performance graphic. Indicators of assessing the economic efficiency: - Comparative analysis of the cost of performing the intervention works compared to the inventory value of the construction. Investment funding sources The investment funding sources are constituted according to the applicable legislation and consist of its own funds, bank loans, funds from the state budget/local budget, external loans guaranteed or contracted by the state, external non-refundable funds and other legally constituted sources. Estimates regarding the occupied workforce by performing the investment: 1. Number of workplaces created in the execution phase; 2. Number of workplaces created in the operation phase. Main technical economic indicators of the investment: 1. Total value (INV), including VAT (thousands lei) (In prices – month, year, 1 Euro = ..... lei), Of which: - Constructions-mounting (C+M); 2. Grading of investment (INV/C+M): - Year I; - Year II; .................; 3. Performance duration (months); 4. Capacities (in physical and value units); 5. Other indicators specific to the field of activity in which the investment is made, by case. Approvals and principle agreements: 1. Urbanism certificate; 2. Principle approvals with regard to ensuring utilities (thermal and electric energy, methane gas, water- channel, telecommunications etc.); 3. Environment agreement; 4. Other approvals and principle agreements specific to the type of intervention. B. Drawn pieces: 1. Plan located in the area (1: 25000-1:5000); 2. General plan (1: 2000-1:500); 241 3. General architecture, resistance, installations plans and sections, including plans to coordinate all specialties that help perform the project; 4. Special plans, longitudinal profiles, transversal profiles, by case. NOTE: The investment expenses related to intervention works are approved by the main loan requester, according to provisions of art. 46 par. (3) of Law no. 500/2002 regarding public finances, with its further changes. 242 Annex 9. Interview Guide INTERVIEW GUIDE – RDA REPRESENTATIVES INTRODUCTION  WB team presentation  Project presentation: objectives, implementation schedule, results  Project objectives: o Identification of the strengths and weaknesses of the current 2007 – 2013 ROP project preparation, evaluation and selection procedures o Recommendations on improved project preparation, evaluation and selection procedures and processes for the 2014 – 2020 ROP PROJECT SELECTION MODELS  General questions  What are the 3-5 main positive aspects of the selection model for the 2007 – 2013 ROP? Compare and contrast first-in-first-out (FIFO) selection model vs. PA 1.1 selection model.  Which of the current elements should be preserved or improved for the selection models for the 2014-2020 ROP?  Is absorption rate or impact the most appropriate indicator to measure the performance of ROP? How to best measure the latter?  How effective is the current selection model in achieving the following goals:  select projects with regional impact, county impact, local impact  select projects with more value for money  select projects more efficiently (less financial and human resources used and more projects and with higher quality of selection)  How to best classify projects for different selection models based on:  the nature of projects: projects for infrastructure investment that involve construction and projects of service contracts  scope of projects (small, medium, or large project size)  location of projects (county, inter-county, city/town, inter-city/town)  type of beneficiaries (region, county, city/town, company, individual)  What do you think about the following tentative options of project categories:  Category A: large projects of regional/national importance that will be drawn from national/regional strategies, and which will be financed without prior selection  Category B: medium-sized projects that will enter a pool, will be analyzed by a committee, and proposed for financing based on expected impact  Category C: small projects that will be selected based on the FIFO rule, and will benefit from simpler and more straight-forward procedures.  How would different selection models work for the different categories of projects?  How should funding be allocated for different categories of projects to different regions?  What categories of project should be selected competitively or not competitively? How to do so?  What categories of project should have definite or indefinite deadlines for project submission?  What agencies should be involved in project concept development and project appraisal management for different categories of projects?  What categories of projects should be screened before going through appraisal?  What agencies should have the authority/delegated authority to evaluate project application?  What agencies should have the authority/delegated authority to select projects after the evaluation of application?  What agencies should have the authority/delegated authority to approve project selection?  For the success of these proposed selection models, what are required conditions for success in terms of: 243  Procedures  Financial resources  Human resources  IT  Political support PROJECT EVALUATION PROCESS AND CRITERIA  Evaluation Process  Relevance of steps: What do you consider as relevant, irrelevant, or missing regarding the steps in the evaluation process? What should be modified or simplified in the next ROP, for different categories of projects?  Relevance of step sequencing: What do you consider as relevant or irrelevant regarding the sequencing of the evaluation process? The quality of project evaluation depends on three main elements: evaluation criteria, evaluators, inputs from project application.  Evaluation Criteria  What criteria are relevant, irrelevant, or missing for evaluating different categories of projects? What criteria should be removed, kept, modified for the next ROP?  Objectivity: Is there any room for subjectivity in evaluation? What modifications in criteria and methodology to assign scores should there be to reduce the subjectivity in evaluation?  Validity: Are evaluators required to check the assumptions underlying the appraisal results and claims in project application before evaluating the results and claims?  Clarity: Is the guidance for evaluators sufficiently clear and inclusive, or is it subject to individual interpretation?  Evaluators:  What advantages and disadvantages does the current model for selection and appointment of evaluators have? What agencies should have the authority to select and appoint evaluators for different categories of projects in the next ROP?  How does the current selection and appointment of evaluators affect the independence, quality, and responsibility of evaluators?  How relevant are the evaluator expertise and qualifications to the evaluation work?  How can the performance of evaluators be improved in the next ROP?  Inputs from project application  Which requirements in the application form are relevant, irrelevant, missing? What is the RDA’s role in the preparation of project application?  In practice, what is the quality and comprehensiveness of information provided by applicants in the application form, on average? How can the quality of data inputs be improved?  Is technical guidance on feasibility study sufficiently clear? What requirements for feasibility studies are appropriate/not appropriate for different categories of projects? What is the quality and comprehensiveness of information provided by applicants in the feasibility studies, on average? How can the quality of data inputs be improved? Do beneficiaries have adequate financial and technical capacity to manage project appraisals and to provide good quality feasibility studies?  How is the cost-benefit analysis tool used by beneficiaries? For which type of projects should this analysis be required?  How is cost effectiveness analysis tool used by beneficiaries? For which type of projects should this analysis be required?  How can the quality of data inputs from feasibility studies be improved? Who should manage feasibility studies for different categories of projects? 244 Annex 10. Selected Case Studies – Project Selection Models in 8 EU Member Countries Germany – Nordrhein-Westfalen (NRW) ROP: Competitive Calls in Business Infrastructure60 The German Land of Nordrhein-Westfalen has started using competitive calls on a large scale in the 2007-13 programming period. This case study will provide insights on calls for different types of interventions (R&D, infrastructure, other), but as for selection criteria it will focus on a specific call in the field of business infrastructure. The selection of project outlines by a jury (involving weighting and scoring) is followed by the submission of full applications. The approach has been successful and will be replicated in 2014-20, but there is scope for improvement, for example regarding target-orientation. Context In Germany, as a federal country, Structural Funds’ management is the responsibility of the Länder. In Nordrhein-Westfalen, the regional ERDF OP is managed by the Land Government and more specifically by the Ministry for Economy, Energy, Industry, SMEs, and the Craft Sector. The following table shows the funding allocations by priority for the 2007-13 programming period. Nordrhein-Westfalen ERDF RCE Programme 2007-13 – Program Allocations (€) National Public National Private Priority Axis EU Investment Total Contribution Contribution 1) Strengthening the 121.3 101.3 20 242.6 entrepreneurial basis 2) Innovation and 881.5 641.5 240 1,762.9 knowledge-based economy 3) Sustainable urban and 268.2 258.2 10 536.3 regional development 4) Technical Assistance 12.5 12.5 0 25 Total 1,283.4 1,013.4 270 2,566.9 Source: Ministerium für Wirtschaft, Energie, Bauen, Wohnen und Verkehr des Landes Nordrhein-Westfalen (2012) Operationelles Programm (EFRE) für das Ziel "Regionale Wettbewerbsfähigkeit und Beschäftigung" für Nordrhein-Westfalen, Version of 05.07.2012, p.167 The majority of funding is allocated through competitive calls. Funding under Priority 1 is allocated through existing domestic funding instruments (e.g., aid or loan schemes for SMEs), where the approach to project generation, appraisal, and selection is generally defined in the legal documents relating to these instruments. Most remaining funding is allocated via competitive calls for tender (see detailed description below). In exceptional cases, some funding under Priority 3 (notably for infrastructure in lagging areas) may be allocated on the basis of decisions by the State Secretaries Committee, generally in the context of well-established sub-regional strategies. Overall, the implementation of the OP is progressing very well. By the end of 2012, ERDF commitments were at 95.1 percent and payments at 47.3 percent. 60 This case study is largely based on the findings of the following evaluation report: Deloitte & Touche GmbH (2012): Evaluierung von wettbewerblichen Auswahlverfahren des Ziel 2-Programms 2007-2013, Ministry for Business, Energy, Industry, SMEs & the Craft Sector of the Land Nordrhein-Westfalen, Final Report. 245 Project Selection Model and Procedures – Competitive Calls Competitive calls have been used on a large scale for the first time in 2007-13 in order to select projects with the highest impact. Although still fairly novel and experimental, the approach covers virtually all of Priority 2 (Innovation and knowledge-based economy) and most of Priority 3 (Sustainable urban and regional development). The intention is multi-fold: to enhance project contribution to regional strengths and improve efficiency and effectiveness of support; to enhance the objectivity and transparency of the selection process, as well as project quality and the mobilization of target groups. The calls cover several project types and involve significant amounts of funding. Calls have been launched to support the Land’s cluster strategy on horizontal themes (e.g., tourism or knowledge- intensive services), and for regional networks or clusters. By February 2012, support had been granted to 1,207 R&D projects, 176 infrastructure projects, and 114 other projects. Overall, ERDF funding of €439 million had been committed through competitive calls. Domestic co -financing amounted to €195 million. The overall amount of eligible project funding was €923 million (i.e., aid rate of approximately 69 %). By February 2012, payments of around €198 million were made (i.e., roughly 31 % of commitments). Following the development of the calls, projects are selected in a multi-stage process: 1. Development of calls by different Land sectoral Ministries and the managing authority further checks whether they respond to the main program objectives; the calls are published on internet sites of individual Land Ministries and on the Land government website, as well as on websites of other relevant support bodies; calls are publicized via information events. 2. Pre-scoring on eligibility of project outlines (application form and project description, 12 pages maximum) to identify issues that may question project eligibility or require detailed checks (non-binding). 3. Appraisal of project outlines by a jury of independent experts: eligibility check, scoring of project outlines in line with call criteria (three categories with call-specific sub-criteria) using indicators, weighing, and scoring. 4. Selection of project outlines (around six months after publication of call) by the jury according to the highest impact relative to program and call-specific objectives – if projects have the same rating, the one with the best contribution to horizontal objectives is preferred. 5. Submission of full application: project description – including work/time/expenditure and financing plan and details regarding indicators; information on applicant (profile, company data, extract from trade register); proofs of solvency and business plan (annual balance sheets, feasibility study, own contribution); statements regarding de-minimis rule, non-existence of prior project start, checked regarding completeness, consistency and eligibility of expenditure; specific focus on solvency of applicant and business plan (notably own contribution); decision on aid rate (around six months after selection of project outlines). The duration of the different phases varied depending on the project type. While R&D projects take longest overall, this is partly due to considerable variation and a high number of outliers. Duration of the different phases of the competitive calls by project type (in days) Project type Call phase Application & Selection Overall duration R&D 180 426 606 Infrastructure 186 395 581 Other 158 356 514 246 During the 2007-13 period, efforts have been made to improve the implementation of the call approach. This included standardization and simplification efforts, moves to enhance the independence of jury members as well as the transparency of its composition, and a greater focus on eligibility during the pre-scoring phase. Selection Criteria and Indicators The focus of this case study is on Measure 2.1: Support for business innovation, clusters and network development, including business infrastructure and, more specifically, on calls on electronic mobility (ElektroMobil.NRW). In this context, support is provided to R&D, infrastructure, and investment projects, over three years, for a variety of themes (energy storage, vehicle, infrastructure and networks, framework conditions, norms and standards). Regarding infrastructure projects, the focus is on: (a) pooling products, services, or process innovations of several businesses to offer new integrated solutions; and (b) projects required to enhance the automotive cluster (e.g., brownfield regeneration, competence and operator centers, investment planning and preparation, promotion measures, concepts and expertise). Under Measure 2.1, a total of €628.4 million has been committed in the context of competitive calls between 2010 and 2012, including €55.7 million under the call on electronic mobility. The underlying purpose of the selection criteria is to allow the appraisal of projects through a transparent and traceable process, in line with the principles and targets set out in the Operational Programme. They should thus specify the statements made regarding individual measures rather than narrowing the room to maneuver. Detailed approaches to project generation, appraisal, and selection are defined in the relevant call for tender documents. On average, assessment is based on ten to twenty criteria. There are a number of standard requirements that apply to all projects:  Compliance with core strategic program goals relating to the improvement of competitiveness, innovation and job creation, and horizontal targets in terms of sustainable development and equal opportunities.  Economic and sectoral criteria: reliability and capacity of project applicant; volume and economic adequacy of project cost; secured financing; compliance with principles of efficient budget management; sectoral suitability.  Time-related criterion: The project plan has to specify that time requirements will be adhered to and that the project applicant is able to ensure the timely delivery of the final proof of funds ’ expenditure. Cost-benefit analyses are seen to be useful in determining whether applications qualify. They are therefore required for selected, strategically important projects, even if they cost less than €50 million, as well as for infrastructure investments of over €10 million. There are different categories of applicable criteria, each broken down at the measure / sub-measure level. For Measure 2.1 Support for business innovation, clusters, and network development and Measure 2.2 Business Technology and Research Infrastructure, these are the following:  Core strategic program goals: Measure-specific sub-criteria are defined under each of the following aspects: o Competitiveness – improvement of business opportunities to position themselves on supra-regional and international markets through investment in new products and processes, which lead to increased productivity and thus sales potential; the application 247 needs to specify the contribution of the project via quantitative (e.g., increased exports) or qualitative statements (targeting local or regional markets is in this respect insufficient); o Innovation – understood as the application of new ideas on the market, referring to products, production processes, and organizational procedures (i.e., wide definition of innovation, including organizational, logistical, financial, HR, marketing and design- oriented innovations, and their application on the market); proof of respective contributions is required and marketability needs have to be elaborated upon; o Employment – projects need to contribute directly to creating new jobs and to safeguarding existing jobs; if a project does not contribute to this goal in the short term, a medium to long-term contribution needs to be specified; employment effects need to be quantified and (as far as possible) differentiated based on gender.  Strategic target of Priority 2: Promotion of innovation and development of a knowledge-based economy  Horizontal criteria: o Equal opportunities – contribution to reduce gender inequalities; if this does not apply, applications have to describe gender-sensitive approaches (e.g., gender-differentiated procedures during tests and field studies; ways to build in gender competences in setting up networks; acknowledgement of needs of those using the new products / services); if women had / will have a decisive role in project development/ implementation; and in how far funding benefits those business segments that have an above average share of women in leading positions / in engineering and natural science areas (alternatively, applicants can refer to existing measures that specifically support the accommodation of work and family life); o Sustainable development – contribution to greater resource and energy efficiency, notably regarding synergies between environmental protection and competitiveness.  Measure-specific criteria: Promotion of the value creation chain; transfer of knowledge and know-how for economic ends.  Call-specific criteria: Enhancing the value creation chain; knowledge and know-how transfer that benefits the economy; state-of-the-art technology and project motivation; cooperation with scientific institutions or other businesses, inter-disciplinary R&D; future and market potential of project results; macro-economic benefits; own contributions of businesses and sustainability of cooperation beyond support period.  Specific criteria for infrastructure concepts: Applications need to position themselves regarding o the concrete need for the infrastructure project; o analyses of strengths and weaknesses and potential in the framework of the cluster structure and the support focus (automotive); o innovative character / impact on the innovativeness and performance of businesses involved in the cluster; o impact on strengthening the value creation chain; o proof of integration of the project in a regional action strategy in line with regional priorities; and o relationship to existing support structures (inter-community, interregional, or cross- border support structures receive preferential treatment). Applications also need to be in line with the regional guidelines on infrastructure support.61 61 Infrastrukturrichtlinien des Regionalen Wirtschaftsfoerderungsprogramms NRW (08.01.2009), see http://www.foerderdatenbank.de/Foerder- 248 Project outlines need to contain descriptions of: a. targets, technical requirements, energy management context, connection of infrastructure projects with electro-mobility; b. new application areas to be developed; c. innovation and comparison with state-of-the-art / existing structures at regional / national / international level; d. technical risks; e. for infrastructure projects pilot initiatives and scientific studies, the use of electric vehicles; f. economic outlook; g. expected specific technological difficulties; h. industrial and economic know-how in the project-specific area in NRW; i. preparatory works that are not part of the application; and j. planned steps / work packages. Three sets of indicators are used and respectively relate to the project’s contribution to the three sets of goals: Weighting of indicators with respect to different types of goals Goal Weighting Strategic goals (competitiveness, innovation, 40% - the three aspects are weighted in line with employment) the orientation of the call (10-20% each) Horizontal goals (equal opportunities, 10% - up to 5% for each horizontal goal sustainable development) Call-specific goals 50% For the individual criteria, scores are attributed between 1 and 5, as follows: 1 = no contribution 2 = limited contribution 3 = satisfactory contribution 4 = high contribution 5 = outstanding contribution By multiplying the weighting with the score for each criterion, a total score is determined and compared to the other project applications. In order to illustrate the approach, the following table shows an example of how projects are assessed under Measure 2.1. Project assessment under Measure 2.1 – Calculation Example Criteria Calculation example Breadth (%) Weight (%) Score Value 1. Overarching target and strategic program 40 150 objectives 1.1 Competitiveness (10-20%) 10-20 15 60 Increasing business productivity (directly and 5-10 7.5 4 30 indirectly) DB/Navigation/Foerderrecherche/suche.html?get=4aa561e46fff16fb87d819d09c769842;views;document&doc=10612 (accessed on 12.12.2013). 249 Supra-regional approach 5-10 7.5 4 30 1.2 Innovativeness (10-20%) 10-20 15 60 Novelty (technical, organizational) 5-10 7.5 4 30 Marketability, economic usability 5-10 7.5 4 30 Employment (10-20%) 10-20 10 3 30 2. Horizontal program objectives 10 25 Equal opportunities 5 5 2 10 Environment 5 5 3 15 3. Call-specific criteria 50 165 3.1 Measure-specific criteria Promotion of the value creation chain 5-10 7.5 4 30 Knowledge and know-how transfer benefiting 5-10 7.5 4 30 the economy 3.2 Call-specific criteria e.g., degree of cooperation, quality of the team variable 7.5 3 22.5 of scientists e.g., integration in regional development variable 7.5 4 30 strategy e.g., quality of SWOT analysis/proof of need variable 7.5 3 22.5 e.g., international visibility variable 7.5 2 15 e.g., sustainability of cooperation after support variable 5 3 15 has ended Total value (example) 100 340 However, the criteria included in call-specific documentation are not accompanied by information on weighting or maximum scores and it is difficult to see how the assessment works in practice (e.g., whether it is cumulative or alternative). In order to monitor program progress at measure level, the managing authority works with standardized (sub-)measure-specific monitoring fiches, which are filled in for each project. For Measure 2.2, monitoring indicators include the following: - Infrastructure capacity (number of training places, room for SME); - Cooperation with business partners (number of network projects with businesses, number of new products and processes); - Number of businesses supported through infrastructure (focus on new creations and technology-intensive sectors); - Job creation (focus on technology-intensive jobs and jobs for women); - Relationship with horizontal objectives; - Relationship with demographic changes; and - Environmental indicators (change in resource and energy usage, change in CO2 emissions, contribution to soil sealing). Performance Assessment The competitive call approach has proven efficient overall, but there have been some difficulties with project eligibility, which have hampered the post-selection stage. Between 2007 and 2010, 54 calls were implemented under 28 themes. Participation was sufficiently high (at least 40 submissions per call) 250 and the success rate for applicants was at approximately 32 percent. Funding allocations were exhausted for most calls. Around 47 percent of committed funding was allocated to the R&D sector, around 39 percent to the private sector (roughly 15 percent for large firms and 24 percent for SME), and around 14 percent to the public sector. There has been an issue over the eligibility of project applications across competitive calls in different thematic areas. This concerned almost one third of those successful applicants who later withdrew their application linked to the missing proof of their own contribution (while eligibility checks are carried out based on the submitted project outlines, the statements and documents allowing for a final assessment are not required at this stage, since it is considered too onerous to provide the complete documentation). Efforts are made to reduce the work related to non-eligible project applications. The vast majority of projects selected through competitive calls are set to achieve their targets, but some think that the pursuit of regional objectives should be strengthened. According to project applicants, almost 87 percent of projects will reach the targets set out in the project application, with only 4 percent stating that the targets will not be achieved (the remaining 9 percent were unable to comment, since they had just started implementing the project). Results were particularly positive for projects in the field of product / process development and resource / process efficiency.62 However, some experts believe that a stronger use of competitive calls to pursue regional objectives (i.e., related to the regional innovation and cluster strategy) is necessary. Others think that competitive calls do not provide a suitable framework to pursue specific themes in a strategic and long-term approach. The targeting of regional strengths is not explicit enough, since sub-regional and horizontal themes are supported as well. It is also thought that the jury is not in a position to assess project contributions to strategic / political targets due to its specialist composition and the fact that evaluators do not come from the region to ensure their impartiality. Some think that the call themes are not consolidated/ coordinated due to sectoral compartmentalization, while others suggest a wider / more open definition of call themes (however, this may hamper the comparability of project outlines and may require a larger jury in order to cover all areas). With the competitive call approach it has been possible to support innovation potentials throughout the region. While there has been a concentration on university cities / towns, this is in line with the aim to support knowledge and technology transfer as long as businesses receive sufficient assistance. Project networking partnerships initiated through calls are stable – projects are sustained after support ends / networks are successful in applying for follow-on projects. Main Lessons Competitive calls are found to be suitable for different types of interventions and target groups. However, a majority of experts view the model as not universally suitable and suggest implementing alternative / complementary selection models in parallel in order to allow submissions for specific objectives, project types or target groups and to be able to react to new developments. Regarding different project types, many experts view competitive calls as a good model to be used in the field of innovation (especially applied research); regarding infrastructure support, this statement needs to be relativized slightly as different areas of infrastructure support (R&D, environment, investment) show specific characteristics which need to be taken into account (see table below). 62 Please note that no impact assessment has been carried out at the time of writing, so this is solely based on funding recipients’ views. 251 Suitability of Competitive Calls for Different Areas of Infrastructure Support Area of infrastructure Supported Suitability of Limitations support interventions competitive calls R&D Measures linked to Good – linked to - R&D activities general suitability for R&D / innovation projects Environment E.g., energy efficiency Calls can provide A single deadline per measures, investments incentives and allow year and a lengthy to go beyond legal the selection of (two-step) procedure requirements projects with the are not suitable since highest impact in a investment decisions situation of limited are usually short-term funding Investment / regional Measures to balance Competitive Suitability depends on development regional weaknesses / procedure based on existence of a disadvantages performance less common point of suitable; still, reference for all comparison of projects applications can allow selecting projects with best cost-benefit ratio Competitive calls are also seen to be generally suitable for different target groups, although there are limitations for SMEs that are not active in R&D due to the duration of procedures and the related burden. Moreover, competitive calls can be used for both, injecting one-off impulses and pursuing longer term strategies. However, there is scope for improvement regarding the implementation of objectives – a clearer strategic orientation and back coupling with political targets are necessary. Regarding the usability of selection criteria, it has been noted by some that the scoring procedure / table is too complex. It appears to have been mainly used at the pre-scoring stage by the administration, while the jury often uses it to a limited extent or in a simplified way. There also seems to be a trade-off between the level of standardization and the specification of criteria. On a related note, there are calls for reducing the number of criteria while making them more concrete and establishing a stronger link with longer term priorities and political objectives – however, it may be difficult to achieve both at the same time. In addition, applicants struggle with providing information on some criteria, e.g., the innovative character of infrastructure projects. Several examples of good practice and conditions for successful implementation can be identified:  Competitive procedures rely on the existence of a common point of reference to allow for the definition of criteria to compare and select projects; if a support of the ‘best’ is desired, then it needs to be ensured that sufficient strengths exist in the targeted field to allow for qualitative comparison;  Project selection is qualified as transparent, traceable and fair, the contribution by the jury is judged to be positive (also regarding the effectiveness of selection), although this could be optimized; 252  The two-stage procedure works well, although there are some limitations for certain target groups and intervention targets (a one-stage procedure may be more attractive for SMEs);  Project quality is high – it is easier to select the ‘best’ projects – provided that there is a critical mass of high-quality submissions; further improvements are possible regarding the advance planning of calls;  Applicants are motivated through strict deadlines to submit well-structured and well-presented project outlines, on condition that there is sufficient lead time;  There has been high mobilization of (new) target groups thanks to the clear thematic orientation, information events, targeted identification of beneficiary groups, and independent project selection by a jury (only the last point is directly linked to the competitive model, the three other aspects can also be found in alternative approaches);  The model encourages sustainable networking between actors – as at February 2012, this concerned over 47% of projects (e.g., business and science); however, in some cases, the requirement to constitute networks can cause delays; and  It has a two-fold function as selection and award (i.e., prize) procedure – an award can be used for image and marketing purposes. At the same time, there are disadvantages to the use of competitive calls, mainly regarding constraints on flexibility and administrative burden:  Rigid deadlines;  Limited interaction between applicants and call administrators in order to guarantee fair treatment;  Lengthy procedures: delays occur mainly in the application and approval phases related to process organization, the number / interplay of involved actors, and the complexity of the legal basis;  High burden during application and approval phases (eligibility checks), as well as during the implementation phase (provision of proofs, intensity of checks). There is therefore scope for improvement, notably regarding the use of criteria, the frequency of deadlines, and transparency. The use of criteria as strategic tool should be enhanced by integrating their definition in the planning process and enforcing their application by the jury. In terms of deadlines, their frequency is not ideal for certain target groups, especially regarding the limited scope for advance- planning (dates and content); more frequent deadlines may attract a greater participation by SME. Finally, transparency is insufficient regarding duties and efforts involved in the project application and implementation phases; this can be linked to the limited suitability of the legal basis, insufficient quality of information materials and uneven treatment due to the high number of involved actors. The 2012 evaluation of the competitive calls approach advances numerous recommendations, which are listed here (the most relevant ones highlighted): 1. Institutionalization of top-down planning of competitive calls (involving, for example, a reduction in the number of themes and calls; a stronger link with political objectives through more specific selection criteria; the strengthening of cross-sectoral initiation, planning, and implementation of calls); 2. Advance-planning of call themes over a timer period of approximately three years and increase in number of deadlines per call; 3. Continuation of competitive calls as project selection model – decision on suitability for individual intervention targets supported by a decision matrix; 253 4. Strengthening of requirements for the application of call-specific selection criteria – mandatory definition of call-specific criteria as part of planning process – and simplification of scoring model – reduction of complexity (criteria and gradation of assessment) and enforcement of application by jury (reasons: call-specific criteria are not evenly applied to the concrete targets of the call, there is scope for an enhanced use of the criteria to take account of political aims and to steer content and quality; the scoring sheet is often not used by the jury (due to its complexity) and it cannot be ascertained that individual criteria are really taken into account in the assessment using appropriate weighting and that the selection process is documented in its entirety); 5. Reduction of number of actors per call and across calls; 6. Optimization of jury composition and procedures63 - it is recommended to compose the jury so that it covers all required (content-related) competences to assess applications, i.e., in addition to scientific-technological expertise, economic and regional know-how should also be present (experts from NRW should not be excluded in principle); in order to exclude bias, potentially biased jury members should be excluded from the vote and the highest and lowest votes should not be taken into account in the assessment; jury members need to receive an induction into regional objectives and the use of scoring methods; the ‘second opinion’ principle should be applied across the board; only eligible project outlines should be presented to the jury; 7. Enhancement of knowledge and quality management; 8. Optimization of interfaces in the payment process; 9. Enabling support on cost basis; 10. Introduction of flat rates for running costs and staff costs; 11. Simplification of requirements related to public procurement law applied at the level of the Land – there is a need to reduce the current burden on beneficiaries (especially SME) and administrators, e.g., by increasing minimum thresholds for ERDF projects; by limiting the number of required offers to three (to be applied to projects with an aid rate of over 50 percent); by invalidating procurement requirements for projects above €100,000 (the latter would lead to considerable time savings – it can be assumed that beneficiaries contributing over 50 percent of project costs make own arrangements to ensure cost effectiveness); 12. Document and spot checks based on electronic records; 13. Document and spot checks based on samples; 14. Introduction of an overarching ERDF guideline; 15. Introduction of binding deadlines for administrators and beneficiaries; 16. Revision of call documentation (including support handbook); 17. Introduction of an electronic system to improve transparency and simplify processes – enables data exchange, provides a communication platform, and reduces administrative effort (e.g., submission of project outlines, validation checks regarding completeness and correctness of statements and documents, checks of electronic receipts, overview of drawn-down versus available funding); 18. Extension of eligibility checks during appraisal phase – extension of the appraisal checklist regarding statements on the financing plan (though no requirement of proof) to allow for more intense eligibility checks and to avoid disappointment (i.e., rejection of application) at a later stage 19. Intensification of content-related project support following approval 63 The evaluation contains further details on how processes at the level of the jury could be improved (pp. 264-266). 254 In conclusion, competitive calls have proved of value, although their target-orientation can be further enhanced. Looking ahead, the unification and structuring of planning processes and a consistent targeting on intervention aims and the regional innovation strategy are required. There is considerable need for improvement regarding the actual selection and implementation procedures (application and approval phases, as well as the implementation phase). The fact that competitive calls will also be used extensively in 2014-20 shows that the approach is seen to work well. Poland - ERDF Regional Operational Programme Śląskie 2007-13: Pre-selection through Sub-regional Development Programs – Urban Revitalization In the Polish region of Śląskie, part of the regional ERDF OP has been successfully delivered through projects pre-selected in the framework of Sub-regional Development Programs. The projects underwent initial checks at the program development stage and were then assessed in the same way as projects identified through competitive calls, drawing on expert advice (however, not involving any weighting and scoring). The present case study provides details on Priority 6 Urban Revitalization. Despite slightly lower than average progress in terms of funding absorption, the programs were more successful than other selection methods, especially in identifying suitable projects in response to complex challenges. Context In Poland, the management of regional ERDF OPs is decentralized. In the Śląskie Voivodship, the Marshal’s Office is in charge of managing the 2007-13 Convergence Regional Operational Programme (ROP). The following table gives an overview of the funding allocated to the ten priorities supported by the program. Śląskie ERDF Convergence Programme 2007-13 – Program Allocations (€) National National Public Priority Axis EU Investment Private Total Contribution Contribution 1) R&D, innovation, 296 238 553 52 277 392 - 348 515 945 entrepreneurship 2) Information society 150 000 000 26 470 588 - 176 470 588 3) Tourism 110 420 000 19 485 882 - 129 905 882 4) Culture 53 274 150 9 401 321 - 62 675 471 5) Environment 180 678 600 31 884 459 - 212 563 059 6) Sustainable urban 312 802 445 55 200 431 - 368 002 876 development 7) Transport 426 327 555 84 244 367 - 510 571 922 8) Education 82 480 000 14 555 294 - 97 035 294 infrastructure 9) Health and recreation 57 759 000 10 192 765 - 67 951 765 10) Technical assistance 43 000 000 - - 43 000 000 Total 1 712 980 303 303 712 499 - 2 016 692 802 Source: Regionalny Program Operacyjny Województwa Śląskiego na lata 2007 – 2013 (version 6/8/2010). Program progress is satisfactory regarding both commitments and certified expenditures. At the end of 2012, total grant commitments amounted to €1,556.24 million from the European Union, which constituted 88.41% of the allocation for the years of 2007-13. The Certification Authority approved the 255 declarations submitted by the Managing Authority for the amount of €853.64 million, which is co- financed from the European Union and corresponds to 48.49% of the allocation. The ROP uses three procedures to select operations, with most funding allocated through competitive calls. According to the December 2006 Act on the Principles of Development Policy, the majority (60%) of EU financing is allocated through the competitive mode; around 28% of total EU financing is allocated through key projects, which are defined as individual projects; and 12% through Sub-regional Development Programs (PRS), which are understood as systemic projects and are the focus of the present case study. Project Selection Model and Procedures – Pre-selection through Sub-Regional Development Programs (PRS) Sub-regional Development Programs were introduced to enhance stakeholder participation, in response to the demand from cities and municipalities for strong participation in the OP implementation process. The Śląskie region has a polycentric structure with four major agglomerations. In order to involve municipal and district authorities in the process of implementing EU funds, four sub- regional platforms were created around these bases: Central (Centralny Śląski), South (Bielsko-bialski), North (Częstochowski) and West (Rybnicko-jastrzębski). Each sub-region developed partnership arrangements involving municipalities, cities, knowledge institutions, and the private sector. The Sub-regional Development Programs cover a variety of themes regarding different types of infrastructure, the environment, regional growth, and urban renewal. They account for €246.48 million (around 12% of the total EU allocation to the ROP) and include the following measures under seven out of the ten program priorities:  Priority 2.1 Infrastructure for information society  Priority 2.2 Development of e-services  Priority 3.2.2 Tourist infrastructure  Priority 4.1 Cultural infrastructure  Priority 5.2 Waste management  Priority 5.3 Clean air and renewable energy  Priority 6.1 Strengthening regional centers of growth  Priority 6.2.1 Revitalization of large towns  Priority 6.2.2 Revitalization of small towns  Priority 7.1.1 Modernization and development of key road networks  Priority 7.1.2 Modernization and development of infrastructure completing key road networks The Sub-regional Development Programs are developed around a list of projects agreed among key stakeholders. They are created in consultation with the local authorities of the four sub-regions. The basic element is an agreed list of projects that have been identified as particularly important for the development of the sub-region and will improve the life of the inhabitants in the territory (by the end of 2012, this involved 214 projects organized into four Sub-regional Programs). The implementation of projects contained in the Sub-regional Development Programs was preceded by four preparatory stages: 1. Initial preparation stage (first half of 2007) – Sub-regions developed a list of potential projects in line with ROP measures, based on agreements among relevant stakeholders. For this, the managing authority provided a project fiche and guidance. 256 2. Program preparation (May 2007 and February 2008) – Sub-regions worked on PRS and submitted them to the managing authority. Each PRS contains:  an analysis of the socio-economic context of the sub-region;  a set of strategic priorities;  a list of the main projects to be implemented;  a financial plan;  a description of the implementation system; and  a description of the consultation process. On this basis, a cooperation agreement between the managing authority and the sub-regions (i.e., municipalities representing at least 75% of the inhabitants of the sub-region) was signed. 3. Assessment of PRS and initial check of projects for eligibility for co-financing (February-May 2008) - This stage involved a formal and meritocratic assessment of the PRS (see further below) and an initial assessment of project eligibility (based on application forms and project fiches). For the PRS to be approved, all criteria had to be met. There was also a quality assessment of the PRS, based on an analysis of the correctness of the individual program components (primarily to meet the requirements set out in the ROP Guidelines and regarding internal consistency between diagnosis – objectives – projects), and of the feasibility and compatibility of the PRS with regional programming documents (the ROP and the Silesian Regional Development Strategy). On the basis of this stage, an agreement for co-financing of the PRS was made between the managing authority and representatives of the sub-region. 4. Implementation stage - This involved the detailed preparation of full project applications (completed by December 2009), the assessment of PRS projects (completed by June 2010) and the signing of project co-financing agreements (completed by December 2010). The projects were assessed according to the selection criteria used for the different ROP priorities and measures, i.e., at this stage the same approach and criteria as for competitive calls were used for PRS projects. Selection Criteria and Indicators – Urban Revitalization Urban revitalization (Priority 6) is an important priority and receives around one fifth of funding under the PRS (€47.29 million). Śląskie is the most urbanized region in Poland (78.6% of people live in urban areas). In the past, industrial plants often invested in urban centers, but more recently there have been concerns over the degradation of urban areas linked to substantial industrial restructuring. The main objective of interventions under this heading is to restore degraded sites of former industrial centers, cities, and also districts subject to the processes of suburbanization, introducing new economic, social, recreational, and educational facets. The revitalization of degraded areas will also contribute to improving the living conditions and increase the attractiveness of the region. Project selection involved two stages: one was specific to the PRS, while the other was the same as for mainstream projects. Specific formal and meritocratic criteria were used at the initial stage of the Sub- Regional Development Programs in order to ensure that the projects had strong links to the sub-regional territory and that the programs as a whole were strategically integrated with the ROP (1). Subsequently, PRS projects were subject to assessment in parallel with other delivery modes (2). 1. The first stage was formal assessment of the PRS. It consisted of verifying whether the PRS contained all the necessary elements and the required documentation in accordance with the Guidelines set out in the ROP. Assessment was completed on the basis of a simple “yes/no,” i.e., “meets - does not meet requirement.” 257 Formal Criteria used to assess Sub-regional Development Programs Criteria Yes No Cooperation agreement 1 Agreement negotiated in accordance with the guidance 2 Agreement signed in accordance with the guidance 3 Provisions of the cooperation agreement are consistent with the relevant provisions of the PRS 4 Fiches for all projects are attached to the PRS 5 Pre-feasibility studies are attached to the project list Diagnosis 6 Analysis of the sub-region against the regional context is included. 7 Analysis of the ‘problem areas’ where the PRS will be implemented is presented 8 Key barriers to socio-economic development in PRS areas are identified Strategy 9 The main aims of the PRS are identified 10 The specific aims of the PRS are identified 11 Indicators are set out for aims List of projects 12 Project title 13 The principles and aims of the PRS that the project will contribute to 14 The Community contribution to the project 15 The Priority/Measure in the Regional Operational Programme under which the project will be implemented Financial plan (the program contains) 16 Table of financial obligations for the PRS divided according to year and measures 16.1 This part of the table is correct in terms of accounting 16.2 The sum of the amounts for activities and program are consistent with the allocation as defined by the ROP 17 The payment table for the PRS is divided up according to years and measures 17.1 This part of the table is correct in terms of accounting 17.2 The sum of the amounts for activities and program are consistent with the PRS allocation as defined by the ROP 18 The financial table for the PRS shows the amount of the total financial appropriation of the Community and national contributions and indicators of the contribution of funds by measure 18.1 This part of the table is correct in terms of accounting 18.2 The sum of the amounts for activities and the PRS are consistent with the PRS allocation as defined by the ROP 258 Management system 19 The project selection system is described 20 The system for PRS implementation is described 21 The monitoring system is described Public consultation 22 Information on public consultation is presented 23 The section contains the main conclusions of the consultations. Source: Urzad Marszalkowski (2010) Regionalny Program Operacyjny Województwa Śląskiego na lata 2007 – 2013 - Kryteria oceny Programów Rozwoju Subregionów This was followed by a ‘quality assessment’ of the PRS based on an analysis of:  correctness of the individual components of the PRS primarily to meet the requirements set out in the guidelines for non-competitive calls in the ROP;  internal consistency between diagnosis - objectives – projects; and  feasibility and compatibility of the PRS in relation to regional programming documents (ROP and the Regional Development Strategy). The assessment of the individual elements of the PRS was based on criteria in the form of statements that were assessed on a five-point scale '0 -4' where '0' is a complete failure to meet the criterion and '4' refers to full compliance with the criterion. Quality Criteria used to assess the Sub-regional Development Programs Reason Criteria Score (optional) Diagnosis 1 Specific barriers to development are consistent with the rest of the content of the diagnosis 2 The scope and content of the analysis enable the identification of problem areas Strategy 3 Specific objectives are directly linked to support areas identified in the PRS 4 Specific objectives pursue the main goal of the PRS 5 PRS aims are in agreement with the principles and objectives of the Regional Development Strategy 6 PRS aims are in agreement with the aims of the ROP 7 The indicators are SMART (specific, measurable, attainable, relevant and time-bound) 8 Indicators describe aims in a complete way: 8.1 Main aim 8.2 Specific aim 1 8.3 Specific aim 2 8.4 … 8.5 List of projects 9 Identified projects realize the aims of the PRS 9.1 Area 1 259 9.2 Area 2 9.3 9.4 9.5 Monitoring system 10 The monitoring system allows for the collection of complete and reliable data Public consultation 11 The consultation covered a sufficient range of sub-regional actors 12 Those consulted had a real influence on the content of the PRS (or substantiated comments were taken into account) Source: Urzad Marszalkowski (2010) Regionalny Program Operacyjny Województwa Śląskiego na lata 2007 – 2013 - Kryteria oceny Programów Rozwoju Subregionów 2. For the next step, PRS projects were assessed in parallel with projects under other ROP implementation modes, i.e., applying the same criteria. In the case of P6 Urban Revitalization, the process included: 1. Formal assessment of PRS projects to verify the completeness and consistency of documentation. Formal assessment was based on the following: Formal Criteria used to assess PRS Projects Criteria Eligibility of the applicant  Type of beneficiary or target group Preparation of the fiche  The fiche is in line with the list of projects attached to the PRS agreement  The fiche was prepared using the relevant form  All required fields have been filled in the fiche  The fiche includes all parties  The fiche was prepared in Polish currency (PLN)  The fiche was signed by an authorized person  The fiche was signed by the applicant  The paper version is compatible with the electronic version The implementation period is in line with the ROP programming period (i.e., up to 2015) The project is in line with the detailed description of ROP priorities Criteria applicable to specific activities / sub-measures, including:  Compliance with the ceiling of the maximum possible match-funding for a given type of project  Requirements regarding minimum / maximum project value  The location of the project is in accordance with the relevant ROP measure/ sub- measure Compliance with the demarcation line for Cohesion policy OPs in Poland (set by the Ministry of Regional Development and including different funding thresholds for sectoral and regional OPs). All attachments are included Source: Urzad Marszalkowski (2013) Załącznik nr 6 do Szczegółowego opisu priorytetów RPO WSL na lata 2007-2013 - Kryteria wyboru projektów 260 2. Quality assessment of PRS projects was carried out based on three sets of criteria shown in the following two tables: general (a), basic essential (b) and priority or measure-specific essential criteria by project type (c). As PRS projects were pre-selected, there was no weighting or scoring involved, but rather a process of checking by experts that gave the opportunity for project applications to be amended (information on weighting is included for information only). General (a) and Basic Essential (b) Criteria used to assess PRS Projects a) General Criteria 1. The appropriateness of technological solutions in the project 2. Appropriateness and feasibility of the level of eligible project costs 3. The feasibility of indicators (whether data can be collected and targets can be achieved) 4. The project guarantees technological neutrality and open access (only to projects involving information society) 5. Properly prepared financial analysis of the project (the validity of the input data) 6. Sustainability of project results b) Basic Essential Criteria 1. The extent to which the project realizes the Priorities, measures or sub-measures, of the ROP, measured in terms of the project's impact on the achievement of indicators of a Priority, measure or sub-measure. 2. The extent to which the project fulfils equal opportunities policy, environmental policy, energy efficiency 3. Extent of complementarity with other projects 4. Extent of organizational readiness of the beneficiary, including staff and technical resources to guarantee the feasibility of the project, organizational stability, experience in project management, feasibility of the schedule of activities within the project 5. The anticipated effectiveness of the project (including additional effect / value of the project, its cost-effectiveness while maintaining high quality-input / output ratio) 6. The anticipated impact of the project in increasing the attractiveness for investment of the region or sub-region while protecting and improving the environment, health, preserving cultural identity and developing territorial cohesion Source: Urzad Marszalkowski (2013) Załącznik nr 6 do Szczegółowego opisu priorytetów RPO WSL na lata 2007-2013 - Kryteria wyboru projektów Priority / Measure-specific Essential Criteria (c) used to assess PRS Projects Measure 6.1 Strengthening Regional Growth Centers Weight (not applied to PRS projects) Project influence on the development of metropolitan functions in regional 3 agglomerations The unique character of the project in a given area 2.5 Character of project effects (local, regional, national, international) 2 Innovativeness of the project 1.5 261 Eliminating disparities in access to public goods for marginalized people 1 Measure 6.2 Revitalization of Degraded Areas in Large and Small Towns  Project type 1: Reconstruction and repair of former military areas  Project type 2: The development of urban areas  Project type 3: Completion and renovation of existing buildings Number of measure-specific aims to be realized in the project (economic, social, 4 cultural, touristic, recreational, educational) Extent to which the project will realize the Local Revitalization Program 2 Innovativeness of the solutions proposed by the project 1.5 Contribution of the project to general accessibility 1.5 Eliminating disparities in access to public goods for marginalized people 1  Project type 4: Complex preparation of an area targeted for economic activity, with the exception of service infrastructure for inhabitants Accessibility of the area for investment 4 Extent to which the project will realize the Local Revitalization Program 2 Project influence on the development of economic infrastructure 3 Location of the project in terms of attracting investment to a given area 1  Project type 5: Creating or rebuilding monitoring systems to increase safety in public areas Functionality of solutions 5 Extent to which the project will realize the Local Revitalization Program 2.5 Joint initiatives (of different actors) for comprehensive protection of the area 2.5 indicated in the project  Project Type 6: Replacing asbestos in apartment blocks Degree of urgency 4.5 Extent to which the project will realize the Local Revitalization Program 2 Number of removed products containing asbestos 3.5 Source: Urzad Marszalkowski (2013) Załącznik nr 6 do Szczegółowego opisu priorytetów RPO WSL na lata 2007-2013 - Kryteria wyboru projektów Sub-regional Development Programs need to include detailed information on the approach taken to monitoring. The following table gives an overview of the indicators and measurements used in the PRS of the Central Śląskie sub-region. Monitoring Indicators used in the PRS of the Central Śląskie Sub-region (Priority 6) Indicator Unit of Indicator Indicator value in Frequency of measurement baseline target year measurement 2013 2015 Output indicators Number of buildings undergoing Number 0 2 2 Annually restoration under at least of the project objectives (e.g., 262 economic, social, cultural, touristic, recreational, educational) Number of buildings rebuilt / Number 0 5 5 Annually renovated under at least two of these objectives Area of buildings rebuilt / m² 0 9 450.88 9 450.88 Annually renovated under at least two of these objectives Number of CCTV devices Number 0 0 0 Annually installed Result indicators Number of people benefitting Number 0 320, 650 320, 650 Annually from the infrastructure support Crime detection rate % 0 Source: PRS Centralny (2011) Programu Rozwoju Subregionu Centralnego Województwa Śląskiego na lata 2007-2013 Performance Assessment Pre-selecting projects under the Sub-regional Development Programs was efficient and allowed for high rates of absorption, albeit slightly lower than the program average. The total number of projects submitted under the PRS by September 2013 was 263, i.e., 97% of the available funding. 247 of these were approved, representing around 92% of the available funding. The number of PRS projects for which co-financing agreements had been signed was 226, amounting to around 80% of the available funding (the average for the program as a whole is 88%). At the outset of the programming period, the managing authority had made it clear that if sub-regional agreements were not reached, then the associated funding would be offered to open, competitive calls. The fact that this has not happened is another proof of the efficiency of the approach. In terms of strategic fit, the PRS allowed for the selection of high quality projects especially through integrated approaches. A 2012 evaluation assessing the impact of the ROP looked at which approach to project generation and selection (i.e., PRS, key projects or competitive calls) was most successful in supporting indicator achievement regarding the relevant ROP priorities.64 For each of the different approaches, the evaluation looked at the percentage of indicators achieved by a specific approach and the percentage of the allocation used by a specific approach. On the basis of this analysis, the evaluation concluded that PRS projects were the most successful. This was particularly the case for certain “network-type” projects developed in response to complex challenges in a given territory. These required integrated approaches based on cooperation and mutual complementarity in fields such as urban revitalization and integrated local transport systems. More specifically, the evaluation concluded that PRS projects are more effective than projects based on competitive calls in the following fields:  Broadband infrastructure;  Infrastructure projects for tourism and recreation areas;  Projects of public cultural infrastructure;  The restoration of degraded areas and the elimination of waste landfills, including dumps for environmental purposes; and  Adaptation of buildings for the purpose of revitalization. 64 PSDB (2012) Wstępna ocena realizacji i efektów Regionalnego Programu Operacyjnego Województwa Śląskiego na lata 2007 - 2013 263 These projects require the cooperation between individual components within a network (e.g., the individual sections of roads or broadband infrastructure), but also engage multiple stakeholders or require multiple actions (e.g., cultural and tourism projects or regeneration). Competitive projects are in turn more effective than the PRS in “linear” rather than “network-based” interventions, for instance relating to stand-alone or one-off interventions to strengthen environmental or transport infrastructure. Main Lessons According to representatives of the managing authority, the effectiveness of PRS projects is a result of the partnership method used during their preparation. Implementation of individual projects is slower and more difficult, and they carry a higher risk of failure. In the context of competitive procedures, beneficiaries tend to set unrealistic indicators and targets in order to receive funding. The PRS creates a comprehensive way to solve a problem existing in a specific area. Therefore, and supported by the results of the above analyses, managing authority representatives expect the PRS to be very important and to receive a higher allocation in the 2014-20 ROP. Leadership of the PRS is vital, and the evaluation suggests that the appointment by local authorities of a separate unit for this task is preferable to one local authority playing a dual beneficiary / leadership role (see also below). Despite the success of the PRS approach, there is also some scope for improvement regarding the following aspects:  Time / labor intensive procedure (including pre-assessment of projects);  Delays in the submission of projects by beneficiaries;  Shifts in timetables for project implementation (concerning 70% of projects);  Resignation from submission, cancellation of contract (e.g., due to changes in rates and costs during the lengthy application procedure);  Lack of regulations (e.g., related to State aid); and  Lack of motivation of beneficiaries (due to guaranteed funding). Looking ahead to the 2014-20 programming period, the following recommendations were made by the 2012 evaluation assessing the impact of the ROP:65  The selection of part of the projects following non-competitive procedures should be continued, because, in comparison to the implementation of competitive projects, EU funds can be spent more effectively. However, taking into account that work involved in selecting non-competitive projects, both on the side of the beneficiaries and the managing authority, lasted many months prior to the submission of the applications and incurred substantial costs, the need for the initial stage of project assessment is questionable.  A mechanism should be created to allow for the selection of more strategic PRS projects, e.g., by increasing the importance of strategic coherence compared to other project assessment criteria.  The model of the “Western” PRS should be disseminated, where the leadership role was taken on by a separate body appointed by local authorities in the sub-region. This solution provides greater objectivity in coordinating the implementation of the PRS and in representing the beneficiaries before the managing authority. Using this model prevents local governments from exercising a dual role as leader of the sub-region and beneficiary.  Reduction of duplication during initial project assessment in the 2014-20 programming period. 65 PSDB (2012) op. cit. 264  The project preparation process should be shortened but better monitored. The guidelines should provide for the preparation of monitoring reports by the leader of the sub-region documenting progress with project preparation (every two months).  The managing authority should specify the demarcation line between PRS and key projects, while ensuring that the non-competitive path retains a complementary character and covers all possible project types. Estonia OP for the Development of Living Environment: Open Calls Most Structural Funds projects in Estonia are selected via open calls and this case study discusses how they are implemented, focusing on the national OP for the Development of Living Environment. More specifically, it looks at support for business infrastructure provided under Priority 4 “Integral and balanced development of regions” (Measure 3). Projects are selected by an Evaluation panel using common and specific criteria organized in three assessment blocks, involving scoring and weighting techniques based on a detailed list of indicators. Open calls, which cover the whole of Estonia, allow choosing the most competitive projects with the highest national impact. However, they have not ensured balanced spatial distribution of the funded projects. Context Estonia has taken a centralized approach to Structural Funds management and implementation based on a centralized administration and National Operational Programmes without explicit regional targeting. The OP for the Development of Living Environment guides the utilization of the ERDF and Cohesion Fund in the area of environmental protection, energy, local and regional development, education and health care and welfare services. It involves c. €1,548.9 million, equal to 45.5 percent of total Structural Funds and Cohesion Fund resources allocated to Estonia for the 2007-13 programming period. The balanced development of regions (Priority 4) is pursued by developing local public services and improving living conditions. The scope for investments is broad ranging from the renovation of schools and constructing sports grounds to building business infrastructure. More than €388 million (ERDF) has been allocated to support for regional development, equal to 11.4 percent of the Structural Funds assistance allocated to Estonia. The Ministry of Finance is the Managing Authority of all three Estonian NOPs. Support for regional development is implemented by the Ministry of the Interior (first level Intermediary Body) and Enterprise Estonia (second level Intermediary Body). Estonia ERDF Convergence NOP Development of Living Environment – Program Allocations (€ million) Priority Axis EU National National Total Investment Public Private Contribution Contribution 1) Development of water and waste 626.3 93.6 16.9 736.86 management infrastructure 2) Development of Infrastructure and 92.0 9.7 0 101.8 support systems for the sustainable use of the environment 3) Development of Energy Sector 28.8 0 44.1 72.9 4) Integrated and balanced 388.6 68.6 0 457.2 development of regions 5) Development of education 212.8 1.2 0 214.0 265 infrastructure 6) Development of health and 169.1 51.5 0 220.6 welfare infrastructure 7) Horizontal Technical Assistance 28.6 5.0 33.6 8) Technical Assistance 2.8 2.8 Total 1,548.9 229.6 61.0 1,839.5 Source: Operational Programme for the Development of the Living Environment. Republic of Estonia. 2007EE161PO002, version May 2010, p. 147. Program progress has been satisfactory. By the end of 2012, commitments were at 87.7% and payments at 54.3%. Most of the priorities have progressed well and as expected (progress of Priority 5 has been only modest) and the financial progress of the OP can be considered good. Several selection models are used for the different priorities. The OP includes eight priority axes that are divided into 16 sub-axes and 39 measures that are carried out through investment plans (17 in total), programs (5), and open applications (15). The measures of (horizontal) technical assistance (2 in total) are carried out as part of drafting the State budget. Apart from Priority 6, all priorities select measures through open applications. The selection of projects through open applications may take place through competitive or FIFO procedures. Priority axes 2, 4 and 6 select projects in the form of investment plans, which are based on the prioritization of requirements by the State or local authorities. In this case, a list of projects or a pipeline to be financed is approved by the Government. Investment plans can also be compiled in order to fund local authority investments based on prioritizing project applications (pre-application) at county level or in selected urban areas. In the case of national public infrastructure, the Government designates investments (projects) to be financed from Structural Funds once a year or less frequently. The decision about which implementation scheme to be used is decided for each supported activity in line with measure-specific requirements based on an internal decree linked to the Structural Assistance Act 2007- 2013. Project Selection Model and Procedures - Open Calls (Measure “Strengthening of the competitiveness of regions”) Open calls are the most commonly used project selection method in Estonia, although their overall use is less than in the 2004-06 programming period. They cover all of Estonia and allow for choosing the most competitive projects with the highest national impact, and are used alongside pre-selected investment plans. The present case study covers Priority 4 “Integral and balanced development of regions,” which is implemented through six measures: 1) Development of local public services 2) Development of urban regions 3) Strengthening of the competitiveness of regions 4) Development of State-owned cultural and tourism objects 5) Development of regional competence 6) Improving the quality of broadband access Measures three, four, and five are funded through open calls. After the calls have been published, applicants can opt for support before submitting their application; project selection itself is carried out by an Evaluation panel:  Development of calls by first-level Intermediary Body (Ministry of Interior). The exact methodology for project selection is developed by the second-level Intermediary Body (Enterprise Estonia) or entity for each action. The calls are published on the websites of the first 266 and second level Intermediary Bodies and in the national newspaper, providing information on the timing and resources to be allocated.  Before submission of applications, the project applicant is entitled to get support from Enterprise Estonia for developing a project idea into an application. The support draws attention to the technical and substantive deficiencies of the application and gives recommendations for removal. However, this support is neither mandatory nor a prerequisite for submitting an application.  Applications can be submitted on an ongoing basis. Applications can be submitted on paper or electronically. They are submitted to the second-level Intermediary Body (Enterprise Estonia). The application must contain a completed application form designed to ensure that the project is in line with the objectives of the measure and the priority area. The application must also include the following documents: 1) project description; 2) project budget; 3) the schedule for the project; 4) documentation on own contribution; 5) background information on the applicant and the project partners (not required from public authorities); 6) project manager’s Curriculum Vitae; 7) documents describing the action plan and a business model for business infrastructure and jobs to be created, wage levels, and taxes paid (including business plans and fiscal forecasts), as a result of the business infrastructure built; 8) applicant’s last annual report and if not available on databases/if the applicant has been in business for more than the current fiscal year, also the balance sheet and the income statement (not older than two months); and 9) statement of the need for infrastructure development to improve the business environment and of the intention to construct the infrastructure.  The applications are evaluated by an expert Evaluation panel (predefined for each measure/program and communicated on the website), which is composed of external and Enterprise Estonia’s internal experts. The assessment will follow the predefined criteria, which are also publicly available on the Enterprise Estonia website. The assessment criteria are grouped in three “assessment blocks.” The assessment will result in a positive funding decision if the project exceeds the predefined threshold in all three blocks.  No later than 30 days after submitting the application the applicant shall be notified of approval, partial approval, conditional approval, or rejection of the application. Duration: Processing applications shall take up to 30 days. If the application contains inaccuracies, the applicant shall be notified immediately in order to correct the deficiencies. The applicant is given 10 days to complement and correct the application. In this case, the period for processing the application is extended by 10 days. Selection Criteria and Indicators: Development of Business infrastructure Measure 4.3 “Strengthening of the competitiveness of regions” aims to support the development of the local business environment and cultural and tourism sites of national importance. The development of local business environment supports the following operations:  Development of regional industrial parks and logistics centers;  Creation and strengthening of regional innovation systems and operations directed to the development of regional business networks and competence centers (in particular on the basis of county centers);  Development of public technical infrastructure essential for business development;  Preparations for community reuse of former military, industrial and agricultural facilities; demolition of low-value facilities;  Adaptation of disused public facilities for new functions; 267  Creation of conditions for stimulating local employment (establishment of local workshops, promotion of the use of traditional know-how, etc.); and  Investments in improving the quality of broadband access (data transmission links in areas of market failure, public broadband access sites and points). The measure is implemented by Enterprise Estonia through the “Development Program for Regional Competitiveness.” The program supports two types of projects: 1) development of tourism and leisure infrastructure; and 2) development of business infrastructure. Business infrastructure development projects include the following eligible activities: 1) construction and development of public technical infrastructure (access roads, water and sewage systems, drainage systems, exterior lighting and power, communication and heating networks); 2) redeveloping former military, industrial and agricultural facilities in rural areas. The minimum amount of grant per project is €50,000. The maximum contribution to the projects is decided on the basis of the contribution of the project to the State budget (i.e., tax paid in the following three years after the end of project), but does not exceed €1 million. The grant can cover up to 85% of eligible project costs. Beneficiaries are mainly local governments and the third sector. The eligibly criteria aim to facilitate transparent and effective project selection. The first-level Intermediary Body specifies project selection criteria and implementation arrangements including monitoring, supervision and other activities related to project implementation. The Managing Authority monitors the definition of selection criteria in order to ensure their alignment with the specific measures. The Managing Authority also prepares a guide for general project selection criteria to enable common understanding and evaluation of the criteria. More specific appraisal criteria are defined in measure decrees approved by the first-level Intermediary Bodies or the Government, and these can elaborate or complement the general criteria approved by the Monitoring Committee to facilitate their application to different measures. In addition, first-level Intermediary Bodies may define specific sector or measure-based criteria to ensure better focus of measures. Both the general selection criteria approved by the Monitoring Committee as well as the measure-specific criteria are made public and are available on the web sites of the intermediary bodies. The following common criteria were developed in cooperation of the intermediary bodies and are applied across all Estonian OPs:  project impact toward the achievement of the objectives of the priority axis;  project feasibility; and  demand for the project. Common selection criteria for Priority 4 “Integral and balanced development of regions”  Project contribution to the goals of local, regional, and sectoral development strategies:  The aims of the project (or the results) are consistent with local government development plans and integrated local and sectoral development strategies;  The aims of the project are consistent with the objectives of the county development plan (of the county in which the project takes place); and  The goals of the project are consistent with national sectoral strategies, particularly when the project has a wider than regional effect.  Project contribution to the aims of the priority axis:  impact on public services and competitiveness of the county/ local center;  impact on the availability of public services in rural areas; 268  impact on sustainable urban development or other specific urban issues;  impact on strengthening regional competitive advantage;  impact on regional bottlenecks hindering the development of the county;  impact in terms of an increase in the number of visitors and length of stay in the region;  impact on employment; and  impact on balanced regional development.  Sustainability of impact:  Long-term impact on the region’s development;  Ensuring critical mass of beneficiaries;  Ensuring competency of project manager and partners;  Ensuring availability of funding;  Different project implementation stages are well planned and implemented;  Implementation of principles of sustainable development; and  Awareness of other external risks for project implementation.  Project feasibility:  Sufficient number of staff to implement project activities and achieve project objectives;  Budget and cost justification;  Know-how and competence of project team; and  Involvement of relevant partners. Specific criteria for “Strengthening of the competitiveness of regions,” projects developing business infrastructure: Three sets of indicators are used (“assessment blocks” – see table). Assessment Blocks – General Project Selection Criteria for Measure 4.3 Goal Weighting Contribution of the project in advancing the goals of the measure 50% (impact on business activity, jobs, sustainability) Relevance and sustainability of infrastructure facilities (benefit for 25% beneficiaries, justification of the proposed site, financial and administrative sustainability of infrastructure facility, technical quality and sustainability of the infrastructure project) Readiness to implement the project (feasibility, security of resources, 25% applicant’s administrative capacity, self-financing, project budget) Projects are assessed using scoring and weighting techniques around a detailed list of criteria. For the individual criteria, scores are attributed between 0-4: 4= Very good 3= Good 2=Mediocre 1=Weak 0= Poor The total score for the individual criteria is the arithmetic average (to two decimals) of the marks awarded by the individual members of the Evaluation panel. The total score of an assessment block is formed by summing up the weighted-average scores. For a positive funding decision, the project must reach the pre-defined score in each assessment block. In the first assessment block, the project must score at least 2.75, and in the second and third blocks at least 2.50. If the average score given by any 269 member of the Evaluation panel is less than 1.00 or more than half of the members of the Evaluation panel assess any individual criterion for 0.00, the project receives a negative funding decision. The following table shows the definition of grades for each individual evaluation criterion: Assessment Blocks – Detailed Criteria and Indicators for Measure 4.3 Assessment Block 1: The impact of the project on achieving the objectives of the measure 1.1 Impact on entrepreneurial activity (25% of the total grade of the first assessment block) The project has a significant impact on the development of entrepreneurship in the county, as a result of 4 starting up or expanding businesses 3 The project does not have a significant impact on the development of entrepreneurship in the county, as 2 a result of starting up or expanding businesses 1 0 The project lacks impact on the entrepreneurship 1.2 Quality of jobs created due to the project (25% of the total grade of the first assessment block) 4 Jobs created due to the project have an average wage which is at least 50% higher than the average wage in the county 3 Jobs created due to the project have at least 21-49% higher average wage compared to county average wage Jobs created due to the project have at least 20% higher average wage compared to county average 2 wage 1 Jobs created due to the project do not exceed the county average wage 0 Wage estimates are not reliable or justified 1.3 Impact on social tax income (25% of the total grade of the first assessment block) 4 The project is estimated to result in an additional social security tax in the first fiscal year following the fiscal year of the completion of the project; The project is estimated to result in an additional social security tax in the second fiscal year following 3 the fiscal year of the completion of the project; The project is estimated to result in an additional social security tax in the third fiscal year following the 2 fiscal year of the completion of the project; The project’s estimated effect on additional social income tax is overestimated and the project will not 1 result in additional social tax income by the end of the third fiscal year after the completion of the project. 0 The projected influence on social income tax is not reliable or justified. 1.4 Beneficiary/Beneficiaries’ sustainability (25% of the total grade of the first assessment block) The beneficiaries' business models are sustainable. There is demand for the beneficiaries' products / 4 services in the long term and the competitiveness of the beneficiary is high. It is likely that the beneficiaries continue to operate in the markets for at least five years after project completion; The beneficiaries' business models are sustainable. There is demand for the beneficiaries' products / 3 services in the long term and the competitiveness of the beneficiary is high. It is likely that the beneficiaries continue to operate in the markets for at least three years after project completion; The beneficiaries' business models work, but the business model is not particularly sustainable given the 2 changes in the external environment. There is market demand for the beneficiaries’ products/services but they lack clear competitive advantage; 270 The beneficiaries' business models are effective, but they are expected to operate no more than two 1 years after the commissioning of the project because of maturing infrastructure. The market for the beneficiaries’ products / services market is fading and they have no clear competitive advantage. 0 The beneficiaries' business models are not sustainable or the beneficiary lacks a business model. Beneficiaries’ products / services have no real competitive advantages and no markets. Assessment Block 2: The relevance and sustainability of infrastructure facilities 2.1 Infrastructure facilities impact the performance of the beneficiaries (25% of the total grade of the second assessment block) 4 The project eliminates key bottlenecks that hinder the beneficiary(ies) entry / expansion in the markets. The project and built infrastructure will also benefit a large number of other potential businesses. 3 The project eliminates key bottlenecks that hinder the beneficiary(ies) entry / expansion in the markets. The project and built infrastructure will also benefit a small number of other potential businesses. 2 The project eliminates key bottlenecks that hinder the beneficiary(ies) entry / expansion in the markets The project eliminates some bottlenecks that hinder the beneficiary(ies) entry / expansion in the 1 markets The project does not eliminate key bottlenecks that hinder the beneficiary(ies) entry / expansion in the 0 markets and the infrastructure is not essential for the beneficiary to enter / expand in the markets. The project is over-dimensioned. 2.2 Location of proposed building site (25% of the total grade of the second assessment block) 4 There are no alternatives to the selected location in the same county enabling beneficiary entry / expansion, or the cost for building public infrastructure is significantly higher in the alternative location. There are alternatives for the selected location in the same county and the cost of building would be 3 equal but would require significant additional financing from the beneficiary to change location. 2 There are alternatives for the selected location in the same county and the cost of building would be equal and it does not require significant additional financing from the beneficiary to change location. 1 There are alternatives for the selected location in the same county and the cost of building would be lower and the additional cost for beneficiary is modest. 0 There are alternatives for the selected location in the same county and the cost of building would be significantly lower and there are no additional costs for the beneficiary. Financial and administrative sustainability of the infrastructure (25% of the total grade of the second 2.3 assessment block) 4 Maintenance of the infrastructure built as a result of the project does not require additional public financing or investments. The competency to manage the infrastructure is demonstrated. Maintenance of the infrastructure built as a result of the project does not require additional public 3 financing but requires additional investments in the long term. The competency to manage the infrastructure is demonstrated. 271 Maintenance of the infrastructure built as a result of the project requires additional public financing. 2 The applicant has secured the funding necessary to maintain and upkeep the infrastructure. The competency to manage the infrastructure is demonstrated. Maintenance of the infrastructure built as a result of the project requires additional public financing and 1 the applicant has not secured the finance necessary for the maintenance and upkeep of the infrastructure. The competency to manage the infrastructure is not demonstrated. The completion of the project with fixed financing cannot be guaranteed. The applicant has not 0 managed to secure the finance needed for the maintenance and upkeep of the infrastructure. The competency to manage the infrastructure is not demonstrated. Suitability, technical quality, and sustainability of the infrastructure project (25% of the total grade of 2.4 the second assessment block) The technical solutions are technologically and economically justified. Technical quality meets current standards and has lasting effects and the infrastructure is resilient to natural disasters (e.g., floods). The 4 proposed technical solution is optimal for achieving the project objectives. Technical solutions support the use of energy-efficient and modern technologies. The technical solutions are technologically and economically justified. Technical quality meets current 3 standards and has lasting effects and the infrastructure is resilient to natural disasters (e.g., floods). The proposed technical solution is optimal for achieving the objectives of the project. The technical solutions are technologically and economically justified. Technical quality meets current standards and has lasting effects and the infrastructure is resilient to natural disasters (e.g., floods). The 2 proposed technical solution is optimal for achieving the objectives of the project, but there are some risks. Building the infrastructure facilities is not based on needs and the infrastructure is significantly over- 1 dimensioned. The proposed new infrastructure does not take the capacity of existing infrastructure into account. Building the infrastructure is not economically or technologically justified. The technical quality does not 0 meet the current standards and solutions are not sustainable or resilient to natural disasters. The proposed technical solution is not optimal for achieving the objectives of the project. Assessment Block 3: Readiness for implementation 3.1 Feasibility and the applicant's administrative capacity (20% of the total grade of the third assessment block) The applicant has the necessary competence to implement the project and the project organization has 4 the required know-how. The project manager or project team has the required experience and knowledge to realize the project. All relevant partners are involved in the project. 3 2 The capability to meet the criterion can be questioned. 1 The capacity and competence to manage and implement the project is weak or not demonstrated. The organizational arrangements are inadequate and the project manager and team lack the capacity to 0 implement the project. The relevant partners, necessary expertise or know-how are not sufficiently involved in the project. 3.2 Own financing for the project (20% of the total grade of the third assessment block) 4 Co-financing is secured and there are no doubts regarding the applicant’s self -financing capacity 3 2 The capacity to meet the criterion can be questioned. 1 272 0 The applicant's financial capacity is weak and project co-financing is not ensured. 3.3 Technical readiness of the project (20% of the total grade of the third assessment block) 4 The project is ready to be implemented. All necessary permits to begin construction work are in place. Project planning is partially completed (e.g., conceptual design). Most of the necessary permits and 3 similar exist. Obtaining the missing permits is likely. The project requires additional permits before construction can begin. Obtaining the missing permits is 2 likely. The project requires additional permits before construction can begin. Obtaining the missing permits 1 before the start date of the project is questionable Readiness to implement the project is low, because the required permits are missing and / or cannot be 0 obtained by the proposed starting date of the project. 3.4 Project budget, quality and relevance (40% of the total grade of the third assessment block) The budget is transparent and linked to planned actions and their outcomes. Costs are detailed and justified. The budget is based on bidding providers. Project costs are comparable with similar projects. 4 The budget is sufficient for the implementation of project activities and is not exaggerated. The budget is transparent and linked to planned actions and their outcomes. Costs are detailed and justified. The budget of the majority of project activities is based on bidding providers. Project costs are comparable with similar projects. The budget is sufficient for the implementation of project activities 3 and is not exaggerated. The budget is transparent and linked to planned actions and their outcomes. Costs are detailed and justified. The budget has been prepared on the basis of calculations and prepared by an authorized HR specialist. The project is very similar to market prices. The budget is sufficient for the implementation of 2 project activities and is not exaggerated. The budget is insufficient to implement project activities, prices are underestimated. There is a risk that fiscal resources are not adequately planned, which could hinder project realization; or the budget is 1 overestimated, the expenditure is not justified and the prices significantly exceed current market prices. 0 Expenditure is partly unjustified and only partially transparent. Estimates are largely unreliable. Applications that meet the pre-defined thresholds for each funding block receive a positive funding decision. If several projects are presented simultaneously and reach the same aggregate assessments, preference is given to the project that receives the highest score in the first assessment block. The applicant can receive approval, partial approval or conditional approval or be rejected. If the application is rejected, the applicant is informed about the reasons for rejection. In terms of monitoring and evaluation, the NOP for development of Living Environment defines indicators at the measure level. The physical progress of the measure “Strengthening of the competitiveness of regions” is measured through the following indicators: 273 Monitoring Indicators for Measure 4.3 Indicator Explanation Current level Reference Target level level Output Indicators Number of business infrastructure facilities Source: Project Not applicable 15 (2010) 50 (2015) created or qualitatively improved implementation reports Number of projects enhancing regional Source: Project Not applicable 10 (2010) 30 (2015) traditional know-how implementation reports Number of visitor sites created or Source: Project Not applicable 30 (2010) 100 (2015) qualitatively improved implementation reports Result Indicator Number of companies that have benefited Source: Survey by Not applicable 90 (2010) 300 (2015) from the created business or visitor Enterprise Estonia infrastructure Performance assessment The project selection system is found to work satisfactorily since it ensures absorption and output achievement; it ensures high levels of transparency, but could be improved regarding its strategic orientation. The project selection system of the Estonian Operational Programmes was evaluated in 2010 in order to assess in how far the selection models ensured funding absorption and how well the selection models and criteria managed to ensure strategic fit of the projects. Commitments and payments of activities aiming to improve the business environment have been as expected. In terms of the effectiveness of the selection model, the following remarks can be made:  The output indicator “Number of business infrastructure facilities created or qualitatively improved,” which measures the achievement of business infrastructure development activities, had reached 40% of the target value for 2015 by the end of 2012. Since many of the projects are still on-going it is expected that the target level will be achieved by the end of 2015.  The effectiveness of the measure was hindered by the vagueness of targets. The aim of the measure was to increase the competitiveness of the regions by making them more attractive for businesses and visitors. However, the concepts of competitiveness and attractiveness are not clearly defined. Clearer targets are essential for purposeful project selection. To ensure this, the content of project applications and supporting documents should be delimited more clearly. For example, applicants provide their own description of the relationship between project goals and the goals of the county strategy. This is relatively straightforward due to the vagueness of strategic objectives. This makes the scoring and assessment of applications more difficult and leaves room for subjective assessment. Furthermore, applicants and evaluators are expected to refer to multiple strategic documents, however no list of key documents is provided, which would increase the objectivity and uniformity of the assessment by the panel members.  The key strength of the project selection process is transparency; information on selection criteria, weighting and the members of the Evaluation panel, etc., is made public and all applications are assessed by all panel members. However, more emphasis should be placed on 274 clear and uniform definitions of key concepts and indicators. More detailed guidelines for the panel members and clearer definitions of indicators would enhance the uniformity of assessment by panel members and increase the quality of funding decisions.  Cost efficiency should be assessed across all activities. In addition, there are no criteria or mechanisms ensuring that funds are evenly distributed across counties. Italy – Lazio ERDF OP: Competitive Calls for Local and Urban Development Plans (P.L.U.S. Lazio)66 The Local and Urban Development Plans (P.L.U.S., Plano Locale e Urbano di Sviluppo) were introduced to the Lazio ERDF OP very late in the programming period in response to problems with program progress and the changed economic situation. Identified through a competitive procedure, they provide a framework for projects in the field of urban development, including economic, social, territorial and cultural activities. They are characterized by a number of procedural innovations and have allowed for close strategic alignment with the ROP strategy (as reprogrammed) and also with the new Europe 2020 strategy and the priorities of the forthcoming round of Cohesion policy, which see urban development as a pillar of Cohesion policy. Context In Italy, the management of regional ERDF OPs is decentralized. The ERDF OP of the Lazio region is managed by the Regional Directorate Economic Development and Productive Activities. It is one of the Italian Regional Competitiveness and Employment ERDF OPs, and entailed, in its original configuration (October 2007), c. €329.9 million (ERDF), equal to c. 1.3 percent of the total ERDF resources allocated to Italy for the 2007-13 period. In a reprogramming exercise of 2011, to take into account low spending performance (by the end of 2010 the program had only spent 8.5 percent of its total allocation) 67 and new needs that emerged as a result of the economic crisis (including in the metropolitan area of Rome and in medium/small urban centers),68 a new emphasis was placed on urban development as an engine for growth.69 As a result, the program was integrated with a new Priority (P5) on urban and local development, which is the subject of the case study.70 66 http://porfesr.lazio.it/PORFESR/home.html 67 See Regione Lazio, Dipartimento Programmazione Economica e Sociale, Direzione Regionale Programmazione Economica, Ricerca e Innovazione (2011) ‘Programma Operativo Regionale cofinanziato dal FESR 2007-2013: motivi ed effetti della revisione’, A cura del Nucleo di valutazione e verifica degli investimenti pubblici, 7 Febbraio 2011. 68 See Regione Lazio, Dipartimento Programmazione Economica e Sociale, Direzione Regionale Programmazione Economica, Ricerca e Innovazione (2011) ‘Programma Operativo Regionale cofinanziato dal FESR 2007-2013: motivi ed effetti della revisione’, A cura del Nucleo di valutazione e verifica degli investimenti pubblici, 7 Febbraio 2011. 69 A new Committee, CIPU - Interministerial Committee for Urban policies, was established in 2012. In March 2013, the Committee issued a document entitled ‘Methods and Contents relating to the priorities in the field of the urban agenda ’. Essentially, the document denounced the fragmentation of the interventions for towns and cities and the lack of an urban regeneration policy intended as an engine for development, arguing in favour of renewed attention on this theme and better (less fragmented) governance, through the lever of Cohesion policy, particularly to tackle the infrastructure bottlenecks and congestion of urban areas; urban sprawl and land usage; building safety and the maintenance of urban land. 70 Priority 5, ‘Urban and local development’ was introduced in a revision proposal of February 2011, with an allocation of €80 million. The identification of the intervention spheres (linee generali di intervento) was made by the MA and the Regional Directorate for Economic Programming, Research and Innovation, together with the regional evaluation unit (Nucleo di Valutazione e Verifica, NUVV) and the regional Environmental Authority. The revision proposal was endorsed by the partnership in a meeting of 13 December 2010 and by the PMC on 18 February 2011. The revised ROP was then approved by the regional executive (Giunta Regionale) with deliberation of 20 May 2011, by the regional Council with deliberation of 28 March 2012 and by the European Commission on 28 March 2012 (source: Regione Lazio, Integrazioni al documento “Descrizione del Sistema di Gestione e Controllo”, POR FESR Lazio 2007-2013, Novembre 2012). 275 Lazio ERDF RCE Programme 2007-13 – Program Allocations (€) National Public National Private Priority Axis EU Investment Total Contribution Contribution 1) Research, innovation, 162.81 162.81 0 352.63 and strengthening of the productive basis 2) Environment & risk 109.93 109.9 3 0 219.87 prevention 3) Accessibility 45.25 45.25 0 90.50 4) Technical assistance 13.76 13.76 0 27.51 5) Urban and local 40.00 40.00 0 80.00 development Total 371.76 371.76 0 743.511 Source: Regione Lazio (2012), Programma Operativo Confinanziato dal F.E.S.R – Fondo Europeo di Sviluppo Regionale 2007- 2013, Regione Lazio, CCI2007IT162PO004, version approved with the reprogramming approved by the Commission on 28 March 2012 (last known reprogramming), Dec C1659/2012 (original version was dated 2 October 2007, Dec 4584/2007). http://porfesr.lazio.it/PORFESR/galleria_allegati/porfesr/POR_FESR_Lazio_riprogrammato.pdf Note: (1) This is the total in the document, but from the figures the total should be €770 million. We can only assume that the grand total has not been updated to reflect the new allocations. The OP relies upon a number of different selection methods, applicable to the various measures (activities). These include: (i) evaluative procedure through open calls (procedura valutativa a sportello), i.e., open calls for tenders where projects are selected if they meet eligibility criteria and score above certain minimum thresholds in relation to pre-set criteria of merit; (ii) evaluative procedure through competitive calls (procedura valutativa a graduatoria), i.e., calls for tenders with specified deadlines, where projects that have passed a first eligibility scrutiny are ranked based on pre-set selection criteria; and (iii) identification of projects based on the regional authority’s own sectoral plans (only for Priorities 2 and 3). Technical Assistance measures are administered through either calls for tenders or direct contracting-out (affidamento diretto). The most used procedure is the evaluative procedure through competitive calls. In some cases, notably under Priority 1, a single activity can be administered through both open and competitive calls. In addition, the regional authority established the use of ‘integrated access procedures’ (for activities I.1, I.2, I.3. I.4, I.5, I.6 and II.1), comprising aspects of ‘negotiated procedure’ (procedura negoziale di accesso alle agevolazioni), to select composite investment projects above €20 million and comprising at least one ‘key’ project of at least €10 million, in order to enact quick and targeted interventions for the economic development of the region inducing spillovers on local production chains (ricadute di filiera). These ‘integrated access procedures’, which were finalized in 2008, proved too complex and contributed to implementation delays, and had to be revised a year later.71 Project selection is undertaken by both the regional authority’s own offices and by a number of Intermediary and Implementing Bodies.72 The selection procedures utilized for each activity or sub- 71 DGR n. 611 of 5 August 2008 and DGR n. 644 of 7 August 2009. See Regione Lazio, Dipartimento Programmazione Economica e Sociale, Direzione Regionale Programmazione Economica, Ricerca e Innovazione (2011) Programma Operativo Regionale cofinanziato dal FESR 2007-2013: motivi ed effetti della revisione, a cura del Nucleo di valutazione e verifica degli investimenti pubblici, 7 febbraio 2011, pp. 13-16. In some cases, the whole selection process for business support interventions had taken more than 500 days, p. 16. 72 These comprise: 12 ‘regional areas’ in seven regional directorates general which have management functions (Responsabili di gestione dell’Attività, RGA); three Intermediary Organsations (Sviluppo Lazio SpA, the in-house development agency of the 276 activity are specified in detail in the calls for tenders of each activity/sub-activity and in the activity- specific documents called ‘Implementing modalities of the program’ (Modalità Attuative del Programma, MAPO), drafted by the managing authority and approved by the program monitoring committee (and which provide the basis for the calls for tenders).73 The program is not progressing well: At 31 October 2013 (last available data), total payments (ERDF and domestic co-financing) stood at 48 percent of financial allocations. It was therefore not able to meet the domestic expenditure target of 55 percent (a target of c. €409.2 million, against a level of actual expenditures of €354.3 million).74 Project Selection Model and Procedures – The P.L.U.S. Local and Urban Development Plan (Piano Locale e Urbano di Sviluppo) The P.L.U.S. were introduced in the ERDF regional program with a reprogramming exercise in 2011 to respond to the changed context in the region’s medium-sized towns and cities, which has emerged as the result of the economic crisis, while also addressing the spending and implementation difficulties that the program was facing. A document was prepared by the regional evaluation unit in February 2011, explaining the reasons for the reprogramming (which did entail also other changes to Priorities 1 to 3, as well as the creation of the new Priority 5).75 The P.L.U.S. are implemented under Priority 5 on urban and local development, Activity I.7 from Priority 1 (Development of telecommunication applications, products, processes, contents and services) and Activity II.1 (in Priority 2) via integrated investment programs for municipalities with more than 25,000 inhabitants, including investments in: via integrated investment programmes for municipalities with more than 25,000 inhabitants including investments in: 1. Public works and acquisition of services and goods; 2. Active labor market policies (e.g., training, traineeships etc.) and aids to individuals; and 3. Aids to SMEs. Eligible expenditures are precisely defined for each of these categories in a regulation for the P.L.U.S.76 and aids to individuals / firms are granted only to residents in the area of each P.L.U.S. (which is a sub- area of the given municipality). regional authority; Filas SpA, a financial company owned 100 percent by Sviluppo Lazio; LAit SpA, another in-house body of the regional authority, utilized for e-health activities of the ROP); and three external Implementing Bodies (the (national) Ministry of Economic Development, through Infratel Italia SpA, for broadband inteventions; a temporary association of enterprises comprising two financial institutions, Confidinsieme and Capofile CoopFidi; and, Fidimpresa Lazio, the latter two for schemes comprising financial engineering). 73 Source: http://porfesr.lazio.it/PORFESR/galleria_allegati/documentazione/SIGECO/DESCRIZIONE_SISTEMA_GESTIONE_E_CONTROLLO.p df 74 See http://www.coesioneterritoriale.gov.it/fondi-europei-litalia-raggiunge-il-475-della-spesa-certificata/. The Italian authorities set up a system of domestic expenditure targets in addition to the regulatory N+2. Together with Lazio, of the RCE regions, also the ERDF programmes of Piedmont, Sardinia and Umbria and the ESF programme of Valle d’Aosta did not reach the target. 75 Regione Lazio, Dipartimento Programmazione Economica e Sociale, Direzione Regionale Programmazione Economica, Ricerca e Innovazione (2011) Programma Operativo Regionale cofinanziato dal FESR 2007-2013: motivi ed effetti della revisione, a cura del Nucleo di valutazione e verifica degli investimenti pubblici, 7 febbraio 2011. 76 Regione Lazio (2013) Programma Operativo F.E.S.R. Lazio 2007-2013, Asse V, Sviluppo Urbano e Locale, Attività V.1 Rigenerazione delle funzioni economiche, sociali e ambientali delle aree urbane, Disciplinare regionale per la realizzazione di interventi integrati di sviluppo urbano approvato con Determinazione dirigenziale N. B00027 DEL 9/1/2012, modificato con Determinazioni Dirigenziali n.B08055 del 24/10/2012, n. B00891 del 13/3/2013 e n. G02878 del 22/11/2013, pp. 1-39. 277 The selection procedure of the P.L.U.S. appears to have a two-fold rationale. On the one hand, the choice of composite plans comprising, for the largest part, ready-to-be-implemented infrastructure projects, was intended to support the achievement of the programme’s spending targets, at the same time as ensuring added value (synergies between different projects) and concentration (territorial focus), and thus high impact. On the other hand, the devolution to the municipalities of project selection responsibilities for aids to businesses/persons and of the public procurement procedures for the realization of public works, achieved a double effect: to lighten the work of the managing authority/regional offices and to enhance skills and instill a performance-oriented mentality within the municipalities (also beyond the preparation phase of the P.L.U.S.), in line with the principle of administrative federalism recently introduced in the Italian Constitution. In relation to this last aspect, it is noteworthy that a reserve was set aside to reward at a later date the best performing P.L.U.S.., supported by procedural innovations allowing for real-time benchmarking of the different plans, notably through an online monitoring system and a related ‘information querying tool’ to compare the financial and procedural advancement across different P.L.U.S. programs. Taken together, a total amount of €147.3 million was assigned to the P.L.U.S. based on an approved ranking list, of which €113.7 million was immediately allocated to projects. The P.L.U.S. involve almost €130 million from Priority 5, plus another €8.86 million under Activity II.1 in Priority 2 (Promotion of energy efficiency and production of renewable energy) and further €8.95 million from Activity I.7 from Priority 1 (Development of telecommunication applications, products, processes, contents and services). Projects supported by the P.L.U.S. are composite projects comprising a variable mix, defined by each municipality. They include public works, SME support, acquisition of goods or services, social inclusion and employability measures (e.g., traineeships), and communication activities. The selection procedure resulted in the allocation of resources to 16 Integrated Investment Programs in 16 municipalities. Allocations are used to fund 153 projects of which: 73 public works, 17 projects for SMEs, 16 projects for the communication of the results of each P.L.U.S., 13 projects for the acquisition of goods or services and 24 projects for social inclusion and employability. Each integrated program is composed of a different mix of projects, depending on the needs and preferences of each municipality, as defined in the P.L.U.S. plan. In illustration, the P.L.U.S. of Rome, called ‘Porta Portese, a resource for Rome’, consists of the following:77 - Modernization of a public building (€3.14 million); - Modernization of the market of Porta Portese (€3.58 million); - Implementation of the PPTU (detailed plan on urban traffic) of the part of Rome covered by the P.L.U.S. (part of the area of the Municipio XVI) (€540,000); - Fund for the development of micro-entrepreneurship and firm start-ups (€230,000); and - Information and communication plan (c. €150,000); In addition, it included investments from Priority 1 of the OP: - Information and telecommunication technologies for the improvement of the quality of urban mobility (€820,000); - Implementation of the PPTU (coordination of traffic lights and LED technologies, €410,000). The P.L.U.S. programs were identified in a two-stage process, followed by a project selection stage in line with the selection of mainstream projects. In July 2011, the regional authority issued a public call 77 http://www.comune.roma.it/PCR/resources/cms/documents/POR_FESR_Lazio2007_2013.pdf 278 for tenders under Activity V.I, “Regeneration of the economic, social, and environmental functions of urban areas,” inviting municipalities with more than 25,000 inhabitants to submit projects to increase their competitiveness, attractiveness, and sustainability. Proposals had to take the form of a ‘Local and Urban Development Plan’ (P.L.U.S.), which had to relate to a confined area within the territory of the municipality and which could include four thematic spheres: (i) regeneration of public areas and buildings; (ii) social inclusion, territorial cohesion, and development of social, cultural, and tourism services; (iii) improvement of the state of the environment, of mobility, and of the urban transport system; and (iv) promotion of entrepreneurship and revitalization of the economic and productive fabric. The requirements that the P.L.U.S. have to fulfill can be seen from the application forms.78 They comprise a number of pre-defined sections (the description below draws from the P.L.U.S. of Fondi): Section A – General Information on the P.L.U.S. - Summary information on the P.L.U.S. (in the form of a table indicating: title, proponent municipality, selected neighborhood within the municipality, population covered, m2 covered, population density of the area covered, office responsible and name of officials involved, responsible person for the whole P.L.U.S., and number of interventions, diversified by type of intervention); - Executive summary (in English); - Changes or integrations made to the P.L.U.S. strategy; - Management arrangements/responsible offices. Section B – Operational and Financial Plan of the P.L.U.S. - Interventions for public works and acquisition of goods and services; - Interventions on active labor market policies and aids to persons; - Interventions for aids to SMEs; - Financial plans (indicating both the specific allocation to each intervention and the overall expenditure target for each year up to 2015). Section C - Interventions for public works and / or acquisition of goods and services - Public works – technical fiches; - Acquisition of goods and services – technical fiches; - Description of procedural advancement; - Procedural strategy - Description of any environmental impact; - Timetable / roadmap, indicating: approval date of the ‘executive project’, approval date and related document (project ready to be implemented, i.e., with all permissions granted etc.), public procurement call (bid invitation date and related document), start date of works, completion dates for the different implementation lots, end date of works, date of testing (collaudo); - Technical, administrative and accounting annexes; - Economic and financial feasibility; - Output indicators. Section D - Interventions regarding active labor market policies and aids to persons and / or SMEs - Active labor market policies and aids to persons – technical fiches; 78 An example of the application form, taken from the P.L.U.S. of the town of Fondi, can be viewed from this link: http://www.P.L.U.S.fondi.it/pdf/Dossier_candidatura.pdf. This is based on the pro-forma included in the call for tender issued by the regional authority/managing authority (Annex I to Deliberation 348/2011) and includes, in line with such pro-forma, the requested executive summary in English. 279 - Aids to SMEs – technical fiches; - Needs analysis; - Objectives and beneficiaries; - Output indicators; - Annexes (e.g., area map of the P.L.U.S., decree by the municipality approving the P.L.U.S. proposal, supporting letters by organizations that will be involved in the activities of the P.L.U.S.). P.L.U.S. projects were then selected through a two-tiered procedure: Phase 1 - Submission by the municipalities of a (preliminary) ‘application dossier’ compliant with the pro-forma included in the invitation to tender. This phase preceded the approval by the European Commission of the revised ROP. - Screening of the applications and pre-selection of P.L.U.S. proposals that were invited to submit a ‘definitive’ proposal (upon approval of the ROP’s revision). Pre-selection was based on three criteria described further below. This first phase resulted in the pre-selection of 16 P.L.U.S. proposals (with scores above 50/100) and the rejection of 13 P.L.U.S. proposals (scoring between 42 and 46/100). Phase 2 - Definitive selection of the P.L.U.S.: The 16 pre-selected P.L.U.S. were definitively approved, but with improvements and changes introduced in response to feedback obtained from the pre- selection stage. The P.L.U.S. were re-scored and re-ranked based on the same A-B-C criteria used in Phase 1; this led to a different ranking order and overall higher scores than during the first phase (56/100 and above). The selection for both Phases 1 and 2 was undertaken by an Appraisal Commission taking approximately nine months. The commission was appointed by decree of the President of the Regional Authority comprising: - The managing authority of the ERDF ROP (or delegated person), with the function of Chair; - The managing authority of the ESF ROP (or delegated person); - The regional environmental authority (or delegated person); - An expert on public works designated by the ERDF ROP managing authority; - An expert on Structural Funds designated by the Agency Sviluppo Lazio; and - A member of the regional authority’s Evaluation and Verification Unit. The definitive selection of the 16 P.L.U.S. took about nine months from the launch date of the call for tenders in July 2011,79 and the ranking list was approved on 24 May 2012. Subsequently, the office(s) responsible for the P.L.U.S. within each municipality applied the standard approaches foreseen for different types of projects: a. For projects involving aids to firms or individuals, select the beneficiaries, based on procedures specified in the P.L.U.S. document or in subsequent calls;80 for aids to SMEs, procedures can cover the standard selection procedures identified in legislative decree no. 123/1998, i.e., the automatic procedure (eligibility-based, no technical appraisal); 79 The deadline for the submission of the definitive P.L.U.S. proposals was extended to 1 March 2012 by Directorial Determination no, B00870 of 13 February 2012, published in the regional OJ of 21 February 2012. 80 For instance, the procedures for the traineeships of the P.L.U.S. of the Municipality of Fondi are illustrated here: http://www.P.L.U.S.fondi.it/faq.html 280 competitive call; open call (i.e., first come first served, based on an eligibility and merit appraisal, and projects are funded if they score above set thresholds). b. For public works projects, select the subjects responsible for the realization of the works, based on standard public procurement procedures. The P.L.U.S. are characterized by a number of procedural innovations. In addition to the full online submission of the application (the full P.L.U.S. project, inclusive of scanned supporting documentary evidence, such as deliberations of approval of infrastructure projects by the given municipal council, supporting letters, maps, etc.), the following procedures apply once a P.L.U.S. has been approved and is implemented: - The responsible office has the obligation to transmit all documentation relating to expenditure claims exclusively in digital format, using certified e-mail with digital signature; - Online monitoring of the procedural and financial progress of the P.L.U.S. through an online system (this allows the managing authority to know in real time the progress of each P.L.U.S. and related activities); - Creation of an ‘information system’ (‘cruscotto informativo’) that elaborates the data included in the online monitoring system and compares and contrasts the financial and procedural progress of the various P.L.U.S., benchmarking them and thus identifying the best and worst performers (including an alert mechanism about the need for corrective measures). Selection Criteria and Indicators The P.L.U.S. were selected based on three macro-criteria, totaling 100 points: A – Urban Regeneration Strategy (40 points):  Coherence between the objectives of the P.L.U.S. and the objectives of Priority 5 of the ROP; quality, innovativeness, and experimental nature of the regeneration and development strategy proposed;  Coherence of the described regeneration needs and the solutions proposed to overcome them;  Ability of the proposed interventions to maximize the benefits for the environment, to exploit fully the cultural, historical, and architectural heritage of the area, and to promote the equal opportunities principle; where applicable, the inclusion of a municipal energy efficiency plan and CO2 reduction represent an element of added value;  Degree of interconnection between interventions included in the P.L.U.S. and appraisal of the approach taken / to be taken to ensure the integration of interventions;  Balance and innovativeness of the types of initiatives foreseen in the various thematic areas (physical, environmental, social, and economic); the development of telecommunication infrastructure and services for the improvement of urban quality are an element of added value. B – Administrative feasibility (40 points)  Completeness of submitted documentation and sufficient level of overall detail/definition of projects;  Coherence of the P.L.U.S. with regional and provincial territorial planning instruments and with urban planning instruments;  Procedural advancement of each specific intervention (a higher score is assigned to the P.L.U.S. that include projects with available or expected authorizations and certificates); 281  Financial sustainability of the interventions – this includes the financing of the works after completion. The replicability and innovativeness of the proposed interventions is an element of added value;  Compatibility of the time-frame of each intervention with the time-horizon of the 2007- 13 ROP and Structural Funds regulations. C – Favorable factors for the completion of the P.L.U.S. (20 points)  Quality of the project management structure and clear / detailed description of the organization within the proponent municipality for both the overall coordination of the P.L.U.S. and the successful delivery of each intervention; participation of the municipality in networks of towns already involved in urban regeneration strategies (e.g., URBACT) and / or strategic programming. The presence and operation of an URBAN center is an element of added value.  Functional integration of the operations foreseen as part of the P.L.U.S. with interventions already realized or being realized in the reference territory.  Financial participation of private actors in the P.L.U.S. initiatives or, alternatively, in complementary initiatives and the delivery stages. Monitoring is carried out as part of the ordinary monitoring activities of the program. In addition, a dedicated ‘cruscotto informativo’ has been established to monitor the financial and procedural progress of each P.L.U.S., allowing to benchmark the 16 plans against each other. No evaluation has been conducted on this initiative yet, understandably given its novelty. It would be premature to attempt to appraise achievements at this stage. Performance Assessment At 20 December 2013,81 46 out of the 73 public works had stipulated the contract for the launch of the works or had already started the construction phase. 27 further works have still to finalize the public procurement procedures; a majority of these, however, are expected to conclude the contracting-out by mid-January 2014. 63 percent of the works approved for funding – representing c. €53.5 million of investments – have been started. Legally binding commitments stand at above €50 million. Certified expenditure amounts to roughly €9.3 million. The P.L.U.S. have allowed for close strategic alignment with the ROP strategy. Given the structure of the P.L.U.S. and the procedure and criteria used to select them, the selected projects closely match the ROP objectives, as amended following the 2011 reprogramming exercise. Main Lessons Composite territorial projects are generally suitable with respect to ERDF regional development goals for the following reasons:  Suitable to address the specific needs of local milieu;  Positive empowerment of municipalities through delegation of functions, while at the same time framing the interventions realized within the regional strategy of the ROP;  Clear description of objectives in the individual P.L.U.S. proposals formulated by the municipalities and specification of how they match those of the ROP. Coherence between the two sets of objectives ensured by this being one of the selection criteria; 81 Online news item dated 20 December 2013, http://www.P.L.U.S.lazio.eu/P.L.U.S.-impegni-vincolanti-per-oltre-50-mln/. 282  Two-tiered project selection appears to have raised the quality of the interventions foreseen in the P.L.U.S. (as manifested by the higher scoring recorded in the second round of appraisals). The following good practice and conditions for successful implementation can be highlighted:  Transparent and traceable project selection according to pre-defined procedures and criteria specified in the call for projects (something which is quite standard in Italy);  Two-stage procedure for the selection of the P.L.U.S. appear to have worked well, in allowing the ROP authorities to have a dialogue with the proponent municipalities so as to improve the proposals;  Assignment of ad-hoc resources for the municipalities to define their P.L.U.S. proposals once pre-selected;  Weight assigned to readiness to be implemented ensures fast absorption of resources and aids meeting spending targets;  High degree of transparency: all the documentation for potential applicants and the general public is readily available online, not least through a dedicated site (http://www.pluslazio.eu/);  High degree of transparency and certainty on selection procedures and criteria appears to depoliticize project selection (at the level of the individual P.L.U.S.);  The benchmarking of the different P.L.U.S. through the dedicated IT platform and the reserve for performing projects should encourage virtuous behaviors and drive performance;  The fact that P.L.U.S. documents have to specify the financial sustainability of infrastructure projects, after their completion, is a very welcomed aspect of the approach. Past evaluations on ERDF programs found that the unavailability of maintenance and running costs for infrastructure realized has been a major problem.82 There is scope for improvement, notably regarding result orientation and transparency of funding sources. Although the P.L.U.S. have to define output indicators and targets, the emphasis appears to be predominantly on financial performance and spending according to timetable (understandably, to an extent, given the critical situation of the ROP). While the list of approved projects describes the two rankings of the 16 selected P.L.U.S. programs, the overall financial plan of the P.L.U.S. (i.e., a summative table providing details on the scores and resources obtained by each P.L.U.S.), and the detailed financial plan of each individual P.L.U.S. (i.e., a table for each P.L.U.S. indicating the requested and obtained contribution to each intervention), it does not provide any information on the outputs/results targets attached to these. This is something that would have to be improved in a future program. The way in which the overall financial plan of the P.L.U.S. relates to the ROP is not entirely clear (different sources state different amounts and it is not clear whether the overall amount assigned to each P.L.U.S. comes entirely from the ROP or also from other sources).83 82 See Bachtler, J., Begg, I., Polverari, L., and Charles, D. (2013) Evaluation of the Main Achievements of Cohesion Policy Programmes and Projects over the Longer Term in 15 Selected Regions (from 1989-1993 Programme Period to the Present (2011.CE.16.B.AT.015), Final Report to the European Commission (DG Regio), European Policies Research Centre, University of Strathclyde (Glasgow) and London School of Economics. 83 For instance, the approval decree indicates an overall amount of funded investments of €147.3 million, of which €113.7 from the ERDF ROP, plus €24.6 million from ‘economies’ from Priority 5 and a further €8.9 million from activity I.7 (which a re both also based on the ROP). At the same time, the dedicated webpage in its update dated 20 December 2013 states that certified expenditure is equal to just 10.5 percent of resources allocated under the ERDF ROP (of €88.7 million). The ROP as reprogrammed in 2011 assigned to Priority 5 €80 million, of which €60 million for urban regeneration and €20 million for the promotion of attractiveness and entrepreneurship. 283 Potential disadvantages of the P.L.U.S. approach relate to the time involved in selecting them, additionality, and issues around the targeting of municipalities that are most in need:  The appraisal and selection procedure was perhaps lengthier than desirable (but it was also experimental to some degree);  Given that the public works were already in the pipeline, would they have been implemented anyway, even without support from the ROP? (Possibly not, given the tightening of public finances, but this would have to be verified);  The procedure may favor more institutionally advanced municipalities and, by the same token, penalize less endowed ones (which may also need more support for regeneration/to address social problems, etc.). In conclusion it can be said that the P.L.U.S. are a promising approach in several respects. The P.L.U.S. are an interesting example of how project application and selection procedures can encourage the pooling of different types of projects into wider plans characterized by both internal strategic coherence and external strategic fit with the goals of the ERDF program (and wider regional programming). It is also an example of how a two-tier selection procedure can drive the quality of the interventions proposed. It is a new type of support which was only introduced in the ROP in 2012, the actual performance of this new approach in terms of selecting projects should thus be appraised upon completion of the realized investments, but the integrated approach appears promising. Slovenia – Strengthening Regional Development Potentials OP (ERDF): Competitive Calls in Promotion of Entrepreneurship84 Competitive calls have been widely used in Slovenia in 2007-13, including for the OP Strengthening Regional Development Potentials, which is considered in this case study . Projects are selected based on a short criteria list by awarding scores in line with indicators. The approach has worked well in the Slovenian context, although high levels of selectivity have been criticized by unsuccessful applicants. Context Structural Funds management in Slovenia is largely centralized. The ERDF OP ‘Strengthening Regional Development Potentials’, which will be considered in this case study, is managed by the Ministry of Economic Development and Technology, and more specifically the EU Cohesion Policy Directorate. The following table shows the funding allocations by priority for the 2007-13 programming period. Slovenia ERDF Convergence Programme ‘Strengthening Regional Development Potentials’ Program Allocations (€ million) Priority Axis EU Investment National Public Total Contribution 1) Competitiveness and research excellence 402.1 71.0 473.1 2) Economic development infrastructure 396.9 70.0 467.0 84 Please note that this case study is largely based on the findings of an evaluation report: Rambøll Management Consulting (2012) Comparative Study of the Project Selection Process Applied in Cohesion Policy Programmes 20017-2013 in a Number of Member States. Final Report (http://ec.europa.eu/regional_policy/sources/docgener/studies/pdf/selection/selection_process.pdf, accessed 12.12.2013). 284 Priority Axis EU Investment National Public Total Contribution 3) Integration of natural and cultural potentials 263.2 46.5 309.7 4) Development of regions 619.4 109.3 728.8 5) Technical Assistance 28.0 4.9 32.9 Total 1,709.7 301.7 2,011.5 Source: Government Office for Local Self-Government and Regional Policy (2007) Operational Programme for Strengthening Regional Development Potentials for Period 2007-2013, 26.07.2007, p. 118. The majority of funding of the OP Strengthening Regional Development Potentials is implemented via competitive calls. There are some exceptions under Priority 3, where tourism projects are preselected. Overall, the implementation of the OP is progressing very well. By the end of 2012, ERDF commitments were at 95.1 percent and payments at 47.3 percent. Priority 1 performed above average (81.1 percent – see Table below) and the measure under which the case study falls stands even at 98.9 percent of funds already paid out. Commitments and payments under the OP ‘Strengthening Regional Development Potentials’ (by end 2012) Priority Axis Commitments, % Payments, % 1) Competitiveness and research excellence 98.0 81.1 2) Economic development infrastructure 86.8 45.0 3) Integration of natural and cultural potentials 83.8 64.3 4) Development of regions 76.0 69.1 5) Technical Assistance 102.0 47.3 Total 86.6 68.5 Source: Ministrstvo za gospodarski razvoj in tehnologijo (2013) Letno poročilo 2012 o izvajanju Operativnega programa krepitve regionalnih razvojnih potencialov za obdobje 2007-2013, September 2013. Project Selection Model and Procedures – Competitive Calls Funding under Priority 1 is allocated via the Ministry of Economy as the responsible intermediate body with calls being administered by subordinate agents . The Ministry tasks agents with the implementation of measures, for instance the Slovenian Technological Agency (now part of SPIRIT, Public Agency for the Promotion of Entrepreneurship, Innovation, Development, Investment and Tourism) and the Slovene Enterprise Fund (SEF, Slovenski podjetniški sklad). For the selected case of project selection, the SEF is the agent responsible for the whole selection procedure, including the preparation and publication of calls, assessment of applications and deciding on project selection. The ‘Public call for co-financing of purchase of new technology equipment for SMEs’ has been carried out in a similar way seven times in the current programming period. Most information in this report relates to the call for 2009-11 from 13 March 2009 and the last call of this kind for 2013-14 from 21 June 2013. The call covers parts of Measure 1.2 (Promotion of entrepreneurship) under Priority 1 (Competitiveness and research excellence). 285 The purpose of this public call is to select SMEs with the best potential to show increased growth and productivity. The aim is to encourage initial investments in tangible and intangible assets, namely those associated with the expansion of an existing business, expanding the company's activities into new products or a fundamental change in the overall production process of an existing business. Funding should enhance the competitiveness of supported firms, as measured by value added per employee. The amounts allocated are roughly €30 million for each call (€29.75 million for the call from 13 March 2009, €31.14 for the call from 21 June 2013), corresponding to around 17 percent of all funding under Measure 1.2. Eligible costs are physical investments (purchase of new machinery and hardware) and investments in intangible assets, namely the costs of purchasing new software up to a maximum of €40,000. Micro and small enterprises can receive up to 50 percent or a maximum of €250,000, medium- sized enterprises up to 40 percent or a maximum of €200,000, and companies from the transport sector up to 30 percent or a maximum of €150,000. Project selection involves the following stages: 1. Development of calls by the agent (SEF). Proposal for calls has to be confirmed by intermediate bodies (Ministry of Economy) and managing authority. 2. SEF publishes the call, which remains open for six weeks. 3. Submission of applications (postal) to SEF. 4. Provision of guidance (email, phone) to applicants by SEF. 5. SEF registers applications. 6. SEF carries out the assessment process. Eligibility and quality of proposal are checked simultaneously. 7. A group of SEF staff members prepares proposal for selection. 8. The intermediate body (Ministry of Economy) checks the proposed projects as well. 9. If the intermediate body agrees with the proposal, the SEF informs successful and unsuccessful applicants. Assessment results are also published online. Applications must contain:  The registration form, which can be found online (http://www.podjetniskisklad.si/elektronski-prijavni-obrazci.html) and needs to be submitted in printed form;  A signed declaration on the number of employees on the last day of the month preceding the application;  A business plan including an analysis of the two previous years, an outlook on future operations for at least three years and a certified balance sheet for the current year;  A proof of creditworthiness (not older than 30 days);  A list of the costs of the planned operations, accompanied by quotes and other supporting documents. The supporting documents are the basis for the assessment of the operation; and  A bank statement confirming a bank guarantee equal to 30 percent of the funding amount. The calls are open for one to three months and applications are processed very quickly. The 2009 call was published on 13 March 2009, the submission deadline was on 24 April 2009 and the conclusion of co-financed operations was expected by 30 March 2011. The 2013 call was published on 21 June 2013 286 and ran until 5 September 2013. In terms of workload for the SEF incurred by the assessment process, the registration of an application takes 10 minutes, while the whole assessment process takes 4-6 hours. Procedures have continuously been adjusted for each call, albeit only to a minor extent. For instance, the number of criteria was reduced from 8 to 7 (see below). Most other changes were due to the changing implementation rules and conditions set by the responsible intermediate body. Selection Criteria and Indicators Under Measure 1.2 ‘Promotion of entrepreneurship’ (in parts), beneficiaries need to be an SME. They can be micro companies (between 1 and 9 employees and annual turnover and/or annual total assets not exceeding €2million), small companies (between 1 and less than 50 employees and annual turnover and/or annual total assets not exceeding €10 million), or medium-sized companies (between 1 and less than 250 employees, with annual turnover less than €50million and/or annual total assets not exceeding €43 million). A limited number of selection criteria are used, and projects are awarded scores under each of them based on indicators. The last call carried out in 2013 involved seven appraisal criteria with different maximum points awarded (see table): Selection Criteria used for Competitive Calls under Measure 1.2 (2013) Criteria Max. points Introduction of modern technology related to new / improved products / 25 services Increase in value added per employee (average of years 2008 and 2012) 20 Innovativeness 15 Project impact on competitiveness of a company / applicant 10 Direct sales outside Slovenia (average of years 2008 and 2012) 10 Market orientation in conjunction with a new project 10 Impact of investment on the environment 10 Total 100 Points are awarded to applications on the basis of specific indicators for each criterion . For instance, full points (15) for ‘Innovativeness’ are awarded when the company has developed a new product / service. If there is only an improvement of a product / service, the company receives 10 points. If there is no new or improved product / service, the application receives 0 points. In the case of ‘Introduction of modern technology related to new / improved products / services’, the applicant receives points on the basis of a combination of whether there is a new / improved product / service and whether it is based on an innovation developed internally or externally. This applicant then receives 25, 17, 12, 5 or no points at all. When the call was launched for the first time, the number of employees in company (5 points) was also a criterion. This has been abolished later on, because this gave medium-sized companies an advantage. Also, it was found that new technology and additional employment do not necessary go together. The 5 points were moved to “Impact of investment on the environment,” which increased from a maximum of 5 to 10 points. 287 In order to receive support, the minimum threshold was 60 points. If there were more projects scoring above 60 points than there were available funds, those with the most points were selected for co- financing. If more applications would receive the same number of points, those with a higher score under criterion ‘Increase in added value per employee (average in years 2008 and 2012)’ would be selected. If even this would not enable a final ranking of projects, projects would be prioritized according to the highest number of points under the following criteria: ‘The introduction of modern technology in conjunction with new or radically improved products / services’, then ‘Innovativeness in connection with a project’, ‘Project impact on competitiveness of a company / applicant’, ‘Direct sales outside Slovenia (average of years 2008 and 2012)’ and, in the end, ‘Market orientation in conjunction with a new project’. Selected projects can only obtain full co-financing as requested. If this is not possible, the remaining sources were not used. In terms of monitoring, projects need to report regularly to the agent and indicators are collected at project level. These include value added per employee, number of jobs created, direct sales, additional income and profit. Sales and value added are part of on-the-spot checks. Performance Assessment The calls were very successful and allowed for the fast absorption of funds. In case of the call from 13 March 2009, 492 applications were submitted, 232 (47 percent) of which were selected. The call of 21 June 2013 resulted in the selection of 173 projects with a total funding of €31.1 million. Strategic fit is assured via the appraisal criteria, which are derived from the OP as the main strategic document. The strategic dimension features strongly in the tender documents. The aim of the tender is to increase the value added per employee, which corresponds to the aims set out in the program. For Priority 1 the impact indicator is ‘Increase in value added per employee in companies receiving financial support’ and the target for 2013 was an increase by 8 to 10 percent (however, results were not available at the time of writing). A general analysis of the effects of the instrument conducted by the SEF based on recipient information shows that some companies were not able to achieve their targets due to the economic crisis. However, it is considered that the upgrading of equipment undertaken by some companies will also have a positive impact in terms of competitiveness and value added. Main Lessons Competitive calls are generally found to be highly suitable but high levels of selectivity give rise to criticism. The calls allow selecting the best projects in a fast and transparent way. However, in each call about two thirds of applications had to be rejected, of which a lot were nevertheless good projects. This creates frustration and criticism among rejected applicants. The threshold to be selected for funding had to be raised over time due to the high quality of applications. There are, nevertheless, some disadvantages to the use of competitive calls, notably relating to the administrative burden and the high level of selectivity. Time and workload pressures for involved staff are high due to applications coming in all at the same time. Due to the high quality of applications many good projects had to be rejected, causing frustration among rejected applicants. Several recommendations apply pertaining to simplification and capacity needs. It is advised to keep tender documents simple and to define clearly what costs are eligible. More generally, application procedures should be made as simple as possible for potential beneficiaries. Finally, a good team is needed that can cover the workload of dealing with c. 500 applications all at once and come to decisions within 90 days. 288 In conclusion, the project selection model applicable to this OP in Slovenia is seen as a useful way to select the best projects in a fast and transparent way. Hence, it will be continued (albeit for other measures) in the forthcoming 2014-20 programming period. However, there are concerns about how to deal with rejected applicants, especially if they represent good projects worth receiving funding. United Kingdom – West Wales and the Valleys ERDF Convergence Programme 2007-13: Open Calls under the Sustainable Transport Strategic Framework This case study focuses on open calls used to deliver the West Wales and the Valleys Convergence OP in the field of sustainable transport. Projects are selected in the context of Strategic Frameworks in a two-step approach (Expression of Interest followed by detailed proposal), using selection and prioritization criteria. Evidence shows that, while the Strategic Frameworks have helped select appropriate projects in an efficient way, their influence on project content and scope has been more limited. Context In the United Kingdom, Structural Funds’ management is a devolved responsibility. In Wales, two ERDF and two ESF OPs are managed by the Welsh European Funding Office (WEFO), which under the Welsh Government. In 2007-13, Wales was awarded around €2.24 billion of EU Structural Funds across the Convergence and Regional Competitiveness and Employment Programmes. The following table gives an overview of the funding allocated to the six priorities supported by the West Wales and the Valleys ERDF Convergence OP. West Wales and the Valleys ERDF Convergence Programme 2007-13 – Program Allocations (€) National EU National Public Priority Axis Private Total Investment Contribution Contribution 1) R&D, innovation, ICT 313,878,019 164,275,165 80,434,302 558,587,486 2) Enterprise and Business 144,778,000 61,196,140 68,876,850 274,850,990 Finance 3) Transport and Strategic 389,823,802 297,383,721 10,426,284 697,633,807 Infrastructure 4) Energy and Environment 229,898,349 123,962,861 32,439,198 386,300,408 5) Building Sustainable 159,000,019 73,929,440 5,266,283 238,195,742 Communities 6) Technical assistance 13,000,000 7,059,046 - 20,059,046 Total 1,250,378,189 727,806,373 197,442,917 2,175,627,479 Source: WEFO (2010) West Wales and the Valleys Convergence Programme Operational Programme for the European Regional Development Fund 2007-2013, Version of 24.09.2010, p. 191. Commitments are over 100% for most priorities, while payments have been progressing at a slower rate. By the end of 2012, 103% of the allocation available under the program was committed. The commitment was highest under Priority 5 at 120%. The percentages for Priorities 1 and 3 stood at 113% and 114%, respectively, and for Priorities 2 and 4 at 75% and 91%, respectively, while the percentage for Priority 6 was 47%. The total payments were at 34%. 289 Projects are selected via two selection models. First, WEFO operates open calls for project proposals under both the Convergence and the Regional Competitiveness and Employment (RCE) programs. Under these, organizations from the public, private, and third sectors can apply for funding, as a lead project sponsor, by submitting project bids directly to, and for assessment by, the managing authority, WEFO. Second, organizations can participate as partners, joint sponsors of projects, or (where the Welsh Government is the lead project sponsor) by bidding to deliver specific activities of approved projects, in response to procurement notices published on the Welsh Assembly Government’s Sell2Wales website. The present case study will be looking at the first route, where Strategic Frameworks are used to ‘help guide the project activities taken forward by sponsor applicants’ and “to assist WEFO in selecting and prioritizing projects.” Projects are also assessed against generic Selection and Prioritization Criteria that apply to all projects submitted under the open call system. Project Selection Model and Procedures – Open Calls organized in the Context of Strategic Frameworks Strategic Frameworks are overarching planning documents. They were introduced at the start of the 2007-13 programming period, defined as “plans to achieve a particular strategic purpose by means of interventions that are strategically linked.” They comprise: five Spatial Frameworks covering the delivery of spatially-focused ERDF themes in the six Wales Spatial Plan Areas, and 15 Thematic Frameworks covering the delivery of the Priorities and Themes in the ERDF and ESF programs. For ERDF programs, there are thematic and spatially driven Strategic Frameworks addressing the areas of business finance and enterprise, ICT, R&D and innovation, community regeneration, sustainable transport, and the environment. For ESF programs, there are thematic frameworks addressing the areas of employment and skills, supporting young people, promoting gender equality, and improving public services. This case study examines the Sustainable Transport Strategic Framework. Strategic Frameworks provide an operational strategy intended to help focus support on the types of interventions that deliver best on program priorities. A total of 20 thematic and spatially-driven Strategic Frameworks have been developed in partnership, to:  improve impact on growth and jobs;  strengthen strategic alignment between EU and Welsh Government policies;  encourage more ”joined up” action;  assist in reducing the overall volume of projects; and  help shape and balance program delivery. The Frameworks have two specific purposes: to guide project applicants on the strategic approach that partners in Wales have agreed upon to deliver on the aims objectives and targets of the Operational Programmes, and to help identify the extent to which project proposals will contribute to these; and to assist the managing authority in selecting and prioritizing projects. There are three stages involved in applying for EU funding as a lead project sponsor under the open call system: 1. Preliminary exploratory activity: Applicants must ensure that their project is eligible for grant support by first checking eligibility rules, project and selection criteria, and other guidance provided by the managing authority. They should also ensure that the proposed project fits within the relevant Strategic Framework (the operational strategy that focuses on the types of intervention that are considered to best deliver on the program priorities). These are available for applicants to browse or download from the managing authority’s website. Applicants must then take part in Collaborative Planning, where the project idea is discussed with stakeholders, 290 partners, and potential partners or match-funders, particularly those who have similar concepts they intend to pursue. This is facilitated through the managing authority’s website, which has a Project Idea section, where other project ideas can be viewed and the applicant’s own idea can be posted for feedback. Applicants must be prepared to evidence that this collaborative planning has taken place – i.e., that they have examined relevant project ideas on the website, that they have discussed the proposal with other organizations seeking to operate in the same or a similar way, and that they have carried out substantial discussions with major stakeholders and secured their support. The applicant can then request to submit an Expression of Interest (EoI) to the managing authority. 2. Submission of an Expression of Interest to WEFO: The applicant must make a request to submit an Expression of Interest (EoI) on the managing authority website. In response to the request to submit an EoI, the relevant Project Development Officer (PDO) in the managing authority will contact the applicant and work with them to make progress on their EoI. The decision to progress, defer, or reject an EOI rests with the Priority Controller within the managing authority, who makes a decision based on the recommendations of the PDO. If the request is successful, the Project Development Officer will contact the applicant to discuss the proposal and guide her through the rest of the process. Following formal submission of the EoI, which provides enough information for the managing authority to judge whether a more detailed project assessment is appropriate, the managing authority will carry out an initial assessment against specified Selection and Prioritization Criteria. A decision will be made to either: take the proposal forward to the next stage of full assessment; hold it on “reserve status” pending further information or wider program management considerations; or reject it. 3. Submission of a detailed project proposal for full assessment: At this stage, the applicant is asked to produce a more detailed Delivery Plan, which will be considered against the Selection and Prioritization Criteria. The PDO will work with the applicant to clarify project details and identify any risks or conditions that will need to be covered. Experts may also be called to assess points of detail in the proposal and to provide evidence in support of judgments made. The managing authority will issue a formal decision on whether support can be offered. If support is offered, a detailed letter of approval will be issued. Over the life of the program, as conditions change, operational strategies set out in the Strategic Framework documents may need to change too. Therefore, each Framework is subject to ongoing monitoring, evaluation, and review. For example, the Frameworks were modified as a result of the introduction of the Welsh Government’s new Economic Renewal Programme in 2010. Changes to the approach are planned for 2014-20 (see below). Selection Criteria and Indicators - Sustainable Transport Strategic Framework Sustainable Transport is covered by Theme 2, including the majority of funding under Priority 3 of the ERDF Convergence Programme. According to the OP, funding allocated to this theme was expected to total around 80% of that allocated to the priority. The Strategic Frameworks are intended to guide prospective project applications as well as the managing authority. The Strategic Framework for Sustainable Transport was developed jointly by the Welsh Government and the Welsh Local Government Association. Other partners were also involved, including regional transport consortia and Wales Spatial Plan Action Groups. It was also developed in parallel with the Wales Transport Strategy (WEFO, 2011). The Strategic Framework is an operational strategy document and it sets out details on the priority and theme, key indicators relevant to sustainable transport, and examples of the type of strategic intervention which are likely to be taken 291 forward under the theme. It also provides background detail on the spatial and financial context for the theme, and on key principles and regulations affecting activity under the theme. The Strategic Framework document also provides some guidance on eligibility issues and on the types of project that will be given priority, for example:  Priority will be given to schemes that promote agglomeration effects;  Priority will be given to innovative schemes that serve as exemplars or models of best practice in sustainable transport solutions. It also outlines activities that will not be supported (e.g., projects that are commercially viable without support). Project proposals then go on to be assessed against a standard set of selection criteria which apply to all projects submitted in response to the open calls. The selection criteria are published on the managing authority’s website. The Selection and Prioritization Criteria matrix introduced in 2007-13 was so-called to emphasize that projects are appraised against each other, i.e., it is a competitive approach, and that projects will be ranked according to their score. The matrix (see below) outlines a set of positive criteria (“contribution factors”) and a set of risk factors (“certainty factors”), and projects are ranked using this matrix. Projects score between 1-5 on the degree of fit and risks. Judgment on scoring is informed by experts. Specialist software is used to score each of the benefits and risk criteria, weigh them against each other, give a score to the project, and rank all projects against each other. Projects are appraised against the Selection and Prioritization Criteria, while additional appraisal guidelines also exist that are used to different degrees (for example, the UK central government’s Green Book guidelines that apply to spending by central government departments and agencies rather than devolved entities, and Welsh Audit Office definitions). The basic principles of the Green Book are based on sound challenge to the perceived need for a project in the first place, whether the objectives can be achieved in any other way through an options appraisal, and, where appropriate, a form of cost- benefit analysis. The key phrase “where appropriate” is related to the size of the project, its degree of complexity, its novelty, or a combination of those factors. The time and cost of making a full cost-benefit analysis must be set against the potential advantages that can be obtained through the results. The Welsh Audit Office defines its expectations for sound project appraisal around several issues. First, there needs to be an assessment of whether a bid or proposed project is worthwhile, considering its objectives, the various options available, their costs and benefits, and the risks and uncertainties. This assessment has to be done before a decision to apply for (or award) funding is made, as it helps ensure that: proposed projects meet the objectives; funding is actually needed for the project to go ahead; financial propriety over public money is observed; value for money and sustainable development will be achieved; and appropriate management arrangements are in place to deliver results and monitor progress. The managing authority in Wales has worked extensively on developing the monitoring system for the 2007-13 period and indicators have been streamlined as compared to the 2000-06 period. The approach taken to monitoring in Wales includes monitoring the context within which the program is being implemented as well as monitoring the specific and attributable outputs and results of the program against which projects will report. The OPs contain high-level tracking indicators (context indicators) and program indicators (at priority level and aggregated to program level). The high-level tracking indicators were derived from the Lisbon indicators and the Welsh Government’s domestic economic development strategy. They are reported against in Annual Implementation Reports (AIRs). Monitoring indicators at priority level are included in the OPs (regarding outputs and results). Impact indicators are also included in the OPs, but these are assessed through evaluation rather than 292 monitoring. Some indicators are quantified to give targets. Progress against targets is also reported in the AIRs. In addition, analyses of program-level monitoring information is undertaken by the managing authority and reported to the Programme Monitoring Committee (PMC). Selection and Prioritization Criteria used in Project Appraisal Indicators were developed by the managing authority using a project logic chain approach, with output, results, and impact indicators for the activities, objectives and aims of the priorities . In this way, it is considered that indicators are clearly linked to the OPs and to each other. Projects must select 293 relevant indicators from the OPs and detail within their monitoring plans how they intend to collect and process monitoring data. The managing authority has produced guidance that provides definitions and details of evidence requirements for the ERDF monitoring and evaluation indicators. Successful applicants must submit a detailed ‘Delivery Profile’ that includes all indicators and associated targets. This is submitted to WEFO online. Projects are encouraged to collect information beyond what they are required to report to the managing authority, which could be used in the project-level evaluations. The Strategic Framework for Sustainable Transport highlights the indicators considered most relevant. These are as follows: Most Relevant Output, Result and Impact Indicators under the Strategic Framework for Sustainable Transport Outputs Public transport services created or improved Intermodal facilities created or improved Railroads created or reconstructed Access route created or reconstructed Footpath or cycleway created or reconstructed Results Gross jobs created Goods on rail or sea freight Gross passenger kilometers on public transport Impacts Net jobs created Net goods on rail or sea freight Net passenger kilometers on public transport Value for time savings Population within travel time threshold of a ‘key center’ Performance Assessment and Main Lessons The Strategic Frameworks have helped select appropriate projects in an efficient way without, however, triggering in-depth change in project content or scope. A 2011 evaluation of the delivery of Structural Funds in Wales concluded that the Strategic Frameworks have “served their purpose” and been useful in assisting WEFO staff to assess individual projects for eligibility and strategic fit. They have also played an important role in letting applicants know what was technically ineligible and not in alignment with the policy. However, the Strategic Frameworks were found to have had a limited role in the post approval stage. While they were seen to have had a positive influence in guiding applicants toward desired areas of activity, playing a role in reducing duplication and competition between projects, they have not fundamentally influenced or altered the design, spatial location or coverage of projects. In addition, whereas the Frameworks do not seem to have turned potential applicants away from applying for funding, some have been perceived as ‘closed shops’ to sponsors outside of the Welsh Government. Specific challenges identified included:  Getting the right balance between providing enough detail in the Frameworks to help applicants develop their projects without becoming too prescriptive;  Interest in the Frameworks reduced as the program progressed, but this was probably largely due to most of the program funding being committed. In addition, they have been overtaken by other key policy developments. 294  There was (a low level of) concern that the Frameworks may have been used in an overly mechanistic way at the start of the programming period by Project Development Officers, but as experience and knowledge built up, they were used in a more sophisticated way. A 2013 report reviewing Structural Funds implementation arrangements in Wales (Guildford, 2013) reported mixed views on the effectiveness of the Strategic Frameworks in practice: from having provided useful direction in early parts of the program, to having a declining value as the programs progressed and there were major changes in the economic context. One negative view was that the Strategic Frameworks: “did not go down to a level of detail sufficient to allow a clear delineation between areas and activities that were a priority for support and those that were not. This may have made it more difficult to achieve an appropriate balance between eligibility (compliance risk) criteria and economic suitability (business risk) criteria, which in turn can affect the appropriate balance between a focused and a more widely distributed expenditure pattern.” Some practical recommendations have been made regarding the 2007-13 and 2014-20 programs. The 2011 evaluation of the delivery of Structural Funds in Wales made a series of recommendations related to Strategic Frameworks. These included, in relation to the 2007-13 programs:  The managing authority, in conjunction with Strategic Framework Coordinators should carry out a stocktaking exercise of projects already approved under each Strategic Framework. WEFO should then publicize specific areas where it would like to see new project ideas come forward.  WEFO should continue to use the Strategic Frameworks as part of the process of assessing the eligibility and strategic fit of project applications. Where funds remain to be committed, the Strategic Frameworks should be refreshed to reflect major policy announcements.  The relationship between WEFO Project Development Officers and Strategic Framework Coordinators needs to be maintained, given a fresh impetus and where necessary re- established. In relation to the 2014-20 programs, the evaluators recommended:  The concept of strategic guidance that complements Operational Programmes and prioritizes fields of intervention is sound. However, the managing authority and the Welsh Government need to consider the exact form this should take for any future programs.  In any future programs where Strategic Frameworks are used, the managing authority should seek as an internal management tool to break down priority level performance indicator targets by areas of eligible activity relevant to each Strategic Framework and should monitor progress against these to enable a clearer understanding of the extent to which the desired balance between different elements of each Strategic Framework is being achieved as the program progresses.  In any future programs, the role and on-going function of Strategic Framework Co-coordinators should be more clearly defined and communicated to the individuals that hold these posts. Looking ahead, a two-phase approach is foreseen, entailing the pre-selection of key projects, followed by a more responsive and innovative approach later in the program. The publication of the 2013 Guildford report, which reviewed Structural Funds implementation arrangements in Wales (Guildford, 2013), is expected to inform the approach to project selection and prioritization in the 2014-20 programs. The report recommends the development of an Economic Prioritization Framework (EPF), which allows the areas to which Structural Funds can contribute most effectively in Wales to be identified, and also identification of synergies with Welsh Government policy. It is intended that the EPF 295 be constructed by the managing authority ‘with enough detail to provide guidance to potential project sponsors, to inform decision making at the time of project selection and support the management of implementation.’ The report recommends that, to help discriminate between “compliance risk” and “business risk,” a two-phase mechanism is created in which “eligibility is seen in the context of a gateway function and in which the EPF subsequently informs a selection process based on fit with its priorities.” The project appraisal system would therefore:  aim for the early establishment of key backbone projects that would form the basis of the program; and  later in the program approval could focus on gap filling, could be more responsive to change and could involve more innovative approaches (including commissioning) and be able to tolerate a higher overall risk profile, given that the risk profile of the backbone projects might be relatively low. Ireland – Border, Midland and Western (BMW) ROP European Union Structural Funds have played an important role in the success of the Irish economy and the dramatic rise in the standard of living across all sectors of Irish society over the past decade. Since 1973, almost 19 billion EUR of EU Structural and Cohesion Funds have been invested in Ireland. The Economic and Social Research Institute estimated that structural funds over the first two programming periods combined (1989-93 and 1994-99) led to a permanent increase in GNP of 2%. They also estimated that funding for the first three years of the 2000-06 programming period added a further 0.7% to GNP.85The government has concentrated Structural Funds on programs designed to strengthen competitiveness through three broad areas: infrastructure, human capital, and the productive sector, including manufacturing, tourism and agriculture. As the Structural Funds allocations increased in the 1990’s, the government increased the share of funds allocated to infrastructure. Ireland has used EU Structural Funds to its benefit for many years and has taken the opportunity to use the availability of these funds as a driver for reforms to its PIM processes over the years. The National Development Plan (NDP) 2007-2013, launched in Dublin Castle on January 23, 2007, is a high level strategic document that sets out, within a sustainable economic and budgetary framework, indicative seven-year investment allocations. These cover various sectors, totaling almost 184 billion EUR, at a much larger level compared to EU contributions over the same period.86The NDP 2007-2013 sets out the key high level objectives of investment for Ireland for 2007-2013, as follows: promotion of sustainable economic and budgetary stability; promotion of national competitiveness; fostering better balance in regional development with a particular focus on assisting all regions to deliver to their maximum potential; delivering economic and social infrastructure in an efficient Value for Money way that improves the quality of life of citizens; ensuring environmental sustainability; and promoting social inclusion. The economic and social progress made by Ireland through previous EU funding now means that structural financing is significantly less than in previous years. It also means that funding from the EU now plays only a tiny role in public investment funding in Ireland. The ERDF actions thus target niche areas that generate high levels of added value: for instance, supporting the Research and Development 85 See the Irish NSRF document 86 Although this has been substantially re-modelled following the economic crisis 296 (R&D) work of institutes of technology in the BMW region, environmental flagship projects, small business development schemes and specific public transport projects. Within the agreed objectives of Cohesion policy, Structural Funds are focused on implementing community policies, namely the renewed Lisbon Agenda 5 and the Göteborg Declaration 6. The specific focus is on innovation and the knowledge economy, environment and risk minimization, accessibility to services of general interest (e.g., broadband, transport infrastructure), increasing adaptability of workers and enterprises, enhancing access to employment, and social inclusion measures. This case study refers only to the BMW OP. The Irish Government, as set out in the National Strategic Reference Framework4 (NSRF), decided that Cohesion policy would be delivered through separate Regional OPs for the Border, Midland and Western (BMW) Region and the Southern & Eastern (SE) Regions, and through a national European Social Fund OP. These programs are part-funded by the ERDF. Project Selection Ireland has a comprehensive set of guidelines for the appraisal and management of capital investment projects, dating from 2005.87These are to a large extent reflective of the UK’s appraisal system and put ‘value for money’ at the heart of the process, focusing on a risk-adjusted balance of price and quality. The guiding principles for selection models are as follows: transparency in the process (all potential beneficiaries should be aware of the process and criteria to be used); incorporation of OP objectives into the selection criteria; incorporation of applicable horizontal principles (e.g., sustainability, equality) into the selection criteria; use of competitive selection processes, where feasible; cost effectiveness and value for money; and consistency in the appraisal of proposals. There are almost as many selection criteria operating under these principles as there are possible interventions. A sample of published selection criteria for proposed action areas is shown in the figures below. The examples are all from the same document88 and show the diversity of selection criteria used, the range of sophistication, and also the degree of transparency ranging from vague (Example 1) to excellent (Example 5). It can be argued that the authorities are allowing enough discretion to awarding bodies to tailor their exact objectives to the selection criteria. Others may argue that transparency suffers and the possibility for highly subjective selection practices increases. Selected examples of selection/evaluation criteria for the Irish OP Example Selection/Evaluation Criteria  Achievement of balanced regional development  Achievement of sectoral balance 1  Mutual agreement between Enterprise Ireland and college authority on appropriate research agenda for the region  Track record to date  Value for money 2  Commitment of the host institution  Contribution to balanced regional development  Commercialization potential 3  Return on investment  Technical feasibility 87 Department of Finance: Guidelines for the Appraisal and Management of Capital Expenditure Proposals in the Public Sector, February 2005 88 Implementation Plans for the Border Midland and Western Regional Operating Plan 2007-2013 Updated November 2011 297 Example Selection/Evaluation Criteria  Track record of researcher(s)  Project management  Overall quality of application  Promotion of national/regional strategies  Alignment of proposals to institutions’ overall missions and strategies  Promotion of knowledge transfer and commercialization opportunities 4  Past performance in developing and exploiting research capabilities  Level of inter-institutional collaboration and resource sharing  Overall benefits and likely impact  Contribution to national and institutional strategy (85 pts. max) o Alignment to national and institutional strategies (40 pts.) o Degree to which delivery of institutional strategies is enabled (20 pts.) o Extent to which proposal facilitates collaboration (25 pts.)  Quality of proposal (40 pts. max) 5 o Clear identification of needs and objectives (20 pts.) o Optimum solution identified to address need (20 pts.)  Implementation (75 pts. max) o Outputs justify proposed investment (40 pts.) o Feasibility of work plan and delivery timeline (20 pts.) o Risks identified and contingencies detailed (15 pts.)  Quality of proposed program  Track record 6  Value for money  Regional impact  Strategy to produce credible pipeline of applicants All examples are from the same implementation and it is notable that they use relatively subjective and hard to measure criteria. For instance, estimating “commitment of the host institution” in Example 2 or “commercialization potential” in Example 3 is not an easy or straightforward task. In general, such qualitative criteria should be better defined in order to ensure fairness and transparency, though it is possible that evaluators have more in-depth grids that help them score on similar scales. Implementation Capacity Ireland benefits from around 30 years of concerted effort to develop its PIM capacity . It also makes extensive use of external consultancy, which has become a thriving industry in the country. Due to the well-developed skills of public sector managers, they are able to fulfill the role of “intelligent customer” when consultants are employed, knowing what to ask for and holding them accountable for results. In this way, capacity requirements are readily achievable. In the case of the vast majority of projects that fall outside the normal definitions of PIM and might be considered short-term service contracts, Ireland is also able to benefit from a well-developed skill set in public procurement. Its officials not only fulfill the administrative and control requirements, but they are required to be enablers and assist project initiators to structure their objectives and align them with the finally agreed selection criteria. Additionally, where these skills are not available – as might be the case in more rural areas– there is a thriving consultancy community that can provide assistance as needed. This allows project initiators and the intermediate bodies to manage the preparation of projects in a satisfactory manner, a recommendation also made for the Romanian ROP in the report on “ROP 2.0: Facilitation of Proactive and Direct Support for Applicants and Beneficiaries of the ROP 2014-2020.” 298 Lithuania – Economic Growth OP Lithuania is one of the smaller EU countries and takes on many of the characteristics of a region within other Member States. Due to this, there is no ROP and instead the study has focused on the Economic Growth Operational Program, which represents almost 50% of the Structural Funds allocation and is supported by the European Regional Development Fund (ERDF) and the Cohesion Fund (CF). The core targets of the Lithuanian National Strategic Reference Framework (NSRF) concern: (a) accelerating long- term economic growth; (b) creating more and better workplaces; and (c) developing social cohesion. To achieve these targets, the NSRF includes three key objectives – namely, promoting productive human resources for the knowledge society, building a competitive economy, and supporting quality of life and cohesion – and four horizontal areas to which the entire strategy must contribute: information society, sustainable development, equal opportunities, and regional development. Lithuania is different from the other countries in this study in that it has moved in the opposite direction from the more common decentralization approach with many smaller projects. It has instead gone for a centralized strategy based on larger investments of national or regional significance. In the OP, five projects were identified or pre-selected, all in excess of 50 million EUR. It is interesting that this approach is in sharp contrast to the one employed in the previous programing round during which the country focused on smaller, localized interventions. Infrastructure has been given a higher priority for the current programming period, as shown in the figure below. To measure the impact of the strategy, Lithuania has proposed a range of target objectives for the end-year of the programming period (2013). These include an increase in the employment rate for the 15-64 year-old population to 70% (compared to 61.2% in 2004), as well as an increase in labor productivity to 65% of the EU average (from 41.7% in 2004). Another objective is to increase R&D expenditure to 2.2% of GDP. In particular, the main objectives of the Operational Programme on Economic Growth are as follows: increase the share of high value added businesses; boost business productivity especially by creating a favorable environment for innovations and SMEs; and increase efficiency of economic infrastructure. Project Selection Starting from the OP programmatic document it is clear that review, scrutiny, and challenge form an important part of the Lithuanian strategy. Although a requirement under EU procedures, the document itself contains many pages in annexes of ex-ante peer review, which contain a number of questions about proposed interventions and challenges their realism and capacity for implementation. The managing authority’s responses and an indication of the degree to which they have enacted these plans are also shown. The selection of projects is heavily influenced by national strategic requirements. Examples of this include: “Establishment and development of a modern North-South transport link (Tallinn–Riga– Kaunas–Warsaw) connecting the Baltic States and Poland” and the “Rehabilitation of the infrastructure of the West-East transport axis.” What is not clear at all is how the specific projects proposed for investment were identified as being more effective and impactful than others. Under EU requirements, a Cost Benefit Analysis for each one would be required in terms of demonstrating financial viability and economic value. However, this would not demonstrate how the projects indicated in the OP would be better than other potential projects not mentioned. No indication is given as to how these projects were selected other than their ability to meet the strategic directions. 299 Breakdown of financing by priority axis (in million EUR) Source: DG REGIO Much like other Member States, Lithuania has two basic models for intervention. First, there are the PIM style infrastructure-based projects that are subject to CBA. Regional Development Councils plan and pre-select projects based on their own high-level strategic goals, but even so pre-selection is based on criteria set up by a Single Monitoring Committee. They ensure that interventions are selected for funding in accordance with the criteria applicable to the operational program. The final decision on selecting projects is made by the relevant managing authority. This model removes significant discretion from local entities and places some distance between the identified “problem” and the optimal “solution.” Second, there are projects that enter competitive ‘open-call’ interventions. These are appraised through two basic selection tools: the degree to which the project proposal meets the strategic objectives in each priority axis and the degree to which the project can demonstrate its capacity to be implemented. The precise selection criteria for each axis are developed by the public body responsible for implementation. For example, for the priority axis on “Development of Trans-European Transport Networks” the Transport Investment Directorate within the Ministry of Transport makes the initial proposal, although the final approval for all PAs comes from the Single Monitoring Committee made up of a number of relevant bodies. Implementation Capacity The highly centralized model ensures that the available resources are concentrated rather than dispersed. Projects are designed and prepared by ministries and institutions with relevant technical skills. A further project management overlay is the existence of the Central Project Management Agency that, in addition to its own direct responsibilities, is able to provide specialized project management advice and assistance. It is also responsible for the Technical Assistance part of the OP, meaning that its skills can be put to use in designing targeted Terms of Reference for consulting firms that assist other implementing bodies. 300 Annex 11. Categorization of investments funded through the ROP 2007-2013 (CI = Connective Infrastructure; LF = Life Standards; MC = Marginalized Communities; QL = Quality of Life) AXIS 1 - SUSTAINABLE URBAN DEVELOPMENT Sub-Axis 1.1 - Integrated Urban Development Plans GROWTH POLES Rehabilitation of urban services infrastructure Public urban infrastructure QL, LF, CI Eligible activities: -> completion and/or renovation of worn-out and/or unused buildings and preparing them for new economic and social uses QL -> demolition of worn-out buildings QL -> conversion of unused or polluted lands to new uses QL -> development and rehabilitation of public spaces (streets, sidewalks, plazas, bridges, underpasses and overpasses, parking spots) QL -> development and rehabilitation of public services infrastructure (public lighting, broadband network, etc.) QL, LS -> brownfields redevelopment QL -> purchase of information and communication equipment (electronic public billboards, interactive billboards, etc.) QL -> traffic management systems QL, CI Transport and mobility QL, CI, MC, LS Eligible activities: -> development of new bus, tram, and trolleybus stations, and rehabilitation of existent ones QL, CI, MC, LS -> development of inter-modal transport hubs QL, CI, MC, LS -> development/extension/rehabilitation of tramway and trolleybus infrastructure QL, CI, MC, LS -> development of dedicated public transport lanes and bike paths QL, CI -> rehabilitation and modernization of county roads CI, MC -> improvement of public transport systems in rural areas CI, MC Urban UNESCO sites QL Eligible activities: -> restoration, protection, consolidation, and consolidation of historic sites QL Development/modernization/extension of tourism infrastructure and connected utilities QL, CI Improving the business environment Business Development/modernization/extension of business support structures Environment Eligible activities: -> construction/extension/ modernization of buildings to be used by economic Business agents (primarily SMEs) for production activities and services Environment -> construction/extension/modernization of road/rail infrastructure within Business business compounds, and development/rehabilitation of access roads Environment -> construction/extension/modernization of utilities within business structures (water treatment stations, energy and natural gas supply, sewage systems, Business cable infrastructure, broadband systems) Environment 301 Business -> purchase of technical equipment for businesses Development Rehabilitation of social infrastructure LS, MC Rehabilitation/modernization of social services buildings (child centers, centers for the youth, centers for disabled persons, centers for the elderly, etc.) LS, MC Purchase and installation of crime prevention equipment (e.g., surveillance cameras) LS URBAN DEVELOPMENT POLES Rehabilitation of urban services infrastructure Public urban infrastructure QL, LF, CI Eligible activities: -> completion and/or renovation of worn-out and/or unused buildings and preparing them for new economic and social uses QL -> demolition of worn-out buildings QL -> conversion of unused or polluted lands to new uses QL -> development and rehabilitation of public spaces (streets, sidewalks, plazas, bridges, underpasses and overpasses, parking spots) QL -> development and rehabilitation of public services infrastructure (public lighting, broadband network, etc.) QL, LS -> brownfields redevelopment QL -> purchase of information and communication equipment (electronic public billboards, interactive billboards, etc.) QL -> traffic management systems QL, CI Transport and mobility QL, CI, MC, LS Eligible activities: -> development of new bus, tram, and trolleybus stations, and rehabilitation of existent ones QL, CI, MC, LS -> extension and/or rehabilitation of tramway and trolleybus infrastructure QL, CI, MC, LS -> development of dedicated public transport lanes and bike paths QL, CI Urban UNESCO sites QL Eligible activities: -> restoration, protection, consolidation, and consolidation of historic sites QL Improving the business environment Business Development/modernization/extension of business support structures Environment Eligible activities: -> construction/extension/ modernization of buildings to be used by economic Business agents (primarily SMEs) for production activities and services Environment -> construction/extension/modernization of road/rail infrastructure within Business business compounds, and development/rehabilitation of access roads Environment -> construction/extension/modernization of utilities within business structures (water treatment stations, energy and natural gas supply, sewage systems, Business cable infrastructure, broadband systems) Environment Business -> purchase of technical equipment for businesses Development Rehabilitation of social infrastructure LS, MC Rehabilitation/modernization of social services buildings (child centers, centers for the youth, centers for disabled persons, centers for the elderly, etc.) LS, MC Purchase and installation of crime prevention equipment (e.g., surveillance cameras) LS 302 URBAN CENTERS Rehabilitation of urban services infrastructure Public urban infrastructure QL, LF, CI Eligible activities: -> completion and/or renovation of worn-out and/or unused buildings and preparing them for new economic and social uses QL -> demolition of worn-out buildings QL -> conversion of unused or polluted lands to new uses QL -> development and rehabilitation of public spaces (streets, sidewalks, plazas, bridges, underpasses and overpasses, parking spots) QL -> development and rehabilitation of public services infrastructure (public lighting, broadband network, etc.) QL, LS -> brownfields redevelopment QL -> purchase of information and communication equipment (electronic public billboards, interactive billboards, etc.) QL -> traffic management systems QL, CI Transport and mobility QL, CI, MC, LS Eligible activities: -> development of new bus, tram, and trolleybus stations, and rehabilitation of existent ones QL, CI, MC, LS -> extension and/or rehabilitation of tramway and trolleybus infrastructure QL, CI, MC, LS -> development of dedicated public transport lanes and bike paths QL, CI Urban UNESCO sites QL Eligible activities: -> restoration, protection, consolidation, and consolidation of historic sites QL Improving the business environment Business Development/modernization/extension of business support structures Environment Eligible activities: -> construction/extension/ modernization of buildings to be used by economic Business agents (primarily SMEs) for production activities and services Environment -> construction/extension/modernization of road/rail infrastructure within Business business compounds, and development/rehabilitation of access roads Environment -> construction/extension/modernization of utilities within business structures (water treatment stations, energy and natural gas supply, sewage systems, Business cable infrastructure, broadband systems) Environment Business -> purchase of technical equipment for businesses Development Rehabilitation of social infrastructure LS, MC Rehabilitation/modernization of social services buildings (child centers, centers for the youth, centers for disabled persons, centers for the elderly, etc.) LS, MC Purchase and installation of crime prevention equipment (e.g., surveillance cameras) LS Sub-Axis 1.2 - Energy Efficiency in Buildings Works on the exterior of apartment blocks QL Eligible activities: -> thermal insulation works QL -> upgrade of window panes QL 303 -> closure and thermal insulation of balconies QL -> thermal insulation of roofs QL -> thermal insulation of basements QL Interior spaces and installations of apartment blocks QL Eligible activities: -> thermal insulation of interior spaces that separate heated spaces from non- heated spaces QL -> rehabilitation of heat and hot water infrastructure QL -> thermostats for radiators and pressure knobs QL -> repair/replacement of the apartment block thermal unit QL -> purchase and installation of renewable energy systems (solar panels, heat pumps, biomass heating units) QL -> improvement of public transport systems in rural areas QL AXIS 2 - REGIONAL and LOCAL ROAD INFRASTRUCTURE Sub-Axis 2.1 - Rehabilitation and modernization of county and urban roads, including ring roads Rehabilitation and modernization of the county road network; Rehabilitation and modernization of urban streets; The construction/rehabilitation/modernization of ring roads that are classified as county roads or urban streets CI, MC, LS Indicative eligible activities (other may be proposed by beneficiaries): -> rehabilitation and modernization of county roads CI, MC -> rehabilitation and modernization of urban streets (category 1, 2, and 3) CI, MC -> construction/rehabilitation/modernization of ring roads classified as county roads or urban roads CI. MC -> construction/modernization/rehabilitation of bridges CI, MC -> construction/modernization/rehabilitation of overpasses CI, MC -> clearing and preparing of land for road development CI, MC -> deviation of water, electricity, gas, sewage, and phone networks CI, MC -> environmental works CI, MC -> modernization and extension of utility infrastructure that is part of the road (water, sewage, electricity, gas, phone) CI, MC, LS AXIS 3 - IMPROVING SOCIAL INFRASTRUCTURE Sub-Axis 3.1 - Rehabilitation/modernization/equipment of health services infrastructure Rehabilitation/modernization/equipment of county hospitals; Rehabilitation/modernization/development/equipment of ambulatory services LS, MC Indicative eligible activities (other may be proposed by beneficiaries): -> rehabilitation/modernization of hospital buildings and ambulatories LS -> rehabilitation/modernization of general and specific utilities of hospitals/ambulatories LS -> creation/modernization of access facilities for people with physical disabilities in hospitals/ambulatories LS, MC -> purchase of equipment for hospitals/ambulatories LS Sub-Axis 3.2 - Rehabilitation/modernization/development and equipment of social services infrastructure Rehabilitation/modernization/development and equipment of buildings for multifunctional social centers; Rehabilitation/modernization and equipment of buildings for residential social centers LS, MC Indicative eligible activities (other may be proposed by beneficiaries): -> modernization/extension of social center buildings LS, MC 304 -> modernization/extension of buildings for the establishment of new social centers LS MC -> modernization of general and specific utilities of social centers LS, MC -> creation/modernization of access facilities for people with physical disabilities in hospitals/ambulatories MC, LS -> purchase of equipment for social centers LS, MC Sub-Axis 3.3 - The equipment of emergency intervention units Purchase of vehicles and specific equipment for emergency intervention units LS Sub-Axis 3.4 - Rehabilitation/modernization/equipment of educational infrastructure for universities, pre- university education units, and education units for continued learning Rehabilitation/modernization/equipment of university and pre-university educational facilities; Creation and development of pre-university campuses; Rehabilitation/modernization/equipment of continued learning centers LS, QL Indicative eligible activities (other may be proposed by beneficiaries): -> consolidation/modernization/extension of buildings (all types of compulsory education facilities, with the exception of campuses for technical and professional learning) LS, QL -> consolidation/modernization/extension and equipment of buildings for special schools: learning units, housing spaces, cafeterias LS, QL -> construction/extension/consolidation and modernization of buildings for technical and professional learning centers LS, QL -> purchase of didactical equipment, equipment for professional learning, IT equipment LS, QL -> consolidation/modernization/extension and equipment of state-funded university centers LS, QL -> consolidation/modernization/extension of buildings for continued leraning centers LS, QL -> modernization of utilities, including special facilities for people with disabilities, for all types of educational infrastructure LS, QL AXIS 4 - SUPPORT for the REGIONAL and LOCAL BUSINESS ENVIRONMENT Sub-Axis 4.1 - Sustainable development of support structures for regional and local businesses Eligible activities: -> construction/extension/ modernization of buildings to be used by economic agents Business (primarily SMEs) for production activities and services Environment -> construction/extension/modernization of road/rail infrastructure within business Business compounds, and development/rehabilitation of access roads Environment -> construction/extension/modernization of utilities within business structures (water treatment stations, energy and natural gas supply, sewage systems, cable infrastructure, Business broadband systems) Environment Business -> purchase of technical equipment for businesses Development Sub-Axis 4.2 - Brownfields redevelopment PHASE A: Rehabilitation of the polluted site QL, LS, MC Eligible activities: -> decontamination of polluted lands QL, LS, MC -> buildings demolition and land leveling QL, LS, MC PHASE B: Preparing rehabilitated site for new activities QL, LS, MC Eligible activities: -> construction/modernization/extension of buildings and building annexes, which will be used by economic agents for production activities and service provision QL, LS, MC -> buildings demolition and land leveling QL, LS, MC -> construction/extension/modernization of road/rail infrastructure within business Business compounds, and development/rehabilitation of access roads Enviornment -> construction/extension/modernization of utilities within business structures (water Business treatment stations, energy and natural gas supply, sewage systems, cable infrastructure, Environment 305 broadband systems) Business -> purchase of technical equipment for businesses Development Sub-Axis 4.3 - Support for micro-enterprises Eligible activities: -> purchase of equipment and new technologies, for production activities, service delivery, or Business construction Development Business -> purchase of IT systems (hardware and/or software) Development Business -> construction/extension/modernization of production spaces and service delivery spaces Development AXIS 5 - TOURISM DEVELOPMENT Sub-Axis 5.1 - Restoration and sustainable capitalization of cultural patrimony, and the development/modernization of connective infrastructure Restoration, protection and conservation of world/national patrimony and the modernization of connective infrastructure QL Sub-Axis 5.2 - Development/modernization of tourism infrastructure for the capitalization of natural resources and enhancing the quality of touristic services Business Development of natural tourist attraction Environment Business Capitalization of mountainous touristic potential Environment Business Development of hot springs tourism Environment Business Development, rehabilitation, and extension of touristic infrastructure, including related utilities Environment Sub-Axis 5.3 - Tourism promotion On-line marketing Marketing Participation at tourism fairs and expositions Marketing Promoting tourism products abroad Marketing Direct marketing (e.g., flyers, letters) Marketing TV marketing Marketing Publicity activities for tourism development Marketing 306 Annex 12. Schemes for proposed scenarios General Scheme for Scenario 1 – “Status Quo+” 307 General Scheme for Type 1 Projects under Scenario 2 308 SCENARIO 2 – “Semi-competitive” Proposed selection procedures for Type 1 projects according to the Public Investment Management Framework Public Investment Management Framework Stage 1 – Investment Guidance, Stage 2 Stage 3 Stage 4 Project Preparation & Preliminary Project Appraisal Independent Review Selection & ROP Screening of Appraisal Contracting Procedures Project Preparation:  Project concept, application form and prefeasibility study  By associations of county councils or by future regional administrative entity  Project must be listed in Regional Development Plan Strategic Screening: Guidelines and  Done by a strategic screening Announcement committee comprising both central and regional representatives  Done by MoF and approval for passing to FS stage and publication in the List of prioritized projects  Based on objective measurable criteria and clear procedures (proposed by the MA and approved by the Monitoring Committee of ROP)  Twice per year A set of 2-4 projects per region within Content: pre-allocated funds  FS  Technical Studies  CBA only for projects > 50 mil Euro  Operation and Application Maintenance Costs Submission  All are submitted in digital copies Project Preparation Costs: sufficiently co-funded by MA and future regional entities Centrally prepared evaluation grid Admin.& Elig Verification 309 Centrally prepared evaluation grid Independent evaluators:  Selected by MA  Payment sufficiently funded by MA  Procurement criteria weighed more on quality Technical and and experience Financial than costs Evaluation  Mandatory tasks: desk review, give scores, site visit and make recommendations for project improvements in writing and in meeting with applicants Applicant improves Improvement project at feasibility of Project at study stage according to Feasibility the recommendations of Study Stage independent evaluators Independent Project passing certain Verif. of evaluators verify the threshold scores gets project improvement of funded improvement project at feasibility & final eval. study stage and give final scores  Financing Contract is signed between MA and applicant with FS attached  Then applicant selects between the following options: Option 1: Red FIDIC: Two contracts in Financing sequence: first Contracting contract for design and second contract for build Option 2: Yellow FIDIC: Applicant contracts both design and build with a company in one contract 310 General Scheme for Type 2 Projects under Scenario 2 311 SCENARIO 2 – “Semi-competitive” Proposed selection procedures for Type 2 projects according to the Public Investment Management Framework Public Investment Management Framework Stage 1 Stage 2 Stage 3 Stage 4 Investment Guidance, Project Project Appraisal Independent Review of Selection, Detailed Technical Preparation & Preliminary Appraisal Design & Contracting ROP Screening Procedures Restricted call of proposals:  Prepared by RDA or a future regional entity and approved by regional monitoring committee  Screening done by MoF and approval for passing to FS stage and publication in the Guidelines List of prioritized projects by and the Government Announce  Project must be derived ment from Integrated Development Plans for municipalities or equivalents for County Councils  Deadline for pre-allocated fund availability in Phase 1 and FIFO in Phase 2 Content:  FS  Technical Studies  Cost Effectiveness Analysis  CBA only for projects > 50 Application mil Euro Submission  Operation and Maintenance Costs  All are submitted in digital copies Preparation Costs: by applicants Admin. & Centrally prepared evaluation Elig. grid Evaluation Centrally prepared evaluation Independent evaluators: grid  Selected by MA or RDA or a future regional entity  Payment sufficiently funded by MA  Procurement criteria weighed more on quality Technical and experience than costs and  Mandatory tasks: desk Financial review, give scores, site Evaluation visit and recommendations for project improvements in writing and in meeting with applicants 312 Improveme Applicants improve project nt of at feasibility study stage Project at according to the Feasibility recommendations of Study Stage independent evaluators Independent evaluators verify Phase 1: projects passing the improvement of project at certain threshold scores get Verification feasibility study stage and give funded. of project final evaluation scores Phase 2: projects passing a improveme certain threshold are nt and final selected using FIFO evaluation approach.  Financing Contract is signed between MA and applicant with FS attached  Then applicant selects between the following options: Option 1: Red FIDIC: Two Financing contracts in sequence: first Contracting contract for design and second contract for build Option 2: Yellow FIDIC: Applicant contracts both design and build with a company in one contract 313 General Scheme for Type 3 Projects under Scenario 2 314 SCENARIO 2 – “Semi-competitive” Proposed selection procedures for Type 3 projects according to the Public Investment Management Framework Public Investment Management Framework Stage 1 Stage 2 Stage 3 Stage 4 Investment Guidance, Project Project Appraisal Independent Review of Selection, Detailed Technical Preparation & Preliminary Appraisal Design & Contracting ROP Screening Procedures Restricted call of proposals:  Prepared by RDA or a future regional entity and approved by regional monitoring committee  Screening done by MoF and approval for passing to FS stage and publication in the List of prioritized projects by Guidelines the Government – for large and projects. Announce  Project must be derived ment from Urban Integrated Development Plans for municipalities or equivalents for County Councils  Deadline for pre-allocated fund availability in Phase 1 and FIFO in Phase 2 Content:  FS  Technical Studies  Cost Effectiveness Analysis  CBA only for projects Application > 50 mil Euro Submission  Operation and Maintenance Costs  All are submitted in digital copies Preparation Costs: by applicants Admin. & Centrally prepared evaluation Elig. grid Evaluation 315 Centrally prepared evaluation Independent evaluators: grid  Selected by MA or RDA or a future regional entity  Payment sufficiently funded by MA  Procurement criteria weighed more on quality and Technical experience than costs and  Mandatory tasks: desk Financial review, give scores, site visit Evaluation and recommendations for project improvements in writing and in meeting with applicants Improveme Applicants improve nt of project at feasibility Project at study stage according to Feasibility the recommendations of Study Stage independent evaluators Independent evaluators verify Phase 1: projects passing the improvement of project at certain threshold scores get Verification feasibility study stage and give funded. of project final evaluation scores Phase 2: projects passing a improveme certain threshold are nt and final selected using FIFO evaluation approach.  Financing Contract is signed between MA and applicant with FS attached  Then applicant selects between the following options: Option 1: Red FIDIC: Two Financing contracts in sequence: first Contracting contract for design and second contract for build Option 2: Yellow FIDIC: Applicant contracts both design and build with a company in one contract. Option 3: Green FIDIC for small projects 316 General Scheme for Type 4 Projects under Scenario 2 317 SCENARIO 2 – “Semi-competitive” Proposed selection procedures for Type 4 projects according to the Public Investment Management Framework Public Investment Management Framework Stage 1 Stage 2 Stage 3 Stage 4 Investment Guidance, Project Appraisal Independent Review of Selection, Detailed Project Preparation & Appraisal Technical Design & Preliminary Screening Contracting ROP Procedures Restricted call for proposals:  Prepared by RDA or a future regional entity and approved by regional monitoring Guidelines and committee Announcement  Implicit strategic screening realized by eligibility and evaluation criteria Content:  Feasibility Study and O&M funding for public sector; Business plan for private sector and NGO  Technical Studies  O&M Costs Application  Digital copies Submission Preparation Costs: by applicants Centrally prepared Admin&Eligibility evaluation grid Evaluation 318 Centrally prepared Independent evaluators: evaluation grid  Selected by MA or RDA or future regional entity  Payment sufficiently funded by MA  Procurement criteria weighed more on quality and experience than costs  Mandatory tasks: desk Technical and review, give scores, site Financial Evaluation visit and recommendations for project improvements in writing and in meeting with applicants Applicants improve project at FS stage according to Improvement of independent evaluators’ Project at Feasibility recommendations Study Stage Independent evaluators Projects passing verify the improvement of certain thresholds get Verification of project at FS stage and give funded. project improvement final evaluation scores and final evaluation  Financing Contract signed btw. MA and applicant, FS attached  Applicant selects between: Option 1: Green FIDIC for small projects Financing Contracting 319 General Scheme for Type 1 Projects under Scenario 3 320 SCENARIO 3 – “Competitive” Proposed selection procedures for Type 1 projects according to the Public Investment Management Framework Public Investment Management Framework Stage 1 – Investment Guidance, Stage 2 Stage 3 Stage 4 ROP Project Preparation & Preliminary Project Appraisal Independent Review of Selection & Contracting Procedures Screening Appraisal Project Preparation:  Project concept, application form and prefeasibility study  By associations of county councils or by future regional administrative entity  Project must be listed in Regional Development Plan Guidelines and Strategic Screening: Announcement  Done by a strategic screening committee comprising both central and regional representatives  Based on objective measurable criteria and clear procedures (proposed by the MA and approved by the Monitoring Committee of ROP)  Twice per year A set of 2-4 projects per region Content: within pre-allocated funds  FS  Technical Studies  Cost Benefit Analysis  Operation and Maintenance Costs Application Submission  All are submitted in digital copies Project Preparation Costs: sufficiently co-funded by MA and future regional entities Admin.& Elig Centrally prepared evaluation grid Verification Centrally prepared evaluation grid Independent evaluators:  Selected by MA  Payment sufficiently funded by MA  Procurement criteria weighed more on Technical and quality and experience Financial than costs Evaluation  Mandatory tasks: desk review, give scores, site visit and make recommendations for project improvements in writing and in meeting with applicants Applicant improves Improvement project at feasibility of Project at study stage according to Feasibility the recommendations of Study Stage independent evaluators Verif. of Independent evaluators Project passing certain project verify the improvement of threshold scores gets improvement project at feasibility study funded & final eval. stage and give final scores 321  Financing Contract is signed between MA and applicant with FS attached  Then applicant selects between the following options: Financing Option 1: Red FIDIC: Two Contracting contracts in sequence: first contract for design and second contract for build Option 2: Yellow FIDIC: Applicant contracts both design and build with a company in one contract 322 General Scheme for Type 2 Projects under Scenario 3 323 SCENARIO 3 – “Competitive” Proposed selection procedures for Type 2 projects according to the Public Investment Management Framework Public Investment Management Framework Stage 1 Stage 2 Stage 3 Stage 4 Investment Guidance, Project Project Appraisal Independent Review of Selection, Detailed Technical ROP Procedures Preparation & Preliminary Appraisal Design & Contracting Screening Restricted call of proposals:  Prepared by RDA or a future regional entity and approved by regional monitoring committee  Implicit strategic screening realized by eligibility and evaluation criteria Guidelines and  Project must be derived Announcement from Urban Integrated Development Plans or equivalents  Deadline for pre-allocated fund availability in Phase 1  Schedule of selection in Phase 2 (e.g., once every 6 months) Content:  FS  Technical Studies  Cost Effectiveness Analysis Application  Operation and Submission Maintenance Costs  All are submitted in digital copies Preparation Costs: by applicants Admin. & Elig. Centrally prepared evaluation Evaluation grid Centrally prepared evaluation Independent evaluators: grid  Selected by MA or RDA or a future regional entity  Payment sufficiently funded by MA  Procurement criteria weighed more on Technical and quality and experience Financial than costs Evaluation  Mandatory tasks: desk review, give scores, site visit and recommendations for project improvements in writing and in meeting with applicants Applicants improve Improvement of project at feasibility Project at study stage according to Feasibility Study the recommendations of Stage independent evaluators Verification of Independent evaluators Phase 1:projects passing project verify the improvement of certain threshold scores get improvement and project at feasibility study funded. 324 final evaluation stage and give final Phase 2:projects are ranked evaluation scores according to selection schedule (e.g., twice per year) and top ranked projects get funded.  Financing Contract is signed between MA and applicant with FS attached  Then applicant selects between the following options: Financing Option 1: Red FIDIC: Two Contracting contracts in sequence: first contract for design and second contract for build Option 2: Yellow FIDIC: Applicant contracts both design and build with a company in one contract 325 General Scheme for Type 3 Projects under Scenario 3 326 SCENARIO 3 – “Competitive” Proposed selection procedures for Type 3 projects according to the Public Investment Management Framework Public Investment Management Framework Stage 1 Stage 2 Stage 3 Stage 4 Investment Guidance, Project Appraisal Independent Review of Selection, Detailed ROP Procedures Project Preparation & Appraisal Technical Design & Preliminary Screening Contracting Restricted and temporary call of proposals:  Prepared by RDA or a future regional entity and approved by regional monitoring committee  Implicit strategic screening realized by Guidelines and eligibility and Announcement evaluation criteria, reflecting the objectives of ROP Priority Axes and including the criteria contained in the GEO 88/2013  Schedule of (e.g., once every six months) Content:  Feasibility Study (FS)  Technical Studies  Cost Effectiveness Analysis Application  Operation and Submission Maintenance Costs  All are submitted in digital copies Preparation Costs: by applicants Administrative & Centrally prepared Eligibility evaluation grid Evaluation Centrally prepared Independent evaluators: evaluation grid  Selected by MA or RDA or a future regional entity  Payment sufficiently funded by MA  Procurement criteria weighed more on quality and experience than costs  Mandatory tasks: desk review, give scores, site Technical and visit and recommendations Financial for project improvements Evaluation in writing and in meeting with applicants  Version A - evaluation criteria, using scoring system and reflecting the objectives of ROP Priority Axes and GEO 88/2013 Version B – evaluation criteria, using qualitative judgment and reflecting 327 the objectives of ROP Priority Axes and GEO 88/2013 Applicants improve project Improvement of at feasibility study stage Project at according to the Feasibility Study recommendations of Stage independent evaluators Independent evaluators Version A: Projects are verify the improvement of ranked using scoring project at feasibility study system, according to stage and give final selection schedule (e.g., evaluation scores twice per year) and top Verification of ranked projects get project funded. improvement and Version B: Projects are final evaluation ranked using qualitative judgment assessment, according to selection schedule (e.g., twice per year) and top ranked projects get funded  Financing Contract is signed between MA and applicant with FS attached  Then applicant selects between the following options: Financing Option 1: Red FIDIC: Contracting Two contracts in sequence: first contract for design and second contract for build Option 2: Yellow FIDIC: Applicant contracts both design and build with a company in one contract 328 General Scheme for Type 4 Projects under Scenario 3 329 SCENARIO 3 – “Competitive” Proposed selection procedures for Type 4 projects according to the Public Investment Management Framework Public Investment Management Framework Stage 1 Stage 2 Stage 3 Stage 4 Investment Guidance, Project Appraisal Independent Review of Selection, Detailed ROP Procedures Project Preparation & Appraisal Technical Design & Preliminary Screening Contracting Restricted and temporary call of proposals:  Prepared by RDA or a future regional entity and approved by regional monitoring committee  Implicit strategic screening realized by Guidelines and eligibility and Announcement evaluation criteria, reflecting the objectives of ROP Priority Axes and including the criteria contained in the GEO 88/2013  Schedule (e.g., once every six months) Content:  Feasibility Study (FS)  Technical Studies  Cost Effectiveness Analysis Application  Operation and Submission Maintenance Costs  All are submitted in digital copies Preparation Costs: by applicants Administrative & Centrally prepared Eligibility evaluation grid Evaluation Centrally prepared Independent evaluators: evaluation grid  Selected by MA or RDA or a future regional entity  Payment sufficiently funded by MA  Procurement criteria weighed more on quality and experience than costs  Mandatory tasks: desk review, give scores, site Technical and visit and recommendations Financial for project improvements Evaluation in writing and in meeting with applicants  Version A - evaluation criteria, using scoring system and reflecting the objectives of ROP Priority Axes and GEO 88/2013 Version B – evaluation criteria, using qualitative judgment and reflecting the objectives of ROP 330 Priority Axes and GEO 88/2013 Applicants improve project Improvement of at feasibility study stage Project at according to the Feasibility Study recommendations of Stage independent evaluators Independent evaluators Version A: Projects are verify the improvement of ranked using scoring project at feasibility study system, according to stage and give final selection schedule (e.g., evaluation scores twice per year) and top Verification of ranked projects get project funded. improvement and Version B: Projects are final evaluation ranked using qualitative judgment assessment, according to selection schedule (e.g., twice per year) and top ranked projects get funded  Financing Contract is signed between MA and applicant with FS attached Financing  Then applicant Contracting selects between the following options: Option 1: Green FIDIC for small projects 331 Annex 13. Ministry of Finance prioritization criteria, according to Government Emergency Ordinance 88/2013 Principle 1: Appropriateness of the Project in the Policy Context Criteria Verification Points (Total 20 points) 1.1: Are the project’s goals and objectives 1.1: Is the project explicitly contained within the Governing still relevant? [10 points] Program and/or other programs/strategies/programmatic documents? [10 points]  Very explicitly and repeatedly contained within the current National Program: [10 points]  Explicitly and repeatedly contained within the current National Program [7.5 points]  Moderately contained within the current National Program [5 points]  Not contained within the current National Program [2.5 points] 1.2: Does the project represent a high 1.2: Is the project explicitly considered a prioritized project at priority at the current national strategies? the current national strategies? [10 points] [10 points]  High priority project [10 points]  Moderate priority project [7.5 points]  Low priority project [5 points] Principle 2: Economic and Social Justification Criteria Verification Points (Total 30 points) 2.1: Is the project (still) economically justified? 2.1.1: The given criterion is useful for comparing different [10 points] projects for different levels of risk. Projects with a rate of return (IRR) higher have higher priority compared to projects with a lowest IRR. [5 points] • IRR > 60 % [5 points] • 60 %> IRR > 40 % [4 points] • 40 %> IRR > 30 % [3 points] • 30 %> IRR > 20 % [2 points] • 20 %> IRR > 10 % [1 point] • IRR <10 % or without calculation [0 points] 2.1.2: Has the demand for the services of the project changed to 332 the extent that the original economic justification is put in question? [2 points] • Substantial evidence of demand change [0 point] • Moderate evidence of demand change [1 point] • No evidence of demand change [2 points] 2.1.3: Have project costs risen to the extent that the original economic justification is put in question? [3 points] • Substantial evidence of cost change [0 point] • Moderate evidence of cost change [1 point] • No evidence of cost change [3 points] 2.2: Is the project socially justified? [10 points] 2.2: The evaluation of a project is based on their suitability with respect to the following aspects of social importance. [10 points] • Increasing employment and reducing unemployment of the population [2 points] • Increased access to quality health services [1 point] • Increased access to quality education [1 point] • Poverty reduction: [6 points] - Provision of sufficient water supply to the population [1 point] - Reduction of mortality rates [1 point] - Increase in the extent of coverage of secondary education [1 point] - Increased economic opportunities for the poor [1 point] - Ensuring access to social service delivery to the poor [1 point] - Coverage of remote rural districts [1 point] - Address vulnerabilities of minorities at risk [1 point] The assessment is based on the sum of specific aspects applicable points. 2.3: Is the environmental impact of the project 2.3: The environmental safety assessment of the project (taking acceptable? [5 points] into account contingencies of environmental pollution and the use of limited natural resources and irreplaceable). [5 points] • Environmental safety of a project [5 points] 333 • A project contingency is avoidable [3 points] • A project means strong danger of environmental risk [-5 points] 2.4: Are (were) the terms of the 2.4: A project involving acquisition of labor, goods or services on acquisition/procurement competitive? [5 the basis of competitive bidding, announces the largest number points] of participants to take part in the tender and has no restriction on the amount or type of participating bidders, will have higher priority than the projects that impose restrictions on bids. • No restriction on bidding [5 points] •Acquisition of labor, goods and services only from domestic sources [2 points] • Non-tender (instead, buys labor, goods or services directly) [0 points] Principle 3: Affordability and Financial Sustainability Criteria Verification Points (Total 20 points) 3.1: Is the total funding requirement for the 3.1: The bulk of project financing has been secured from [10 project consistent with a realistic forecast of points] forward resource availability in the • EU financing [10 points] sector/sub-sector? [10 points] • Reimbursable sources [7.5 points] • Own or beneficiary revenues [5 points] • Government revenues only [3 points] 3.2: Does the total funding/financing require 3.2: In view of certain difficulties in the provision of domestic domestic Government co-financing? [5 Government co-financing, the absence of the requirement of points] the Government requiring mandatory participation in the co- financing part of the cost of a project is considered an advantage. [5 points] • Up to 10 % of the total cost of a project [5 points] • 10% to 20 % of total project cost [3 points] • Over 20 % of the total cost of a project [0 points] 3.3: Are credible arrangements in place for 3.3: Evaluate the current status of institutional arrangements 334 meeting the resulting operating and for meeting the resulting operating and maintenance costs on a maintenance costs on a sustainable basis sustainable basis. [5 points] once the project is completed? [5 points] • Well-arranged [5 points] • Moderately arranged [4 points] • Poorly arranged [2 points] Principle 4: Implementation Performance Criteria Verification Points (Total 30 points) 4.1: How well is the current preparation of 4.1: Evaluate the quality of preparation for a project based on the project? [5 points] the availability of detailed project documentation (terms of reference) and the results of the audit of the project. The lack of detailed terms of reference and negative results of the audit, considered as a disadvantage when evaluating a project. [5 points] • Terms of reference available and positive audit results [5 points] • Terms of Reference not available and are being developed [3 points] • Terms of reference available but negative audit results [0 points] 4.2: Is there an identified project 4.2.1: Availability within the management body management body accountable for (ministry/department) of a structural unit responsible for implementation progress? [5 points] managing investment projects in the sector associated with the project in question. [3 points] • Explicit appointment of a project manager [3 points] • No explicit appointment of a project manager [1 points] 4.2.2: If the management body has experience in the implementation of different international projects, the project is considered an advantage when evaluating the project. [2 points] • Availability of this body of relevant experience in international project implementation [2 points] 335 • Lack of this experience [0 points] 4.3: How well is the current project 4.3: Based on the special indicators below, each project is management performing? [15 points] evaluated in terms of its implementation and the achievement of the tasks and goals established under the project. Unsatisfactory implementation of a project will result in lower priority compared to projects implemented successfully. [15 points] 4.3.1: Quality of acquisition [3 points] • Satisfactory [3 points] • Unsatisfactory [1 point] 4.3.2: Compliance with the project implementation schedule [3 points] • Compliance [3 points] • Failure [1 point] 4.3.3: Rates of development of project financing [3 points] • Satisfactory [3 points] • Unsatisfactory [1 point] 4.3.4: Quality of work , goods and services [3 points] • Satisfactory [3 points] • Unsatisfactory [1 point] 4.3.5: Compliance with the tasks and goals of a project [3 points] • Compliance [3 points] • Failure [1 point] Auxiliary Principle: Nearness to Completion Criteria Verification Points A.1: How near is the project in terms of time A.1.1: Consider number of remaining years left for construction remaining (how many years left) or physical completion. A small number of remaining years results in high 336 status (how many implementation progress priority. [2 points] left)? [additional 5 points] • Remaining years > 5 years [0 points] • 5 years > Remaining years > 3 years [1 points] • Remaining years < 3 years [2 points] A.1.2: Consider number of remaining implementation progress (percentage) for construction completion. A high number of remaining progress results in high priority. [3 points] • Remaining progress > 50% [0 points] • 50% > Remaining progress > 20% [1 point] • Remaining progress < 20% [3 points] A.2: What would be the costs involved in A.2: Probability and size of the costs involved in restructuring or restructuring or closing the project? closing the project. [-5 points] [additional -5 points] • Substantially high probability and large size [-5 points] • Moderately high probability and moderately large size [-2 points] • Low probability and small size [0 points] 337 Annex 14. ROP 2007-2013 screening procedures for Integrated Development Plans (IDPs) Screening of an Urban Center’s Integrated Development Plans (IDP) 338 Screening of an Urban Development Pole’s Integrated Development Plan (IDP) 339 Screening of a Growth Pole’s Integrated Development Plan (IDP) MA-ROP 340 Annex 15. ROP 2007-2013 funds accessed by public authorities (counties and localities) by August 2013 (by axis and by county) Data Source for all maps: Ministry of Regional Development and Public Administration: www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html ROP 2007-2013 funds accessed for Axis 1 (Integrated Urban Development) ROP 2007-2013 funds accessed for Axis 2 (Road Infrastructure) 341 ROP 2007-2013 funds accessed for Axis 3 (Social Infrastructure) ROP 2007-2013 funds accessed for Axis 4 (Business Infrastructure) 342 ROP 2007-2013 funds accessed for Axis 5 (Tourism Infrastructure) 343 Annex 16. The availability of development strategies at the county residence and county council level The availability of development strategies at the COUNTY RESIDENCE level (GREEN means a strategy is in place; BLUE indicates a strategy is in place but some key elements are missing; RED indicates a functional strategy si missing) Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding 1. Strategia de Dezvoltare a 1. Alba-Iulia Municipiului Municipality Alba-Iulia; 2. Development 1.... Alba-Iulia Strategia Strategy; 2. 2. 2009- YES YES YES YES YES YES YES YES dezvoltarii zonei Urban Area 2020 de Development actiune urbana Strategy (PIDU) 1. 2008-2013 2014-2020 Arad City 1. Strategia de Development dezvoltare a Strategy (770 municipiului Arad 2008- pages, County 2008-2013 2014- 2013 Arad Development YES YES YES YES YES YES YES YES 2020; 2. Plan 2014- Strategy Integrat de 2020 included); 2. Dezvoltare Integrated Urbana Urban Development Plan 344 Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding 1. Pitesti City Post- 1. Strategia post- Accession aderare a Startegy, 2nd municipiului Chapter - Pitesti, Capitolul Economic II - Dezvoltare si Development competitivitate and Pitești economica; YES YES YES YES YES YES YES YES Competitivity; Strategia de Municipality Dezvoltare a Development municipiului; 2. Strategy; 2 Plan Integrat de Integrated Dezvoltare Urban Urbana Development Plan 1. Bacau 1. Strategia de Municipality Dezvoltare Sustainable Durabila a Development Municipiului Bacău Startegy; 2 YES YES YES YES YES YES YES YES Bacau, PID; 2. Integrated Plan Integrat de Urban Dezvoltare Development Urbana Plan 1. Planul 1. Oradea Urbanistic General General Oradea, 2009- Oradea Urbanistic YES YES YES YES YES YES YES YES strategii 2013 Plan, 2. dezintegrate; Masterplan 2.Masterplan 345 Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding 1. 2010-2030 1. Strategia de Bistrita dezvoltare locală Municipality a municipiului Local Bistriţa pentru Development Bistrița perioada 2010- YES YES YES YES YES YES YES YES Startegy; 2. 2030; 2. Plan Integrated Integrat de Urban Dezvoltare Development Urbana Plan 1. Botosani 1. Strategia de Municipality Dezvoltare a Development Municipiului Strategy; 2. Botoșani NO YES YES YES YES YES YES YES Botosani 2. Plan Integrated de Dezvoltare Local Locala Integrata Development Plan 1. Braila 1. Strategia de Municipality Dezvoltare a Development Municipiului 2008- Brăila Startegy; 2. YES YES YES YES YES YES YES YES Braila; 2. Plan de 2013 Integarted Dezvoltare Development Integrata Plan Planul Intergrat Brasov de Dezvoltare Growth Pole Urbana (PIDU) Integrated 2009- Brașov YES YES YES YES YES YES YES YES pentru Polul de Urban 2015 Crestere (PC) Development Brasov Plan 346 Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding Buzău Projects only Projects only 2012-2016 Strategia de Calarasi City dezvoltare socio- Not Socio- 2012- Călărași economică a YES YES YES YES YES YES YES precisel Economic 2016 orașului Călărași y Development 2012-2016 Strategy YES, 2007- Reșița Strategie Strategy NO NO NO NO YES YES Little more or 2013 less 1. Cluj-Napoca 1. Strategia de Municipality Dezvoltare a Development Municipiului Cluj- Startegy; 2. Cluj-Napoca Napoca; 2. Plan YES YES YES YES YES YES YES YES Integrated Integrat de Urban Dezvoltare Development Urbana Plan 1. Sustainable 1. Plan local de Development dezvoltare Local Plan; 2 durabila; 2. Plan Constanța Growth Pole YES YES YES YES YES YES YES YES integrat de Integrated dezvoltare pol Development crestere Plan Sfântu Gheorghe 347 Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding Strategia Locală Targoviste pentru Municipality Dezvoltare Târgoviște Sustainable YES YES YES YES YES YES NO NO Durabilă a Development Municipiului Local Strategy Târgovişte 1. Craiova 1. Strategia de Municipality Dezvoltare Locala Local a Municipiului Development Craiova; 2. Plan Strategy; 2. 2007- Craiova Intergrat de Craiova 2013 YES YES YES YES YES YES YES YES Dezvoltare Growth Pole (PIDU) Urbana pentru Integrated Polul de Crestere Urban Craiova Development Plan Integrated Plan Integrat de Urban Galați Dezvoltare YES YES YES YES YES YES YES YES Development Urbana Plan 1. Strategia 1. Giurgiu Locala de Municipality Dezvoltare Sustainable Durabila a Development Giurgiu Municipiului NO YES NO YES YES YES NO NO Local Strategy; Giurgiu; 2. Plan 2. Sustainable local de Development dezvoltare Local Plan durabila 348 Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding Planul Local de Targu Jiu Dezvoltare Municipality Târgu Jiu Durabilă a Sustainable NO YES YES YES YES YES YES - Municipiului Development Târgu - Jiu Local Plan Strategia locala Sustainable Miercurea de Dezvoltare Development NO YES NO YES YES NO NO NO Ciuc durabila Local Strategy 1.Planul de 1. 2007-2013 Dezvoltare al Deva Municipiului Municipality Deva pentru Development 2007- Deva perioada 2007- Plan; 2. YES YES YES YES YES YES YES YES 2013 2013; 2. Planul Integrated integrat de Urban dezvoltare Development urbana Plan Integrated PIDU - incadrare Urban Slobozia in strategii NO YES YES YES YES NO NO NO Development dezvoltare Plan 1. 2020 Iasi 1. Strategia de Municipality dezvoltare Sustainable economico – Economic and socială durabilă a Social more or Iași Municipiului Iaşi - 2020 YES, PID YES YES YES YES YES YES Development less ORIZONT 2020; Strategy; 2. 2. Plan Integrat Growth Pole de Dezvoltare Pol Integrated Crestere Development 349 Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding Plan 2035 Conceptul Bucharest București strategic YES YES YES YES YES - - - Strategic Bucuresti 2035 Concept 1. Baia Mare 1. Strategia de Municipality Dezvoltare Sustainable Durabila a Development municipiului Baia Baia-Mare Strategy; 2. 2020 YES YES YES YES YES YES YES YES Mare; 2. Plan Integrated Integrat de Urban Dezvoltare Development Urbana Plan Integrated Drobeta Plan Integrat de Urban Turnu- Dezvoltare NO YES YES YES YES YES YES YES Development Severin Urbana Plan Planul Integrat Targu-Mures de Dezvoltare Integrated Târgu 2007- Urbana al Urban YES YES YES YES YES YES YES YES Mureș 2013 Municipiului Development Targu Mures Plan 350 Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding 1. 2008-2015 1. Strategia de Piatra Neamt Dezvoltare Municipality Durabila a Urban Municipiului Piatra Development 2008- Piatra Neamt YES, PIDU YES YES NO NO Neamț Strategy; 2. 2015 2008 - 2015; 2. Integrated Plan Integrat de Urban Dezvoltare Development Urbana Plan Integrated Plan Integrat de Urban Slatina Dezvoltare YES YES YES YES YES YES YES YES Development Urbana Plan 1. Ploiesti 1. Strategia de Municipality Dezvoltare a Development Municipiului Strategy; 2. 2007- Ploiești Ploiesti; 2. Plan YES YES YES YES YES YES YES YES Integrated 2025 Integrat de Urban Dezvoltare Development Urbana Plan Strategia de Municipality dezvoltare Sustainable 2008- Zalău YES YES YES YES YES YES YES YES durabila a Development 2013 municipiului Strategy Plan Strategic de Development Satu-Mare 2010 YES YES YES YES YES YES YES YES Dezvoltare Strategic Plan 1.Plan Integrat 1.Integrated 1. 2009- de Dezvoltare Urban Sibiu 2015 YES YES YES YES YES YES YES YES Urbana; Development 2. 2004 2.Agenda Locala Plan; 2.Local 351 Strategy Multi- List of Sources Strategy name in Strategic General SWOT City name in Vision sectoral Objectives projects/ of ROP Romanian period Analysis Analysis English Analysis programs funding 21 Agenda 21 Strategia de Municipality Dezvoltare Sustainable 2009- Suceava YES YES YES YES YES YES YES YES Durabila a Development 2015 Municipiului Strategy Strategia de Municipality NOt more or Alexandria Dezvoltare a Development NO YES YES YES YES YES precisel less Municipiului Strategy y 1. Masterplan; 1. Masterplan; 2. 2. Integrated Plan Integrat de Timișoara Urban 2030 YES YES YES YES YES YES YES YES Dezvoltare Development Urbana Plan Strategia de Municipality 2014- Tulcea Dezvoltare a Development YES YES YES YES YES YES YES YES 2020 municipiului Strategy Strategia de Municipality Râmnicu dezvoltare Sustainable 2008- YES YES YES YES YES YES YES YES Vâlcea durabila a Development 2013 municipiului Strategy Vaslui NONE NONE 1. Integrated 1. Plan Integrat Urban de Dezvoltare Development Urbana; 2. Plan; 2. Focșani Strategia pentru YES YES YES YES YES YES YES YES Focsani Dezvoltarea Municipality Municipiului Development Focsani Strategy 352 The availability of development strategies at the COUNTY COUNCIL level (GREEN means a strategy is in place; BLUE indicates a strategy is in place but some key elements are missing; RED indicates a functional strategy si missing) Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis Strategia de 2007-2013 Dezvoltare Alba County a Judetului Development Alba pentru Strategy. 2007- Alba perioada YES YES YES YES YES YES YES YES Economic 2013 2007-2013. Development Dezvoltarea - Second economica Phase Final - Faza a II- a Finala 2007-2013 Arad County Strategia Development de Strategy (770 2007- Arad dezvoltare pages, NO YES YES YES YES YES YES YES 2013 a judetului County 2007-2013 Development Strategy included) 1. Studiu de 1. Arges Dezvoltare County Durabila Sustainable 2013- judetul Development 2020 Arges 2. Study; 2. The Startegy Argeș Dezideratul Desideratum YES YES YES NO YES YES YES YES in draft - unei of an - final in Dezvoltari Integrated 2014 Integrate Development “Arges - "Arges- Muscel” Muscel" 353 Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis Strategia de 2010-2030 Dezvoltare Bacau economica County durabila a 2010- Bacău Sustainable YES YES YES YES YES YES YES YES judetului 2030 Economic Bacau in Development orizontul Startegy de timp 2010-2030 Planul de 2007-2013 Dezvoltare Bihor County 2007- Bihor YES YES YES YES YES YES YES YES al judetului Development 2013 Bihor Plan Bistrița- NO NO Năsăud Strategia 2008-2013 de Botosani Dezvoltare County Economico- 2008- Botoșani Socio- YES YES YES YES YES YES YES YES Sociala a 2013 Economic Judetului Development Botosani Strategy 2008-2013 Strategia de 2010-2015 Dezvoltare Braila County 2010- Brăila Durabilă a Sustainable YES YES YES YES YES YES YES YES 2015 Judeţului Development Brăila Strategy 2010-2015 354 Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis Strategia 2013-2020- de 2030 Brasov Dezvoltare 2013- County Not Brașov a Judetului 2020- YES YES YES YES YES NO Not precisely Sustainable precisely Brasov 2030 Development 2013-2020- Strategy 2030 Strategia 2007-2013 de Buzau dezvoltare County 2007- Buzău durabilă a YES YES YES YES YES YES YES YES Sustainable 2013 Judetului Development Buzău 2007 Strategy – 2013 Planul de 2007-2013 Dezvoltare Calarasi 2007- Călărași al Judetului County YES YES YES YES YES YES YES YES 2013 Calarasi Development 2007-2013 Plan Strategia 2007-2013 de Caras- dezvoltare Severin Caraș- durabilă a County YES YES YES YES YES NO NO NO Severin Judeţului Sustainable Caras- Development Severin Strategy 2007-2013 Strategia de 2014-2020 Dezvoltare Cluj County 2014- Cluj YES YES YES YES YES NO Not precisely a Judeţului Development 2020 Cluj pentru Strategy perioada 355 Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis 2014-2020 Constanța Deseuri Dezvoltarea 2009 serviciilor Transport Covasna de Services transport Development 2009 Strategia 2012-2020 de Dambovita Dezvoltare County 2012- Dâmbovița Durabila a YES YES YES YES YES YES YES YES Sustainable 2020 JudetuluiI Development Dambovita Strategy 2012-2020 Strategia de Dolj County Dezvoltare Social- 2007- Dolj Economico- Economic YES YES YES YES YES YES YES YES 2013 Sociala a Development Judetului Strategy Dolj Strategia Galati de County 2010- Galați Dezvoltare YES YES YES YES YES YES NO NO Development 2015 a judetului Strategy Galati 356 Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis Strategia de Dezvoltare 2008-2013 a Judetului Giurgiu 2008- Giurgiu Giurgiu County YES YES NO YES YES YES NO NO 2013 pentru Development perioada Strategy 2008 – 2013 Strategia de Dezvoltare 2011-2020 Durabila a Gorj County 2011- YES - not Gorj Judetului Sustainable YES YES YES YES YES YES YES 2020 precisely Gorj pentru Development perioada Strategy 2011 – 2020 Strategia Harghita de County 2002- Not Harghita dezvoltare NO YES YES YES YES YES Not precisely Development 2013 precisely judetul Startegy Harghita Planul de Regional Hunedoara Dezvoltare Develpment NO YES YES YES YES YES YES YES Regionala Plan Strategia 2007-2013 to de 2013-2020 2007- dezvoltare Ialomita 2013- Ialomița a judetului YES YES YES YES YES YES YES YES County 2013- Ialomita pe Development 2020 perioada Strategy 2007-2013, 357 Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis cu orizont 2013-2020 Strategia de Iasi County dezvoltare Economic 2009- Iași economica and Social YES YES YES YES YES YES YES YES 2014 si sociala a Development judetului Strategy Iasi Strategia de Dezvoltare 2007-2013 Economico- Ilfov County Sociala a Social- Ilfov YES YES YES YES YES YES YES YES judetului Economic Ilfov pentru Development perioada Strategy 2007 – 2013 Strategia Maramures 2009- de YES County 2014 / more or Maramureș dezvoltare YES YES YES YES YES (Development YES Development 2020- less a judetului Plan) Strategy 2030 Maramures Mehedinți NO NO 358 Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis Stategia de Dezvoltare 2014-2020 a judetului Mures Mures County pentru Development 2014-2020 Strategy (not 2014- NOt Mures - NU YES YES NO YES YES NO NO the full 2020 really complet ; version) ; Strategia Territorial de Development Dezvoltare Strategy Teritoriala - - PATJ Strategie Sustainable Locala de Development NOt Neamt NO YES YES YES YES YES YES Dezvoltare Local precisely Durabila Strategy Olt in lucru Under work Planul de Prahova Dezvoltare County 2007- Prahova Durabila a Sustainable NO YES YES YES YES YES YES YES 2013 Judetului Development Prahova Plan Plan de County 2007- Sălaj Dezvoltare Development YES YES YES YES YES YES YES YES 2013 Judetean Plan Strategia County de Development Dezvoltare Strategy + 2007- Satu-Mare YES - 2027 YES YES YES YES YES YES YES a Judetului Regional 2013 + Startegie Marjeting de Startegy 359 Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis Marketing regional Stategia de 2007- Sibiu County dezvoltare 2013 / Sibiu Development YES YES YES YES YES YES YES YES a judetului 2014- Strategy Sibiu 2020 Strategie Suceava de County Dezvoltare Economic 2011- Suceava Economica YES YES YES YES YES YES YES YES and Social 2020 si sociala a Development judetului Strategy Suceava Strategia County de Sustainable 2010- Teleorman Dezvoltare YES YES YES YES YES YES YES YES Development 2020 Durabila a Strategy judetului Strategia County de Economic dezvoltare 2009- Timiș and Social YES YES YES YES YES YES YES YES economico- 2020 Development sociala a Strategy judetului Strategie Development Not Tulcea NO YES YES YES YES YES YES dezvoltare Strategy precisely Strategia County YES, de Economic 2009- more or Vâlcea YES YES YES YES more or YES YES dezvoltare and Social 2013 less less economico Development 360 Strategy Strategy Multi- Strategic Development General SWOT List of Sources of County name in name in sectorial Objectives ROP period vision Analysis Analysis projects funding Romanian English Analysis sociala a Strategy judetului Strategia Vaslui de County dezvoltare Social- 2013- Vaslui socio- YES YES YES YES YES YES YES YES Economic 2020 economica Development a judetului Strategy Vaslui Strategia County de 2007- Vrancea Development NO YES YES YES YES YES YES YES dezvoltare 2013 Strategy a judetului 361 Annex 17. Capital Investment Planning/Case study: Nis, Serbia89 Population: 255.500 inhabitants Annual city budget: 384 USD per capita (2011, planned) Annual city capital investment: 149 USD per capita (2011, planned) The Draft Capital Investment Plan (CIP) of the City of Nis is a document of strategic importance that defines the city’s midterm development guidelines. It is a five-year plan of budget investments in infrastructure, public buildings, and facilities and equipment. This document was prepared by the CIP Working Group (21 members plus 1 CIP coordinator), appointed by the mayor, and made up of the staff of the Local Economic Development Office and representatives of the Nis administration. The process was facilitated by advisors from the US Agency for International Development (USAID)-sponsored project, Municipal Economic Growth Activity. The CIP is designed to bring together the City Development Strategy and the City’s financial capabilities. One of important final results of the strategic planning process is a list of potential capital projects that will be implemented in the future. The CIP Working Group developed the following guidelines, assessments, and forms:  Period to be covered by CIP: 2010–15  Definition of a capital investment project: Sectors that may be included; minimal value: 100,000 Euro; period of implementation: Not fewer than 2 years  Possible sources of finance: City budget, the National Investment Plan and other central government programs, loans, donations, own resources of municipal public companies, and concessions (public-private partnerships)  CIP calendar, according to which the document had to be adopted in December 2010  System for rating projects  Assessment of city’s financial capacity and budget projections through 2015 prepared by Department of Finance  Project Proposal Form (project request) was adopted, including: (1) general project information; (2) project goals and strategic framework; (3) project description; (4) degree of project readiness for implementation; and (5) project management structure/project sustainability. Budget institutions and public utility companies filled out the forms and sent them back to the working group. Once the project proposals were submitted, the working group eliminated those that did not comply with the defined criteria (cost was below 100,000 Euro or was not from the targeted sectors, or 89 Kaganova, O. 2011, Guidebook on Capital Investment Planning for Local Governments, Urban Development Series, World Bank and Local Government Unit of the Sustainable Development Network 362 implementation period was fewer than 2 years). Next, the working group rated the remaining projects according to 16 criteria, with a weight (from 1 to 3) assigned to each criterion. Each project was scored against each of the 16 criteria; the points were multiplied by the weights and added to produce the total score for the project. Content of the Capital Investment Plan: 1. City’s organizational structure, composition of the CIP Working Group, and city development vision and general data about the city (population, economy, employment, education, natural and cultural resources, and transportation infrastructure); 2. Methodology used to draft the CIP, including methodology formulating the CIP, calendar, project selection criteria, assessment of city’s financial capacities, and selection of project proposal form; 3. Project descriptions and ratings (project value, budget share of funding, and degree of readiness for implementation). A total of 33 projects were selected and ranked. The top priority projects in the CIP were for utility infrastructure and general infrastructure: reconstruction and expansion of the local airport; parking garage; central waste water treatment system; and a farmers’ market. In addition, three social housing projects (housing at subsidized prices) also received high ratings. Lessons of the case study:  The CIP process is a complex set of activities that requires almost 12 months of engagement. Unfortunately, LGs do not always understand this, and after a period of initial enthusiasm, only a couple of employees followed through on the entire process.  The experience of using the National Investment Fund application form for CIP project requests produced mixed results. On the one hand, it is a good form that requires solid data and information. On the other hand, many projects still lack technical and engineering documents. For this reason, it often was impossible to complete the form.  Even when technical documents (for example, surveys, engineering studies and design) have been prepared for a project, an economic feasibility assessment usually is lacking, and often there is no financial analysis. Consequently, it is impossible to specify the timing of project expenditures. The LG should allocate attention and resources to prepare feasibility studies.  A CIP calendar should allocate more time for budget beneficiaries and public utility companies to fill out and return the project proposal forms and for members of the CIP Working Group to evaluate and rate projects.  It would be better to include the entire CIP Working Group in evaluating projects. Doing so would secure a broader based evaluation, less influenced by judgment of the Mayor, his deputy, and other members of the 5-member evaluation group.  Although there is no “standard” for selecting CIP Committee members, in addition to budget beneficiaries and public utility companies established by the LG, it is desirable to include businesses, institutions, and citizens in the CIP process. It is especially desirable to include 363 citizens’ and businesses’ vision of investment priorities in the city. In Nis, for instance, no public participation was brought into the process.  The City of Nis CIP is based on well-prepared strategic documents, which clearly identified the long-term needs of the city.  Lack of documents such as pre-feasibility studies often makes it impossible to identify all of the investment cost components.  The project request form did not include a section for estimates of future maintenance and repair, investment maintenance, and operating costs of the projects. As a result, except for 2010, this information was not collected, making it impossible to evaluate the impact of future annual costs on the city budget.  Midterm budget projections (for five years) often are unreliable due to uncertain fiscal and monetary policies and an unpredictable inflation growth. It also is impossible to rely on such sources of revenue as donations and funds provided by central ministries.  CIP preparation and adoption often depend on the distribution of political power in the City Assembly. Obstruction by political parties has occurred. To overcome such obstacles, a clear CIP methodology must be in place. 364 Annex 18. Draft Proposal for ROP 2014-2020 Axes and how they would align to the Competitive Scenario proposed in this report Development challenge 1 - Competitiveness Pre-allocated funds Pre-allocated funds Competitive allocation FIFO Thematic objectives (>100 mil RON) (> 30 mil RON) <100 mil >1,5 mil RON ( 1,5 mil RON) Beneficiary IT centers (HG 406/2003) / SMEs Thematic objective 1: Strengthening research, technological development and innovation Priority axis 1: Strengthening X X X X X X research, technological development and innovation Thematic objective 3: Enhancing the competitiveness of small and medium sized enterprises, the agricultural sector and the fisheries and aquaculture sector Priority axis 2: Enhancing the competitiveness of small and X X X X X X medium‐sized enterprises 365 Development challenge 2 - People and Society Pre-allocated funds Pre-allocated funds Competitive allocation FIFO Thematic objectives (>100 mil RON) (> 30 mil RON) <100 mil >1,5 mil RON ( 1,5 mil RON) Beneficiary County City IDA County City IDA County City IDA County City IDA Thematic objective 8: Promoting employment and supporting IDA labor mobility = Inter- Priority axis 6: Promoting Community 90 91 92 93 94 95 employment and supporting labor development X X X X X X X X mobility association 6.1. Business incubators support Thematic objective 9: Promoting social inclusion and combating poverty Priority axis 7: Development of the health and social infrastructure X X X X X X X X 7.1. Investment in health and social infrastructure Priority axis 8: Social inclusion and X X X X X X fighting poverty 8.1. Economic regeneration support for urban disadvantaged communities Thematic objective 10: Investing in education, skills and lifelong learning Priority axis 9: Investment in education, skills and long life learning 90 Micro/SMEs firms are eligible 91 Micro/SMEs firms are eligible 92 Micro/SMEs firms are eligible 93 Micro/SMEs firms are eligible 94 Micro/SMEs firms are eligible 95 Micro/SMEs firms are eligible 366 9.1. Development of the X X X X X X education and training infrastructure Development challenge 3 - Infrastructure Pre-allocated funds Pre-allocated funds Competitive allocation FIFO Thematic objectives (>100 mil RON) (> 30 mil RON) <100 mil >1,5 mil RON ( 1,5 mil RON) Beneficiary County City IDA County City IDA County City IDA County City IDA Thematic objective 7: Promoting sustainable transport and removing bottlenecks in key network infrastructures Priority axis 5: Promoting sustainable transport 5.1. Enhance regional mobility, 96 connection of the regional road X X X X X X X X X system to TEN-T infrastructure Development challenge 4 - Resources Pre-allocated funds Pre-allocated funds Competitive allocation FIFO Thematic objectives (>100 mil RON) (> 30 mil RON) <100 mil >1,5 mil RON ( 1,5 mil RON) Beneficiary County City IDA County City IDA County City IDA County City IDA Thematic objective 4: Supporting the shift toward a low‐carbon economy in all sectors Priority axis 3: Urban 100 101 102 104 Development X X X X 3.1. Energy efficiency support in 99 103 105 public/private buildings X X X X X X 3.2. Reducing emission including 96 Growth poles and Bucharest are eligible 367 transport 97 98 106 X X X X X X X 3.3. Polluted land and noise pollution Thematic objective 6: Protecting the environment and promoting resource efficiency Priority axis 4: Protecting the environment and promoting resource efficiency 4.1. Conservation, protection, 107 108 109 110 111 112 113 114 promotion and development of X X X X X X X X X X X the natural and cultural heritage Development challenge 5 -Administration & Government Pre-allocated funds Pre-allocated funds Competitive allocation FIFO Thematic objectives (>100 mil RON) (> 30 mil RON) <100 mil >1,5 mil RON ( 1,5 mil RON) Beneficiary County City IDA County City IDA County City IDA County City IDA 115 Thematic objective 11 X X X X 99 For example major metropolitan transport system, or rapid public transport connection between two urban areas. 100 For administrative, health and education buildings including the public heating network. Energy production and distribution organizations are eligible 101 For administrative, health and education buildings including the public heating network. Energy production and distribution organizations are eligible 102 Public and private owners, religious organizations, are eligible. Energy production and distribution organizations are eligible 103 Private operators are eligible 104 Private owners/condominium associations are eligible. Energy production and distribution organizations are eligible 105 Private transport operators are eligible (i.e. taxi) 97 For historical polluted land (i.e. industries/mines) 98 For example, major brownfields redevelopment project in a Growth Pole 106 Private owners of polluted land are eligible 107 One found per region 108 Investment in a large regionally representative cultural patrimony site, situate within a city. 109 Major investment to rehabilitate cultural patrimony that covers several jurisdictions (e.g., painted monasteries in Moldova or fortified churches in Transylvania) 110 Public and private owners, religious organizations, are eligible 111 Public and private owners, religious organizations, are eligible 112 Public and private owners, religious organizations, are eligible 113 Public and private owners, religious organizations, are eligible 114 Public and private owners, religious organizations, are eligible 368 Annex 19. ROP 2007-2013: Synthesis of types of investments and eligible beneficiaries Local Public Authorities Type of investment Local Council County Council IDAs (LPAs) in partnership Urban Growth Poles (Axis 1) X (only selected municipalities – 7 growth poles) Urban Development Poles (Axis 1) X (only selected municipalities – 13 development poles) Urban Centers (Axis 1) X (only municipalities over 10,000 inhabitants) Energy efficiency of residential X buildings (Axis 1) (only 41 capital municipalities and Bucharest sectors) County roads (Axis 2) X X X Urban Streets network (Axis 2) X Ring roads (Axis 2) X X Health Infrastructure – only for the 15 county hospitals/ambulatory care X X X X services (Axis 3) Social Infrastructure (Axis 3) X X (through the General (through Public Social Assistance Directorates of Social Service in in partnership with a Assistance and licensed social services provider) Childcare) Emergency equipment (Axis 3) X (only IDAs covering all counties in a region) Business infrastructure (Axis 4) X X X Brownfields redevelopment (Axis 4) X X X Tourism infrastructure (Axis 5) X X X X 115 For counties without proper cadaster records 369 Annex 20. Financial Situation for County Residences and County Councils Financial situation for 2012 (in Current RON 2012)* Ratio non- Ratio of earmarked Shared PIT Total Capital Non- Earmarked Grants – revenues Own (Personal Other Expenses in Total Earmarked Reimbursements from EU Capital Expenses for current Revenues Income Revenues Non- Grants funds and Other Donors expenses Tax) Earmarked from total Revenues revenues Other Local EU Funds EU Funds Donors Budgets County Residences (2012) Alba Iulia 167,543,402 29,765,276 43,046,117 4,677,240 89,997,319 25,832,548 57,450 5,847,488 64,332,796 91% 20% Arad 377,205,642 140,500,102 118,374,578 1,480,660 116,850,302 24,236,692 0 76,060,819 53,879,226 50% 41% Pitesti 328,302,912 74,030,485 95,143,956 2,624,998 106,012,601 5,576,158 50,490,872 90,356,533 24,103,047 51% 35% Bacau 342,686,340 72,361,409 96,629,267 52,231,717 121,123,964 5,974,701 339,983 62,066,156 56,076,751 53% 32% Oradea 428,239,098 113,067,188 109,851,364 29,623,598 163,053,854 15,677,821 12,643,094 49,618,774 29,694,576 30% 47% Bistrita 149,834,702 39,136,200 46,316,583 1,286,190 63,095,729 8,753,107 0 17,827,813 19,199,478 43% 39% Botosani 241,105,973 38,034,846 39,420,523 6,909,720 156,273,872 32,890,522 467,012 97,010,467 0 114% 9% Brasov 512,103,556 189,211,984 170,350,441 3,828,357 147,038,452 14,124,352 1,674,322 102,060,285 52,169,333 42% 44% Braila 383,255,087 68,477,439 65,239,660 11,499,020 162,591,436 34,783,114 75,447,532 91,697,502 78,676,039 77% 22% Buzau 246,728,505 57,947,949 69,990,638 33,064,229 85,672,941 1,747,618 52,748 7,233,475 6,751,912 9% 60% Resita 123,022,553 26,010,510 27,311,697 23,896,178 45,797,668 2,061,243 6,500 4,474,872 4,254,976 11% 57% Calarasi 104,888,276 31,249,195 28,478,379 3,467,076 41,679,368 0 14,258 18,719,684 0 30% 42% Cluj-Napoca 647,195,785 159,310,244 291,228,174 2,619,860 193,458,876 36,384,091 578,631 69,787,174 88,668,359 35% 51% Constanta 513,256,582 188,809,112 179,464,367 4,947,992 139,975,111 6,700,463 60,000 114,170,537 9,405,023 33% 50% Sfantu Gheorghe 104,067,432 22,598,700 31,162,225 5,071,969 45,168,265 3,954,881 66,273 17,639,137 8,229,512 44% 36% Targoviste 172,285,577 46,992,891 50,234,762 6,076,410 68,811,877 4,991,397 169,637 18,773,823 15,805,256 33% 43% Craiova 561,934,962 115,433,744 145,395,491 5,719,018 219,246,682 48,259,570 76,140,027 103,759,493 87,784,061 56% 35% Galati 476,946,587 140,234,586 122,448,221 5,282,998 136,755,332 6,348,848 72,225,450 110,576,245 33,970,566 42% 42% Giurgiu 89,812,216 20,730,458 24,205,502 6,713,851 37,560,371 506,325 602,034 9,514,055 233,886 19% 48% Targu Jiu 171,704,370 35,902,693 41,792,310 8,992,674 84,887,206 8,385,861 129,487 13,257,130 16,039,731 34% 38% Miercurea Ciuc 94,674,416 20,082,374 28,137,550 1,514,403 44,874,389 4,670,527 65,700 19,647,403 25,803,910 91% 10% Deva 142,523,597 33,327,846 39,445,024 5,169,125 46,303,074 5,085,403 18,278,528 27,452,264 15,943,253 45% 41% Slobozia 79,674,347 22,635,528 25,228,222 1,786,412 29,517,185 136,212 507,000 22,082,805 993,484 46% 34% Iasi 538,048,921 120,129,244 166,228,830 22,614,780 228,904,221 36,967,016 171,846 58,179,566 58,063,110 38% 43% Baia Mare 259,790,869 64,679,738 59,315,713 707,219 91,825,332 4,773,362 43,262,867 53,131,914 4,625,853 34% 44% 370 Turnu Severin 251,212,686 41,202,861 40,794,625 19,615,735 113,269,175 32,643,018 36,330,290 70,341,094 28,103,328 71% 29% Targu Mures 321,981,179 74,871,607 99,765,676 26,047,501 98,025,566 5,691,164 23,270,829 39,487,800 19,403,850 26% 53% Piatra Neamt 197,255,111 41,002,133 37,708,730 24,926,650 91,322,955 18,063,538 2,294,643 36,472,969 44,478,295 76% 22% Slatina 183,521,954 46,432,905 49,570,030 1,435,066 86,068,903 19,890,698 15,050 22,983,077 32,715,216 57% 34% Ploiesti 414,412,865 135,646,730 150,067,689 2,192,510 125,761,373 1,819,690 744,563 44,683,902 85,615 16% 59% Satu Mare 185,837,562 46,421,233 54,674,794 1,096,340 80,648,295 5,703,787 2,996,900 7,751,155 7,921,425 15% 51% Zalau 111,190,048 28,058,538 29,327,590 502,395 53,105,552 5,908,003 195,973 2,772,644 14,703,957 30% 42% Sibiu 304,565,543 101,650,962 124,546,971 2,980,000 75,245,263 102,990 142,347 46,819,958 4,810,282 23% 58% Suceava 292,448,039 59,771,383 43,482,849 36,272,830 101,068,570 19,336,366 51,852,407 66,350,559 37,230,021 54% 37% Alexandria 97,264,245 21,837,146 29,976,291 2,321,806 42,806,438 6,807,400 322,564 18,370,147 14,927,788 61% 29% Timisoara 751,571,185 160,327,341 249,517,846 115,895,821 181,594,047 14,499,654 44,236,130 89,533,014 72,949,969 29% 56% Tulcea 137,685,793 42,777,408 40,699,364 936,270 53,153,751 2,586,213 119,000 14,936,563 182,000 18% 52% Vaslui 119,199,603 21,768,364 26,847,317 2,127,129 67,975,069 13,494,346 481,724 16,338,652 27,047,538 85% 18% Ramnicu Valcea 233,260,496 54,969,000 61,153,924 9,432,096 79,650,634 6,477,336 28,054,842 51,599,138 13,930,861 43% 41% Focsani 165,924,084 37,398,277 42,755,152 6,920,106 77,850,549 12,267,480 1,000,000 11,415,649 36,489,400 54% 32% County Councils (2012) Alba 168,259,500 3,940,503 29,208,372 22,820,000 112,184,826 1,172,951 105,799 11,275,053 3,936,384 27% 25% Arad 245,428,203 12,067,075 44,748,428 36,428,021 151,246,234 37,975,894 938,445 12,542,762 47,042,876 63% 30% Arges 265,012,521 4,864,597 60,871,889 38,968,849 160,285,686 15,094,972 21,500 18,168,894 12,960,019 30% 33% Bacau 276,278,998 3,193,334 41,294,329 35,123,731 193,242,225 41,498,884 3,425,379 8,159,147 76,384,390 102% 14% Bihor 235,490,049 3,418,907 46,227,521 36,175,020 149,668,601 4,615,434 0 16,743,091 18,478,636 41% 23% Bistrita-Nasaud 228,087,886 13,566,263 17,294,089 18,637,556 150,186,254 33,701,266 28,403,724 46,827,004 51,387,161 126% 6% Botosani 231,182,308 2,168,324 15,665,389 27,408,510 170,245,608 39,401,771 15,694,477 28,167,830 93,753,902 200% -9% Brasov 223,209,992 16,580,878 66,622,521 37,037,185 101,599,055 234,794 1,370,353 9,540,990 633,745 8% 50% Braila 130,577,316 7,578,461 21,809,092 18,884,000 79,227,345 6,029,332 3,078,418 14,291,326 12,583,915 52% 23% Buzau 184,561,673 1,785,523 26,729,720 26,454,896 127,153,022 4,986,029 2,438,512 18,913,682 12,522,062 55% 17% Caras Severin 155,448,451 3,044,078 15,784,914 24,196,733 111,919,329 14,612,203 503,397 11,639,717 26,927,682 89% 13% Calarasi 152,919,256 163,335 14,345,465 18,158,195 118,839,203 26,058,850 1,413,058 2,964,556 62,653,289 193% -4% Cluj 356,121,796 12,410,738 97,947,472 55,778,000 187,136,458 24,192,694 2,849,128 13,301,767 42,021,120 33% 39% Constanta 393,805,667 32,182,688 73,836,689 49,066,935 175,660,351 19,690,868 63,059,004 66,341,958 42,244,522 50% 33% Covasna 102,205,898 2,425,390 14,640,795 17,718,513 67,365,076 17,766,875 56,124 3,121,403 22,019,937 72% 27% Dambovita 199,266,281 7,177,821 27,558,760 34,700,141 129,829,559 12,328,987 0 24,401,428 21,266,536 66% 18% Dolj 217,498,977 3,679,020 49,817,487 37,544,292 124,468,280 20,251,755 1,989,898 15,997,557 35,285,026 55% 29% Galati 186,052,017 1,976,871 38,907,241 30,064,815 114,573,368 7,096,754 529,722 21,583,315 8,652,593 42% 26% Giurgiu 177,522,731 4,385,628 13,152,277 16,103,337 142,008,330 55,919,684 1,873,159 9,852,936 68,722,037 221% 7% Gorj 154,310,764 2,072,986 30,963,908 21,841,320 99,344,676 8,951,253 87,874 6,370,118 15,537,679 40% 27% Harghita 131,172,159 3,404,998 20,046,030 21,662,441 85,683,083 11,731,995 375,607 17,140,373 11,899,007 64% 21% 371 Hunedoara 192,631,400 1,566,729 37,115,212 26,750,155 126,093,125 7,833,944 1,106,179 12,245,990 25,722,147 57% 19% Ialomita 91,491,668 6,115,762 14,457,247 15,850,020 54,975,710 125,603 92,929 10,580,363 63,769 29% 28% Iasi 311,911,213 4,457,130 55,506,282 42,740,998 208,357,458 11,099,082 849,345 18,843,313 44,030,561 61% 17% Ilfov 187,903,698 1,135,557 73,005,504 37,133,806 76,294,046 9,940,615 334,785 9,945,371 240,570 9% 59% Maramures 184,265,248 2,283,006 26,536,379 29,234,574 126,211,289 9,350,269 0 2,254,665 12,047,464 25% 29% Mehedinti 141,217,610 2,155,246 15,641,570 19,766,182 103,442,326 6,976,769 212,286 15,910,802 33,916,011 132% -4% Mures 278,656,070 3,015,817 46,443,929 38,610,104 189,472,595 34,736,236 1,113,625 5,402,224 63,958,862 78% 20% Neamt 215,113,461 2,022,894 23,155,102 30,808,866 157,498,937 19,614,296 1,627,662 5,662,362 46,286,101 90% 12% Olt 183,736,687 3,066,967 22,006,263 24,187,781 134,373,417 30,365,351 102,259 3,345,937 55,029,841 118% 12% Prahova 365,606,828 7,130,698 73,053,681 44,426,097 240,409,773 16,276,536 586,579 38,866,174 26,306,797 52% 21% Satu Mare 144,936,291 4,465,285 23,080,775 26,027,864 83,448,455 2,339,700 7,913,912 7,105,897 1,771,089 14% 38% Salaj 150,318,498 2,707,355 13,260,692 13,427,747 120,108,715 19,532,846 813,989 13,620,716 43,651,539 190% -5% Sibiu 249,551,851 9,759,255 52,548,171 31,125,550 155,794,725 27,400,565 324,150 20,198,734 43,268,409 68% 23% Suceava 288,387,781 2,663,344 27,898,006 37,223,744 217,425,437 12,564,705 3,177,250 5,071,055 23,538,598 40% 19% Teleorman 136,592,682 2,004,905 17,758,793 26,711,410 90,000,735 1,445,670 116,839 21,577,189 11,759,202 72% 11% Timis 353,127,219 7,786,971 92,174,356 52,707,415 200,428,842 17,747,464 29,635 29,353,696 37,156,273 44% 29% Tulcea 123,508,937 7,111,276 15,748,745 17,536,441 81,961,597 5,041,365 1,150,878 9,901,229 11,050,929 50% 21% Vaslui 172,480,669 2,214,644 16,826,145 25,129,425 127,783,139 5,260,084 527,316 12,008,046 12,238,453 54% 15% Valcea 176,838,545 5,246,272 24,884,641 24,696,263 121,625,070 2,181,830 386,299 20,038,150 7,695,324 50% 17% Vrancea 175,188,430 1,827,149 17,848,850 20,876,847 133,020,317 2,372,326 1,615,267 31,320,043 6,914,422 91% 4% Source: Ministry of Regional Development and Public Administration – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html *Note: An explanation of the terms used here, as well as a short description of Intergovernmental Fiscal Relations in Romania is included at the end of this Annex. 372 Financial situation for 2011 (in Current RON 2011) Ratio Ratio of non- Total earmarked Shared PIT Non- Earmarked Grants – Capital Own Other revenues Total (Personal Earmarked Reimbursements from EU Capital Expenses Expenses Revenues Revenues for current Income Tax) Grants funds and Other Donors in Non- expenses Earmarked from total Revenues revenues Other Local EU Funds EU Funds Donors Budgets County Residence (2011) Alba Iulia 161,714,441 31,368,491 43,560,274 943,000 85,676,964 40,729,088 165,712 6,290,741 73,862,827 105% 23% Arad 382,745,664 133,419,137 114,293,079 3,895,033 131,138,415 9,725,030 0 93,787,881 29,880,276 49% 36% Pitesti 267,679,492 73,226,894 101,033,819 1,468,923 91,896,856 1,141,441 53,000 38,989,968 799,269 23% 51% Bacau 354,036,723 74,046,780 115,514,600 5,178,520 110,999,947 3,822,293 48,296,876 91,188,949 15,208,251 44% 40% Oradea 422,155,631 111,399,066 110,147,317 6,860,699 188,458,104 10,576,392 5,290,445 69,303,837 21,207,105 39% 36% Bistrita 141,870,962 40,174,444 48,114,223 143,887 53,438,408 4,464,739 0 28,838,765 13,324,012 48% 36% Botosani 217,243,082 45,134,713 40,596,968 4,853,027 126,542,184 36,187,623 116,190 42,526,728 24,821,406 74% 27% Brasov 597,828,047 182,614,663 167,016,782 2,862,997 227,835,476 10,653,185 17,498,129 184,826,453 21,557,456 56% 29% Braila 275,315,634 68,680,286 65,743,204 7,476,925 132,014,504 35,349,684 1,400,715 13,220,718 74,377,548 61% 33% Buzau 237,792,523 57,152,995 74,513,528 3,257,398 77,906,908 2,501,543 24,961,694 34,208,974 1,738,529 22% 53% Resita 101,669,727 27,216,691 30,914,317 2,047,046 41,427,608 769,774 64,065 15,158,042 2,905,391 30% 42% Calarasi 103,204,333 31,792,179 29,076,522 5,023,922 36,510,310 0 801,400 19,720,803 0 30% 46% Cluj-Napoca 642,735,997 161,417,454 296,271,023 1,459,603 183,199,271 32,944,878 388,646 50,216,112 47,122,602 21% 61% Constanta 523,387,185 175,004,161 183,059,616 2,404,699 162,918,709 3,139,836 0 101,780,584 4,554,738 29% 49% Sfantu Gheorghe 99,562,821 22,176,663 29,655,659 1,807,721 45,921,376 3,855,639 1,402 22,967,501 6,736,057 55% 28% Targoviste 197,827,191 47,243,017 51,946,316 2,427,507 96,122,241 30,366,186 88,110 13,310,590 21,965,138 35% 49% Craiova 445,333,369 117,587,245 158,624,958 671,087 164,983,250 15,425,430 3,466,829 26,038,120 37,750,414 23% 52% Galati 411,588,835 133,659,411 136,086,703 934,991 140,902,730 12,325,379 5,000 48,172,745 13,731,669 23% 54% Giurgiu 95,025,393 23,935,516 24,693,243 5,788,006 35,771,106 0 4,837,522 21,888,627 68,275 37% 39% Targu Jiu 153,595,050 37,190,859 42,097,017 3,272,618 70,952,191 13,362,685 82,365 14,371,628 25,250,381 48% 37% Miercurea Ciuc 108,427,871 22,618,960 28,392,624 1,522,206 55,825,081 9,234,891 69,000 24,270,803 14,826,359 74% 21% Deva 130,130,935 33,451,352 52,710,451 638,209 43,319,423 7,005,707 11,500 23,071,214 6,268,459 34% 50% Slobozia 78,946,599 24,519,566 26,346,115 2,028,796 26,048,072 374,229 4,050 21,144,146 795,806 41% 40% Iasi 504,353,610 124,798,738 166,292,646 10,748,177 202,249,716 31,722,373 264,333 49,142,474 28,049,956 26% 51% Baia Mare 203,291,410 58,338,486 65,077,285 1,163,085 78,567,895 4,073,059 144,659 25,902,969 5,197,938 25% 48% Turnu Severin 166,779,499 36,588,828 41,671,225 1,887,191 86,349,295 18,652,917 282,960 20,939,177 38,909,786 74% 24% Targu Mures 307,031,995 76,622,402 101,818,356 3,880,326 113,208,706 14,861,392 11,502,205 41,364,923 18,013,207 31% 49% 373 Piatra Neamt 231,366,710 49,069,820 43,927,458 11,778,322 124,616,753 34,763,503 1,974,357 62,100,711 69,830,422 124% 4% Slatina 160,040,858 45,905,155 51,739,413 423,977 61,947,163 9,305,453 25,150 20,992,841 18,635,159 40% 42% Ploiesti 406,071,056 124,705,739 159,097,115 4,334,790 117,837,676 127,482 95,736 46,282,964 198,424 16% 60% Satu Mare 204,980,132 47,722,353 55,132,730 596,616 68,901,644 2,073,708 32,626,789 39,739,771 7,533,039 35% 44% Zalau 100,723,661 29,187,774 29,319,223 690,329 41,279,727 2,230,188 246,608 3,361,228 3,788,675 12% 54% Sibiu 332,950,247 108,417,201 122,441,692 3,981,700 63,963,230 783,174 34,146,424 89,767,462 693,340 34% 54% Suceava 214,366,058 63,789,353 47,653,963 9,410,725 93,506,917 7,539,650 5,100 15,179,632 24,329,929 33% 41% Alexandria 81,471,641 18,724,141 27,930,322 630,362 33,983,282 5,090,145 203,534 9,276,010 3,950,497 28% 48% Timisoara 595,544,689 168,257,698 242,022,763 2,111,648 181,337,470 41,658,193 1,815,110 54,253,470 22,393,975 19% 64% Tulcea 144,078,306 43,454,672 41,568,821 1,923,004 56,976,072 1,758,848 155,737 24,041,787 5,245,063 34% 41% Vaslui 110,206,398 20,551,652 27,131,375 2,130,754 60,058,048 16,497,495 334,569 11,610,678 21,935,526 67% 30% Ramnicu Valcea 222,986,305 53,656,703 65,211,238 462,320 103,468,608 5,762,668 187,436 22,710,320 10,804,870 28% 41% Focsani 159,961,923 39,644,032 39,167,149 1,480,968 64,455,123 7,182,843 15,214,651 24,073,486 829,034 26% 49% County Councils 2,011 Alba 177,540,701 3,634,686 27,920,558 23,998,000 121,905,057 7,502,733 82,400 5,870,815 7,940,162 25% 28% Arad 227,082,093 17,480,718 41,717,506 31,462,450 124,976,001 22,757,317 11,445,418 34,930,020 58,153,136 91% 14% Arges 260,896,295 2,487,793 61,157,419 42,201,507 155,014,286 6,710,896 35,290 20,873,848 38,543,255 56% 20% Bacau 280,462,040 4,719,102 46,367,304 42,200,389 183,731,214 43,035,087 3,444,031 21,560,867 67,369,931 92% 18% Bihor 252,383,259 3,212,376 45,247,145 36,267,408 167,656,330 11,789,346 0 10,396,641 28,603,631 46% 23% Bistrita-Nasaud 195,934,355 2,706,844 18,490,802 19,453,383 154,461,680 35,763,802 821,646 21,310,680 112,627,988 323% -29% Botosani 198,096,473 2,772,283 16,056,229 26,534,467 152,219,159 43,253,042 514,335 14,712,249 45,550,932 131% 15% Brasov 239,475,757 6,967,094 64,664,005 38,510,977 103,202,116 241,534 26,131,565 43,229,279 1,522,062 33% 38% Braila 148,335,464 7,250,078 21,658,417 20,228,308 99,198,661 18,356,539 0 4,299,294 50,875,112 112% 8% Buzau 189,805,870 1,942,113 27,530,127 27,618,555 132,715,075 4,586,045 0 28,523,566 16,603,060 79% 9% Caras Severin 175,563,116 4,735,633 16,500,543 21,138,350 133,188,590 20,417,607 0 24,736,980 14,615,671 93% 13% Calarasi 149,459,275 1,448,030 13,667,584 19,339,947 113,266,059 25,502,949 1,737,655 6,807,129 51,454,223 161% 2% Cluj 368,927,814 11,393,568 99,057,665 57,908,668 157,555,326 2,196,419 43,012,587 145,054,149 18,722,155 77% 13% Constanta 388,784,411 11,226,168 75,250,992 48,547,048 253,230,783 27,452,027 529,420 111,696,803 55,109,834 123% -1% Covasna 93,819,193 2,593,687 14,096,295 14,970,038 62,012,058 13,275,594 147,115 7,942,400 40,744,929 153% -4% Dambovita 222,566,723 4,779,377 27,146,731 25,551,642 165,088,973 27,097,812 0 36,808,750 41,060,635 135% 3% Dolj 228,388,504 3,510,035 51,125,166 41,628,535 130,499,089 12,588,671 1,625,679 6,684,203 36,779,310 44% 29% Galati 201,336,593 3,094,180 42,408,098 33,815,257 122,016,240 16,266,767 2,818 26,967,558 33,410,369 76% 17% Giurgiu 163,603,417 27,003,682 12,927,250 15,501,706 106,941,065 35,052,704 1,229,714 29,993,077 55,370,873 151% 4% Gorj 161,708,179 6,957,408 30,469,667 23,487,761 100,719,436 6,844,918 73,907 6,616,276 19,110,618 42% 26% Harghita 136,762,832 3,262,679 19,413,204 20,198,326 93,436,097 18,220,864 452,526 24,066,103 35,619,729 138% 1% Hunedoara 204,812,356 2,069,693 37,290,258 34,001,179 131,451,226 7,100,620 0 32,301,584 21,246,022 73% 13% Ialomita 94,450,981 6,068,191 14,283,451 16,612,401 57,462,231 946,375 24,707 16,685,973 4,368,524 57% 18% Iasi 389,352,712 19,082,232 55,459,356 44,441,817 220,673,639 23,473,393 49,695,668 56,362,872 48,258,575 62% 22% 374 Ilfov 231,485,453 1,202,499 72,079,721 36,999,246 120,581,629 40,948,661 622,358 18,516,478 376,761 17% 57% Maramures 190,312,125 2,232,571 27,177,803 29,109,422 131,767,039 13,662,664 25,290 3,680,440 29,572,314 57% 20% Mehedinti 184,369,432 1,541,519 16,044,534 16,548,860 150,111,033 32,523,031 123,486 11,356,427 69,691,493 237% -8% Mures 271,558,854 3,065,648 46,730,890 36,190,151 185,550,765 25,246,063 21,400 16,924,959 38,156,607 64% 21% Neamt 249,394,045 2,671,654 24,025,107 31,698,603 158,001,097 19,691,452 32,997,584 45,318,435 33,792,427 87% 13% Olt 172,144,744 2,578,866 22,261,634 26,104,514 121,189,730 19,807,318 10,000 4,518,238 22,463,052 53% 25% Prahova 359,509,226 10,529,710 75,170,370 49,273,443 224,374,776 2,340,887 160,927 24,180,677 17,121,932 31% 27% Satu Mare 143,791,625 2,307,382 22,609,531 21,124,801 92,132,467 1,744,162 5,617,444 15,673,128 10,726,150 51% 19% Salaj 181,192,244 2,120,851 12,807,079 14,156,515 151,675,448 50,789,567 432,351 14,976,970 100,264,702 390% -19% Sibiu 257,000,723 6,255,673 51,499,656 32,941,932 164,452,560 21,026,004 1,850,902 14,146,015 59,728,990 80% 15% Suceava 344,205,371 2,274,463 28,583,998 40,226,263 268,171,646 50,506,613 4,949,001 4,506,840 61,601,088 87% 18% Teleorman 169,142,709 1,730,607 15,691,413 23,412,240 128,307,449 41,521,019 1,000 10,521,878 51,417,845 152% 12% Timis 426,120,925 8,147,551 87,666,166 54,013,000 276,244,632 63,675,696 49,576 20,939,573 95,024,029 77% 23% Tulcea 172,605,024 8,875,882 15,186,406 17,817,002 124,847,494 25,703,229 5,878,240 35,154,988 60,965,233 201% -13% Vaslui 211,722,994 2,625,949 17,129,567 26,256,336 165,487,059 28,898,232 224,083 16,679,597 17,048,269 73% 20% Valcea 194,453,370 6,408,697 25,753,191 24,048,260 138,243,222 3,328,723 0 25,845,366 3,337,599 52% 16% Vrancea 197,333,504 856,220 16,430,828 21,486,184 157,752,077 9,668,897 808,195 49,725,378 12,073,007 156% -6% Source: Ministry of Regional Development and Public Administration – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html *Note: An explanation of the terms used here, as well as a short description of Intergovernmental Fiscal Relations in Romania is included at the end of this Annex. 375 Financial situation for 2010 (in Current RON 2010) Ratio Ratio of non- Total Shared PIT earmarked Non- Capital Own (Personal Earmarked Other revenues Total Earmarked Capital Expenses Expenses Revenues Income Grants Revenues for current Grants in Non- Tax) expenses Earmarked from total Revenues revenues Local EU Funds Budgets County Residence (2010) Alba Iulia 158,642,481 25,295,156 42,777,821 3,565,000 49,665,457 37,339,047 5,201,384 27,939,288 30.41% 47.80% Arad 158,642,481 144,396,972 108,385,302 4,834,509 164,617,353 9,318,042 58,729,371 17,747,917 28.65% 44.13% Pitesti 291,963,672 69,801,497 111,063,113 3,157,998 107,883,290 57,774 39,157,468 0 21.27% 49.64% Bacau 359,691,370 56,830,359 112,350,813 2,309,862 140,171,267 48,029,069 28,091,016 19,525,142 21.69% 47.79% Oradea 412,945,568 95,653,813 112,754,747 5,413,441 178,873,277 20,250,290 14,080,484 17,729,435 13.59% 48.98% Bistrita 156,427,983 28,217,148 63,428,946 5,755,000 52,783,081 6,243,808 19,032,745 5,121,984 23.31% 50.82% Botosani 170,211,466 30,361,879 40,890,606 6,975,848 86,458,772 5,524,361 18,543,958 329,286 22.53% 38.12% Brasov 572,764,587 182,262,092 175,672,007 4,639,326 200,216,086 9,975,076 76,187,821 2,386,213 21.09% 51.33% Braila 258,961,232 55,354,730 71,377,348 5,653,692 113,177,918 13,397,544 14,278,655 15,661,233 20.54% 44.73% Buzau 233,524,753 51,437,957 72,056,316 5,358,586 103,329,108 1,342,786 11,518,831 2,328,575 10.64% 49.82% Resita 112,846,594 30,052,214 34,260,269 1,908,467 45,225,526 1,400,118 7,150,333 130,590 10.77% 53.47% Calarasi 103,244,429 26,582,661 29,538,234 4,521,499 42,555,345 46,690 13,804,393 23,000 22.78% 45.39% Cluj-Napoca 590,752,220 138,669,109 263,847,722 8,470,232 179,383,347 381,810 42,452,564 2,772,020 10.99% 61.98% Constanta 510,029,597 155,269,667 201,359,750 1,807,640 148,985,687 2,606,853 7,435,169 1,580,752 2.50% 69.02% Sfantu Gheorghe 113,259,925 19,838,888 30,450,794 4,950,223 54,505,684 3,514,336 26,860,466 3,381,603 51.47% 25.17% Targoviste 185,997,926 37,717,283 52,964,852 4,766,875 72,881,267 17,667,649 16,593,978 8,958,677 22.59% 47.08% Craiova 450,374,257 116,722,204 152,545,716 2,294,029 167,419,681 11,392,627 84,258,223 20,315,424 36.96% 39.61% Galati 414,869,012 115,832,126 137,818,160 1,672,998 146,360,404 13,185,324 56,003,069 18,785,040 27.85% 46.69% Giurgiu 98,201,945 20,084,403 26,137,553 6,580,721 44,774,168 625,100 11,718,362 277,842 22.45% 42.19% Targu Jiu 162,780,919 33,606,066 47,498,004 2,528,703 67,207,674 11,940,472 12,506,472 1,493,065 14.65% 50.11% Miercurea Ciuc 101,190,020 21,336,930 30,620,261 1,567,770 43,273,853 4,391,206 12,187,670 3,187,284 26.55% 42.04% Deva 130,818,301 30,725,115 55,206,170 1,659,660 41,708,162 1,519,194 32,744,722 1,066,518 37.94% 42.27% Slobozia 79,990,897 20,477,107 27,689,954 1,740,131 29,701,513 382,192 21,329,204 161,755 42.73% 36.00% Iasi 522,593,044 102,258,300 178,088,906 15,430,384 187,443,982 39,371,472 57,966,202 4,559,893 18.66% 52.17% Baia Mare 207,084,103 50,050,225 66,506,197 1,509,782 86,671,725 2,346,174 25,766,637 3,406,718 24.23% 44.06% Turnu Severin 170,172,781 31,563,147 43,753,687 1,652,274 72,820,144 20,383,529 19,020,266 809,426 20.37% 45.56% Targu Mures 313,467,141 69,943,214 107,754,598 5,175,861 114,939,330 15,654,138 21,149,402 14,603,892 18.01% 51.93% 376 Piatra Neamt 205,143,127 41,975,330 46,492,719 12,929,347 79,808,316 23,937,415 56,688,077 13,242,729 55.80% 27.01% Slatina 164,672,031 44,910,690 49,101,315 1,450,461 69,183,640 25,925 22,689,914 70,150 23.84% 44.17% Ploiesti 425,144,437 105,610,318 170,623,300 6,951,398 141,845,280 114,141 71,228,506 141,763 25.19% 49.85% Satu Mare 185,998,801 47,732,456 56,979,505 431,234 77,708,972 3,146,634 8,864,376 279,702 8.44% 53.30% Zalau 185,998,801 47,732,456 56,979,505 431,234 77,708,972 3,146,634 8,864,376 279,702 8.44% 53.30% Sibiu 290,317,550 93,228,091 112,129,646 1,934,880 81,854,785 1,170,148 58,290,715 1,712,104 28.78% 51.14% Suceava 222,870,449 54,324,491 51,173,000 16,151,997 85,457,936 15,763,025 30,681,067 3,162,957 24.63% 46.47% Alexandria 86,976,962 18,992,589 27,539,197 1,979,298 37,990,643 475,235 7,395,520 163,806 15.43% 47.63% Timisoara 583,434,636 147,490,669 242,369,457 5,189,989 187,358,908 1,025,613 37,555,617 1,264,321 9.80% 61.23% Tulcea 150,272,050 39,952,409 41,476,625 3,531,901 57,280,299 8,030,816 16,006,732 6,951,663 24.69% 46.60% Vaslui 103,273,844 24,711,018 27,816,076 2,574,908 47,270,133 901,709 12,768,740 3,113,567 28.36% 38.85% Ramnicu Valcea 198,552,261 50,049,119 63,599,745 3,273,899 75,790,749 5,838,749 27,319,996 707,599 22.83% 47.71% Focsani 150,509,854 35,412,237 44,206,882 871,992 69,090,900 927,843 12,155,307 1,944,624 17.32% 44.73% County Councils (2010) Alba 189,192,369 3,210,465 27,511,633 25,781,000 120,735,579 11,953,692 15,453,032 16,232,819 46.29% 19.44% Arad 227,210,209 13,024,925 46,159,101 27,888,391 91,659,909 48,477,883 3,028,565 18,697,894 16.03% 50.10% Arges 278,300,399 2,766,383 62,931,311 39,167,769 159,609,384 13,825,552 23,589,901 9,057,135 27.51% 30.92% Bacau 236,390,276 5,367,065 47,818,429 37,867,593 128,846,329 16,490,860 16,770,175 20,115,357 34.30% 29.89% Bihor 259,382,651 2,267,954 46,742,651 32,573,044 163,236,078 14,562,924 478,073 9,515,106 10.39% 33.21% Bistrita-Nasaud 190,489,072 2,394,916 22,969,879 21,215,000 97,949,409 45,959,868 5,816,408 32,621,987 41.54% 28.40% Botosani 170,970,434 1,135,115 16,573,010 23,862,381 116,081,373 13,318,555 13,721,204 8,333,802 40.18% 19.20% Brasov 211,443,896 6,743,676 67,218,685 32,957,617 102,822,501 1,701,417 23,931,554 70,559 22.10% 40.02% Braila 140,671,223 2,735,743 23,564,667 18,390,337 75,868,229 20,112,247 3,788,920 19,209,523 35.49% 29.72% Buzau 189,136,190 1,162,290 28,245,292 24,726,926 116,708,091 18,293,591 17,717,807 23,188,641 56.48% 16.67% Caras Severin 157,342,660 3,263,479 18,646,203 26,851,880 104,666,173 3,914,925 24,881,607 646,618 48.46% 17.25% Calarasi 137,771,715 472,052 13,629,864 16,384,467 76,709,943 30,575,389 3,538,335 9,268,461 20.97% 35.03% Cluj 388,538,945 11,977,219 92,590,732 112,958,948 156,306,742 14,705,304 18,337,434 4,988,417 10.04% 53.77% Constanta 323,176,652 6,645,876 81,445,441 47,665,522 171,634,976 15,784,837 85,750,748 10,217,293 63.33% 17.20% Covasna 87,501,385 2,863,992 14,510,169 16,796,031 45,537,778 7,793,415 2,091,278 54,728 5.11% 45.51% Dambovita 202,728,614 2,703,961 28,765,882 33,849,794 112,968,016 24,440,961 21,319,267 12,100,980 37.23% 27.79% Dolj 287,493,098 1,987,921 55,884,338 38,472,395 123,905,691 67,242,753 11,879,718 85,530,956 59.55% 23.02% Galati 210,125,755 1,838,716 43,768,197 30,324,700 110,293,445 23,900,697 23,021,850 22,736,628 45.84% 25.73% Giurgiu 125,191,122 4,430,245 13,130,555 13,200,654 59,164,731 35,264,937 6,562,370 3,581,657 15.36% 44.64% Gorj 160,607,724 1,809,278 34,057,178 22,906,510 88,287,879 13,546,879 5,959,783 14,355,758 28.09% 32.38% Harghita 145,921,189 3,156,951 20,535,278 21,692,042 79,031,931 21,504,987 23,837,656 23,060,521 70.11% 13.70% Hunedoara 230,631,234 1,683,553 42,703,221 35,075,087 115,660,000 35,509,373 29,137,729 8,477,212 32.72% 33.54% Ialomita 98,269,100 5,547,693 14,976,658 14,933,181 58,629,477 4,182,091 11,716,055 7,984,117 49.70% 20.29% 377 Iasi 324,547,516 8,907,398 58,742,849 42,814,263 188,619,940 25,463,066 13,374,397 19,123,171 23.91% 31.87% Ilfov 252,757,906 1,346,155 70,264,023 35,877,149 70,197,737 75,072,842 19,354,659 68,539,951 48.15% 37.45% Maramures 234,377,056 3,622,878 28,524,367 30,630,441 114,591,929 57,007,441 3,541,471 26,329,545 24.94% 38.36% Mehedinti 198,606,088 1,171,773 17,575,673 16,569,730 103,271,746 60,017,166 14,661,431 45,871,611 63.50% 17.52% Mures 227,319,683 2,933,762 48,590,408 33,840,078 140,930,656 1,024,779 15,002 1,563,360 1.83% 37.31% Neamt 243,353,437 2,514,858 25,370,015 28,840,145 143,306,042 43,322,377 8,146,135 38,174,529 46.30% 22.08% Olt 161,386,240 2,141,582 22,912,991 22,165,476 93,001,325 21,164,866 108,122 18,472,346 27.17% 30.86% Prahova 374,886,611 11,109,324 78,850,688 47,069,698 223,975,655 13,881,246 31,021,327 20,412,795 34.08% 26.54% Satu Mare 160,804,144 5,517,504 23,674,556 19,760,451 102,139,284 9,712,349 31,765,442 2,422,539 58.28% 15.22% Salaj 147,422,749 2,960,195 13,365,726 12,228,109 81,975,653 36,893,066 2,860,457 3,670,212 9.98% 39.96% Sibiu 248,431,003 8,213,754 48,878,994 29,501,297 132,731,424 29,105,534 4,572,930 9,576,804 12.23% 40.88% Suceava 350,787,705 2,021,047 30,694,337 45,075,658 203,027,924 69,968,739 3,506,181 50,005,392 36.22% 26.87% Teleorman 191,400,226 1,492,711 17,438,718 20,521,087 82,938,494 69,009,216 16,631,017 71,048,525 80.84% 10.86% Timis 339,977,738 10,458,478 88,434,996 51,253,856 174,753,392 15,077,016 11,710,476 2,339,639 8.50% 44.47% Tulcea 209,457,172 6,604,726 15,769,669 19,646,524 109,397,334 58,038,919 47,315,929 47,979,461 95.24% 2.27% Vaslui 213,029,019 2,034,046 16,755,163 22,175,821 139,156,504 32,907,485 13,170,468 43,333,042 76.49% 8.15% Valcea 208,427,502 6,408,588 26,098,087 16,070,365 156,030,388 3,820,074 1,041,385 3,627,184 8.91% 22.90% Vrancea 264,636,729 1,240,683 18,266,793 30,125,302 147,717,119 67,286,832 26,656,552 33,292,230 51.27% 21.53% Source: Ministry of Regional Development and Public Administration – www.dpfbl.MRDPA.ro/sit_ven_si_chelt_uat.html *Note: An explanation of the terms used here, as well as a short description of Intergovernmental Fiscal Relations in Romania is included at the end of this Annex. 378 Short Description of Intergovernmental Fiscal Relations in Romania Own Revenues: first tier local government revenues generated by local fiscal revenues (property tax, vehicle tax, company income tax for the companies owned by local authorities, judicial and notary taxes, special taxes/betterment taxes, other minor local taxes) and non-fiscal revenues (concessions, rents, penalties/fines, revenues from capital). The second tier local governments (county councils) have a much more narrow capacity on own revenues: company income tax for the companies owned by county authorities, tax on transport vehicles, some license fees, capital revenues, concessions and rents, and other very small sources. Shared Personal Income Tax (PIT): revenues generated by sharing the PIT based on area of collection, following the formula: 41.75% for the budget of tier 1 local governments (municipalities, small towns, communes), 11.25% for the budget of tier 2 local governments (county councils), 18.5% for equalizing at the level of the county using the same equalization formula used by the center government. This share of 13% should be considered non-earmarked grants – analyzed below. Non-Earmarked Grants: equalization transfers supported by PIT or VAT, allocated by counties and localities in two rounds: first round Ministry of Finance allocates equalization transfers based on VAT by counties, using a formula with two indicators: area (30% weight) and fiscal capacity per capita (calculated only based on PIT and used inverse proportional to obtain an equalization effect) (70% weight). From the total non-earmarked transfer at county level (what is allocated by MoF based on VAT and 18.5% from collected PIT within county administrative boundaries) the county council budget receives 27% and the rest is allocated among localities in the following manner: - 80% is allocated among localities using a formula with three indicators: fiscal capacity per capita (calculated on PIT and used inverse proportionally), population and urban area, excluding the agricultural land). The effective allocation is done by the deconcentrated office of the MoF and it is realized in two steps. In the first step the transfers are allocated only for the local government units with the fiscal capacity per capita (calculated based on PIT) less than the average fiscal capacity per capita calculated at county level, using a formula with two indicators: population (75% weight) and permitted to build area (25%). In the second step, the non-allocated money are distributed to all local governments from the county, inverse proportionally with their fiscal capacity per capita; - 20% is allocated among localities by county councils for three purposes, but without formula: the pay the arrears, to support the local development plans, and to support the co-financing of investment projects. Earmarked Grants are state transfers with specific destinations: for the first tier of local governments they are used for pre-university education and social welfare (Minimum Income Guarantee program, disabled people, heating subsidies for poor families). At the county level, the earmarked grants are essentially used to finance the social assistance services (for children, disabled persons, and the elderly). The allocation by local government units is done generally based on number of beneficiaries – in pre-university education it is calculated mainly on the number of teachers. The reimbursements of EU funds are also an important component of earmarked grants, used essentially for investments – the value of EU non-investment projects at the level of local government is only marginal. Other Revenues contains a number of small other revenue sources – with the exception of 2010, when apparently the reimbursements from EU funds have been included in this category (for this reason the share of this category is very high in 2010). This analysis is done based on international practice and OECD classification and specific categories for local government finance. 379 380 Annex 21. Economic and financial appraisal criteria A. High level purpose of criteria 1. Self-financing or commercially viable projects (User charges or fees exceed operating and maintenance (O&M) expenses such that they are contributing to payment of capital expenditures) Appraisal Perspective Appraisal Method Appraisal Purpose Sponsor (owner or equity Financial appraisal based on sponsor or 1. Determine whether financially viable holder) equity holders cost of capital (cash 2. Determine whether sponsor needs capital flows include loan disbursement and grant to make viable or sponsor willing to service specifically raised to finance pay for right to implement project (basic project) PPP issue): estimate size of capital grant or payment Combined financiers or Financial appraisal based on weighted Assessment of capacity of project to raise total investment average cost of capital debt finance Economic Economic appraisal including all Assessments of projects economic impact or external cost and benefits discounted aggregate contribution to net wealth of at the economic opportunity cost of economy capital Distributive or stakeholder Disaggregation of economic appraisal Identification of net winners and losers to identify net benefits or costs for amongt stakeholders to determine need to major groupings of stakeholders: reallocate gains away from excessive winners sponsors, users/beneficiaries, or compensate major losers from project government, etc. 2. Non-self-financing or revenue-dependent projects (Government revenues required for supporting capital and O&M expenses; any user charges or fees are insufficient to cover O&M expenses) Appraisal Perspective Appraisal Method Appraisal purpose Sponsor (owner or equity Financial appraisal based on equity Determine size of grant if no use of sponsor holder) : Typically, local holders cost of capital (cash flows funds, and government in case of ROP include loan disbursement and service a. Grant if O&M is covered by budget = specifically raised to finance project) capital expenditure a. Include fees and revenue b. Grant if O&M and share of capital is contribution of sponsor to cover covered by budget = capital cost less O&M expenses sponsor government contribution b. Include contribution to co-finance capital expenditure Sponsor budget supply Net revenue forecast (Growth in Assessment of financial capacity of revenues net of all other new O&M government entity to financially sustain commitments relative to required project operation essential to service delivery project O&M ) for economic impact Economic Same as above, but benefits of direct Assessments of projects economic impact or users of project services become key aggregate contribution to net wealth of external benefit economy Distributive or stakeholder Disaggregation of economic appraisal Identification of net winners and losers to identify net benefits or costs of among stakeholders to determine need to major groupings of stakeholders: reallocate gains away from excessive winners sponsors, users/beneficiaries, or compensate major losers from project government, etc. 381 B. Technical criteria for financial and economic appraisal 1. Self-financing or commercially viable projects Appraisal Perspective & Criteria Comment purpose Financial appraisal from perspective of 1. FNPV(K) > 0 at real discount rate 1. Determine whether financially viable sponsor (owner or equity holder) to equal to cost of investment funds 2. Determine whether sponsor needs capital grant determine attractiveness of project and to sponsor and NCF of sponsor (FNPV(K)<0) or sponsor willing to pay for right need for grant or willingness to pay for including debt disbursements and to implement project (FNPV(K) > 0): FNPV give project. service amounts 2. FRR(K) > real discount rate equal 3. FRR(K) > real discount rate indicates financial to cost of investment funds to viability, but not useful in making comparisons sponsor between projects unless of similar size, 3. FRR(K)-discount rate duration, NCF pattern, etc. 4. Gap of FRR(K) over discount rate is a measure of risk coverage. Financial appraisal from perspectives of 1. FNPV(C) >0 and FRR(C) > WACC 1. FNPV(C) = FNPV(K) and hence does not add combined financiers or total 2. Debt service ratios by year and information investment for analysis of financing cumulatively over remaining 2. Debt service ratios measure the annual and requirements and capacity to repay project life. cumulative ability of the project to cover debt debt service Economic appraisal to access the 1. ENPV > 0 at real social discount 1. ENPV measure the present value of economic economic net benefit or impact rate (EU recommended at 5.5%) benefits minus the present value of economic 2. ERR > 5.5% costs generated by the project or a measure of 3. B/C > 1 the net increase in economic wealth 2. Measures based on ERR or B/C are both based on ENPV measure and can be used for project selection, but not selection between projects. Distributive or stakeholder ENPV is disaggregated to measure the Identification of net winners and losers among NPV earned by all stakeholders stakeholders to determine need to reallocate gains away from excessive winners or compensate major losers from project 2. Non-self-financing or revenue dependent projects Appraisal Perspective & Criteria Comment purpose Financial appraisal from perspective FNPV(K) measured after inclusion of Where O&M (net of any fees or user charges) is of sponsor (owner or equity holder) : fees, user charges and debt financing of required to be covered by the sponsor budget, Typically, local government in case project and after budget contribution by there is no need for any NPV calculations as it is of ROP sponsor to cover co-financing of project clear that the project needs a capital grant to cover and O&M over life of project = capital the capital expenditures in excess of the ineligible expenditure – ineligible costs and cost and required co-financing amount. required co-financing Ability of sponsor to sustain project Net revenue forecast (increase in 1. New or rehabilitated assets need O&M over project life from budget revenues net of all other new O&M expenditures over life time to ensure service resources commitments) - project O&M delivery. Required O&M varies by asset type expenditures >0 annually and (road by type, school, health facility, IT cumulatively equipment, etc.) 2. Where old assets have been sustained by high O&M, O&M savings can accrue through asset rehabilitation Economic & distributional appraisal As above As above as above 382 C. Cost-effectiveness or unit cost measures Cost-effectiveness measures are often employed when it is difficult to place a value on the output or services delivery by a project. They are also used to estimate the unit costs of production for a service. The measure can either be expressed in financial cost terms or economic cost terms as needed. Financial unit costs are required for budgeting or contracting purposes where the financial costs of a service provider need to be covered. For example, the financial cost of treating a patient, educating a student to a specified level, delivering a cubic meter of water, maintaining or rehabilitating a kilometer of road, etc. are all useful budgeting measures. Economic unit costs can be a substitute selection criterion. For example, the economic cost of extending a patient’s life by a year or saving an hour per vehicle on per 1000 vehicle-kilometer. Conceptually cost-effectiveness measures are derived from the net present value criterion whether from a financial or economic perspective for a project or program: NPV = PV (price or unit value of stream quantities or effects over life of project) – PV (stream of capital and operating costs over life of project) If NPV = PV (p * stream of quantities) – PV (costs) = 0 and p is a constant real price (value or unit cost) indexed for inflation, then p = PV(costs) / PV(stream of quantities) = real unit cost or value. A few key points should be noted about this unit cost measure (p): i. The unit cost applies to the pattern of costs incurred and pattern of quantities or effects produced over some specific project life time. ii. The unit cost will be sensitive to the timing of the costs (increase if costs are front loaded) and the quantities (increase if the achievement of the effects are delayed) iii. The unit cost effectively smooth out the accrued annual costs that may vary from year to year over the project life time. 383 Abbreviations (consistent with EU manual notation) FNPV(K) = financial net present value based on real net cash flows (including all user charges and fees and receipts of debt disbursement on loan raised to finance project and interest and principal payments) discounted by real cost of investment funds of sponsor (cost of equity if private investor). FRR(K) = real financial internal rate of return based on real net cash flows for NPV(K) FNPV(C) = financial net present value based on real net cash flows to total investment before inclusion of debt financing discounted by the weighted average cost of capital (WACC) FRR(C) = real financial internal rate of return based on real net cash flows for NPV(C) WACC = weight average of the costs (real interest rates or required rates of return) of the shares of debt and equity in invested in the total value of the project assets. ENPV = Net present value of real cash flow to total investment plus all external costs and benefits arising to users of project services or suppliers of project inputs or changes in government revenues discounted by the real social discount rate ERR = Economic rate of return is the internal rate of return earned on the flows of net benefits or costs B/C = Benefit-cost ratio = PV( economic benefits) over PV (economic costs) 384 Annex 22. Limitations to the use of the Cost-Benefit Analysis and the Internal Rate of Return in the competitive selection of projects The Cost-Benefit Analysis (CBA) and the Internal Rate of Return (IRR) are two indicators that are now used by the MA-ROP to evaluate and select projects. The indicators are not used to compare projects, but rather as measures of individual project performance. They are however inadequate measures for comparing projects with each other. In what follows, we will outline some of the limitations of these tools. The Limitations of the IRR The IRR is the discount rate (K) at which the present value of benefits are just equal to the present value of costs for the particular project It is important to note that the IRR is a mathematical concept, not an economic or financial criterion, and it has two common uses: (a) If the IRR is larger than the cost of funds then the project should be undertaken (b) Often the IRR is used to rank mutually exclusive projects. The highest IRR project should be chosen An advantage of the IRR is that it only uses information from the project Put in graph form, the IRR looks as follows: Next, we will provide four examples that highlight the limitations of the IRR, as a tool for comparing projects. 1. Having multiple rates of return for a project over time 385 As the graph below indicates, an internal rate of return can vary over time. For the example above, if we set the discount rate at 0% or at 100%, we get the same Net Present Value (NPV), as indicated in the two solutions proposed below: Solution 1: K = 100%; NPV= -100 + 300/(1+1) + -200/(1+1)2 = 0 Solution 2: K = 0%; NPV= -100+300/(1+0)+-200/(1+0)2 = 0 2. Having projects of different sizes and strict alternatives In this situation one could have an NPV and an IRR that provide different conclusions, as indicated in the example below: Opportunity cost of funds = 10% NPV : 600/0.10 - 2,000 = 6,000 - 2,000 = 4,000 NPV : 4,000/0.10 - 20,000 = 40,000 - 20,000 = 20,000 Hence, NPV > NPV IRRA : 600/KA - 2,000 = 0 or KA = 0.30 IRRB : 4,000/KB - 20,000 = 0 or KB = 0.20 Hence, KA>KB 3. Having projects of different lengths of life and strict alternatives Opportunity cost of funds = 8% Project A: Investment costs = 1,000 in year 0 Benefits = 3,200 in year 5 Project B: Investment costs = 1,000 in year 0 Benefits = 5,200 in year 10 IRRA : -1,000 + 3,200/(1+KA)5 = 0 which implies that KA = 0.262 IRRB : -1,000 + 5,200/(1+KB)10 = 0 which implies that KB = 0.179 386 Hence, KA>KB 4. Having the same project but started at different times Project A: Investment costs = 1,000 in year 0 Benefits = 1,500 in year 1 Project B: Investment costs = 1,000 in year 5 Benefits = 1,600 in year 6 IRRA : -1,000 + 1,500/(1+KA) = 0 which implies that KA = 0.5 IRRB : -1,000/(1+KB)5 + 1,600/(1+KB)6 = 0 which implies that KB = 0.6 Hence, KB>KA In sum, the IRR can be used to compare investments when they have the same:  Scale/size;  Timing  Term/length  Pattern of benefits This is very hard to do in practice, as ROP projects, even within the same axis, tend to be quite different with respect to the four items listed above. The Limitations of the CBA The Benefits-Cost Ratio is calculated as follows: Benefit-Cost Ratio (R) = Present Value Benefits/Present Value Costs If the benefit-cost ratio (R) is larger than 1, then the project should be undertaken. There are however many cases when it is not possible to rank projects with the Benefit-Cost Ratio: • Mutually exclusive projects of different sizes • Mutually exclusive projects and recurrent or operating costs subtracted out of benefits and costs versus benefits reported gross of operating costs • Not necessarily true that if RA>RB project “A” is better These issues can be explained easily in mathematical form using the following example: Project A: PV0of Costs = $5.0 M, PV0 of Benefits = $7.0 M NPVA = $2.0 M RA = 7/5 = 1.4 0 0 Project B: PV of Costs = $20.0 M, PV of Benefits = $24.0 M NPVB = $4.0 M RB = 24/20 = 1.2 According to the Benefit-Cost Ratio criterion, project A should be chosen over project B because RA>RB, but the NPV of project B is greater than the NPV of project A. So, project B should be chosen. 387 A second problem is that the Benefit-Cost Ratio does not adjust for mutually exclusive projects with recurrent or operating costs subtracted out of benefits and costs versus benefits reported gross of operating costs. For example: Project A: Total Costs = $5.0 M Recurrent Costs = $1.0 M (i.e., Fixed Costs = $4.0 M) PV0 of Gross Benefits= $7.0 M RA = (7-1)/(5-1) = 6/4 = 1.5 (compared to 1.4 above) Project B: Total Costs = $20.0 M Recurrent Costs = $18.0 M (i.e., Fixed Costs = $2.0 M) PV0 of Gross Benefits= $24.0 M RB = (24-18)/(20-18) = 6/2 =3 (compared to 1.2 above) Hence, project B should be chosen over project A under Benefit-Cost Criterion. Consequently, the CBA cannot be used to rank projects. 388 Annex 23. ROP Performance Crosswalk based on US example Challenge Thematic Objective ERDF Expected Results ROP Priorities for Funding US Performance Goals for Comparison US Performance Indicators for US Performance US Performance US Comparison (All Indicators) Indicators - Results Indicators - Output Performance Indicators Indicators Indicators - Key Implementati on Steps Competitivenes 1. Strengthening research, Higher levels of investment in Developing technology‐transfer support i (NASA - New Missions) Enable bold new missions and (NASA - New Missions) Total number of new (NASA - New Missions) Total s technological developmen RDI, both in the public and priv nfrastructure in the public and private se make new technologies available to Government agencies technology reports, software usage agreements, number of new technology t and innovation ate sectors; greater translation ctors, particularly in Romania's less develo and U.S. industry - By September 30, 2013, document the new patent applications filed, technology licenses, reports, software usage of research findings into com ped regions, informed by the principles of s maturation of new technologies by completing 4,065 documented technology spinoffs and NASA agreements, new patent mercial applications; an impro mart specialization. technology-related products, including patents, licenses, technology mission use documents. applications filed, technology ved distribution of RDI activity and mission use agreements. licenses, documented across Romania's regions. technology spinoffs and NASA (NSF - Public Access) Increase opportunities for research technology mission use and education through public access to high-value digital documents. products of NSF-funded research. By September 30, 2013, NSF will have established policies for public access to high- value data and software in at least two data-intensive scientific domains. Competitivenes 2. Enhancing access to, an More dynamic and competitiv N/A s d use and quality of, infor e markets across Romania, wit mation and communicatio h associated gains for consume n technologies rs, arising from increased use o f e‐commerce; Improvements t o public services and reduced a dministrative burden, both on businesses and citizens, arising from the implementation of e‐ government 2.0. Competitivenes 3. Enhancing the competiti Sustainable growth of the Rom • (SBA - Business Loans) Process business loans as efficiently (SBA - Business Loans) Percentage of 7(a) (SBA - Small Business (SBA - Business Loans) s veness of small and mediu anian economy, reflected in in Actions to enhance enterprise culture and as possible By September 30, 2013, increase the use of Electronic Loan Applications. Contracting) Percentage of 7(a) Electronic m‐sized enterprises, the ag creased formal economic activi to raise awareness of the option paperless processing in the 7(a) program from 72% to 90% (SBA - Business Loans) Number of 7(a) Electronic Percentage of federal Loan Applications. ricultural sector and the fis ty and employment across all o and potential support in starting business. and in the 504 program from 55% to 75% to improve the Applications. prime contracts heries and aquaculture sec f Romania's less developed reg efficiency, effectiveness, and level of service in its business (SBA - Business Loans) Total Number of 7(a) Loan awarded to small (SBA - Business Loans) Number tor ions; a higher proportion of em • loan programs. Applications business (23%). of 7(a) Electronic Applications. ployment in internationally co Actions to improve the productivity of S mpetitive industrial and higher MEs and to provide to growth opportuni (SBA - Small Business Contracting) Increase Small Business (SBA - Small Business Contracting) Percentage of (SBA - Small Business (SBA - Business Loans) Total value‐adding service sectors; a ties and innovation, including supporting participation in government contracting Increase small federal prime contracts awarded to small business Contracting) Dollar Number of 7(a) Loan n improved contribution to gro business participation in government contracting. By (23%). amount of federal Applications wth and employment from SM advanced technological investments, pro September 30, 2013, SBA will increase small business contract dollars Es; increased international trad moting creative industries, enhancing acc participation in federal government contracting to meet the (SBA - Small Business Contracting) Dollar amount awarded to small (SBA - Access to Capital) The e; improved environmental sus ess to business planning support, to tech government wide goal that 23 percent of all prime of federal contract dollars awarded to small businesses. Debenture program level tainability of the business sect nical consultancy, international markets, contracting dollars go to small businesses, and continue to businesses. obligated in a given fiscal year. or. to exporting advice and support. ensure that the benefits of SBA’s small business contracting These funds are used by SBIC programs flow to the intended recipients. (SBA - Access to Capital) The Debenture program licensees to invest in small • level obligated in a given fiscal year. These funds businesses. Actions to improve access to finance, inclu (SBA - Access to Capital) Expand Access to Long Term are used by SBIC licensees to invest in small ding through the provision of an appropriat Capital From FY 2012 through September 30, 2013, commit businesses. (SBA - Access to Capital) The e range of financial instruments. at least $4.3 billion of capital via the Small Business amount of funding provided to Investment Company program in order to facilitate access (SBA - Access to Capital) The amount of funding small businesses in a given FY. • to capital for high growth companies and enhance job provided to small businesses in a given FY. These These funds are used to invest Actions to network businesses, to enhan creation and retention by these companies. funds are used to invest in and grow small in and grow small businesses. ce knowledge spillovers and to facilitate t businesses. heir participation in supply chains, includin (SBA - Access to Capital) The g those of international scale. (SBA - Access to Capital) The number of SBIC number of SBIC funds receiving funds receiving a new license in a given FY. These a new license in a given FY. • Integrated actions funds (SBIC licensees) will invest capital in small These funds (SBIC licensees) will sites, skills, investment support businesses. invest capital in small to attract investment into Romania's less d businesses. eveloped regions 389 Challenge Thematic Objective ERDF Expected Results ROP Priorities for Funding US Performance Goals for Comparison US Performance Indicators for US Performance US Performance US Comparison (All Indicators) Indicators - Results Indicators - Output Performance Indicators Indicators Indicators - Key Implementati on Steps People and 8. Promoting employment Increased economic activity an Creation of new small enterprises through Society and supporting labor mobi d employment, particularly in business start‐up aid in urban areas lity Romania's less developed regi ons and rural areas. People and 9. Promoting social inclusi Improved public satisfaction wi • (HUD - Vacancy Rates) Reduce vacancy rates. By September (HUD - Vacancy Rates) Percent of Neighborhood (HUD - Vacancy Rates) (HUD - Vacancy Rates) Number Society on and combating poverty th Romania's health and social Development of basic infrastructure of s 30, 2013, reduce average residential vacancy rate in 70% of Investment Clusters (NICs) with vacancy rate Percent of of units of service completed services, and better health out ocial services, focusing community‐based the neighborhoods hardest hit by the foreclosure crisis* outcomes over at least one comparable area. Neighborhood and occupied using NSP2 funds, comes, particularly in Romania infrastructure. relative to comparable areas. *Hardest hit neighborhoods Investment Clusters as reported by NSP2 grantees. 's less developed regions. are defined as NSP 2 Neighborhood Investment Clusters (HUD - Vacancy Rates) Number of units of service (NICs) with vacancy rate [Units of service are the the • (NICs). completed and occupied using NSP2 funds, as outcomes over at least number of units produced Investments in hospitals and other public h reported by NSP2 grantees. [Units of service are one comparable area. within each eligible activity. ealth infrastructure. (HUD - Affordable Housing) Preserve affordable rental the the number of units produced within each Many units may be produced housing. By September 30, 2013, preserve affordable rental eligible activity. Many units may be produced with (HUD - Vacancy Rates) with multiple activities (e.g., • housing by continuing to serve 5.4M families and serve an multiple activities (e.g., acquisition and rehab) and Average number of acquisition and rehab) and Support for physical, economic and social r additional 61,000 families through HUD's affordable rental therefore the measure “units of service” is not days it takes to list an therefore the measure “units of egeneration of deprived communities in ur housing programs. directly translatable into number of total FHA real-estate owned service” is not directly ban areas. individual housing units.] (REO) property to translatable into number of market. total individual housing units.] (HUD - Vacancy Rates) Average number of days it takes to list an FHA real-estate owned (REO) (HUD - Vacancy Rates) (HUD - Affordable Housing) property to market. The average number of Total HUD Families Served days a FHA real-estate October 1, 2011 to September (HUD - Vacancy Rates) The average number of owned (REO) property 30, 2013. days a FHA real-estate owned (REO) property stays stays in inventory. in inventory. (HUD - Affordable (HUD - Affordable Housing) Total HUD Families Housing) Number of Served October 1, 2011 to September 30, 2013. occupied rental units in Multifamily Housing (HUD - Affordable Housing) Number of occupied properties, including rental units in Multifamily Housing properties, Project Based Section 8, including Project Based Section 8, tax credit/low tax credit/low income income housing tax credit (LIHTC) financed units, housing tax credit HUD insured privately owned properties, housing (LIHTC) financed units, for elderly and disabled (202 and 811), and legacy HUD insured privately housing programs like Rental Assistance Program owned properties, (RAP) and Rent Supplement properties. Many of housing for elderly and these programs are being preserved, so targets disabled (202 and 811), are negative, reflecting an expected loss of units, and legacy housing with the overall strategy of trying to decrease the programs like Rental number of units lost. While units are expected to Assistance Program be lost, a successful trend would be positive, not (RAP) and Rent negative. Supplement properties. Many of these programs are being preserved, so targets are negative, reflecting an expected loss of units, with the overall strategy of trying to decrease the number of units lost. While units are expected to be lost, a successful trend would be positive, not negative. 390 Challenge Thematic Objective ERDF Expected Results ROP Priorities for Funding US Performance Goals for Comparison US Performance Indicators for US Performance US Performance US Comparison (All Indicators) Indicators - Results Indicators - Output Performance Indicators Indicators Indicators - Key Implementati on Steps People and 10. Investing in education, Good quality education and tra Supporting the educational infrastructure (Education - Improve Outcomes) Improve outcomes for all (Education - Improve Outcomes) States (Education - Improve Society skills and lifelong learning ining at all levels, delivered in a and resources development in ECEC, prim children from birth through third grade. By September 30, implementing high-quality plan to collect and Outcomes) States n attractive environment. ary and secondary education, tertiary edu 2013, at least nine states will implement a high-quality plan report disaggregated data on the status of children implementing high- cation and VET to improve the education a to collect and report disaggregated data on the status of at kindergarten entry. quality plan to collect nd training quality and relevance for the L children at kindergarten entry. and report M. (Education - Effective Teachers) By September 30, disaggregated data on (Education - Effective Teachers) Improve learning by 2013, the majority of states will have statewide the status of children at ensuring that more students have an effective teacher. By requirements for comprehensive teacher and kindergarten entry. September 30, 2013, at least 500 school districts will have principal evaluation and support systems. comprehensive teacher and principal evaluation and (Education - Effective Teachers) By September 30, (Education - Effective support systems and the majority of states will have 2013, at least 500 school districts will have Teachers) By statewide requirements for comprehensive teacher and comprehensive teacher and principal evaluation September 30, 2013, principal evaluation and support systems. and support systems. the majority of states (Education - Effective Teachers) By September 30, will have statewide (Education - School Turnaround) Demonstrate progress in 2013, at least 500 school districts will have requirements for turning around the nation's lowest-performing schools. By comprehensive principal evaluation and support comprehensive teacher September 30, 2013, 500 of the nation's persistently systems. and principal evaluation lowest-achieving schools will have demonstrated significant and support systems. improvement and will have served as potential models for (Education - School Turnaround) Lowest-achieving future turnaround efforts. schools demonstrating significant improvement (Education - Effective and serving as potential models for future Teachers) By (Education - Informed Decisions) Make informed decisions turnaround efforts. September 30, 2013, at and improve instruction through the use of data. By least 500 school September 30, 2013, all states will implement (Education - Informed Decisions) Number of districts will have comprehensive statewide longitudinal data systems (SLDS). States implementing K-12 data systems. comprehensive teacher and principal evaluation (Education - College Preparedness) Prepare all students for (Education - College Preparedness) Number of and support systems. college and career. By September 30, 2013, all states will States adopting internationally-benchmarked adopt internationally-benchmarked college- and career- college-and career-ready standards. (Education - Effective ready standards. Teachers) By (Education - College Affordability) Number of September 30, 2013, at (Education - College Affordabilty) Improve students' ability states with college completion goals in place. least 500 school to afford and complete college. By September 30, 2013, the districts will have Department will develop a college scorecard designed to (HHS - Childhood Education) Increase the number comprehensive improve consumer decision-making and transparency about of states that implement Quality Rating and principal evaluation and affordability for students and borrowers by streamlining Improvement Systems (QRIS) that meet high support systems. information on all degree-granting institutions into a single, quality benchmarks. comparable, and simplified format, while also helping all (Education - School states and institutions develop college completion goals. (HHS - Childhood Education) Reduce the Turnaround) Lowest- proportion of Head Start grantees receiving a achieving schools (HHS - Childhood Education) Improve the quality of early score in the low range on the basis of the demonstrating childhood education. By September 30, 2013, improve the Classroom Assessment Scoring System (CLASS: significant quality of early childhood programs for low-income children Pre-K). [For the purposes of this measure, low is improvement and through implementation of the Quality Rating and defined as below a 2.5 on a 7-point scale. Grantees serving as potential Improvement Systems in the Child Care and Development must have an average score of at least a 2.5 to be models for future Fund, and through implementation of the Classroom out of the low range.] turnaround efforts. Assessment Scoring System in Head Start. (HHS - Childhood Education) Increase the number (Education - Informed (DOL - Industry Credentials) Improve opportunities for of states and territories with professional Decisions) Number of America's workers to succeed in a knowledge-based development systems that include core knowledge States implementing K- economy through industry-recognized credentials. By and competencies, career pathways, professional 12 data systems. September 30, 2013, increase the percent of training development capacity assessments, accessible program exiters who earn industry-recognized credentials professional development opportunities, and (Education - College by 10 percent. financial supports for child care practitioners. Preparedness) Number of States adopting (DOL - Industry Credentials) Increase the percent internationally- of training program exiters who earn industry- benchmarked college- recognized credentials by 10 percent. and career-ready 391 Challenge Thematic Objective ERDF Expected Results ROP Priorities for Funding US Performance Goals for Comparison US Performance Indicators for US Performance US Performance US Comparison (All Indicators) Indicators - Results Indicators - Output Performance Indicators Indicators Indicators - Key Implementati on Steps standards. (Education - College Affordability) Number of states with college completion goals in place. (HHS - Childhood Education) Increase the number of states that implement Quality Rating and Improvement Systems (QRIS) that meet high quality benchmarks. (HHS - Childhood Education) Reduce the proportion of Head Start grantees receiving a score in the low range on the basis of the Classroom Assessment Scoring System (CLASS: Pre-K). [For the purposes of this measure, low is defined as below a 2.5 on a 7- point scale. Grantees must have an average score of at least a 2.5 to be out of the low range.] (HHS - Childhood Education) Increase the number of states and territories with professional development systems that include core knowledge and competencies, career pathways, professional development capacity assessments, accessible professional development opportunities, and financial supports for child care practitioners. (DOL - Industry Credentials) Increase the percent of training program exiters who earn industry- recognized credentials 392 Challenge Thematic Objective ERDF Expected Results ROP Priorities for Funding US Performance Goals for Comparison US Performance Indicators for US Performance US Performance US Comparison (All Indicators) Indicators - Results Indicators - Output Performance Indicators Indicators Indicators - Key Implementati on Steps by 10 percent. Infrastructure 2. Enhancing access to, an Extensive coverage of NGA bro N/A (DOC - Rural Broadband) Expand Broadband Service to (DOC - Rural Broadband) Miles of broadband (DOC - Rural (DOC - Rural Broadband) Miles d use and quality of, infor adband, of at least 30 Mbps in Communities. By September 30, 2013, the Department will networks deployed (Infrastructure Projects). Broadband) New of broadband networks mation and communicatio cluding in rural areas; high leve increase the nation's broadband infrastructure developed (DOC - Rural Broadband) New and upgraded household and business deployed (Infrastructure n technologies l of take‐up of NGA broadband through the Broadband Technology Opportunities Program public computer workstations (Public Computer subscribers to Projects). services both by business and (BTOP) from 29,200 miles at the end of FY 2011 to 75,000 Centers Projects). broadband (Sustainable domestic customers. miles. (DOC - Rural Broadband) Community anchor Broadband Adoption (DOC - Rural Broadband) New institutions connected (infrastructure projects). Projects). and upgraded public computer (DOC - Rural Broadband) New household and workstations (Public Computer business subscribers to broadband (Sustainable Centers Projects). Broadband Adoption Projects). (DOC - Rural Broadband) Community anchor institutions connected (infrastructure projects). 393 Challenge Thematic Objective ERDF Expected Results ROP Priorities for Funding US Performance Goals for Comparison US Performance Indicators for US Performance US Performance US Comparison (All Indicators) Indicators - Results Indicators - Output Performance Indicators Indicators Indicators - Key Implementati on Steps Infrastructure 7. Promoting sustainable tr Improved traffic safety, particu Increasing accessibility of urban areas locat (DOT - Passenger Rail) Advance the development of (DOT - Passenger Rail) Number of corridor (DOT - Roadway (DOT - Passenger Rail) Number (DOT - Air Traffic) ansport and removing bott larly on road and rail; reducing ed in proximity of TEN‐T network, through passenger rail in the United States By September 30, 2013, programs achieving initial construction. Fatalities) Reduce the of corridor programs achieving Achieve Initial lenecks in key network infr travel time, more sustainable u construction and modernization of second initiate construction on all 7 high-speed rail corridors and 36 (DOT - Passenger Rail) Number of individual roadway fatality rate initial construction. Operating astructures rban transport and consequent ary and tertiary connections to the networ individual high-speed rail projects. construction projects achieving initial per 100 million Vehicle (DOT - Passenger Rail) Number Capability (IOC) on ly reduced pollution. k, including by‐passes, having in view bottl construction. Miles Travelled (VMT) of individual construction En Route enecks removing and traffic fluidization; (DOT - Air Traffic) Air traffic control systems can improve (DOT - Passenger Rail) Number of monitoring to 1.03 in 2013. projects achieving initial Automation the efficiency of airspace. Air traffic control systems can reviews. construction. Modernization improve the efficiency of airspace. By September 30, 2013, (DOT - Roadway (DOT - Passenger Rail) Number (ERAM) program replace a 40-year old computer system serving 20 air traffic (DOT - Air Traffic) Achieve Initial Operating Fatalities) Reduce of monitoring reviews. at Air Route control centers with a modern, automated system that Capability (IOC) on En Route Automation passenger vehicle Traffic Control tracks and displays information on high altitude planes. Modernization (ERAM) program at Air Route occupant fatalities per Centers (ARTCCS). Traffic Control Centers (ARTCCS). 100 million passenger (DOT - Roadway Fatalities) Reduce the rate of roadway vehicle Miles Travelled (DOT - Aviation fatalities. Reduce the rate of roadway fatalities from 1.26 in (DOT - Roadway Fatalities) Reduce the roadway (VMT) to 0.82 in 2013. Safety) Implement 2008 to 1.03 per 100 million vehicle miles traveled (VMT) by fatality rate per 100 million Vehicle Miles Travelled 80% of approved December 31, 2013. (VMT) to 1.03 in 2013. (DOT - Roadway interventions to (DOT - Roadway Fatalities) Reduce passenger Fatalities) Reduce mitigate the top 5 (DOT - Aviation Safety) Reduce risk of aviation accidents. By vehicle occupant fatalities per 100 million motorcycle rider hazards associated September 30, 2013, reduce aviation fatalities by passenger vehicle Vehicle Miles Travelled (VMT) to fatalities per 100,000 with airborne addressing risk factors both on the ground and in the air. 0.82 in 2013. motorcycle registrations losses of Commercial aviation (i.e., U.S. Carriers): Reduce fatalities to (DOT - Roadway Fatalities) Reduce motorcycle to 63 in 2013. separation. no more than 7.4 per 100 million people on board. General rider fatalities per 100,000 motorcycle aviation (i.e., private planes): Reduce fatal accident rate per registrations to 63 in 2013. (DOT - Roadway 100,000 flight hours to no more than 1.06. (DOT - Roadway Fatalities) Reduce non-occupant Fatalities) Reduce non- (pedestrian and bicycle) fatalities per 100 million occupant (pedestrian (Army Corps - Commercial Navigation) Help facilitate VMT to 0.16 in 2013. and bicycle) fatalities commercial navigation by providing safe, reliable, highly (DOT - Roadway Fatalities) Reduce roadway per 100 million VMT to cost-effective, and environmentally-sustainable waterborne fatalities involving large trucks and buses per 100 0.16 in 2013. transportation systems. Through 30 September 2013, limit million VMT to 0.114 in 2013. annual lock closures due to mechanical failures of main lock (DOT - Roadway chambers on high and moderate use waterways to no more (DOT - Aviation Safety) Reduce U.S. commercial Fatalities) Reduce than 46 for closures lasting more than 1 day and no more air carrier fatalities per 100 million persons on roadway fatalities than 26 for closures lasting more than one week. board to 7.6 in FY 2012 and to 7.4 in FY 2013. involving large trucks (DOT - Aviation Safety) Reduce the general and buses per 100 aviation fatal accident rate to no more than 1.07 million VMT to 0.114 in fatal accidents per 100,000 flight hours in FY 2012 2013. and no more than 1.06 fatal accidents per 100,000 flight hours in FY 2013. (DOT - Aviation Safety) (DOT - Aviation Safety) Reduce Category A & B Reduce U.S. commercial (most serious) runway incursions to a rate of no air carrier fatalities per more than .395 per million operations, and 100 million persons on maintain or improve through FY 2013. board to 7.6 in FY 2012 (DOT - Aviation Safety) Implement 80% of and to 7.4 in FY 2013. approved interventions to mitigate the top 5 hazards associated with airborne losses of (DOT - Aviation Safety) separation. Reduce the general aviation fatal accident (Army Corps - Commercial Navigation) Lock rate to no more than closures due to mechanical failiures on main locks 1.07 fatal accidents per of moderate and high use waterways lasting more 100,000 flight hours in than one day. FY 2012 and no more (Army Corps - Commercial Navigation) Lock than 1.06 fatal closures due to mechanical failures of main locks accidents per 100,000 on moderate and high use waterways lasting more flight hours in FY 2013. than one week. (DOT - Aviation Safety) Reduce Category A & B (most serious) runway 394 Challenge Thematic Objective ERDF Expected Results ROP Priorities for Funding US Performance Goals for Comparison US Performance Indicators for US Performance US Performance US Comparison (All Indicators) Indicators - Results Indicators - Output Performance Indicators Indicators Indicators - Key Implementati on Steps incursions to a rate of no more than .395 per million operations, and maintain or improve through FY 2013. (Army Corps - Commercial Navigation) Lock closures due to mechanical failiures on main locks of moderate and high use waterways lasting more than one day. (Army Corps - Commercial Navigation) Lock closures due to mechanical failures of main locks on moderate and high use waterways lasting more than one week. Resources 4. Supporting the shift tow Reduced greenhouse gas emi • (HUD - Energy Retrofits) Increase the energy efficiency and (HUD - Energy Retrofits) Number of HUD Cost (HUD - Energy Retrofits) ard a low‐carbon economy ssions; actions to improve to the thermal insulatio health of the nation's housing stock. By September 30, effective, healthy, energy-efficient, and green Number of HUD Cost effective, in all sectors more sustainable urban transp n of residential stock, and to public buildin 2013, HUD will enable a total of 159,000 cost effective retrofits or new housing. healthy, energy-efficient, and ort and consequently reduced gs to be prioritised on the basis of systema energy efficient or healthy housing units, as a part of a joint green retrofits or new housing. pollution; tic appraisal taking into account the cost‐ef HUD-DOE goal of 520,000 in 2012-2013 and a total goal of ficient reduction of GHG emissions and soci 1.2 million units from 2010 through 2013. improved efficiency of energy etal benefits including addressing energy p use in industry, agriculture, fis overty. heries, the residential sector a nd the public realm. • actions to rehabilitation and renewal of ma ss transit systems within the framework of sustainable urban mobility plans, where th ese will make significant contribution to air quality and energy efficiency. • actions for replacement/improvement of li ghting systems in the public realm, particul arly in urban areas. Resources 5. Promoting climate chan N/A N/A ge adaptation, risk prevent ion and management 395 Challenge Thematic Objective ERDF Expected Results ROP Priorities for Funding US Performance Goals for Comparison US Performance Indicators for US Performance US Performance US Comparison (All Indicators) Indicators - Results Indicators - Output Performance Indicators Indicators Indicators - Key Implementati on Steps Resources 6. Protecting the environm Reduction in the area of dereli Protecting and sustainable valorisation of (EPA - Contaminated Sites) Clean up contaminated sites (EPA - Contaminated Sites) Number of Superfund (EPA - Contaminated Sites) ent and promoting resourc ct and polluted former industri natural sites, restoration and valorisatio and make them ready for use. By September 30, 2013, an sites ready for anticipated use (RAU) site-wide. Number of Superfund sites e efficiency al sites; increasing the visitor’s n of cultural heritage, including valorisati additional 22,100 sites will be ready for anticipated use. ready for anticipated use (RAU) number. on of the local specific tourism potential, (EPA - Contaminated Sites) Number of site-wide. the rehabilitation of historical (urban) area (EPA - Drinking Water) Improve public health protection brownfields properties ready for reuse. s, sustainable capitalization of the cultural for persons served by small drinking water systems by (EPA - Contaminated Sites) patrimony, measures for urban environme strengthening the technical, managerial, and financial (EPA - Contaminated Sites) Number of Resource Number of brownfields nt (including rehabilitation of industrial poll capacity of those systems. By September 30, 2013, EPA will Conservation and Recovery Act (RCRA) corrective properties ready for reuse. uted sites) engage with twenty states to improve small drinking water action facilities made ready for anticipated use system capability through two EPA programs, the (RAU). (EPA - Contaminated Sites) Optimization Program and/or the Capacity Development Number of Resource Program. (EPA - Contaminated Sites) Number of leaking Conservation and Recovery Act underground storage tank (LUST) cleanups (RCRA) corrective action (EPA - Water Quality) Improve, restore, or maintain water completed that meet risk-based standards for facilities made ready for quality by enhancing nonpoint source program human exposure and groundwater migration. anticipated use (RAU). accountability, incentives, and effectiveness. By September 30, 2013, 50% of the states will revise their (EPA - Water Quality) Number of waterbodies (EPA - Contaminated Sites) nonpoint source program according to new Section 319 identified by states (in 1998 -2000 or subsequent Number of leaking underground grant guidelines that EPA will release in November 2012. years) as being primarily nonpoint source impaired storage tank (LUST) cleanups that are partially or fully restored. (cumulative). completed that meet risk-based standards for human exposure and groundwater migration. (EPA - Water Quality) Number of waterbodies identified by states (in 1998 -2000 or subsequent years) as being primarily nonpoint source impaired that are partially or fully restored. (cumulative). Governance 11. Enhancing institutional Reduced obstruction to develo Support to develop a national cadastral sys capacity and an efficient p pment and consolidation of ag tem to provide certainty of title, promotion ublic administration ricultural holdings arising from of land reform and effective land asse land ownership disputes. mbly in support of Romania's developme nt goals Governance 2. Enhancing access to, an Improved efficiency of public a N/A d use and quality of, infor dministration arising from enh mation and communicatio anced use of ICTs and e‐govern n technologies ment; improved public satisfac tion with public administration and public services. 396 Annex 24. The LHDI as an Overall Measure of Public Investments for the ROP’s M&E The purpose of this section is to design and test a version of the human development index (HDI) and to use it for monitoring the development of Romania over a set period of time. The standard HDI developed by the UNDP measures development on the basis of three dimensions - education, life expectancy, and GDP per capita - usually measured country-wide or for large regions (UNDP, 2013). The Local Human Development Index (LHDI) is an adaptation of the HDI designed to capture the development at the commune and city level in Romania. It measures development by looking at four key dimensions: human, health, and vital and material capital. The figure below provides an overview of the individual indicators collected to measure the LHDI. Figure 50. The Mechanics of the LHDI Indicators Dimensions ( forms of Community community capital) capital Education stock at locality HUMAN CAPITAL level (EDSTOCK) Life expectancy at birth HEALTH CAPITAL LOCAL HUMAN DEVELOPMENT INDEX (LHDI) (LIFEXP) Medium age of adult population of 18 years old VITAL CAPITAL and over. Complementary value to 1 (CompAGE) Living floor area by house (locality average) Distribution of gas for MATERIAL CAPITAL household consumption (MATCAP, geometric by locality inhabitant mean of component indicators) Private cars to 1000 inhabitants Different versions of the index have been designed and implemented in Romania since the mid-90s. Four waves (see Annex 24A) of measuring human development at the community level followed:  commune (i.e., basic rural administrative unit) level, in a research project for the World Bank (1998);116 116 Dumitru Sandu (coordinator). 1998. Community poverty in Romania rural areas .Foundation for poverty alliviation by Romania Social Development Fund. World Bank, Bucuresti (http://dumitru.sandu.googlepages.com/CommunitypovertyinRomaniaruralareas1.pdf) and Dumitru Sandu, 1999, Community and Regional Poverty in Rural Romania, in Romanian Journal of Sociology, vol. X. 397  village (i.e., socio-territorial unit without an administrative status) level, in a national UNDP report (2005);117  basic administrative units level, favoring comparisons between similar localities or between different residential communities (2008);118  basic administrative units level (communes or cities), also allowing for comparisons between data collected in different censuses (2013, based on data from the 2011 and 2002 censuses). This analysis represents the fifth wave of measuring local human development and it is designed to allow for a yearly computation of the LHDI, irrespective of census moments. The methodology for normalizing the constituent indicators and for aggregating the four dimensions is the same as the one for HDI: each indicator is normalized by dividing its deviation from the minimum value of the data series to the range between minimum and maximum (mini-max method); normalized values are aggregated by geometric means. The analysis is structured in three parts, focusing on the steps in computation of the LHDI, the computation algorithm, and validating the index. Annexes 24 A, B, and C contain further details about all issues discussed in this section. The LHDI could be computed for all localities in Romania, for each successive year or at a two-year interval, using the data provided by the National Institute of Statistics. The only constituent indicator that is reported only to census moments is the education stock, which equals the average number of graduated years at the locality level. Due to its high stability as a stock measure, this indicator will be updated through estimations every three years. EXTRALOCAL CONNECTIVITY by out-commuting , migration abroad and by urban accesibility The diagram and the associated analysis show clearly that the POPULATION CONCENTRATION (locality size and urban status) LHDI incorporates directly EDUCATION DIFFUSE MODERNITY as MATERIAL CAPITAL reflected by low fertility relevant information on human, LOCAL HUMAN material, heath, and vital AGE DEVELOPMENT capitals of localities. INDEX Connectivity and concentration factors are also highly relevant HEALTH for LHDI. This is in line with the three D approach of the World Development Report 2009. LOCAL LABOUR MARKET SUPPLY The validation of the index is done by first testing the stability over time of the relation between local development as a latent variable and its measurement indicators (see Annex 24B). Second, the index is validated by considering the LHDI in the context of its key predictors 117 UNDP. (2005). National Human Development Report (NHDR) for Romania, p. 65. A previous form of LHDI , from 1998, working mainly with non-census data. 118 The data from this version of the index (called in Romanian indicele dezvoltarii sociale locale, IDSL, Sandu 2011) have been largely used for the foundation of the document laying the foundation for the designing the administrative regionalisation of Romania in 2013 (Sandu 2013). 398 related to extra-local connectivity, local labor supply, population concentration, and diffuse modernity of the population as reflected in demographic behaviors (see figure above). Steps in computing the LHDI 1. Computing the values for the component indicators for the reference locality L and a reference moment: 1.1. Life expectancy at birth (LIFEXP) is estimated for the last three years of available data. Its value for 2011, for example, involved estimations for the period 2009-2011 and the measure for 2012 was based on the 2010, 2011, and 2012 values. 1.2. Stock variables referring to mean age (MEANAGE), size of the houses and private cars are based on data from the last available year. 1.3. Education stock (EDSTOCK) is the only indicator that is measured only at census moments. In the first three years after the census moment the same data can be used for the estimation of the education stock. For later years, the update of this indicator could be done by considering regional values taken from large surveys conducted by the National Institute of Statistics (e.g., Living Force Survey, known as AMIGO) and intraregional education stock hierarchies of localities. 2. Checking data quality for each indicator by: - comparing the most recent data to the previous data at locality and county levels - examining maximum and minimum values. It is particularly important to verify the quality of the data for private cars, as this data is usually not aggregated for territorial administrative units (UAT), but rather collected for individual localities (villages, communes, cities). 3. Estimating upper limits for each of the six component indicators (I) for 2020. The principle at the foundation of this activity is to consider two inputs: the elasticity of the means for the reference indicator to the increase of GDP/capita between 2002-2011 and its expected upper limit for 2020.The GDP/capita in Romania as a percentage of the EU average was equal to 47% in 2011. Its target value for 2020 is approximately 80%of the EU average119. 3.1. Elasticity coefficient for indicator I= (I2011 – I2002 )/ (GDP2011-GDP2002) 3.2. Expected increase of mean I2011-2020=(GDP2020-GDP2011)*elasticity coefficient. 3.3. Expected mean for I2020 = mean of I2011 + Expected increase of mean I2011-2020. 3.4. Estimating maximum limit for I2020 by considering the trend in the change of maximum/medium value between 2002 and 2011: max I2020 =mean I2020 *(max I2020 / mean2020). 4. Identifying minimum value for index I as equal to the minimum recorded in 2002 and 2011. 5. Normalizing each indicator I through the mini-max procedure: dividing (observed deviation value of index I for locality L, in the reference year)/ difference between maximum and minimum values for the whole period 2002-2020. 119 The reference level of 80% for Romania’s GDP/capita2020 is adopted from National Sustainable Development Strategy Romania 2013-2020-2030, GOR, UNDP 2008. 399 6. Aggregating the three normalized indicators of the material capital (CAPMAT) – private cars, living floor area per household and natural gas consumption per inhabitant – to generate the partial index of material capital for each locality. 7. Computing LHDI as a geometric mean of the four component dimensions for each territorial administrative unit (UAT) in Romania with more than 1000 inhabitants in 2011: LHDI= (EDSTOCK*CAPMAT*LIFEXP*(1-MEANAGE))(1/4) . 8. Testing the validity of the new LHDI estimates, compared to: 8.1. Measures from previous years, 8.2. Predictability of the new value functions of external criteria (regression models based on predictors of urban connectedness, locality size, immigration, commuting etc. – see relevant chapter later in the analysis). Computing algorithm Normalizing variables using the mini-max method with empirically determined fixed ranges Where : i – indicator i, l –reference locality, t – reference time for current estimation, t0 – starting year for the period of range stability, tn – end year for the period of range stability t0 min – average of the five smallest values of the indicator for the year t0 The starting year for the data series of LHDI is t0= 2002 (the year of the second to last census) and tn=2020. The expected maximum value for the ith indicator is determined by the value from the last census plus the expected increase function of the known variation between the two last successive censuses. The maximum values for the last two censuses are averages of the five highest values in the corresponding series. (1) (2) The normalization of the indicators composing the LHDI for a series of successive measurements involves the identification of stable minimum and maximum limits over time (see equation 1). What would the maximum limit be for the stock of education at the level of locality in 2020? What is the best rule for predicting such a limit for Romanian localities in 2020? Answers to these questions have to be formulated for all the indicators composing the LHDI. Education stock (EDSTOCK) is the average number of years that people from the same locality graduated within formal education cycles. The number of graduated years in different cycles differs according to the age of the person. One way to get a proxy for the local education stock 400 is to multiply people in each educational segment by the maximum number of years of the reference cycle. People that only graduated primary school, for example, would receive 4 years of education. Following this logic, the computation weights (wi) for the number of people (pi) that graduated cycle i would be 8 for gymnasium, 10 for vocational or apprenticeship schools, 12 for high school, 14 for post-secondary school and 16 for higher education. Where ∑pi =all the population living for more than 10 years in the respective locality. The education stock in Cluj-Napoca city, for example, increased from 10.9 in 2002 to 12.1 in 2011. The EDSTOCK at the national level grew in the same period from 8.6 to 9.8. In order to set the upper limit for 2020 we will have to first consider the expected mean of the constitutive indicator function of the elasticity coefficient of its change for one unit of change in the GDP/capita of the country from the mean of European Union. In the 2002-2011 timeframe the relative GDP/capita of the country increased from 30% to 47%120 and the education stock from 8.6 to 9.8. By dividing the increase of EDSTOCK to the increase of GDP we get an elasticity coefficient of 0.071. Expected increase of mean EDSTOCK2011-2020=(GDP2020-GDP2011)*elasticity coeff.=(80- 47)*0.068=2.24 By adding the resulted value to 9.8, the average stock for 2011 that results is 12.03 =9.79+2.24, representing the expected average level for the education stock in Romania 2020. The ratio of the maximum education stock in the series of data for all the localities to the national average remained constant in both census years of 2002 and 2011, equal to 1.23. If the same ratio for 2020 is adopted, the resulting maximum value for EDSTOCK in 2020 is 14.79. elasticity coeff. maximum* LIFEXP/GDP per capita Mean Minimum Maximum Range /mean relative to EU mean EDSTOCK 2002 8.63 4.13 10.72 6.78 1.24 EDSTOCK 2011 9.79 3.85 12.08 8.44 1.23 0.068 expected EDSTOCK2020 12.03 3.85 14.79 10.94 1.23 Data source: National Institute of Statistics (NIS). Note: National averages for 2002 and 2011 take the locality population as a weigh factor. The cumulated data base includes only the localities that had more than 1000 inhabitants in 2011 (2,839 for 2002 and 3,059 for 2011). The maximum values for 2002 and 2011 represent the averages of the first five highest values in the year series. An important hypothesis used is that of the constant ratio between maximum/mean. The shadows indicate figures derived through elasticity coefficients or expected ration between maximum and mean. Life expectancy at birth (LIFEXP) is the average number of years a person born in the reference year would live under the same unchanged life conditions as the current ones. The index is computed by aggregating mortality rates for all the segment of population by age or age-groups. 120 Data source for GDP/capita as percentage from EU mean: EUROSTAT http://epp.eurostat.ec.europa.eu/tgm/table.do?tab=table&init=1&plugin=1&language=en&pcode=tec00114 . The reference level of 80% for Romania_GDP/capita2020 is adopted from National Sustainable Development Strategy Romania 2013-2020-2030, GOR, UNDP 2008. 401 elasticity coeff. LIFEXP/GDP per capita Mean Minimum Maximum Range max./mean relative to EU mean LIFEXP1999-2001 70.6 53.3 77.5 26.3 1.10 LIFEEXP2009-2011 73.6 57.7 88.4 32.6 1.20 0.177 LIFEXP2018-2020 79.5 53.3 91.4 38.1 1.15 Data source: NIS Note: National averages for 2002 and 2011 use locality population as a weigh factor. The cumulated data base includes only the localities that had more than 1000 inhabitants in 2011 (2,839 for 2002 and 3,059 for 2011). Hypothesis of a decrease of the ratio between maximum/mean toward the average between 2002 and 2011. This could be the result of higher concentration of the population and of the reduction in health disparities through regional policies. Maximum values for 2002 and 2011 are averages of the first five highest values in the year series. The shadows indicate figures derived through elasticity coefficients or expected ration between maximum and mean. The average age for the adult (18 and 18+ years old) population in a locality is included into LHDI index as a measure of the vital capital of the locality. The higher the mean age in the community, the lower its vital capital and, implicitly, its human development level. The aging process in the Romanian society is inevitable. Its variation in territory, among regions or among localities, indicates the variation in the development potential as given by its younger working- age population or vital capital. Due to the negative correlation between local human development and the mean age of adult population, this indicator will be included into the aggregation process for the LHDI as a complement to one, after its normalization through the mini-max procedure. The average age of the adult population is expected to move from 45.6 in 2011 to about 49.4 years of age in 2020. A slight increase of the upper average limit per locality is also expected. It is presented in the table below and follows the same reasoning as in previous tables: elasticity coeff. Average age of population in maximum* locality/GDP per capita M ean M inimum M aximum Range /mean relative to EU mean average age adults 2002 43.67 34.00 59.40 27.30 1.36 average age adults 2011 45.63 37.20 58.82 22.90 1.29 0.115 expected average age 2020 49.44 34 60.31 26.31 1.22 Data source: NIS. The maximum in the local data series is expected to come closer to the average, as indicated by the trend from 2002 to 2011. That could be a consequence of the expansion of the aging process in more and more communities. Emigration abroad contributed and will very likely continue to contribute to the same trend. The shadows indicate figures that are derived through elasticity coefficients or expected ration between maximum and mean. The indicator is correlated in a negative way with the local human development. This is the reason why its complementary to one values (after rescaling it through the min-max formula) is included in the algorithm for the LHDI. Living floor area per dwelling. The disparities in the housing conditions among localities are expected to increase up to 2020. It will be particularly true for communes that are close to rich cities to have larger houses (Dumbravita and Ghiroc near Timisoara, Corbeanca and Voluntari close to Bucharest, and Corunca in the proximity of Targu Mures, for example) and for poor rural localities with low connectivity to dynamic centers to keep their traditional small houses 402 (Necsesti and Scurtu Mare in Teleorman county, Lemnia in Covasna, Birda in Timis and Alexandru Vlahuta in Vaslui, for example). elasticity coeff. Living floor area/GDP per capita Mean Minimum Maximum Range maximum* /mean relative to EU mean living floor area per dwelling 37.79 20.78 64.16 45.94 1.70 2002 living floor area per dwelling 39.65 16.72 93.36 95.03 2.35 2011 0.109 expected living floor area per dwelling 2020 43.26 16.72 129.98 113.26 3.00 D ata source: NIS Private cars to 1000 inhabitants. The low elasticity of variation in the number of car per 1000 inhabitants could be an effect of the weak measurement at the raw data level: some of the cars in Romania are recorded abroad due to tax reasons, and also due to the fact that the official evidence is better at the county level rather than at the locality level. Data series for 2011 involved several estimations as some localities had cars in 2007 and no car in 2012 etc. A detailed check of the raw data is necessary in the case of repeated analysis for the LHDI. elasticity coeff. Private cars/GDP per capita Mean Minimum Maximum Range maximum* /mean relative to EU mean private cars to 1000 inhabit 2007, 7.94 0.00 12.99 12.99 1.64 ln transformation private cars to 1000 inhabit 2012, 8.17 1.10 13.22 12.12 1.62 ln transformation 0.014 expected no. of private cars to 1000 inhabitants 2020 8.98 0 14.55 1.62 Data source: Ministry of Interior The natural gas distributed to households had a rather small increase between 2002 and 2011. The range among maximum and minimum consumption decreased. Repeated use of the measure for LHDI computations would involve, as in the case of private cars, a more detailed examination of the raw data quality (as recorded into locality data base BDL at the NIS level). elasticity coeff.natural gas distributed to hhds./GDP per capita relative to EU Mean Minimum Maximum Range maximum* /mean mean natural gas distributed to 131.84 0.00 1212.60 1212.60 9.20 population ( m3 per inhabitant) 2002 natural gas distributed to 139.26 0.00 1000.00 1000.00 7.18 households (m3 per inhabitant) 2011 0.4360 natural gas distributed to households (m3 per inhabitant, 2020, estimation) 153.65 0 1220.00 1213 5.16 Data source: NIS All the normalized indicators and the generated geometric means are multiplied by 100 for an easier reading. 403 Validating the LHDI There are several ways to validate the Local Human Development Index. First of all, one can consider the stability of the relations between local development as latent variable and its indicators measuring the four forms of vital, human, material and health capitals. Annex 24B argues with technical reasons, and by comparing data from 2002 and 2011, that the criterion of ”metric invariance” is met to a partial, but acceptable degree, according to the mainstream methodological practices. Human capital is the most relevant indicator for local human development in both periods of reference. Material capital tends to have a higher relevance for development compared to vital capital during the period of reference. The process is consistent with the stable nature of the age structure compared to material capital stocks. The ways of using variables for measuring human development at the local level keep a rather consistent configuration in their inter-relationships. One of the most relevant reasons for proving the validity of LHDI is the predictability function of the development factors that are not included into its computation (criterion validity, Babbie 2010). From the degree to which the index is really measuring local development one can expect to get a good estimation of its value function of indicators related to local connectivity, local employment and diffuse cultural modernity of the local population. The data in the figure below entirely supports the idea that local human development is strongly dependent on local connectivity through-out – commuting, migration abroad, and distances to the nearest cities. Higher development goes also together with higher local labor supply, population concentration (given by urban status and demographic size) and higher cultural modernity, as reflected into demographic behaviors. The above mentioned factors have different roles, depending on the residence status of the locality: local population concentration seems to have a larger impact on rural local development than on urban development; the share of employed people from the same locality has a higher impact for the development of cities than for that of communes; cultural modernity of the population is a significant predictor just for communes (see table A26c in Annex 24C). The above findings demonstrate that the LHDI is a powerful development measure that includes information both on the four forms of community capital (education, health, material and vital) and on important conditioning factors of development (extra-local connectivity, labor supply, territorial concentration and diffuse local modernity). In order to test the effect of the aggregation method on the index we used as well a more sophisticated procedure (a factor score) with the same four constituting indicators. The factor score procedure generates an index that is highly correlated with the LHDI, based on aggregation through the geometric mean (r= 0.98). The disadvantage of the factor score procedure is that the generated index is less comparable across time. The map below highlighting the development level for counties in 2012 could be taken as a reference for the dynamics of the LHDI. This is largely consistent with another map that circulated extensively in Romania when the foundations for regionalization project in 2013 were under debate (Sandu 2013). The location of the most developed areas remained the same: Sibiu-Brașov-Timiș-Cluj-București-Ilfov. This is the case for poverty as well, which is mainly located in both maps in the South and East, and in the lower middle-developed areas in West Moldova. Middle developed areas “migrated” from West Transylvania in 2002 to North and South-West Transylvania in 2012. 404 Table 28. Predictors of Human Local Development in Romania, 2012 High Labor Supply, High Connectivity of Locality to Urban Centers, Region of Location and Other Counties bring High Local Development spatial connectivity of the 0.061 0.328 rate of people salaried in the urban local localityy to the neighbouring same locality , 2011, LOCAL HUMAN DEVELOPMENT INDEX LHDI 2012 distances employment cities (IURCON) LOCSALARIED 0.402 -0.028 general fertility rate (born diffuse LOCALITY CONNECTIVITY by rate of commuting to urban children to 1000 fertile age cultural localities 2011 URBCOMUTTING women), 2011, FERTILITY modernity commuting 0.084 rate of commuting to rural localities 2011 RURCOMUTTING rate of salaried people of 0.168 0.036 locality working abroad , 2011, urban locality (1 yes, 0 no) WORKABROD temporary controle emigration variables rate of people that live abroad 0.115 0.325 temporarily for more than one locality population 2011 year, 2011, TEMIGABROD Data source: NIS. 2 Results of OLS multiple regression. N=2955. The model explains 69% of the variation in LHDI (R =0.689). Figures are for beta coefficients. The higher the value of the coefficient, the higher the influence of the reference factor on LHDI. All the predictors are significant for p=0.01, excepting urban staus of locality with p=0.03. Only FERTILITY is connected in a negative way to development. This relation means that higher local diffuse modernity (indicated by low values of FERTILITY) brings higher local development. All the five indicators that are rates are computed by dividing the indicator values to local population in 2011 and converting this value by a ln transformation. Locality population as controle variable is also transformed by ln. 405 Figure 51. Human development level of the counties, LHDI 2012 Figure 52. Human development level of the counties, LHDI 2002 Data sources: NIS Note: Computing and design: Dumitru Sandu. Grouping intervals: poor <40, lower middle developed 40-42, middle developed 41-43, upper middle developed 44-45, developed >45. Limits of intervals are settled by natural brake method for 2012. The upper middle developed interval had no case in 2002. 406 Annex 24A. Design and use of the LHDI in applied research in Romania The Local Human Development Index (LHDI) is an adaptation of the Human Development Index (HDI) developed by UNDP. Different versions of the index were designed and implemented in Romania since the mid-90s. Four waves of measuring human development at local or community level followed:  Commune (i.e., basic rural administrative unit) level, in a research project for the World Bank (1998)121  Village (i.e., socio-territorial unit without an administrative status) level, in a national UNDP report (2005),122  Basic administrative units level, favoring comparisons between similar localities or between different residential communities (2008)123,  Basic administrative units level (communes or cities), also allowing for comparisons between data collected in different censuses (2013, based on data from 2011 and 2002 censuses). Commune level. The first design and use of LHDI in Romania dates back to a research project for the World Bank124. The Commune Development Index was generated as a factor score from 11 indicators (see figure below). Village level. The second design and large scale use of LHDI in Romania was for the 2003-2005 National Human Development Report (NHDR) for Romania (UNDP, 2005), and it was based on data from the 1992 and 2002 censuses: ”The NHDR is using a so-called Local Human Development Index to attempt to aggregate for the first time a number of human, social and material indicators (inhabitability or inhabitancy conditions), to measure development and potentials. Given that this is an initial attempt, the Local Human Development Index presented in this Report should be seen as a proxy measure for human development at the regional, county and village levels. As such, it does not attempt to use the same exact variables as does the Human Development Index and neither does it focus on defining poverty rates through estimations of incomes and consumption spending. Instead it focuses on human capacity, using the educational stock and the occupational characteristics (agricultural /non-agricultural, employment/ unemployment). The degree of isolation, which is considered to be directly proportional to the distance to the closest city, provided that the village is an outlying one (the city is not within its territory). The Local Human Development Index proves to be valid also in explaining other economic and cultural trends.” 125 121 Dumitru Sandu (coordinator). 1998. Community poverty in Romania rural areas .Foundation for poverty alliviation by Romania Social Development Fund. World Bank, Bucuresti (http://dumitru.sandu.googlepages.com/CommunitypovertyinRomaniaruralareas1.pdf) and Dumitru Sandu, 1999, Community and Regional Poverty in Rural Romania, in Romanian Journal of Sociology, vol. X. 122 UNDP. (2005). National Human Development Report (NHDR) for Romania, p. 65 (a previous form of LHDI, from 1998, working mainly with non-census data). 123 The data from this version of the index (called in Romanian Indicele Dezvoltarii Sociale Locale, IDSL, Sandu 2011) has been largely used in the document laying down the foundation for designing the administrative regionalisation of Romania in 2013 (Sandu 2013). 124 Dumitru Sandu , 1998, op.cit. 125 UNDP. (2005). National Human Development Report (NHDR) for Romania, p. 65 (a previous form of LHDI, working mainly with non-census data). 407 Commune development as a first index of local human development in Romania, 1998 The LHDI 2002 and the LHDI 1992 were based on the set of indicators specified in the figure below. 408 Components of the LHDI 1992 and the LHDI 2002 measuring local human development at village level (DEVSAT) Index of human Figures indicate development factor score 0.34 UMANVILL 0.36 education stock coefficients that were used as 0.39 sallaried people to 1000 inhabitants weight to % population employed in generate -0.4 agricultura indices. Index of village development, DEVSAT 2002 0.47 village population(ln transform) Index of biological % pop. of the village in commune 0.33 capital DEMPOT 0.45 population 0.3 % working age population 0.43 % dwelling with running watter Index of housing stock development 0.32 QBUILD 0.43 % dwelling having sewerage living floor area per house in the 0.27 village peripheral village in the commune 0.67 (1 yes, 0 no) Degree of isolation of -0.3 the village IZOLAT 0.67 distance to the nearest village The functions and design of DEVSAT index are similar both to HDI and to the multiple deprivation index that is used in United Kingdom at the level of small areas (wards)126. The methodological premises for building the LHDI at the village level are summarized below:127 “Village poverty is considered (…) to be a complex latent variable that cannot be measured directly. It is a kind of hidden nucleus within several spheres of more or less visible signs of poverty or development. The most visible aspects of community poverty are location factors; such location signs of poverty include: being isolated in relation with the regional road system or being a peripheral village within a commune. But these are rather week signs as there are villages which are quite well off without being commune centers or without being located very close to a highly modernized road. A family of signs that is closer to the real nucleus of village development or poverty is formed by a set of structural factors as human capital, social capital, public infrastructure, housing infrastructure, production units and housing durable goods. All these aspects indicate clear signs of community development. Having more educated people in a community, better housing and better public infrastructure, more developed economic units and more durable goods in the households is a clear sign of a wealthy community. Reaching by measurement these structural aspects of community development or poverty will give long lasting outcomes as the reference structures are characterized by high inertia. They do not change from day to day. The layer of signs that has the maximum proximity to community development/poverty is formed by the following set of phenomena: migration, fertility, 126 DETR, Indices of Deprivation 2000, Regeneration Research Summary no. 31, 2000. 127 Dumitru Sandu (Coord.) 2000. Villages of Romania: Development, Poverty and Social Capital. Updating targeting for RSDF, World Bank, Bucuresti, pp. 10-11. 409 production and consumption. Knowing their key parameters allows for our understanding of the up to date state of the community poverty or well-being.” Only part of the signs of the community well-being is available for direct measurement with statistical data for all the villages of the country. They are highlighted through bold letters in the figure below. Some other variables could be estimated by using the grid system that is already used by the Romanian Social Development Fund. Blocks of variables that can be used for measuring village development/ poverty Note: The categories of variables that function with available data and that have been used to construct the index of village development for each of the existing villages of the country are highlighted through bold letters. Public infrastructure and housing durable goods indicators are involved in assessing village poverty through a set of ten poverty criteria. Village isolation and village centrality variables are marked by italic letters and are used for testing the validity of the proposed methodology of measuring village development or poverty. Average level of human development by villages (DEVSAT) and by counties, 1992 and 2002 DEVSAT1992 DEVSAT2002 DEVSAT 2002*100/ Cultural areas Counties DEVSAT 1992 BC NT SV VR Vrancea 52.8 51.1 96.9 Bacau 55.1 52.3 95.1 Neamt 59.9 53.9 90.0 Suceava 60.3 56.7 94.0 GL IS Iasi 51.4 49.4 96.0 Galati 59.1 53.8 91.0 BT VS Vaslui 47.1 45.5 96.6 Botosani 48.9 46.9 95.8 AG DB PH Arges 60.8 62.9 103.5 Dambovita 62.8 65.4 104.2 Prahova 66.9 67.8 101.4 410 DEVSAT1992 DEVSAT2002 DEVSAT 2002*100/ Cultural areas Counties DEVSAT 1992 BZ BR Braila 56.1 59.6 106.3 Buzau 55.4 61.8 111.5 G TL IL CL Giurgiu 57.3 62.8 109.6 Teleorman 57.4 61.9 107.7 Ialomita 60.4 66.1 109.5 Calarasi 63.5 65.3 102.8 DJ MH OT Mehedinti 51.9 57.4 110.8 Dolj 55.9 62.1 111.1 Olt 57.1 59.7 104.5 GJ VL Gorj 58.7 60.5 103.1 Valcea 53.5 57.0 106.7 DOBROGEA Constanta 69.4 64.8 93.3 Tulcea 62.7 58.7 93.7 AB HD Alba 60.3 60.5 100.3 Hunedoara 61.0 58.8 96.5 BV SB Brasov 77.6 71.2 91.8 Sibiu 72.4 68.6 94.6 CJ MS Cluj 61.5 63.8 103.8 Mures 63.6 62.8 98.8 CV HG Covasna 69.6 70.3 101.1 Harghita 72.3 70.1 97.0 BN SJ Bistrita-Nasaud 61.3 67.4 110.0 Salaj 59.3 60.1 101.3 MM SM Maramures 64.8 69.7 107.6 Satu Mare 61.8 64.6 104.6 AD BH Arad 66.3 65.3 98.5 Bihor 60.3 62.5 103.7 BANAT Caras-Severin 62.4 62.5 100.1 Timis 66.8 62.3 93.2 BUCURESTI Ilfov 78.0 75.0 96.1 60.4 60.7 100.5 DEVSAT is computed using the indicators from the second figure in this annex. Factor scores are transformed by Hull score H=50+14*factor score. DEVCOM is a weighted average of the human development index at the village level (DEVSAT2002) and at commune level. 411 Average level of communes development, 2002 The same LHDI algorithm was then computed for communes and cities. LHDI 2008128 is computed so as to allow comparisons between cities’ and communes’ development starting from a set of seven indicators.129 Its novelty resides in the ability to measure by the same set of indicators the level of local human or social development for both cities and communes. LHDI 2011 (UMANLOC acronym, in Romanian) keeps six out of seven indicators from LHDI 2008, as well as its ability to compare human development among localities of the same or of different residential status (communes and cities). The improvement is related to the possibility to compare data from several censuses by putting them in the same data base for estimating the index. 128 LHDI 2008 , built on seven indicators, was promoted under the name of index of local social development (Indicele Dezvoltării Sociale Locale, IDSL). 129 Sandu, D. ( 2011). Social disparities in the regional development and policies of Romania. International Review of Social Research, I, 1-30. 412 Annex 24B. Comparability of LHDI across time Short presentation of issues in measurement invariance analyses Since LHDI is a multi-item measure, containing indicators of material capital, educational stock, life expectancy at birth, and the average age of the adult (18 years of age or older) population, we can treat it as a latent variable with multiple indicators. This allows us to formally test hypotheses regarding the comparability of this measure across time. The degree of comparability is established using tests of measurement invariance (Steenkamp & Baumgartner, 1998). The main challenge in testing measurement invariance is to provide an answer to the question whether the instrument (the LHDI index in this case) operates equivalently across groups or occasions or, in other words, if we measure the same construct over time. The tests are designed to provide information regarding: (a) comparability of LHDI effects on other variables across occasions (in our case 2002 and 2011) and (b) comparability of the mean of local development levels, computed using LHDI across occasions. The first situation (labeled as “metric invariance” in measurement invariance studies) is useful if the LHDI is used as a predictor for other variables (or, conversely, if one would like to test the effects of other variables on LHDI). The second situation (labeled as “scalar invariance” in measurement invariance studies) would apply when one would like to study changes across time in the average local development levels in Romania across occasions (computed based on the LHDI). Any research using comparisons of multi-item measures assumes (but often does not directly test) that these multi-item measures are invariant (across cultures in cross-cultural research, or across occasions in longitudinal research) (Pentz & Chou, 1994; Widaman, Ferrer, & Conger, 2010). The computation of the LHDI index gives the same weights to indicators, regardless of the year in which the index is computed, so the LHDI is constructed on the assumption that all four indicators are similarly related to the underlying construct of interest (local development) at all occasions. The next analysis evaluates whether this assumption is tenable or not. There are three levels of interest considering measurement invariance in longitudinal comparisons, each more restrictive than the next. The first, least restrictive level is the configural invariance and implies that the same indicators measure the same latent variables across occasions (the variables load in the same way on the latent variable). This is a necessary, but not sufficient condition of being able to make comparisons involving latent variables across occasions. The next level, metric invariance, keeps the constraints of the configural invariance model and further imposes the requirement that the loadings should have the same sizes across occasions. In other words, items have the same importance in defining the underlying constructs or latent variables at each occasion. This is a necessary and sufficient condition for comparing the effects of (or on) latent variables across occasions, but it is not sufficient for comparing the means of latent variables across occasions. In order to be able to achieve comparability of latent variable means across occasions, a third level of invariance is required: scalar invariance. This level of invariance keeps the constraints from the previous two levels of invariance and further imposes the constraint that observed variable intercepts are the same across occasions (Byrne, 2012; E. Davidov, 2008; E Davidov, Datler, Schmidt, & Schwartz, 2010; Steenkamp & Baumgartner, 1998; Widaman et al., 2010). We are mainly interested in the degree of cross-temporal comparability of the LHDI as a predictor for other variables, so we evaluate tests regarding metric invariance for a latent variable measured by the four indicators of LHDI; we will label this latent variable LHDlat in order to differentiate between the LHDI index (resulting from aggregation based on the geometric mean of the four indicators) and the LHD lat (the latent variable measured by the four indicators). The LHDI and LHDlat are alternative ways of 413 constructing multi-item measures, with the indicator LHDI having the advantage of simplicity of computation and the latent variable LHDlat having the advantage of accounting for measurement error and allowing for formal tests of measurement invariance. We also conducted tests of scalar invariance, but this is of secondary importance since we are less interested in estimating and comparing the 2002 national average of local development and the 2011 national average of local development. Furthermore, we expect that the development indicators (educational stock, life expectancy at birth, mean age of adult population, and material capital) will change over time in association with changes in the underlying dimension (local human development), but their change will be dependent on other factors as well. This would automatically lead to a lack of scalar invariance (Steenkamp & Baumgartner, 1998) Recent cross-national research has suggested that the requirements of metric (and scalar) invariance as described above are very hard to fulfill. This might be the case in longitudinal research as well, depending on the construct under investigation. Consequently, more recent recommendations suggest that if full metric or full scalar invariance does not hold, one needs to be able to demonstrate at least partial invariance in order to be able to proceed with cross-group comparisons. Partial measurement invariance requires that at least two loadings per factor are invariant (for partial metric invariance) and at least two intercepts per factor are invariant (for partial scalar invariance) (E Davidov et al., 2010; Steenkamp & Baumgartner, 1998). Models The latent variable model used to capture local human development is presented in the figure below. LHDlat is the latent variable or factor, measured by the four indicators used for the construction of the LHDI index. This configuration of the factor is used both in the 2002 and in the 2011 data. The figure presents the size of the factor loadings (in standardized form) resulting from estimating this model on the pooled 2002 and 2011 data. This is a preliminary model used to check how the indicators are related to the underlying factor. All factor loadings in this model have moderate to high sizes and two of the indicators (educational stock and material capital) are particularly highly associated with the underlying local human development dimension. Since this model is estimated on pooled over-time data (with a single set of coefficients being estimated for both 2002 and 2011), it implicitly assumes (but does not directly test) that the four indicators130 are invariant measures of the latent variable across time. The model in the figure below is equivalent to the model used in computing LHDI. The only differences in the latent variable model are that the weights of the observed indicators are freely estimated (as opposed to using the geometric mean to aggregate the indicators for LHDI) and that the latent variable model accounts for measurement error in the four indicators by including error terms in the model (e1 through e4). 130 We use the normalized version of the indicators in these models to replicate the model used in computing LHDI as closely as possible. 414 Local Human Development Latent Variable Model Notes: N=5,887 (2,831 localities with complete data on all four indicators in 2002 and 3,056 localities with complete data on all four indicators in 2011); pooled data for 2002 and 2011; the data is arrayed so that each case represents one locality in one of the years. The figure above follows structural equation models (SEM) conventions: rectangles = observed variables ellipses = unobserved/latent variables (factors measuring unobserved constructs such as “local human development” - LHDIlat and error terms in the model – e1, e2, e3, a.s.o.) straight arrows= causal relationships (in this case, factor loadings or the relationships between the observed indicators and the latent variable) curved arrows = associations (covariances) between variables Next, we explicitly test the assumptions of configural, metric, and scalar invariance for the LHD lat factor. The initial model for testing longitudinal invariance (the configural invariance model) is presented in Figure A6, below.131 Testing Longitudinal Invariance of LHDlat – The Configural Invariance Model Notes: N=2,830 (localities with valid data for the four indicators in both 2002 and 2011); the data is re-arrayed so that one case represents one locality. 131 All models for testing longitudinal measurement invariance are estimated using a longitudinal dataset in which missing data is treated using list-wise deletion (we analyze localities with valid data in both 2002 and 2011 occasions of measurement). 226 localities present in the 2011 database were excluded from these analyses due to their absence from the 2002 data. The technical specification of the configural invariance model is as follows: the mean of the factor at the first occasion is set to 0, the variance of the factor at the first occasion is set to 1, the loading of the EDSTOCK indicator on the factor is estimated as invariant over time (lambda1), the intercept of the EDSTOCK indicator is estimated as invariant over time (alpha1), and the means of the error terms are fixed at 0. Error terms for each indicator are correlated across occasions of measurement to account for non-random error introduced by having the same indicator measured at different time points. Finally, the two latent variables (Local Human Development in 2002 and Local Human Development in 2011) are correlated to account for the stability of the underlying factor over time. 415 The model for testing metric invariance adds the constraint that loadings should be equal across occasions and the scalar invariance model further adds the constraint that intercepts should be equal across occasions. We also explored partial metric invariance (relaxing some of the constraints on the loadings) and partial scalar invariance (relaxing some of the constraints on the intercepts) models, in accordance with the recent literature recommendations mentioned above and suggesting that partial measurement invariance is sufficient to demonstrate the comparability of latent factors across groups. Decisions regarding whether the constraints are appropriate (supported by the data) or not are made based on comparisons of model goodness of fit – if the fit of the model worsens significantly with the addition of constraints, which suggests that the constraints are not appropriate, and if the fit of the model does not worsen significantly, which suggests that the constraints are supported by the data132. Results and Discussion The Table below presents model goodness of fit for the series of models (briefly described above) used to test measurement invariance over time of the LHDlat. The initial model, the configural invariance model has an adequate (moderate) fit to the data according to measures of goodness of fit, suggesting that a model in which the structure of the local development factor is the same across occasions is plausible. Tests of Measurement Invariance across Occasions (2002 and 2011), LHD lat Model Goodness of Fit Model Comparisons 2 2 χ df p RMSE CFI SRM Δχ Δ Δp ΔRMSE Δ Δ A R df A CFI SRMR Configural Invariance 334.76 15 0.0 0.087 0.9 0.02 1 00 81 7 Full Metric Invariance 1076.3 18 0.0 0.144 0.9 0.07 741.62 3 0.0 0.057 0.0 0.051 82 00 36 8 1 00 45 Partial Metric Invariance (free 338.27 17 0.0 0.082 0.9 0.02 3.51 2 0.1 0.005 0.0 0.001 compAGE loading) 1 00 81 8 73 00 Partial Scalar Invariance (free 1762.3 19 0.0 0.18 0.8 0.17 1424.0 2 0.0 0.098 0.0 0.148 compAGE intercept) 16 00 95 6 45 00 86 Partial Scalar Invariance (free 754.28 18 0.0 0.12 0.9 0.07 416.01 1 0.0 0.038 0.0 0.042 compAGE and MATCAP 1 00 56 0 00 25 intercepts) Note: The shaded row refers to the model with a set of constraints supported by the data. Model comparison measures (goodness of fit “difference” tests) are presented in the right panel of the table. The metric invariance models are compared with the baseline configural invariance model. The scalar invariance models are assessed and compared in relation to the partial metric invariance model. 132 Decisions regarding measurement invariance were made based on a series of model comparisons, using criteria for choosing models proposed by Chen, 2007. Chen’s simulations suggested that: - Δ CFI >=0.01 supplemented by ΔRMSEA>=0.015 or Δ SRMR >=0.03 suggest metric non-invariance (constraining loadings to be equal across occasions is not supported by the data) - Δ CFI >=0.01 supplemented by ΔRMSEA>=0.015 or Δ SRMR >=0.01 suggest scalar non-invariance (constraining intercepts to be equal across occasions is not supported by the data) 2 and among the tests Δ CFI is the main criterion for judging measurement invariance. Δ χ (with its corresponding Δdf and Δp) is also presented, but not primarily used in the interpretation of results, due to its serious limitations related to its sensitivity to sample size, noted in previous literature (Bentler & Bonett, 1980, p. 591). Decisions regarding which loadings and intercepts to free in the partial measurement invariance models were made based on modification indices. 416 The second row of the table presents measures of goodness of fit for the metric invariance model in which loadings for each indicator are constrained to be equal across occasions. The results show that the goodness of fit of this model is significantly worse compared to the configural invariance model, suggesting that the condition of full metric invariance is not tenable. However, there is a single indicator whose non-invariance in loadings over time drives this conclusion: this is the indicator capturing the average age of the adult population of 18 years old and over (compAGE). If we allow this indicator to have unequal loadings over time, but keep the equal loadings constraints for the other three indicators in the model (this is essentially a partial metric invariance model), the resulting model, compared to the configural invariance model does not significantly worsen in terms of goodness of fit. Consequently, even though the local development index was not characterized by full metric invariance, it does fulfill the requirement of partial metric invariance. While the indicators capturing the educational stock, life expectancy at birth, and material capital are related in the same way to the underlying measure of local human development over time, average age had a slightly stronger association with the underlying factor in 2002 (the unstandardized loading was 6.59), compared to 2011 (the unstandardized loading is 3.67). Given that three of the four indicators used to construct the measure of local human development have invariant loadings over time, the measure fulfills the requirement of partial metric invariance. Consequently, this supports its value as a comparable predictor for other variables over time. If one is interested in the effects of other variables on local human development and in comparisons of these effects over time, these effects will be comparable, as well. The coefficients estimated in this model (which is ultimately the model that best approximates the data and fulfills the requirements of factor comparability over time) are presented in the figure below. LHDlat, Partial Metric Invariance - Results Note: Unstandardized coefficients presented on the diagram. Standardized coefficients in parentheses. Cov = covariance; r = correlation. All presented coefficients are statistically significant at the p=0.001 level. Estimated latent variable means are presented for informative purposes, but are not strictly comparable from a statistical point of view. 417 Since full metric invariance is not supported, full scalar invariance will not be a tenable assumption either, but we may proceed by testing the weaker hypothesis of partial scalar invariance. Unfortunately, none of the possible partial scalar invariance models is supported by the data. It is very likely that including some additional control variables in the model (potentially important predictors of the observed indicators) would remove the scalar non-invariance that is apparent in this model. Further tests are needed for the use of the LHDI measure in order to compare average local human development levels for Romania across time and make inferences about such trends over time. In sum, the local human development index is a measure that is appropriate in those cases in which researchers use it to compare its effects on other variables (or effects of other variables on it) over time. The models presented above referred strictly to comparison between 2002 and 2011. However, the comparability of the measure extends most likely over a longer period of time. The measure is also highly stable over time (the correlation between local human development in 2002 and local human development in 2011, controlling for error autocorrelations between the same indicators measured at different time points is extremely high: 0.96). Furthermore, the constructed measures based on the partial metric invariance model (factor scores based on estimated factor score coefficients in the model presented in the Error! Reference source not found.) are highly correlated with the proposed LHDI index computed using a simpler algorithm based on the geometric mean of the four observed indicators. In 2002, the correlation between LHDI and LHDlat is 0.89 and in 2011, the correlation between LHDI and LHDlat is 0.86.133 133 Both correlations are statistically significant at the p=0.001 level. 418 Annex 24C. LHDI at local and county levels, 2002, 2011 and 2012 COUNTY LEVEL Table 29 .Local human development index by counties and residence, 2002 and 2011 Devel opment l evel of county LHDI2002 LHDI2011 LHDI2012 ILFOV 38 50 50 BUCURESTI 47 50 50 Developed BRASOV 46 47 48 SIBIU 46 47 48 CLUJ 46 47 48 TIMIS 42 46 46 ALBA 43 44 44 developed middle Upper MURES 43 43 44 ARGES 40 43 44 PRAHOVA 41 44 44 HUNEDOARA 40 42 43 CONSTANTA 39 42 43 Middle developed IASI 40 42 43 ARAD 39 43 43 GORJ 40 42 43 BISTRITA NASAUD 41 42 43 MARAMURES 39 43 43 HARGHITA 41 41 43 SATU MARE 39 42 43 COVASNA 41 41 42 DAMBOVITA 38 41 42 SUCEAVA 38 41 42 Lower middle developed BIHOR 37 41 42 CARAS SEVERIN 38 41 42 GALATI 39 41 42 SALAJ 37 41 41 DOLJ 38 41 41 VALCEA 38 40 41 BACAU 39 40 41 BRAILA 38 41 41 NEAMT 39 40 41 VRANCEA 36 39 40 TULCEA 36 39 39 BUZAU 36 39 39 OLT 36 38 39 MEHEDINTI 36 38 39 IALOMITA 34 37 38 Poor VASLUI 35 37 38 BOTOSANI 34 37 37 CALARASI 33 37 37 GIURGIU 32 36 37 TELEORMAN 32 35 35 Total 40 42 43 Note: Figures are weighted by population averages, based on counties and residence for 2002 and 2011 LHDI. Data source and partial computations: National Institute of Statistics. Design and final computations: D.S. 419 LOCALITY LEVEL Table 30. Predictors of LHDI 2012 by rural and urban residence Communes Cities B Beta P B Beta P (Constant) -4.531 .000 -25.654 .000 URBANCOMUTTING .044 .471 .000 .045 .510 .000 LOCSALARIED 2.992 .246 .000 7.511 .592 .000 LOC_POPULATION 2.366 .230 .000 1.812 .404 .000 WORKABROAD .977 .197 .000 1.084 .179 .000 TEMIGABROAD .546 .133 .000 .541 .104 .002 RURALCOMUTTING .555 .090 .000 .905 .117 .001 IURCON .688 .060 .000 1.936 .166 .000 RGF2011 -.008 -.027 .041 -.003 -.006 .857 R2 0.600 0.700 N 2640 315 Computing urban connectivity index (IURCON) An index of commune urban connectivity (IURCON) measures how close is each locality to its nearest neighbor from each category of city: small, medium, large and very large. It is constructed as a geometric mean (numerator in each fraction represents the minimal limit of population, in thousands, for each city category): For the case of the most developed locality according to LHDI measurement (commune of Dumbravita from Timis), the nearest cities are Timisoara, Arad, Resita and Lugoj. With the above formula and appropriate data, the index is 420 Table 31. Leading and lagging localities in Romania by LHDI values 30 Most developed localities in 2012 30 least developed localities in 2012 COUNTY LOCALITY LHDI2002 LHDI2011LHDI2012 COUNTY LOCALITY LHDI2002 LHDI2011 LHDI2012 TIMIS DUMBRAVITA 42 67 64 TELEORMAN SACENI 16 21 23 ILFOV CORBEANCA 34 63 61 TELEORMAN VARTOAPE 18 24 23 ILFOV CHIAJNA 45 59 60 OLT RADOMIRESTI 22 23 23 ILFOV ORAS OTOPENI 44 59 58 VRANCEA SLOBOZIA BRADULUI 22 23 23 ILFOV ORAS BRAGADIRU 45 57 57 GIURGIU STOENESTI 17 23 23 ILFOV ORAS VOLUNTARI 42 57 57 VRANCEA CIORASTI 17 23 23 TIMIS GIROC 43 56 57 TELEORMAN BALACI 17 21 23 CLUJ FLORESTI 44 58 56 GIURGIU SCHITU 17 21 23 IASI VALEA LUPULUI 57 56 IALOMITA COCORA 27 26 23 BRASOV ORAS PREDEAL 51 55 56 DOLJ SOPOT 24 25 23 BRASOV ORAS GHIMBAV 49 54 55 MEHEDINTI POROINA MARE 15 19 23 MURES CORUNCA 54 55 CLUJ PALATCA 25 22 23 ILFOV MOGOSOAIA 40 57 54 IALOMITA VALEA MACRISULUI 25 24 23 SALAJ MUNICIPIUL ZALAU 50 53 54 IALOMITA BALACIU 23 23 22 ORAS POPESTI ILFOV LEORDEN 40 55 53 CARAS SEV. BERLISTE 29 25 22 SIBIU SELIMBAR 44 53 53 CARAS SEV. LAPUSNICEL 31 20 22 ALBA MUNICIPIUL ALBA IULI 53 54 53 CLUJ SANMARTIN 24 24 22 MUNICIPIUL CLUJ- CLUJ NAPO 55 54 53 IALOMITA BRAZII 26 18 22 SIBIU MUNICIPIUL SIBIU 54 53 53 OLT TOPANA 22 22 22 TIMIS GHIRODA 41 52 53 DOLJ BRABOVA 20 21 21 ILFOV BALOTESTI 39 52 53 MEHEDINTI VLADAIA 22 22 21 BISTRITA NAS. MUNICIPIUL BISTRITA 54 53 53 CALARASI GURBANESTI 22 25 20 PRAHOVA ORAS BUSTENI 48 52 53 VASLUI BLAGESTI 18 24 20 PRAHOVA ORAS SINAIA 51 52 52 CALARASI ULMU 17 21 20 ILFOV DOBROESTI 40 51 52 GIURGIU RASUCENI 16 18 19 GORJ MUNICIPIUL TARGU JIU 52 48 52 TELEORMAN SLOBOZIA MANDRA 23 15 19 ILFOV SNAGOV 36 52 52 GIURGIU TOPORU 15 20 19 BRASOV MUNICIPIUL BRASOV 49 52 52 MEHEDINTI PADINA 7 18 17 IASI MUNICIPIUL IASI 50 51 52 DOLJ SEACA DE PADURE 16 10 14 DAMBOVIT MUNICIPIUL A TARGOVIST 49 47 52 TELEORMAN NECSESTI 15 18 13 421 Annex 25. The new European Commission requirements on Project Selection models The new approach of the EC on programming, monitoring, and evaluation Up until 2013, the European Commission primarily required from Member States a focus on absorption and a number of output indicators. Staring with the 2014-2020 Programming Period the EU Cohesion Policy is more performance-oriented, with a shift of focus toward outcomes. Operational Programmes are encouraged to draft project selection models that focus on the specific dimensions of people’s well- being and progress (outcomes) through the use of appropriate indicators rather than of financial inputs and on actions: i.e., on ends rather than means134. The result orientation of the future European Structural and Investment Funds is based on three pillars: a. A clear articulation of the objectives of the programs with a strong intervention logic (the result orientation of programmes) b. The definition of ex-ante conditionalities to ensure that the necessary prerequisites are in place for the effective and efficient use of the EU support c. The establishment of clear and measurable milestones to ensure progress is made as planned (performance framework) The following logical framework is capital for understanding the change of perspective over outputs, results and impact in relation to programming, monitoring and evaluation of ERDF and Cohesion Funds for 2014 – 2020. 135 Source: Guidance Document on Monitoring and Evaluation 134 http://ec.europa.eu/regional_policy/sources/docgener/evaluation/doc/performance/outcome_indicators_en.pdf 135 http://ec.europa.eu/regional_policy/sources/docoffic/2014/working/wd_2014_en.pdf 422 The most important change as compared to the former Guidance of the EC for 2007 – 2013 period is the emphasis on clear articulation of the policy objectives. This is key to implement a results oriented policy and moving away from an excessive focus on the absorption of funding. 1.1 Programming Programmes with a clear identification of changes sought, concentrated on a limited number of interventions are a decisive condition for achieving tangible results. The concentration of financial means will also facilitate a programming process of good quality, in which the limited administrative resources can be dedicated to the careful preparation of interventions. One of the criteria/conditions that are considered to be critical for a sucessful programming excercise is to have clear objectives and to carefully select result and output indicators. In this sense, each priority axis will include one or more investment priorities selected by the Member State/region, according to its specific needs and context. The specific objective is the expression of what each investment priority aims to achieve. The change sought by the specific objective is expressed in some (although the push is for very few) result indicators. Each result indicator requires a baseline value (the value at the beginning of the programming period). If the programme is not in the position to deliver the baseline for a result indicator for ERDF and CF, this can be seen as a case of not meeting the ex-ante conditionality on result indicators. Investment priorities will be implemented through projects. Result indicators are an expression of an investment priority. Consequently, result indicators can inform the decision on project selection criteria because projects should be able to demonstrate how they will contribute to the achievement of the objectives of a priority. It should be underlined that project selection is a task of the Member States; however, the decision upon a project selection model should take into consideration the need to maximize achievement of results and demonstrate progress (outcomes) against the targets assumed by each Member State under the main coordinated of ERDF/CF: - Productive investment; - ICT - Transport - Environment - Research, Innovation - Energy and Climate Change - Social Infrastructure - Urban Development The outputs and results indicators used should be the ones listed in the List of common indicators136. 1.2 Ex-ante evaluation 136 http://ec.europa.eu/regional_policy/sources/docoffic/2014/working/wd_2014_en.pdf, p. 20 423 The ex-ante evaluations – analysis helping to understand the data needs for the envisaged interventions – should be carried out in interaction with the establishment of the programme and shall be submitted to the Commission at the same time as the Operational Programme with an executive summary. The Member States will integrate, where appropriately, the strategic environmental assessment into the ex- ante evaluation. 1.3 Monitoring Annual monitoring reports will be provided for each Operational Programmes, based on a given structure mandatory starting with the report for 2016. 1.4 Evaluation The Commission recommends Member States to start drawing up the evaluation plan together with the operational programme. The evaluation plan will be submitted to the monitoring committe no later than one year after the approval of the programme. 1.5 Performance Framework The performance framework is one of the tools to achieve a result-orientation of the ESI Funds. It is a table in which a set of milestones and targets is defined for each priority in a programme. The achievement of milestones will be assessed in 2019 and will form the basis for the allocation of the performance reserve and possibly the suspention of payments. The achievement of targets will be assessed in 2025 and might form the basis of financial corrections. A priority will be presumed to have seriously failed to achieve the milestones if two or more of the indicators related to a priority fail to achieve 65% of their intermediate target. An example of a Performance Framework for ERDF can be seen below: 424 Based on a Report on Pilot Tests in 23 regions/15 Member States across the EU with regard to the Results Indicators 2014+137, among the most important lessons learnt provided further as Guidance by the Commission are the following: - The results focus must become part of the development of the programme, which need a stronger and more explicit intervention logic; this cannot be added afterwards. - The main change required is concentration. Many of the priorities examined were in fact an amalgamation of more or less related sub-priorities which could not be expressed in one or two result indicators. - Concentration is more than thematic concentration as defined in the Common Provisions Regulation. Resources should be concentrated where they can have measurable effects; this requires identification in a very precise way of needs to be tackled or potentials to be exploited and the targeting of resources on these. - Concentration has to be the outcome of a process of deliberation and policy choice. This emphasises the importance of political debate on the choices which should drive programme design and starting this process urgently. - If there is concentration, this means that there will be fewer indicators. Whatever result indicator is selected, baselines and targets are essential. These can be quantitative or qualitative. 137 http://ec.europa.eu/regional_policy/sources/docoffic/2014/working/result_indicator_pilot_report.pdf 425 Annex 26. Proposed evaluation and selection grids for the ROP 2014-2020 Grids proposed for the Status Quo+ 426 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Status Quo+ Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 6 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub-criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 6 or 1 to 6 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub-criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. After the technical and financial evaluation, a qualifying score is equal to 3.5 or more. A score of 0 for any of the sub-criteria results in the disqualification of the project. Note: This grid only covers SMEs. Once the axes/key areas of intervention are finalized, the same grid will be adaptable to the other types of eligible enterprises (e.g., same general criteria, with some adjusted thresholds). 427 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Status Quo+ Scenario Max Criterion Weight points 1. Project contribution to fulfilling ROP objectives 6 points 60% 1.1 Creating new permanent jobs a. The project leads to the creation of new permanent jobs within the enterprise, and the number of newly created 6 points permanent (full-time) jobs for the EUR 10,000 grant obtained is greater than or equal to 0.30. b. The project leads to the creation of new permanent jobs within the enterprise, and the number of newly created 3 points permanent (full-time) jobs for the EUR 10,000 grant obtained 65% is equal to 0.20. c. The project leads to the creation of new permanent jobs within the enterprise. The number of newly created 2 points permanent (full-time) jobs for the EUR 10,000 grant obtained is equal to 0.15. d. The project does not lead to the creation of new permanent 1 point jobs within the enterprise. 1.2 Using local resources a. The enterprise mostly uses local resources/raw materials (produced 6 points locally – same city/county) and manpower. b. The enterprise partially uses local resources/raw materials 3 points 35% (produced locally – same city/county) and/or manpower. c. The enterprise does not use local resources/raw materials 1 point (produced locally – same city/county) or manpower. 2. Project quality, maturity, and sustainability 6 points 40% 2.1 The applicant’s capacity to implement the proposed project a. Material and human resources (the project team) are clearly 6 points defined and are adequate for the project’s implementation. The applicant has strong experience in the field that forms the object of the proposed project. The turnover for the 10% latest fiscal year is at least equal to value of the grant. 3 points b. Material and human resources (the project team) are adequate for the project’s implementation. The applicant has relative experience in the field that forms the object of the 428 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Status Quo+ Scenario Max Criterion Weight points proposed project. The turnover for the latest fiscal year is at least equal to 50% of the value of the grant. 1 point c. Material and human resources (the project team) are inadequate. The applicant has no experience in the field that forms the object of the proposed project. The turnover for the latest fiscal year is at least equal to 30% of the value of the grant. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the 6 points attributions of project team members, and the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning 3 points 10% of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not 1 point correlated with the implementation calendar, with the attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 429 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Status Quo+ Scenario Max Criterion Weight points 2.3 The project’s budget a. The budget is complete and perfectly correlated with proposed activities, with material resources involved in the project’s implementation, with anticipated results. Expenditures have been correctly differentiated into the categories of eligible and non-eligible expenditures, and the thresholds for certain expenditures have been observed, according to the Eligible Expenditures Order and the 6 points provisions of the applicable de minimis aid scheme. There is full correlation between the budget and the financing sources. Costs are realistic (correctly estimated), sufficient and necessary for project implementation. The budget observes the standard model for the key intervention area, is accurately calculated, and the VAT is calculated with two decimal places. The total project value is within the limits established for the key intervention area. b. The budget is satisfactorily correlated with proposed activities, with material resources involved in the project’s implementation, with anticipated results. There is an imperfect correspondence between each activity/result and 10% budget lines, and some activities and costs are treated as “global.” Costs are necessary and sufficient for the project’s implementation. Expenditures have been correctly differentiated into the categories of eligible and non-eligible 3 points expenditures, and the thresholds for certain expenditures have been observed, according to the Eligible Expenditures Order and the provisions of the applicable de minimis aid scheme. There is full correlation between the budget and the financing sources. Costs are realistic (correctly estimated), sufficient and necessary for project implementation. The budget observes the standard model for the key intervention area, is accurately calculated, and the VAT is calculated with two decimal places. The total project value is within the limits established for the key intervention area. c. The budget is not perfectly correlated with proposed activities, with material resources involved in the project’s implementation, with anticipated results. The budget has 1 point certain errors. The total value of the project is within the limits set for this key intervention area. Expenditures have been correctly differentiated into the categories of eligible 430 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Status Quo+ Scenario Max Criterion Weight points and non-eligible expenditures, and the thresholds for certain expenditures have been observed, according to the Eligible Expenditures Order and the provisions of the applicable de minimis aid scheme. 2.4 The quality of the business plan a. Data are sufficient, correct, and justified. The market analysis demonstrates a clear need/demand for the products/services offered. The competitive analysis identifies competitors, their strong and weak points, the competitive advantage of the applicant (in marketing, operations, organizational structure, etc.). The marketing 6 points strategy is feasible (it identifies adequate and efficient instruments) in the context of available resources. Financial projections are correlated with the marketing strategy; they are correct and realistic. Cost and revenue estimates are correct, realistic, and justify the financial projections. b. Data are sufficient and largely justified. The market analysis demonstrates a need/demand for the products/services offered. The competitive analysis identifies competitors, but provides only a partial assessment of their strong and weak points, and the competitive advantage of the applicant (in 40% marketing, operations, organizational structure, etc.). The marketing strategy is partly feasible (it identifies adequate 3 points and efficient instruments) in the context of available resources. Financial projections are partly correlated with the marketing strategy; they are partly correct and realistic. Cost and revenue estimates are partly correct, realistic, and justify to a limited extent the financial projections. c. Data are insufficient and/or not justified. The market analysis does not demonstrate a need/demand for the products/services offered. The competitive analysis identifies competitors, but does not assess their strong and weak points, and/or the competitive advantage of the applicant (in marketing, operations, organizational structure, etc.). The marketing strategy is not feasible (it identifies 1 point adequate and efficient instruments) in the context of available resources. Financial projections are not correlated with the marketing strategy; they are incorrect and/or 431 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Status Quo+ Scenario Max Criterion Weight points unrealistic. Cost and revenue estimates are incorrect and/or unrealistic. 2.5 Investment sustainability a. The applicant has a solid capacity to ensure that the investment/ enterprise is preserved, maintained, and operational during and after the project’s implementation 6 points and after the grant disbursement stops. The investment generates profit (at least starting in Year 2), and the risk of failing to ensure a functional investment is minimal or nonexistent. The data provided (and working hypotheses) are correct, realistic, and justified. b. The applicant has an average capacity to ensure that the investment/enterprise is preserved, maintained, and 3 points operational during and after the project’s implementation and after the grant disbursement stops. The investment 20% generates profit (at least starting in Year 3), and the risk of failing to ensure a functional investment is low. The data provided (and working hypotheses) are correct, realistic, and justified. c. The applicant has a weak capacity to ensure the 1 point investment/enterprise is preserved, maintained, and operational during and after the project’s implementation and after the grant disbursement stops. The investment generates profit (at least starting in Year 5), and the risk of failing to ensure a functional investment is significant. The data provided (and working hypotheses) are correct, realistic, and justified. 2.7 Sustainable development and energy efficiency a. The project describes in detail the measures for improving the quality of the environment, along with measures for 6 points improving energy efficiency. 5% b. The project abides by minimum legal requirements regarding environmental protection and energy efficiency. 3 points 2.8 Equality of opportunity and nondiscrimination 5% a. The applicant has a nondiscrimination policy applicable to the 6 points 432 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Status Quo+ Scenario Max Criterion Weight points project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. b. The applicant abides by minimum legal standards in this field. 1 point 433 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Status Quo+ Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 6 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 6 or 1 to 6 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. After the technical and financial evaluation, a qualifying score is equal to 3.5 or more. A score of 0 for any of the sub-criteria results in the disqualification of the project. 434 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Status Quo+ Scenario Max. Criterion Weight points 1. Project contribution to fulfilling ROP objectives 6 points 60% 1.1 Project relevance for the objectives of the priority axis and the key intervention area: a. The investment has a high potential to boost the energy efficiency of urban transport systems (min. 10% reduction in 6 points kWh equivalent). It relies on eco-friendly transport modes (bike paths, tram lines, etc.), efficient rolling stock, modern public spaces, and improved ticketing systems. b. The investment has a moderate potential to boost the energy 3 points 40% efficiency of urban transport systems (min. 5-10% reduction in kWh equivalent). It relies on eco-friendly transport modes (bike paths, tram lines, etc.), efficient rolling stock, and modern public spaces. c. The investment has a low potential to boost the energy efficiency of urban transport systems (under 5% reduction in 1 point kWh equivalent). It relies on eco-friendly transport modes (bike paths, tram lines, etc.) and efficient rolling stock. 1.2 Investment need a. The project will lead to a reduction of travel times by over 15% and an increase by over 10% in the number of people 6 points who use public transport or non-motorized means (e.g., bikes) for urban travel. b. The project will lead to a reduction of travel times by 5-10% 3 points and an increase by 5-10% in the number of people who use 60% public transport or non-motorized means (e.g., bikes) for urban travel. c. The project will lead to a reduction of travel times by under 1 point 5% and an increase by under 5% in the number of people who use public transport or non-motorized means (e.g., bikes) for urban travel. 2. Project quality, maturity, and sustainability 6 points 40% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or 10% has completed the procedure for public procurement of the 6 points DTD, and the DTD completion is planned for a maximum of 3 435 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Status Quo+ Scenario Max. Criterion Weight points months after the approval of the financing request, it has all the required approvals, and it has purchased the land (if needed) 3 points b. The applicant has launched the procedure for public procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and 1 point it has purchased the land (if needed) c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 6 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning 10% of public procurement activities. The results of the project and the indicators are partly correlated with the planned 3 points activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the attributions of project team members, or with the planning 1 point of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 436 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Status Quo+ Scenario Max. Criterion Weight points 2.3 Coherence of technical documentation 25% a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical 6 points solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 3 points The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical 1 point solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 Coherence of economic documentation a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)138 is 6 points lower than 30% OR the expenditure sustainability indicator (ESI)139 is higher than 30%. The basis for calculating these indicators is the average for the last five years. 25% b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 4 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for 138 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 139 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 437 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Status Quo+ Scenario Max. Criterion Weight points calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly 2 points estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2.5 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is 6 points complete and fully correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. 10% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is 3 points complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.6 Sustainable development and energy efficiency a. The project describes in detail the measures for improving 6 points the quality of the environment, along with measures for improving energy efficiency. 5% b. The project abides by minimum legal requirements regarding 3 points environmental protection and energy efficiency. 2.7 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the 5% project, and there are mechanisms for ensuring equality of 6 points opportunity also within public procurement contracts. The 438 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Status Quo+ Scenario Max. Criterion Weight points applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 3 points b. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 1 point c. The applicant abides by minimum legal standards in this field. 2.8 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s 6 points implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 10% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear 1 point strategy. The organization does not have specific procedures for verifying/supervising the project team’s activities. 439 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Status Quo+ Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 440 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Status Quo+ Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 6 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 6 or 1 to 6 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. After the technical and financial evaluation, a qualifying score is equal to 3.5 or more. A score of 0 for any of the sub-criteria results in the disqualification of the project. 441 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Status Quo+ Scenario Max Criterion Weight points 1. Project contribution to fulfilling ROP objectives 6 points 60% 1.1 Project relevance for the goals of the priority axis/key intervention area: a. The project generates new permanent jobs following the site’s rehabilitation and the development of new activities, 6 points and the number of newly created permanent jobs for the EUR 100,000 grant obtained is greater than or equal to 0.8. Rehabilitated surface for the EUR 100,000 grant obtained140 is greater than or equal to 0.5ha. b. The project generates new permanent jobs following the site’s rehabilitation and the development of new activities, 3 points 35% and the number of newly created permanent jobs for the EUR 100,000 grant obtained is greater than or equal to 0.6. Rehabilitated surface for the EUR 100,000 grant obtained is greater than or equal to 0.35ha. c. The project generates new permanent jobs following the site’s rehabilitation and the development of new activities, and the number of newly created permanent jobs for the 1 point EUR 100,000 grant obtained is greater than or equal to 0.5. Rehabilitated surface for the EUR 100,000 grant obtained is greater than or equal to 0.25ha. 1.2 Opportunity to rehabilitate the polluted site and to create/expand a business support structure a. In the target area there is a well-defined need for the business support structure and for the services offered. The facilities and/or services correspond to/are adequate for the needs of the companies present within the 6 points structure/potential (space, utilities, specific services, etc.). The accessibility degree: the structure location allows for 65% easy access to at least 2 types of transport (road, railroad, naval). The activity of building/modernizing/ expanding the access road to the business support structure, as well as the related expenditures, are fully justified by the current opportunities to connect to county/national roads, railways, trading ports etc. b. The opportunity for the business support structure and the 140 The total grant obtained for both implementation phases of the project shall be taken into consideration. 442 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Status Quo+ Scenario Max Criterion Weight points offered services is not fully justified. The need for such structures in the target area has not been/cannot be identified or is not clear. The provided facilities and/or services correspond to/are adequate to a small extent for the needs of the companies present within the 3 points structure/potential (space, utilities, specific services, etc.). The accessibility degree: the structure location allows for easy access to one type of transport (road, railroad, naval). If included, the activity of building/modernizing/expanding the access road to the business support structure, as well as related expenditures, are partially justified by current possibilities to connect to county/national roads, railways, trading ports, etc. c. The opportunity of the business support structure and of the offered services is not justified. The provided facilities and/or services do not correspond to/are not adequate for the needs of the companies present within the structure/potential (space, utilities, specific services, etc.). 1 point The accessibility degree: the structure location does not allow for easy access to any type of transport (road, railroad, and/or naval). If provided for, the activity of building/modernizing/expanding the access road to the business support structure is not justified by the current possibilities to connect to county/national roads, railways, trading ports etc. 2. Project quality, maturity, and sustainability 6 points 40% 2.1 Applicant’s capacity to implement the project a. Material and human resources (the project team) are clearly defined and are adequate for the project’s implementation. The applicant has a clear strategy for monitoring the project’s 6 points implementation; there is a clear distribution of tasks; there are procedures and there is a calendar of monitoring activities. The applicant has experience in managing at least 2 10% projects financed from public funds, with a total value similar to the proposed investment. b. Material and human resources (the project team) are 3 points sufficient. The responsibilities of the project team members do not entirely cover the activities required for the project’s management and implementation. The applicant has a series 443 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Status Quo+ Scenario Max Criterion Weight points of procedures for monitoring project implementation, and a calendar of monitoring activities, but there is no clear strategy. The applicant has managed at least one project financed from public funds. 2.2 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the DTD, and the DTD completion is planned for a maximum of 3 months after the approval of the financing request, it has all 6 points the required approvals, and it has purchased the land (if needed) b. The applicant has launched the procedure for public 5% procurement of the DTD, and the DTD completion is planned 3 points for a maximum of 6 months after the approval of the financing request, it some of the required approvals, and it has purchased the land (if needed) c. The applicant estimates the completion of the DTD in a 1 point maximum of 12 months after the approval of the financing request 2.3 Methodology for project implementation a. The objectives of the project are clear and can be achieved. 6 points The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 5% b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the 3 points attributions of project team members and with the planning of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). 444 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Status Quo+ Scenario Max Criterion Weight points c. The objectives of the project are not realistic. The activities 1 point of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.4 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 6 points b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 20% The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not 3 points entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 1 point The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 Coherence of economic documentation a. Operation and maintenance (O&M) costs are correctly 20% estimated. The investment sustainability indicator (ISI)141 is 6 points 141 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 445 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Status Quo+ Scenario Max Criterion Weight points lower than 30% OR the expenditure sustainability indicator (ESI)142 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 4 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly 2 points estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR t`he expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2.5 Marketing and management strategy a. The market analysis demonstrates a clear need/demand for the products/services offered. The competitive analysis 6 points identifies competitors, their strong and weak points, the competitive advantage of the applicant (in marketing, operations, organizational structure, etc.). The marketing strategy identifies adequate and efficient instruments, appropriate for the structure type. The management strategy is appropriate, given the specifics of the structure 20% and the services it offers. b. The market analysis demonstrates a potential need/demand for the products/services offered. The competitive analysis partially identifies competitors, their strong and weak points, the competitive advantage of the applicant (in marketing, operations, organizational structure, etc.). The marketing strategy identifies some adequate and efficient instruments. 3 points The management strategy is not fully aligned with the 142 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 446 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Status Quo+ Scenario Max Criterion Weight points specifics of the structure and the services it offers. 2.6 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the 6 points implementation of the project. 5% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, 3 points with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.7 Sustainable development and energy efficiency a. The project describes in detail the measures for improving 6 points the quality of the environment, along with measures for improving energy efficiency. 5% b. The project abides by minimum legal requirements regarding 3 points environmental protection and energy efficiency. 2.8 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The applicant identifies aspects/risks related to discrimination in 6 points the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 5% b. The applicant identifies aspects/risks related to 3 points discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 1 point c. The applicant abides by minimum legal standards in this field. 447 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Status Quo+ Scenario Max Criterion Weight points 2.9 Information society a. The project implements modern IT systems, modern solutions/ applications, etc. 6 points 5% b. The project implements minimum standards with respect to 3 points IT equipment. OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 448 Regional Operational Programme 2014-2020 Priority Axis 4 – Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Status Quo+ Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 6 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 6 or 1 to 6 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. After the technical and financial evaluation, a qualifying score is equal to 3.5 or more. A score of 0 for any of the sub-criteria results in the disqualification of the project. 449 Regional Operational Programme 2014-2020 Priority Axis 4 – Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Status Quo+ Scenario Max Criterion Weight points 1. Project contribution to fulfilling ROP objectives 6 points 60% 1.1 Project contribution to increasing the number of tourists by 6 points promoting the local and regional tourism potential a. After the project’s completion, the annual average number of tourists will increase by over 5% 3 points b. After the project’s completion, the annual average number of 60% tourists will increase by up to 5% c. After the project’s completion, the annual average number of tourists will not decrease compared to current levels 1 point 1.2 Project contribution to jobs creation a. The project will generate new permanent jobs 6 points 40% b. The project will generate new temporary jobs 3 points c. The project involves maintaining the current number of jobs 1 point 2. Project quality, maturity, and sustainability 6 points 40% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the DTD, and the DTD completion is planned for a maximum of 3 months after the approval of the financing request, it has all 6 points the required approvals, and it has purchased the land (if needed) b. The applicant has launched the procedure for public 15% procurement of the DTD, and the DTD completion is planned 3 points for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed) c. The applicant estimates the completion of the DTD in a 1 point maximum of 12 months after the approval of the financing request 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. 6 points 10% The activities of the project are clearly identified, detailed, 450 Regional Operational Programme 2014-2020 Priority Axis 4 – Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Status Quo+ Scenario Max Criterion Weight points and closely correlated with the implementation calendar, the attributions of project team members, and the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the 3 points attributions of project team members and with the planning of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the attributions of project team members, or with the planning 1 point of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 6 points The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 20% b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and 3 points according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are 451 Regional Operational Programme 2014-2020 Priority Axis 4 – Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Status Quo+ Scenario Max Criterion Weight points complete and correspond fully to written parts. 1 point c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 Coherence of economic documentation a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)143 is lower than 30% OR the expenditure sustainability indicator (ESI)144 is higher than 30%. The basis for calculating these 6 points indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for 4 points calculating these indicators is the average for the last five years. 20% c. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability 2 points indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 1 point over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2.5 The marketing strategy used for promoting the tourist potential of 6 points the heritage site 10% a. The marketing plan identifies adequate instruments to 143 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 144 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 452 Regional Operational Programme 2014-2020 Priority Axis 4 – Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Status Quo+ Scenario Max Criterion Weight points highlight the heritage site, at the international and national level. The tourism potential of the site is certain. Financial projections are correlated with the marketing strategy and they are accurate and realistic. 3 points b. The marketing plan identifies adequate instruments to highlight the heritage site, at national and/or regional and/or local level. The tourism potential of the site is certain. Financial projections are correlated with the marketing strategy and they are accurate and realistic. c. The marketing plan identifies adequate instruments to 1 point highlight the site at regional and/or local level. The tourism potential o is poorly demonstrated. Financial projections are not correlated with the marketing strategy, and they are not accurate/realistic. Cost and income calculations are not accurate/realistic. 2.6 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the 6 points implementation of the project. 5% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, 3 points with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.7 Sustainable development and energy efficiency a. The project describes in detail the measures for improving 6 points the quality of the environment, along with measures for improving energy efficiency. 5% b. The project abides by minimum legal requirements regarding 3 points environmental protection and energy efficiency. 2.8 Equality of opportunity and nondiscrimination 5% a. The applicant has a nondiscrimination policy applicable to the 453 Regional Operational Programme 2014-2020 Priority Axis 4 – Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Status Quo+ Scenario Max Criterion Weight points project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The 6 points applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 3 points b. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the 1 point current structure to facilitate disabled people’s access. c. The applicant abides by minimum legal standards in this field. 2.9 Applicant’s capacity to implement the project a. Material and human resources (the project team) are clearly defined and are adequate for the project’s implementation. The applicant has a clear strategy for monitoring the project’s 6 points implementation; there is a clear distribution of tasks; there are procedures and there is a calendar of monitoring activities. 10% b. Material and human resources (the project team) are sufficient. The responsibilities of the project team members do not entirely cover the activities required for the project’s 3 points management and implementation. The applicant has a series of procedures for monitoring project implementation, and a calendar of monitoring activities, but there is no clear strategy. 454 Regional Operational Programme 2014-2020 Priority Axis 4 – Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Status Quo+ Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 455 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 6 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 6 or 1 to 6 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. After the technical and financial evaluation, a qualifying score is equal to 3.5 or more. A score of 0 for any of the sub-criteria results in the disqualification of the project. 456 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario Max Criterion Weight points 1. Project contribution to fulfilling ROP objectives 6 points 60% 1.1 Project relevance for the objectives of the priority axis and the key intervention area: a. County roads  The project ensures a direct connection to the TEN-T 6 points network, covers at least two counties, and also includes bike paths within the localities it crosses.  The project ensures a direct connection to the TEN-T 3 points network, covers at least one county, and also includes bike paths within the localities it crosses. b. Beltways  The project covers major traffic flows that cross localities and contributes to facilitating rapid connections between 6 points 60% residential and functional areas (economic, social, etc.).  The project does not cover major traffic flows that cross localities but contributes to facilitating rapid connections 3 points between residential and functional areas (economic, social, etc.) c. Urban streets 6 points  The project connects a city’s major traffic flows to a TEN- T road. 3 points  The project connects only part of a city’s major traffic flows to a TEN-T road. 1.2 Project contribution to reducing the time required for travel by improving traffic flows Average time savings (hours per 1000 km) < 3 hours 3-6 hours 6-9 hours > 9 hours Average < 600 1 pt. 1 pt. 1 pt. 3 pts. 40% daily 600-1200 1 pt. 3 pts. 3 pts. 6 pts. traffic 1200-2400 3 pts. 6 pts. 6 pts. 6 pts. flows > 2400 6 pts. 6 pts. 6 pts. 6 pts. 457 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario Max Criterion Weight points 2. Project quality, maturity, and sustainability 6 points 40% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the DTD, and the DTD completion is planned for a maximum of 3 6 points months after the approval of the financing request, it has all required approvals, and it has purchased the land (if needed) b. The applicant has launched the procedure for public 10% procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the 3 points financing request, it has some of the required approvals, and it has purchased the land (if needed) c. The applicant estimates the completion of the DTD in a 1 point maximum of 12 months after the approval of the financing request 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the 6 points attributions of project team members, and the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the 15% attributions of project team members and with the planning of public procurement activities. The results of the project 3 points and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the 1 point attributions of project team members, or with the planning of public procurement activities. The results of the project 458 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario Max Criterion Weight points and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 6 points b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of 25% resources efficiently and sustainably. Technologies are not 3 points entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 1 point The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 Coherence of economic documentation a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)145 is 6 points lower than 30% OR the expenditure sustainability indicator 20% (ESI)146 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly 145 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 146 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 459 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario Max Criterion Weight points estimated. The investment sustainability indicator (ISI) is 4 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly 2 points estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2.5 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the 6 points implementation of the project. 10% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, 3 points with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.6 Sustainable development and energy efficiency a. The project describes in detail the measures for improving the quality of the environment, along with measures for 6 points improving energy efficiency. 5% b. The project abides by minimum legal requirements regarding environmental protection and energy efficiency. 3 points 460 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario Max Criterion Weight points 2.7 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The 6 points applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 5% b. The applicant identifies aspects/risks related to 3 points discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. c. The applicant abides by minimum legal standards in this field. 1 point 2.8 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s 6 points implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 10% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear 1 point strategy. The organization does not have specific procedures for verifying/supervising the project team’s activities. 461 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 462 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 463 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 15% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly 10 points estimated. The investment sustainability indicator (ISI)147 is lower than 30% OR the expenditure sustainability indicator (ESI)148 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly 7 points estimated. The investment sustainability indicator (ISI) is between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. 4 points c. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2 QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 25% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the 25% DTD, and the DTD completion is planned for a maximum of 3 10 points months after the approval of the financing request, it has all 147 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 148 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 464 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS the required approvals, and it has purchased the land (if needed) b. The applicant has launched the procedure for public 7 points procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed) 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning 7 points of public procurement activities. The results of the project 15% and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not 4 points correlated with the implementation calendar, with the attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 465 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives, 10 points ensuring that health authorization requirements are fulfilled and there is a clear intention to obtain a certain classification. The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical 7 points solutions are feasible and contribute to reaching objectives, 30% ensuring that health authorization requirements are fulfilled. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ ecofriendly. The drawn parts are complete and correspond fully to written parts. 4 points c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives, ensuring that health authorization requirements are fulfilled. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 466 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with 10 points allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. 15% b. Estimated cost breakdowns are partially clear, complete, and 5 points realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.5 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of 10 points monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 15% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear strategy. The organization does not have specific procedures 5 points for verifying/ supervising the project team’s activities. 3. ECONOMIC IMPACT 10% 3.1 Project impact in terms of reducing patient diagnostic and treatment times (including reducing the number of in-patient care days, reducing the number of situations where patients are redirected due to lack of proper equipment, and/or other ways of reducing costs): a. Major impact. 10 points b. Average impact. 7 points c. Low impact. 4 points 467 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 4. SOCIAL IMPACT 40% 4.1 Project relevance for the goals of the priority axis and the key intervention areas: a. The investment has a high potential to offer new/innovative/ modern medical services. The need for the investment (demographic evolution, actual vs. needed capacity over the next five years, profile of diseases in the area vs. available 10 points treatments, etc.) is certain. b. The investment has an average potential to offer new/innovative/ modern medical services. The need for the 60% investment (demographic evolution, actual vs. needed capacity over the next five years, profile of diseases in the 7 points area vs. available treatments, etc.) is relative. c. The investment has a low potential to offer new/innovative/ modern medical services. The need for the investment (demographic evolution, actual vs. needed capacity over the next five years, profile of diseases in the area vs. available treatments, etc.) is limited. 4 points 4.2 Project impact on poor and marginalized groups: a. Major increase in poor and marginalized groups’ access to 10 points integrated health services (e.g., significant share of poor and marginalized groups within the total beneficiary population, expected % increase or expected additional number of 30% patients from poor and marginalized communities, etc.). 4 points b. No clear change in the accessibility of poor and marginalized groups to integrated health services. 4.3 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The 10 points applicant identifies aspects/risks related to discrimination in 10% the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 468 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on 7 points some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. c. The applicant abides by minimum legal standards in this field. 5 points 5. ENVIRONMENTAL IMPACT 10% 5.1 Sustainable development and energy efficiency a. The project describes in detail measures for improving the environment and increasing energy efficiency (The cost 10 points breakdown includes measures to protect the environment, energy efficiency works and alternative energy uses; the technical/ economic documentation includes EC certified materials; there is an energy audit with recommendations reflected in the technical solution; the latest standards for buildings’ energy efficiency are fulfilled; the project includes measures for proper disposal of medical waste, fulfilling the latest standards in the field, etc.). 5 points b. The project fulfills minimum legal requirements with respect to environmental protection and energy efficiency. 469 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 470 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Status Quo+ Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 6 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 6 or 1 to 6 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. After the technical and financial evaluation, a qualifying score is equal to 3.5 or more. A score of 0 for any of the sub-criteria results in the disqualification of the project. 471 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Status Quo+ Scenario Max Criterion Weight points 1. Project contribution to fulfilling ROP objectives 6 points 60% 1.1 Project relevance for the objectives of the priority axis and the key intervention area: quality of services a. The investment has a high potential to offer higher quality social services. The need for the investment (demographic 6 points trends, actual vs. needed capacity, share of marginalized groups) is certain. b. The investment has an average potential to offer higher 3 points 70% quality social services. The need for the investment (demographic trends, actual vs. needed capacity, share of marginalized groups) is relative. c. The investment has a small potential to offer higher quality social services. The need for the investment (demographic 1 point trends, actual vs. needed capacity, share of marginalized groups) is limited. 1.2 Project relevance for the objectives of the priority axis and the key intervention area: capacity of services a. The investment has a high potential to offer social services 6 points for a larger number of beneficiaries. b. The investment has an average potential to offer social 3 points 30% services for a larger number of beneficiaries. c. The investment has a low potential to offer social services for a larger number of beneficiaries. 1 point 2. Project quality, maturity, and sustainability 6 points 40% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the 6 points DTD, and the DTD completion is planned for a maximum of 3 months after the approval of the financing request, it has 10% some of the required approvals, and it has purchased the land (if needed) 3 points b. The applicant has launched the procedure for public procurement of the DTD, and the DTD completion is planned 472 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Status Quo+ Scenario Max Criterion Weight points for a maximum of 6 months after the approval of the financing request, it has all the required approvals, and it has 1 point purchased the land (if needed) c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 6 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning 10% of public procurement activities. The results of the project and the indicators are partly correlated with the planned 3 points activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the attributions of project team members, or with the planning 1 point of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical 25% solutions are feasible and contribute to reaching objectives. 6 points The proposed technical solutions involve the use of resources 473 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Status Quo+ Scenario Max Criterion Weight points efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and 3 points according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 1 point c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 Coherence of economic documentation a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)149 is 6 points lower than 30% OR the expenditure sustainability indicator (ESI)150 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly 25% estimated. The investment sustainability indicator (ISI) is 4 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly 2 points estimated. The investment sustainability indicator (ISI) is 149 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 150 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 474 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Status Quo+ Scenario Max Criterion Weight points between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2.5 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the 6 points implementation of the project. 10% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, 3 points with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.6 Sustainable development and energy efficiency a. The project describes in detail the measures for improving 6 points the quality of the environment, along with measures for improving energy efficiency. 5% b. The project abides by minimum legal requirements regarding 3 points environmental protection and energy efficiency. 2.7 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of 6 points opportunity also within public procurement contracts. The 5% applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to 475 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Status Quo+ Scenario Max Criterion Weight points facilitate disabled people’s access. 3 points b. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 1 point c. The applicant abides by minimum legal standards in this field. 2.8 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s 6 points implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 10% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear 1 point strategy. The organization does not have specific procedures for verifying/supervising the project team’s activities. 476 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Status Quo+ Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 477 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Status Quo+ Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 6 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 6 or 1 to 6 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. After the technical and financial evaluation, a qualifying score is equal to 3.5 or more. A score of 0 for any of the sub-criteria results in the disqualification of the project. 478 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Status Quo+ Scenario MAX CRITERION WEIGHT POINTS 1. Project contribution to fulfilling ROP objectives 20% 1.1 Project relevance for the objectives of the priority axis and the key intervention area: a. The project has a high potential to improve the quality of education and human capital development. The project 6 points contributes to the full endowment of the educational facility with proper equipment and connecting it to high-speed Internet. b. The project has an average potential to improve the quality 3 points of education and human capital development. The project 35% contributes to the full endowment of the educational facility with proper equipment and connecting it to high-speed Internet. c. The project has a low potential to improve the quality of 1 point education and human capital development. The project contributes to the full endowment of the educational facility with proper equipment and connecting it to high-speed Internet. 1.2 The real need for the educational infrastructure in the region/county/ locality where the project is implemented: a. The demographic and economic trends in the locality/county/ region (current situation and the forecast for the next 10 years) justify to a large degree the planned investment. The average use of the facility is at 70% minimum. The educational facility is in a 6 points rural area or at least 50% of the total student population belongs to marginalized groups. b. The demographic and economic trends in the locality/county/ region (current situation and the forecast for the next 10 years) 65% justify to a moderate degree the planned investment. The average use of the facility is between 50% and 70%. The educational facility is in a rural area or between 25% and 50% of the total student 3 points population belongs to marginalized groups represent. c. The demographic and economic trends in the locality/county/ region (current situation and the forecast for the next 10 years) justify to a small degree the planned investment. The average use of the facility is under 50%. The educational facility is in a rural area or under 25% of the total student population belongs to 1 point marginalized groups. 479 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Status Quo+ Scenario MAX CRITERION WEIGHT POINTS 2. Project quality, maturity, and sustainability 6 points 40% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the DTD, and the DTD completion is planned for a maximum of 3 months after the approval of the financing request, it has all 6 points the required approvals, and it has purchased the land (if needed) b. The applicant has launched the procedure for public 10% procurement of the DTD, and the DTD completion is planned 3 points for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed) c. The applicant estimates the completion of the DTD in a 1 point maximum of 12 months after the approval of the financing request 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, 6 points and closely correlated with the implementation calendar, the attributions of project team members, and the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and 10% partly correlated with the implementation calendar, with the attributions of project team members and with the planning 3 points of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not 1 point correlated with the implementation calendar, with the attributions of project team members, or with the planning 480 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Status Quo+ Scenario MAX CRITERION WEIGHT POINTS of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 6 points b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of 20% resources efficiently and sustainably. Technologies are not 3 points entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 1 point The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 Coherence of economic documentation a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)151 is 6 points lower than 30% OR the expenditure sustainability indicator 20% (ESI)152 is higher than 30%. The basis for calculating these indicators is the average for the last five years. 151 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 152 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 481 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Status Quo+ Scenario MAX CRITERION WEIGHT POINTS b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 4 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly 2 points estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2.5 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the 6 points implementation of the project. 10% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, 3 points with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.6 Sustainable development and energy efficiency a. The project describes in detail the measures for improving the quality of the environment, along with measures for 6 points improving energy efficiency. 5% b. The project abides by minimum legal requirements regarding 3 points environmental protection and energy efficiency. 482 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Status Quo+ Scenario MAX CRITERION WEIGHT POINTS 2.7 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The 6 points applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 5% b. The applicant identifies aspects/risks related to 3 points discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. c. The applicant abides by minimum legal standards in this field. 1 point 2.8 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s 6 points implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 10% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear 1 point strategy. The organization does not have specific procedures for verifying/supervising the project team’s activities. 2.9 Information society a. The project implements modern IT systems, modern solutions/applications, etc. 6 points 10% b. The project implements minimum standards with respect to 3 points IT equipment. 483 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Status Quo+ Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 484 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Semi-Competitive / Competitive Scenario Grids proposed for the Semi-Competitive and Competitive Scenarios 485 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. Note: This grid only covers SMEs. Once the axes/key areas of intervention are finalized, the same grid will be adaptable to the other types of eligible enterprises (e.g., same general criteria, with some adjusted thresholds). 486 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 15% 1.1 The applicant demonstrates through a rigorous business plan that the ROP investment does not negatively affect the firm’s financial sustainability: a. The applicant has a solid capacity to ensure that the investment/enterprise is preserved, maintained, and 10 points operational during and after the project’s implementation and after the grant disbursement stops. The investment generates profit (at least starting in Year 2), and the risk of failing to ensure a functional investment is minimal or nonexistent. The data provided (and working hypotheses) are correct, realistic, and justified. b. The applicant has an average capacity to ensure that the 7 points investment/enterprise is preserved, maintained, and operational during and after the project’s implementation and after the grant disbursement stops. The investment generates profit (at least starting in Year 3), and the risk of failing to ensure a functional investment is low. The data provided (and working hypotheses) are correct, realistic, and justified. c. The applicant has a weak capacity to ensure the 4 points investment/enterprise is preserved, maintained, and operational during and after the project’s implementation and after the grant disbursement stops. The investment generates profit (at least starting in Year 5), and the risk of failing to ensure a functional investment is significant. The data provided (and working hypotheses) are correct, realistic, and justified. 2. PROJECT QUALITY, MATURITY, AND SUSTAINABILITY 25% 2.1 The applicant’s capacity to implement the proposed project a. Material and human resources (the project team) are clearly 10 points defined and are adequate for the project’s implementation. The applicant has strong experience in the field that forms the object of the proposed project. The turnover for the 20% latest fiscal year is at least equal to value of the grant. b. Material and human resources (the project team) are 7 points adequate for the project’s implementation. The applicant has relative experience in the field that forms the object of the 487 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS proposed project. The turnover for the latest fiscal year is at least equal to 50% of the value of the grant. c. Material and human resources (the project team) are inadequate. The applicant has no experience in the field that 4 points forms the object of the proposed project. The turnover for the latest fiscal year is at least equal to 30% of the value of the grant. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the 7 points attributions of project team members and with the planning 15% of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not 4 points correlated with the implementation calendar, with the attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 The project’s budget a. The budget is complete and perfectly correlated with proposed activities, with material resources involved in the project’s implementation, with anticipated results. 15% Expenditures have been correctly differentiated into the categories of eligible and non-eligible expenditures, and the thresholds for certain expenditures have been observed, 10 points 488 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS according to the Eligible Expenditures Order and the provisions of the applicable de minimis aid scheme. There is full correlation between the budget and the financing sources. Costs are realistic (correctly estimated), sufficient and necessary for project implementation. The budget observes the standard model for the key intervention area, is accurately calculated, and the VAT is calculated with two decimal places. The total project value is within the limits established for the key intervention area. b. The budget is satisfactorily correlated with proposed activities, with material resources involved in the project’s implementation, with anticipated results. There is an imperfect correspondence between each activity/result and budget lines, and some activities and costs are treated as 7 points “global.” Costs are necessary and sufficient for the project’s implementation. Expenditures have been correctly differentiated into the categories of eligible and non-eligible expenditures, and the thresholds for certain expenditures have been observed, according to the Eligible Expenditures Order and the provisions of the applicable de minimis aid scheme. There is full correlation between the budget and the financing sources. Costs are realistic (correctly estimated), sufficient and necessary for project implementation. The budget observes the standard model for the key intervention area, is accurately calculated, and the VAT is calculated with two decimal places. The total project value is within the limits established for the key intervention area. 4 points c. The budget is not perfectly correlated with proposed activities, with material resources involved in the project’s implementation, with anticipated results. The budget has certain errors. The total value of the project is within the limits set for this key intervention area. Expenditures have been correctly differentiated into the categories of eligible and non-eligible expenditures, and the thresholds for certain expenditures have been observed, according to the Eligible Expenditures Order and the provisions of the applicable de minimis aid scheme. 489 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 2.4 The quality of the business plan a. Data are sufficient, correct, and justified. The market analysis demonstrates a clear need/demand for the products/services offered. The competitive analysis identifies competitors, their strong and weak points, the 10 points competitive advantage of the applicant (in marketing, operations, organizational structure, etc.). The marketing strategy is feasible (it identifies adequate and efficient instruments) in the context of available resources. Financial projections are correlated with the marketing strategy; they are correct and realistic. Cost and revenue estimates are correct, realistic, and justify the financial projections. b. Data are sufficient and largely justified. The market analysis demonstrates a need/demand for the products/services offered. The competitive analysis identifies competitors, but provides only a partial assessment of their strong and weak points, and the competitive advantage of the applicant (in marketing, operations, organizational structure, etc.). The 7 points 50% marketing strategy is partly feasible (it identifies adequate and efficient instruments) in the context of available resources. Financial projections are partly correlated with the marketing strategy; they are partly correct and realistic. Cost and revenue estimates are partly correct, realistic, and justify to a limited extent the financial projections. c. Data are insufficient and/or not justified. The market analysis does not demonstrate a need/demand for the products/services offered. The competitive analysis identifies competitors, but does not assess their strong and weak points, and/or the competitive advantage of the 4 points applicant (in marketing, operations, organizational structure, etc.). The marketing strategy is not feasible (it identifies adequate and efficient instruments) in the context of available resources. Financial projections are not correlated with the marketing strategy; they are incorrect and/or unrealistic. Cost and revenue estimates are incorrect and/or unrealistic. 3. ECONOMIC IMPACT 40% 3.1 Creating new permanent jobs a. The project leads to the creation of new permanent jobs 60% within the enterprise, and the number of newly created 10 points 490 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS permanent (full-time) jobs for the EUR 10,000 grant obtained is greater than or equal to 0.30. b. The project leads to the creation of new permanent jobs 7 points within the enterprise, and the number of newly created permanent (full-time) jobs for the EUR 10,000 grant obtained is equal to 0.20. c. The project leads to the creation of new permanent jobs 4 points within the enterprise. The number of newly created permanent (full-time) jobs for the EUR 10,000 grant obtained is equal to 0.15. 1 point d. The project does not lead to the creation of new permanent jobs within the enterprise. 3.2 Using local resources a. The enterprise mostly uses local resources/raw materials (produced 10 points locally – same city/county) and manpower. b. The enterprise partially uses local resources/raw materials 7 points 10% (produced locally – same city/county) and/or manpower. c. The enterprise does not use local resources/raw materials 4 points (produced locally – same city/county) or manpower. 3.3 Innovative character a. The project has a high potential for the implementation of a new or significantly improved product (good or service), or 10 points process, a new marketing method, or a new organizational method in business practices, workplace organization or external relations. 30% b. The project implements a new or improved product, process, 4 points marketing method, or organizational method, but it is only new to the applicant firm (and otherwise available on the local market) 4. SOCIAL IMPACT 10% 4.1 Opportunities for poor/marginalized communities: a. Through the nature of the business, location, and human 10 points resources policies, the project demonstrates a major 70% potential for hiring/ training of low-income/marginalized people. 491 Regional Operational Programme 2014-2020 Priority Axis 2 – Improving the competitiveness of small and medium-sized enterprises Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 4 points b. Through the nature of the business, location, and human resources policies, the project demonstrates a weak focus on hiring/training of low-income/marginalized people. 4.2 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of 10 points opportunity also within public procurement contracts. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms 30% to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. b. The applicant abides by minimum legal standards in this field. 5 points 5. ENVIRONMENTAL IMPACT 10% 5.1 Sustainable development and energy efficiency a. The project describes in detail the measures for improving 10 points the quality of the environment, along with measures for improving energy efficiency. 80% b. The project abides by minimum legal requirements regarding 5 points environmental protection and energy efficiency. 5.2 Uses of alternative energy a. The proposed investment utilizes alternative energy sources 10 points in its current activity to cover at least 50% of needs b. The proposed investment utilizes alternative energy sources 7 points 20% in its current activity to cover at least 20% of needs c. The proposed investment utilizes alternative energy sources 4 points in its current activity to cover at least 5% of needs 492 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban Development Investment Priority 3.1 – Support to energy efficiency and use of renewable energy in public infrastructure, including public buildings and residential buildings – Residential Buildings Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub-criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub-criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 493 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban Development Investment Priority 3.1 – Support to energy efficiency and use of renewable energy in public infrastructure, including public buildings and residential buildings – Residential Buildings Semi-Competitive / Competitive Scenario MAX CRITERIA WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 20% 1.1 The household association demonstrates that it has the required resources to cover planned investments and future operations and 10 points maintenance (O&M) costs. a. The household association can demonstrate clearly and convincingly that it can cover both needed co-financing and future O&M costs. 7 points 60% b. The household association can demonstrate satisfactorily that 4 points it can cover both needed co-financing and future O&M costs. c. The household association cannot demonstrate that needed co-financing and future O&M costs can be covered in full. 1.2 The local authority demonstrates that sufficient resources are available to cover costs and do not affect the capacity of the locality to complete other needed public investments: a. The investment sustainability indicator (ISI)153 is lower than 10 points 30%. The basis for calculating this indicator is the average for the last five years. b. The investment sustainability indicator (ISI) is between 30% 7 points and 40%. The basis for calculating this indicator is the average 40% for the last five years. c. The investment sustainability indicator (ISI) is between 40% and 50%. The basis for calculating this indicator is the average 4 points for the last five years. d. The investment sustainability indicator (ISI) is over 50%. The 1 points basis for calculating this indicator is the average for the last five years. 2 QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 20% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the DTD, 10 points 25% and the DTD completion is planned for a maximum of 3 months after the approval of the financing request, it has all 153 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 494 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban Development Investment Priority 3.1 – Support to energy efficiency and use of renewable energy in public infrastructure, including public buildings and residential buildings – Residential Buildings Semi-Competitive / Competitive Scenario MAX CRITERIA WEIGHT POINTS the required approvals, and it has purchased the land (if needed). b. The applicant has launched the procedure for public 7 points procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed). 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the 7 points attributions of project team members and with the planning 20% of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the 4 points attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 495 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban Development Investment Priority 3.1 – Support to energy efficiency and use of renewable energy in public infrastructure, including public buildings and residential buildings – Residential Buildings Semi-Competitive / Competitive Scenario MAX CRITERIA WEIGHT POINTS 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete 10 points and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The 25% 7 points proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions 5 points are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are 10 points realistic (correctly estimated) and necessary for the implementation of the project. 10% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is 5 points complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 496 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban Development Investment Priority 3.1 – Support to energy efficiency and use of renewable energy in public infrastructure, including public buildings and residential buildings – Residential Buildings Semi-Competitive / Competitive Scenario MAX CRITERIA WEIGHT POINTS 2.5 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; 10 points there is a clear division of tasks, procedures, and a timeline of monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 20% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no 5 points clear strategy. The organization does not have specific procedures for verifying/supervising the project team’s activities. 3. ECONOMIC IMPACT 20% 3.1 Project contribution for reaching the objectives of the priority access and key intervention area. a. The project ensures the thermal insulation/thermal rehabilitation of residential buildings built after 1950, larger 10 points than 50 square meters and having a high potential of energy saving, and reduces the energy consumption by more than 35%. b. The project ensures the thermal insulation/thermal 7 points 50% rehabilitation of residential buildings built after 1950, larger than 50 square meters, and reduces the energy consumption between 20 and 30%. 4 points c. The project ensures the thermal insulation /thermal rehabilitation of residential buildings built after 1950, larger than 50 square meters, and reduces the energy consumption up to 10%. 3.2 The project has impact on a large number of beneficiaries (people who live in the residential building). a. The number of beneficiaries is beyond 200 people. 10 points 50% b. The number of beneficiaries is between 100 and 200 people. 7 points c. The number of beneficiaries is below 50 people. 4 points 497 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban Development Investment Priority 3.1 – Support to energy efficiency and use of renewable energy in public infrastructure, including public buildings and residential buildings – Residential Buildings Semi-Competitive / Competitive Scenario MAX CRITERIA WEIGHT POINTS 4. SOCIAL IMPACT 20% 4.1 Project impact on the poor a. At least 60% of the beneficiaries have lower incomes than the 10 points average income in the country. b. Between 30% and 60% of the beneficiaries have lower 7 points 60% incomes than the average income in the country. c. Up to 30% of the beneficiaries have lower incomes than the 4 points average income in the country. 4.2 Project impact on marginalized communities a. The project has a solid impact on marginalized communities. 10 points 30% b. The project does not have an impact on marginalized 4 points communities. 4.3 Equality of opportunity and nondiscrimination c. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to 10 points address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. d. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on some mechanisms 10% to address these potential issues. The project involves the 7 points creation of facilities/ adapting the current structure to facilitate disabled people’s access. e. The applicant abides by minimum legal standards in this field. 5 points 498 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban Development Investment Priority 3.1 – Support to energy efficiency and use of renewable energy in public infrastructure, including public buildings and residential buildings – Residential Buildings Semi-Competitive / Competitive Scenario MAX CRITERIA WEIGHT POINTS 5. ENVIRONMENTAL IMPACT 20% 5.1 Project impact on reducing greenhouse gas emissions and its contribution to living conditions in apartments and to improving the environment. a. The project has a significant contribution to reducing greenhouse gas emissions and improving living conditions in apartments, as well as improving the environment. In addition 10 points to thermal insulation and replacement of the woodwork, the project ensures not only the insulation of the building, but also the replacement of electrical, thermal, and water networks, in compliance with eligible activities. b. The project has a moderate contribution to reducing greenhouse gas emissions and improving living conditions in apartments, as well as improving the environment. In addition to thermal insulation and replacement of the woodwork, the 7 points project ensures not only the insulation of the building, but also the replacement of electrical, thermal, and water networks, in compliance with eligible activities. c. The project has a small contribution to reducing greenhouse gas emissions, improving living conditions in the apartments, 4 points as well as improving the environment. The project involves only the thermal insulation of the building and change of woodwork. 499 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban Development Investment Priority 3.1 – Support to energy efficiency and use of renewable energy in public infrastructure, including public buildings and residential buildings – Residential Buildings Semi-Competitive / Competitive Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 500 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 501 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 20% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly 10 points estimated. The investment sustainability indicator (ISI)154 is lower than 30% OR the expenditure sustainability indicator (ESI)155 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly 7 points estimated. The investment sustainability indicator (ISI) is between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. 4 points c. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2 QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 20% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the 25% DTD, and the DTD completion is planned for a maximum of 3 10 points months after the approval of the financing request, it has all the required approvals, and it has purchased the land (if 154 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 155 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 502 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS needed) b. The applicant has launched the procedure for public 7 points procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed) 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning 7 points 20% of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not 4 points correlated with the implementation calendar, with the attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility 25% studies and the technical expertise are complete and 503 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 10 points The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical 7 points solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 4 points c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with 10 points allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. 15% b. Estimated cost breakdowns are partially clear, complete, and 5 points realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 504 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 2.5 Applicant capacity to monitor the project’s implementation and post- implementation c. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of 10 points monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 15% d. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear strategy. The organization does not have specific procedures 5 points for verifying/supervising the project team’s activities. 3. ECONOMIC IMPACT 30% 3.1 Need for the investment and facilitating access to opportunities (jobs, education, health, culture, entertainment, etc.): a. By implementing the project, a reduction of over 15% in travel time is expected, along with an increase by over 10% in 10 points the number of public transport users and/or bikers. There is solid evidence that the investment is needed (current/expected ridership vs. current/future capacity) b. By implementing the project, a reduction of over 5-15% in 7 points travel time is expected, along with an increase by 5-10% in 70% the number of public transport users and/or bikers. There is partial evidence that the investment is needed (current/expected ridership vs. current/future capacity) c. By implementing the project, a reduction of less than 5% in travel time is expected, along with an increase by less than 4 points 5% in the number of public transport users and/or bikers. There is little evidence that the investment is needed (current/expected ridership vs. current/future capacity) 505 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 3.2 Reduction in operations and maintenance (O&M) costs (only for upgrading existing infrastructure) per user: a. By over 30% 10 points 30% b. By 10-20% 7 points c. By under 10% 5 points 4. SOCIAL IMPACT 15% 4.1 Project involvement of poor and marginalized communities and its contribution to facilitating their access to opportunities (jobs, education, culture, entertainment): a. The project demonstrates a solid impact on poor and 10 points marginalized communities and involves clear measures for 70% improving their access to opportunities. b. The project does not clearly address issues related to poor 4 points and marginalized communities. 4.2 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The 6 points applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 30% b. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on 3 points some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 1 point c. The applicant abides by minimum legal standards in this field. 5. ENVIRONMENTAL IMPACT 15% 5.1 Sustainable development and energy efficiency: a. The investment has a high potential to boost the energy efficiency of urban transport systems (i.e., over 10% savings 10 points in kWh equivalent). It relies on eco-friendly transport modes (bike paths, tram lines, etc.), efficient rolling stock, modern 506 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment priority 3.2 – Developing an ecofriendly public transport, particularly electric, and stimulating non-motorized urban travel Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS public spaces, and improved ticketing systems. b. The investment has a moderate potential to boost the energy 7 points efficiency of urban transport systems (i.e., 5-10% savings in kWh equivalent). It relies on eco-friendly transport modes (bike paths, tram lines, etc.), efficient rolling stock, and modern public spaces. c. The investment has a low potential to boost the energy efficiency of urban transport systems (i.e., under 5% savings 5 points in kWh equivalent). It relies on eco-friendly transport modes (bike paths, tram lines, etc.) and efficient rolling stock. OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 507 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 508 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 20% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)156 is 10 points lower than 30% OR the expenditure sustainability indicator (ESI)157 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 7 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly 4 points estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2 QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 20% 2.1 Applicant’s capacity to implement the project a. Material and human resources (the project team) are clearly defined and are adequate for the project’s implementation. 10% The applicant has a clear strategy for monitoring the 10 points project’s implementation; there is a clear distribution of 156 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 157 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 509 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS tasks; there are procedures and there is a calendar of monitoring activities. The applicant has experience in managing at least 2 projects financed from public funds, with a total value similar to the proposed investment. b. Material and human resources (the project team) are sufficient. The responsibilities of the project team members do not entirely cover the activities required for the project’s 7 points management and implementation. The applicant has a series of procedures for monitoring project implementation, and a calendar of monitoring activities, but there is no clear strategy. The applicant has managed at least one project financed from public funds. 2.2 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or 10 points has completed the procedure for public procurement of the DTD, and the DTD completion is planned for a maximum of 3 months after the approval of the financing request, it has all the required approvals, and it has purchased the land (if needed) 7 points b. The applicant has launched the procedure for public 25% procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it some of the required approvals, and it 4 points has purchased the land (if needed) c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request 2.3 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and 15% the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The 510 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning 7 points of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities 4 points of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.4 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 10 points The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and 7 points according to current norms and standards. Technical 20% solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 5 points c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources 511 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.5 Marketing and management strategy a. The market analysis demonstrates a clear need/demand for the products/services offered. The competitive analysis identifies competitors, their strong and weak points, the competitive advantage of the applicant (in marketing, 10 points operations, organizational structure, etc.). The marketing strategy identifies adequate and efficient instruments, appropriate for the structure type. The management strategy is appropriate, given the specifics of the structure and the services it offers. 20% b. The market analysis demonstrates a potential need/demand for the products/services offered. The competitive analysis partially identifies competitors, their strong and weak points, the competitive advantage of the applicant (in marketing, operations, organizational structure, etc.). The marketing 7 points strategy identifies some adequate and efficient instruments. The management strategy is not fully aligned with the specifics of the structure and the services it offers. 2.6 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with 10 points allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. b. Estimated cost breakdowns are partially clear, complete, and 5 points 10% realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 512 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 3. ECONOMIC IMPACT 20% 3.1 Project relevance for the goals of the priority axis/key intervention area: a. The project generates new permanent jobs following the site’s rehabilitation and the development of new activities, and the number of newly created permanent jobs for the 10 points EUR 100,000 grant obtained is greater than or equal to 0.8. Rehabilitated surface for the EUR 100,000 grant obtained158 is greater than or equal to 0.5ha. b. The project generates new permanent jobs following the site’s rehabilitation and the development of new activities, 50% and the number of newly created permanent jobs for the 7 points EUR 100,000 grant obtained is greater than or equal to 0.6. Rehabilitated surface for the EUR 100,000 grant obtained is greater than or equal to 0.35ha. c. The project generates new permanent jobs following the site’s rehabilitation and the development of new activities, and the number of newly created permanent jobs for the 4 points EUR 100,000 grant obtained is greater than or equal to 0.5. Rehabilitated surface for the EUR 100,000 grant obtained is greater than or equal to 0.25ha. 3.2 Opportunity to rehabilitate the polluted site and to create/expand a business support structure a. In the target area there is a well-defined need for the business support structure and for the services offered. The facilities and/or services correspond to/are adequate for the needs of the companies present within the 10 points structure/potential (space, utilities, specific services, etc.). The accessibility degree: the structure location allows for 35% easy access to at least 2 types of transport (road, railroad, naval). The activity of building/modernizing/ expanding the access road to the business support structure, as well as the related expenditures, are fully justified by the current opportunities to connect to county/national roads, railways, trading ports etc. 158 The total grant obtained for both implementation phases of the project shall be taken into consideration. 513 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The opportunity for the business support structure and the offered services is not fully justified. The need for such 7 points structures in the target area has not been/cannot be identified or is not clear. The provided facilities and/or services correspond to/are adequate to a small extent for the needs of the companies present within the structure/potential (space, utilities, specific services, etc.). The accessibility degree: the structure location allows for easy access to one type of transport (road, railroad, naval). If included, the activity of building/modernizing/expanding the access road to the business support structure, as well as related expenditures, are partially justified by current possibilities to connect to county/national roads, railways, 4 points trading ports, etc. c. The opportunity of the business support structure and of the offered services is not justified. The provided facilities and/or services do not correspond to/are not adequate for the needs of the companies present within the structure/potential (space, utilities, specific services, etc.). The accessibility degree: the structure location does not allow for easy access to any type of transport (road, railroad, and/or naval). If provided for, the activity of building/modernizing/expanding the access road to the business support structure is not justified by the current possibilities to connect to county/national roads, railways, trading ports etc. 3.3 Correlation with economic and real estate dynamics at the local level: a. The project demonstrates a solid understanding of economic 10 points dynamics and clearly justifies the need to use public grants. 10% b. The project demonstrates a limited understanding of 4 points economic dynamics and only partially justifies the need to use public grants. 3.4 Information society and new technologies: a. The project describes in detail how it implements modern IT 10 points systems, modern solutions/ applications, etc. (the project has 5% a high innovative potential). 514 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The proposed investment maintains (does not increase) the level of use of new/modern technologies and IT systems (the 4 points project has a low innovative potential). 4. SOCIAL IMPACT 20% 4.1 Project contribution to improved living standards and living environment: a. The project has a major contribution to improving the soil, 10 points water, and air for communities nearby. The project clearly demonstrates that poor and marginalized communities in the area are not negatively affected by the project (e.g., forced to move because of real estate pressures, etc.) or proposes a detailed strategy for addressing potential negative consequences for these groups. 7 points b. The project has a moderate contribution to improving the 40% soil, water, and air for communities nearby. The project adequately demonstrates that poor and marginalized communities in the area are not negatively affected by the 4 points project or proposes some measures for addressing potential negative consequences for these groups. c. The project has a low contribution to improving the soil, water, and air for communities nearby. The project does not demonstrate that poor and marginalized communities in the area are not negatively affected by the project and does not propose measures for effectively addressing potential negative consequences for these groups. 4.2 Project contribution to the revitalization of disadvantaged areas and to the creation of new opportunities for poor and marginalized communities: 10 points a. Major impact 7 points 30% b. Average impact 4 points c. Low impact 515 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 4.3 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The 10 points applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 30% b. The applicant identifies aspects/risks related to 7 points discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 5 points c. The applicant abides by minimum legal standards in this field. 5. ENVIRONMENTAL IMPACT 20% 5.1 Sustainable development and energy efficiency a. The project describes in detail the measures for improving the quality of the environment, along with measures for improving energy efficiency. It includes the decontamination 10 points of polluted sites, the proper storage of toxic waste, and adequate measures for recyclable materials. b. The project abides by minimum legal requirements regarding 5 points environmental protection and energy efficiency. 516 Regional Operational Programme 2014-2020 Priority Axis 3 – Urban development Investment Priority 3.3 – Improving the urban environment, revitalizing cities, regenerating and decontaminating polluted sites, and promoting measures for reduing noise Semi-Competitive / Competitive Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 517 Regional Operational Programme 2014-2020 Priority Axis 4 - Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 6 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 6 or 1 to 6 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 518 Regional Operational Programme 2014-2020 Priority Axis 4 - Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Semi-Competitive / Competitive Scenario CRITERIA MAX WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 20% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)159 is 10 points lower than 30% OR the expenditure sustainability indicator (ESI)160 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 7 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly 4 points estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. 1 point d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2. QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 20% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the 10 points 25% DTD, and the DTD completion is planned for a maximum of 3 months after the approval of the financing request, it has all 159 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 160 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 519 Regional Operational Programme 2014-2020 Priority Axis 4 - Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Semi-Competitive / Competitive Scenario CRITERIA MAX WEIGHT POINTS the required approvals, and it has purchased the land (if needed). b. The applicant has launched the procedure for public 7 points procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed). 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning 10 points of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with 7 points the attributions of project team members and with the 20% planning of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the 4 points attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 520 Regional Operational Programme 2014-2020 Priority Axis 4 - Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Semi-Competitive / Competitive Scenario CRITERIA MAX WEIGHT POINTS 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 10 points The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical 7 points solutions are feasible and contribute to reaching objectives. 25% The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The 5 points feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with 10 points allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. 15% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is 5 points complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 521 Regional Operational Programme 2014-2020 Priority Axis 4 - Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Semi-Competitive / Competitive Scenario CRITERIA MAX WEIGHT POINTS 2.5 Applicant capacity to monitor the project’s implementation and post-implementation a. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; 10 points there is a clear division of tasks, procedures, and a timeline of monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 15% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear 5 points strategy. The organization does not have specific procedures for verifying/supervising the project team’s activities. 3. ECONOMIC IMPACT 30% 3.1 Project contribution to increasing the number of tourists by promoting the local and regional tourism potential. a. After the project’s completion, the annual average number 10 points of tourists will increase by over 5%. b. After the project’s completion, the annual average number 7 points 30% of tourists will increase by up to 5%. c. After the project’s completion, the annual average number of tourists will not decrease compared to current levels. 4 points 3.2 Project direct and induced economic impact on local economy. a. The project demonstrates a good understanding of the local HORECA base (hotels, restaurants, and catering industry) 10 points and presents in a clear manner how the local economy will benefit from the project. b. The project demonstrates a good understanding of the local HORECA base (hotels, restaurants, and catering industry), 40% but it does not present in a clear manner how the local 7 points economy will benefit from the project. c. The project demonstrates a partial understanding of the local HORECA base (hotels, restaurants, and catering industry) and the economic potential of the region, but it is 522 Regional Operational Programme 2014-2020 Priority Axis 4 - Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Semi-Competitive / Competitive Scenario CRITERIA MAX WEIGHT POINTS not clear how the project will help the local economy. 4 points 3.3 The marketing strategy used for promoting the tourist potential of the heritage site. a. The marketing plan identifies adequate instruments to highlight the heritage site, at the international and national level. The tourism potential of the site is certain. Financial 10 points projections are correlated with the marketing strategy and they are accurate and realistic. b. The marketing plan identifies adequate instruments to highlight the heritage site, at national and/or regional and/or local level (not international). The tourism potential 7 points of the site is certain. Financial projections are correlated 30% with the marketing strategy and they are accurate and realistic. The plan demonstrates the touristic potential of the site. Financial projections are correlated with the marketing strategy and they are accurate and realistic. c. The marketing plan identifies adequate instruments to highlight the heritage site, at national and/or regional 4 points and/or local level. The tourism potential is poorly demonstrated. Financial projections are not correlated with the marketing strategy, and they are not accurate /realistic. Cost and income calculations are not accurate /realistic. 4. SOCIAL IMPACT 15% 4.1 Project contribution to jobs creation a. The project demonstrates in a satisfactory manner the contribution to new jobs creation, in a direct, indirect, and 10 points induced way, of which poor and marginalized communities will benefit as well. b. The project demonstrates in a satisfactory manner the 80% contribution to new jobs creation, in a direct, indirect, and 7 points induced way, but they will not be directed to poor and marginalized communities. c. The project will only maintain the existing jobs. 4 points 523 Regional Operational Programme 2014-2020 Priority Axis 4 - Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Semi-Competitive / Competitive Scenario CRITERIA MAX WEIGHT POINTS 4.2 Equality of opportunity and nondiscrimination n a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. 10 points The applicant identifies aspects/risks related to discrimination in the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 20% b. The applicant identifies aspects/risks related to 7 points discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. c. The applicant abides by minimum legal standards in this 5 points field. 5. ENVIRONMENTAL IMPACT 15% 5.2 Sustainable development and energy efficiency a. The project describes in detail the measures for improving the quality of the environment, along with measures for 10 points improving energy efficiency. b. The project abides by minimum legal requirements 5 points regarding environmental protection and energy efficiency. 524 Regional Operational Programme 2014-2020 Priority Axis 4 - Environmental protection and promotion of efficient resource use Investment Priority 4.1 – Conservation, protection, promotion, and development of the natural and cultural heritage Semi-Competitive / Competitive Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 525 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 526 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 20% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)161 is lower than 30% OR the expenditure sustainability indicator 10 points (ESI)162 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is between 30% and 40% OR the expenditure sustainability 7 points indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 4 points between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. d. Operation and maintenance (O&M) costs are correctly 1 point estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2 QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 20% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the DTD, and the DTD completion is planned for a maximum of 3 10 points 15% months after the approval of the financing request, it has all the required approvals). There are no land ownership issues. 161 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 162 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 527 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The applicant has launched the procedure for public procurement of the DTD, and the DTD completion is planned 7 points for a maximum of 6 months after the approval of the financing request, it has some of the required approvals. There are no land ownership issues. 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning 7 points 20% of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the 4 points attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and 35% according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The 10 points 528 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions 7 points are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 4 points c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with allocated 10 points estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. 15% b. Estimated cost breakdowns are partially clear, complete, and 5 points realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 529 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 2.5 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of 10 points monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 15% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear strategy. The organization does not have specific procedures 5 points for verifying/supervising the project team’s activities. 3. ECONOMIC IMPACT 30% 3.1 Project relevance for the objectives of the priority axis and the key intervention area: a. County roads  The project improves the direct connection to the TEN-T 10 points network for at least 10,000 people (within a 10-minute driving buffer from the proposed project), covers at least two counties, and also includes bike paths within one or more of the localities it crosses. 6 points  The project improves the direct connection to the TEN-T network for fewer than 10,000 people (within a 10-minute driving buffer from the proposed project), covers at least one county, and also includes bike paths within the 10 points localities it crosses. 30% b. Beltways  The project covers major traffic flows that cross localities 6 points and connect to a TEN-T network, is correlated with the Urban Mobility Plan, and contributes to facilitating rapid connections between residential and functional areas (economic, social, etc.).  The project does not cover major traffic flows that cross localities and connect ot a TEN-T network but does not contribute to facilitating rapid connections between residential and functional areas (economic, social, etc.) 530 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 3.2 The project improves access to opportunities (jobs, higher education, hospitals and social services, major centers of culture and entertainment, airport infrastructure, railway infrastructure, etc.): a. The project improves access to a growth pole or an urban 10 points development poles, and is within at most 60km from that growth pole; 7 points 30% b. The project improves access to a county capital and is within at most 40 km of that county capital; 4 points c. The project improves access to an urban center of at least 10,000 people and is within at most 20 km away from that urban center; 3.3 Project contribution to reducing the time required for travel by improving traffic flows: Average time savings (hours per 1000 km) < 3 hours 3-6 hours 6-9 hours > 9 hours Average < 600 4 pts. 4 pts. 4 pts. 7 pts. 40% daily 600-1200 4 pts. 7 pts. 7 pts. 10 pts. traffic 1200-2400 7 pts. 10 pts. 10 pts. 10 pts. flows > 2400 10 pts. 10 pts. 10 pts. 10 pts. 4. SOCIAL IMPACT 10% 4.1 Project improves safety conditions: a. The project proposes a significant increase in safety conditions 10 points (as measured by expected decrease in death, injury, and vehicle damage rates). 40% b. The project meets minimum safety standards. 5 points 4.2 Project impact on poor and marginalized communities a. The project demonstrates a high impact on improving poor 10 points and marginalized groups’ access to opportunities (jobs, 40% education, health). 531 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The project demonstrates an average impact on improving 7 points poor and marginalized groups’ access to opportunities (jobs, education, health). c. The project demonstrates a low impact on improving poor and 4 points marginalized groups’ access to opportunities (jobs, education, health). 4.3 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The applicant identifies aspects/risks related to discrimination in 10 points the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 20% b. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on some mechanisms 7 points to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 5 points c. The applicant abides by minimum legal standards in this field. 5. ENVIRONMENTAL IMPACT 20% 5.1 Sustainable development and energy efficiency: a. (1) The project includes non-motorized alternative transportation (bicycle lanes or sidewalks where feasible). (2) The project contributes substantially to improving traffic 10 points conditions. (3) The project includes at least one public transport route. (4) The project is connected to intermodal transportation centers and park-and-ride facilities. (5) The project enables lower fuel consumption rates through an improved road surface. b. (1) The project includes non-motorized alternative 7 points transportation (bicycle lanes or sidewalks where feasible). (2) The project contributes only partially to improving traffic conditions. (3) The project includes at least one public transport route. (4) The project is not connected to intermodal 532 Regional Operational Programme 2014-2020 Priority Axis 5 – Promoting sustainable transport systems Priority investment 5.1 – Promoting regional mobility by connecting regional road infrastructures to the TEN-T infrastructure Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS transportation centers and park-and-ride facilities. (5) The project enables lower fuel consumption rates through an improved road surface. 4 points c. (1) The project does not include non-motorized alternative transportation (bicycle lanes or sidewalks where feasible). (2) The project contributes partially to improving traffic conditions. (3) The project does not overlap with any public transport routes. (4) The project is not connected to intermodal transportation centers and park-and-ride facilities. (5) Proposed road surface improvements do not facilitate lower fuel consumption rates. OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 533 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 534 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 15% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly 10 points estimated. The investment sustainability indicator (ISI)163 is lower than 30% OR the expenditure sustainability indicator (ESI)164 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is between 30% and 40% OR the expenditure sustainability 7 points indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability 4 points indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 1 point over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2 QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 25% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the 25% DTD, and the DTD completion is planned for a maximum of 3 10 points months after the approval of the financing request, it has all 163 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 164 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 535 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS the required approvals, and it has purchased the land (if needed) b. The applicant has launched the procedure for public 7 points procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed) 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning 7 points of public procurement activities. The results of the project 15% and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not 4 points correlated with the implementation calendar, with the attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 536 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives, 10 points ensuring that health authorization requirements are fulfilled and there is a clear intention to obtain a certain classification. The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical 7 points solutions are feasible and contribute to reaching objectives, 30% ensuring that health authorization requirements are fulfilled. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives, 4 points ensuring that health authorization requirements are fulfilled. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 537 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with 10 points allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. 15% b. Estimated cost breakdowns are partially clear, complete, and 5 points realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.5 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of 10 points monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 15% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear strategy. The organization does not have specific procedures 5 points for verifying/ supervising the project team’s activities. 3. ECONOMIC IMPACT 10% 3.1 Project impact in terms of reducing patient diagnostic and treatment times (including reducing the number of in-patient care days, reducing the number of situations where patients are redirected due to lack of proper equipment, and/or other ways of reducing costs): d. Major impact. 10 points e. Average impact. 7 points f. Low impact. 4 points 538 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 4. SOCIAL IMPACT 40% 4.1 Project relevance for the goals of the priority axis and the key intervention areas: a. The investment has a high potential to offer new/innovative/ modern medical services. The need for the investment (demographic evolution, actual vs. needed capacity over the next five years, profile of diseases in the area vs. available 10 points treatments, etc.) is certain. b. The investment has an average potential to offer new/innovative/ modern medical services. The need for the 60% investment (demographic evolution, actual vs. needed capacity over the next five years, profile of diseases in the 7 points area vs. available treatments, etc.) is relative. c. The investment has a low potential to offer new/innovative/ modern medical services. The need for the investment (demographic evolution, actual vs. needed capacity over the next five years, profile of diseases in the area vs. available treatments, etc.) is limited. 4 points 4.2 Project impact on poor and marginalized groups: c. Major increase in poor and marginalized groups’ access to 10 points integrated health services (e.g., significant share of poor and marginalized groups within the total beneficiary population, expected % increase or expected additional number of 30% patients from poor and marginalized communities, etc.). 4 points d. No clear change in the accessibility of poor and marginalized groups to integrated health services. 4.3 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The 10 points applicant identifies aspects/risks related to discrimination in 10% the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 539 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on 7 points some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. c. The applicant abides by minimum legal standards in this field. 5 points 5. ENVIRONMENTAL IMPACT 10% 5.1 Sustainable development and energy efficiency a. The project describes in detail measures for improving the environment and increasing energy efficiency (The cost 10 points breakdown includes measures to protect the environment, energy efficiency works and alternative energy uses; the technical/ economic documentation includes EC certified materials; there is an energy audit with recommendations reflected in the technical solution; the latest standards for buildings’ energy efficiency are fulfilled; the project includes measures for proper disposal of medical waste, fulfilling the latest standards in the field, etc.). 5 points b. The project fulfills minimum legal requirements with respect to environmental protection and energy efficiency. 540 Regional Operational Programme 2014-2020 Priority Axis 7 – Development of health and social infrastructure Investment Priority 7.1.1 – Investments in health infrastructure to contribute to national, regional, and local development, reducing health status inequalities Semi-Competitive / Competitive Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 541 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 542 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION Weight POINTS 1. FINANCIAL SUSTAINABILITY 20% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly 10 estimated. The investment sustainability indicator (ISI)165 is points lower than 30% OR the expenditure sustainability indicator (ESI)166 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 7 between 30% and 40% OR the expenditure sustainability points indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 4 between 40% and 50% OR the expenditure sustainability points indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 1 point over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years. 2 QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 20% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the 25% DTD, and the DTD completion is planned for a maximum of 3 10 months after the approval of the financing request, it has all points 165 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 166 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 543 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION Weight POINTS the required approvals, and it has purchased the land (if needed) b. The applicant has launched the procedure for public procurement of the DTD, and the DTD completion is planned 7 points for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed) 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 public procurement activities. The results of the project and points the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the attributions of project team members and with the planning of public procurement activities. The results of the project 7 points 20% and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the 4 points attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 544 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION Weight POINTS 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 10 The proposed technical solutions involve the use of resources points efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 7 points The proposed technical solutions involve the partial use of 25% resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 4 points The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with allocated 10 estimated resources, and with cost breakdowns. Costs are points realistic (correctly estimated) and necessary for the implementation of the project. 15% b. Estimated cost breakdowns are partially clear, complete, and realistic, and closely tied to drawn parts. The budget is 5 points complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- 545 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION Weight POINTS or underestimated budget lines). 2.5 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; there is a 10 clear division of tasks, procedures, and a timeline of points monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 15% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear 5 points strategy. The organization does not have specific procedures for verifying/supervising the project team’s activities. 3. ECONOMIC IMPACT 10% 3.1 Promoting labor market (re)integration: a. The project makes a strong case for how facilities will be used 10 to facilitate beneficiaries’ (re)integration on the labor market. points b. The project makes a weak case for how facilities will be used to facilitate beneficiaries’ (re)integration on the labor market. 4 points 4. SOCIAL IMPACT 40% 4.1 Project relevance for the objectives of the priority axis and the key intervention area: quality of services a. The investment has a high potential to offer higher quality 10 social services. The need for the investment (demographic points trends, actual vs. needed capacity, share of marginalized groups) is certain. b. The investment has an average potential to offer higher 7 points quality social services. The need for the investment 50% (demographic trends, actual vs. needed capacity, share of marginalized groups) is relative. c. The investment has a small potential to offer higher quality social services. The need for the investment (demographic 4 points trends, actual vs. needed capacity, share of marginalized groups) is limited. 546 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION Weight POINTS 4.2 Project relevance for the objectives of the priority axis and the key intervention area: capacity of services a. The investment has the potential to offer improved social 10 services for at least 35 new beneficiaries. points b. The investment has the potential to offer improved social 30% services for 10-35 new beneficiaries. 7 points c. The investment has the potential to offer improved social services for less than 10 new beneficiaries. 4 points 4.3 Project addresses challenges faced by marginalized communities: a. Project improves to a significant degree the access to social 10 services for marginalized groups. points b. Project improves to a moderate degree (only partially) the access to social services for marginalized groups. 7 points 10% c. Project improves to a low degree the access to social services for marginalized groups. 4 points 4.4 Equality of opportunity and nondiscrimination: a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The 10 applicant identifies aspects/risks related to discrimination in points the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. 10% b. The applicant identifies aspects/risks related to discrimination in the context of the project and relies on 7 points some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. c. The applicant abides by minimum legal standards in this field. 5 points 547 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION Weight POINTS 5. ENVIRONMENTAL IMPACT 10% 5.1 Sustainable development and energy efficiency a. The project describes in detail measures for improving the environment and increasing energy efficiency (The cost breakdown includes measures to protect the environment, 10 the technical/economic documentation includes EC certified points materials, and fulfills the latest standards for buildings’ energy efficiency). b. The project fulfills minimum legal requirements with respect 5 points to environmental protection and energy efficiency. OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 548 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 549 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 15% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)167 is 10 points lower than 30% OR the expenditure sustainability indicator (ESI)168 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 7 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 4 points between 40% and 50% OR the expenditure sustainability indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. d. Operation and maintenance (O&M) costs are correctly 1 points estimated. The investment sustainability indicator (ISI) is over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years 2. QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 25% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the 10 points 25% DTD, and the DTD completion is planned for a maximum of 3 months after the approval of the financing request, it has all the required approvals, and it has purchased the land (if 167 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 168 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 550 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS needed). b. The applicant has launched the procedure for public 7 points procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed). 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the 7 points attributions of project team members and with the planning 20% of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the 4 points attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility 25% 551 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 10 points The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and 7 points according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility 5 points studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with 10 points allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. 15% b. Estimated cost breakdowns are partially clear, complete, and 5 points realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 552 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 2.5 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; there is a clear division of tasks, procedures, and a timeline of 10 points monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 15% b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear strategy. The organization does not have specific procedures 5 points for verifying/supervising the project team’s activities. 3. ECONOMIC IMPACT 20% 3.1 Project relevance for the objectives of the priority axis and the key intervention area: a. The project has a high potential to improve the quality of education and the area’s human capital development, serving a student population 20-30% above the national average. The 10 points project contributes to the full endowment of the facility with proper equipment and connecting it to high-speed Internet. b. The project has an average potential to improve the quality of education and the area’s human capital development, 7 points 75% serving a student population 5-10% above the national average. The project contributes to the full endowment of the educational facility with proper equipment and connecting it to high-speed Internet. c. The project has a low potential to improve the quality of 4 points education and the area’s human capital development, serving a student population below the national average. The project contributes to the partial endowment of the educational facility with proper equipment. 3.2 IT/Information Society a. The projects implements modern IT systems, solutions/ 10 points 25% applications, etc., significantly increasing students’ access to this infrastructure. 553 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The projects implements IT equipment at minimum 5 points standards. 4. SOCIAL IMPACT 30% 4.1 The real need for the educational infrastructure in the region/county/ locality where the project is implemented: a. The demographic and economic trends in the locality/county/ 10 points region (current situation and the forecast for the next 10 years) justify to a large degree the planned investment. The average use of the facility is at 70% minimum. b. The demographic and economic trends in the locality/county/ 70% region (current situation and the forecast for the next 10 years) justify to a moderate degree the planned investment. The average 7 points use of the facility is between 50% and 70%. c. The demographic and economic trends in the locality/county/ region (current situation and the forecast for the next 10 years) justify to a small degree the planned investment. The average use 4 points of the facility is under 50%. 4.2 Integration of poor and marginalized communities a. The educational facility is in a rural area and/or marginalized 10 points groups represent at least 50% of the total student population. 7 points b. The educational facility is in a rural area and/or marginalized 15% groups represent between 25% and 50% of the total student population. 5 points c. The educational facility is in a rural area and/or marginalized groups represent fewer than 25% of the total student population. 4.3 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of 15% opportunity also within public procurement contracts. The applicant identifies aspects/risks related to discrimination in 10 points 554 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS the context of the project and relies on specific mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. The project allows the community access to the recreation facilities of the school. b. The applicant identifies aspects/risks related to 7 points discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. c. The applicant abides by the minimum legal standards in the 5 points field. 5. ENVIRONMENTAL IMPACT 10% 5.1 Sustainable development and energy efficiency a. The project provides a detailed description on how to improve the environment and increase energy efficiency. Such measures include: thermal rehabilitation of buildings, 10 points intelligent heating and lighting systems, encouraging commuting by non-pollutant mean of transportation (for example docking stations for bikes), easy access to public transport, etc. 5 points b. The project abides by minimum legal standards in the field of environment and energy efficiency. 555 Regional Operational Programme 2014-2020 Priority Axis 7 – Developing health and social infrastructure Investment Priority 7.1.2 – Investments in social infrastructure that contribute to national, regional, and local development, reducing inequalities in terms of health status and ensuring the switch to community-based health services Semi-Competitive / Competitive Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 556 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Semi-Competitive / Competitive Scenario Technical and Financial Evaluation Grid Each sub-criterion below will be scored on a scale from 0 (minimum) to 10 (maximum). For each sub-criterion, there are 1-3 hypothetical scenarios for which a project could receive a corresponding score. These hypotheses are meant as reference points in the technical and financial evaluation of the project. The score assigned to a particular project, for a certain sub- criterion, will take into account where the project fits among the various scenarios presented. For each sub-criterion, a score of 0 to 10 or 1 to 10 (with one decimal point) can be assigned. The score for a main criterion is equal to the weighted average of scores corresponding to sub- criteria. The final score obtained by the project is equal to the weighted average of scores corresponding to the main criteria. For the “semi-competitive” scenario, projects are selected based on the first-in-first-out (FIFO) rule: any project receiving a score equal to or higher than 5 is eligible for financing, as long as there is still financing to be disbursed. For the “competitive” scenario, the same grid is used, except all projects submitted within a set time interval are ranked with respect to their absolute score. The highest scoring applications receive financing, going down the ranking until the financial allocation (for that time interval) is depleted. In either scenario, a score of 0 for any of the five major criteria results in the disqualification of the project. 557 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS 1. FINANCIAL SUSTAINABILITY 15% 1.1 The beneficiary proves that he/she has sufficient resources to cover planned investments and future operations and maintenance (O&M) costs. a. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI)169 is 10 points lower than 30% OR the expenditure sustainability indicator (ESI)170 is higher than 30%. The basis for calculating these indicators is the average for the last five years. b. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 7 points between 30% and 40% OR the expenditure sustainability indicator (ESI) is between 20% and 30%. The basis for calculating these indicators is the average for the last five years. c. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is between 40% and 50% OR the expenditure sustainability 4 points indicator (ESI) is between 15% and 20%. The basis for calculating these indicators is the average for the last five years. d. Operation and maintenance (O&M) costs are correctly estimated. The investment sustainability indicator (ISI) is 1 points over 50% OR the expenditure sustainability indicator (ESI) is under 15%. The basis for calculating these indicators is the average for the last five years 2. QUALITY, MATURITY, AND SUSTAINABILITY OF THE PROJECT 25% 2.1 Project preparedness and maturity a. The applicant has the detailed technical designed (DTD) or has completed the procedure for public procurement of the 10 points DTD, and the DTD completion is planned for a maximum of 3 25% months after the approval of the financing request, it has all the required approvals, and it has purchased the land (if needed). 169 (Capital Expenditures) / (Total Non-Earmarked Revenues). See endnotes for more details. 170 (Total Non-Earmarked Revenues for Current Expenses) / (Total Revenues). See endnotes for more details. 558 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The applicant has launched the procedure for public 7 points procurement of the DTD, and the DTD completion is planned for a maximum of 6 months after the approval of the financing request, it has some of the required approvals, and it has purchased the land (if needed). 4 points c. The applicant estimates the completion of the DTD in a maximum of 12 months after the approval of the financing request. 2.2 Methodology for project implementation a. The objectives of the project are clear and can be achieved. The activities of the project are clearly identified, detailed, and closely correlated with the implementation calendar, the attributions of project team members, and the planning of 10 points public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). b. The objectives of the project can be partly achieved. The activities of the project are clearly identified, detailed, and partly correlated with the implementation calendar, with the 7 points attributions of project team members and with the planning 20% of public procurement activities. The results of the project and the indicators are partly correlated with the planned activities and the set targets are partly feasible. Indicators are specific, measurable, achievable, relevant, and time- bound (SMART). c. The objectives of the project are not realistic. The activities of the project are clearly identified, detailed, but are not correlated with the implementation calendar, with the 4 points attributions of project team members, or with the planning of public procurement activities. The results of the project and the indicators are correlated with the planned activities and the set targets are not feasible. Indicators are specific, measurable, achievable, relevant, and time-bound (SMART). 2.3 Coherence of technical documentation a. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and 25% according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. 10 points 559 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS The proposed technical solutions involve the use of resources efficiently and sustainably. Technologies are new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. b. The data are sufficient, correct, and justified. The feasibility studies and the technical expertise are complete and 7 points according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions involve the partial use of resources efficiently and sustainably. Technologies are not entirely new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. c. The data are sufficient, correct, and justified. The feasibility 5 points studies and the technical expertise are complete and according to current norms and standards. Technical solutions are feasible and contribute to reaching objectives. The proposed technical solutions do not use resources efficiently and sustainably. Technologies are not new/innovative/ecofriendly. The drawn parts are complete and correspond fully to written parts. 2.4 General cost breakdown and budget of the project a. Estimated cost breakdowns are clear, complete, and realistic, and closely tied to drawn parts. The budget is complete and fully correlated with foreseen activities, with 10 points allocated estimated resources, and with cost breakdowns. Costs are realistic (correctly estimated) and necessary for the implementation of the project. 15% b. Estimated cost breakdowns are partially clear, complete, and 5 points realistic, and closely tied to drawn parts. The budget is complete but only partly correlated with foreseen activities, with allocated estimated resources, and with cost breakdowns. Costs are partly realistic (there are some over- or underestimated budget lines). 2.5 Applicant capacity to monitor the project’s implementation and post- implementation a. The applicant has a clear strategy for monitoring the project’s implementation and post-implementation phase; there is a 15% clear division of tasks, procedures, and a timeline of 10 points monitoring activities. The organization has procedures for verifying/supervising the project team’s activities. 560 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS b. The applicant has a series of procedures for monitoring the project’s implementation and post-implementation phase, and a calendar for monitoring activities, but there is no clear strategy. The organization does not have specific procedures 5 points for verifying/supervising the project team’s activities. 3. ECONOMIC IMPACT 20% 3.1 Project relevance for the objectives of the priority axis and the key intervention area: a. The project has a high potential to improve the quality of education and the area’s human capital development, serving a student population 20-30% above the national average. The 10 points project contributes to the full endowment of the facility with proper equipment and connecting it to high-speed Internet. b. The project has an average potential to improve the quality of education and the area’s human capital development, 7 points 75% serving a student population 5-10% above the national average. The project contributes to the full endowment of the educational facility with proper equipment and connecting it to high-speed Internet. c. The project has a low potential to improve the quality of 4 points education and the area’s human capital development, serving a student population below the national average. The project contributes to the partial endowment of the educational facility with proper equipment. 3.2 IT/Information Society a. The projects implements modern IT systems, solutions/ 10 points applications, etc., significantly increasing students’ access to 25% this infrastructure. b. The projects implements IT equipment at minimum 5 points standards. 4. SOCIAL IMPACT 30% 4.1 The real need for the educational infrastructure in the region/county/ locality where the project is implemented: 70% a. The demographic and economic trends in the 10 points 561 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS locality/county/ region (current situation and the forecast for the next 10 years) justify to a large degree the planned investment. The average use of the facility is at 70% minimum. b. The demographic and economic trends in the 7 points locality/county/ region (current situation and the forecast for the next 10 years) justify to a moderate degree the planned investment. The average use of the facility is between 50% and 70%. 4 points c. The demographic and economic trends in the locality/county/ region (current situation and the forecast for the next 10 years) justify to a small degree the planned investment. The average use of the facility is under 50%. 4.2 Integration of poor and marginalized communities a. The educational facility is in a rural area and/or marginalized 10 points groups represent at least 50% of the total student population. 7 points b. The educational facility is in a rural area and/or marginalized 15% groups represent between 25% and 50% of the total student population. 5 points c. The educational facility is in a rural area and/or marginalized groups represent fewer than 25% of the total student population. 4.3 Equality of opportunity and nondiscrimination a. The applicant has a nondiscrimination policy applicable to the project, and there are mechanisms for ensuring equality of opportunity also within public procurement contracts. The applicant identifies aspects/risks related to discrimination in 10 points the context of the project and relies on specific mechanisms 15% to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. The project allows the community access to the recreation facilities of the school. b. The applicant identifies aspects/risks related to 7 points 562 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Semi-Competitive / Competitive Scenario MAX CRITERION WEIGHT POINTS discrimination in the context of the project and relies on some mechanisms to address these potential issues. The project involves the creation of facilities/ adapting the current structure to facilitate disabled people’s access. c. The applicant abides by the minimum legal standards in the 5 points field. 5. ENVIRONMENTAL IMPACT 10% 5.1 Sustainable development and energy efficiency a. The project provides a detailed description on how to improve the environment and increase energy efficiency. Such measures include: thermal rehabilitation of buildings, 10 points intelligent heating and lighting systems, encouraging commuting by non-pollutant mean of transportation (for example docking stations for bikes), easy access to public transport, etc. 5 points b. The project abides by minimum legal standards in the field of environment and energy efficiency. 563 Regional Operational Programme 2014-2020 Priority Axis 9 – Investments in education, competencies, and lifelong learning Investment Priority 9.1 – Learning and professional development infrastructures Semi-Competitive / Competitive Scenario OECD Intergovernmental Fiscal Categories OECD – Intergovernmental Fiscal Categories Explanation (1) Own Revenues Revenues from own sources corresponding to a particular public organization (e.g., City Hall, County Council, etc.) (2) Shared Personal Income Tax Set proportion of the personal income tax that is redirected from the national level (where it is collected) to the corresponding public institution at the local level (3) Non-Earmarked Grants Sums allocated to a local public authority that can be spent on any chosen priority (4) Earmarked Grants Sums allocated to a local public authority that are to be used for specific purposes (e.g., teacher salaries) (5) Other Revenues Revenues from other sources (6) Non-Earmarked Revenues = (1) + (2) + (3) + (5) (7) Non-Earmarked Revenues for Capital Expenses = (6) + EU reimbursements – Capital investments 564