(',.()VERNMENT OF ROMANIA MINISTRYC 'I! ()PUENT ANOP\. N 'RATlON 84328 The findings, interpretations, and conclusions expressed in this report do not necessarily reflect the views and position of the Executive Directors of the World Bank, the European Union, or the Government of Romania. Project co-financed from the European Regional Development Fund through the Regional Operational Programme 2007-2013. December 20, 2013 Table of Contents List of Acronyms 7 Phase-Specific Support 41 Introduction 9 Phase I: Pre-Application 41 Phase II: Application, Evaluation, Horizontal Cross-Themes 27 43 and Contracting A. Regionalization 27 A. Applicant Guides 43 Table of Contents B. Eligible vs. Ineligible Expenses 43 B. Human Resources 28 C. Helpdesk 44 D. Training 45 C. Access to Finance 33 E. Ongoing Communication 45 D. Electronization 35 Phase III: Implementation 47 E. Targeted Support: Engaging Marginalized A. Public Procurement 47 Communities 39 B. The Design Stage: Technical Documentation 48 C. The Execution Stage: Contracts, Monitoring, and Reimbursements 50 Phase IV: Post-Implementation 52 Top Recommendations 54 5 List of Acronyms ACSI Authority for the Coordination of Structural Instruments CLLD Community-Led Local Development EC European Commission EU European Union GEO Government Emergency Ordinance List of Acronyms IB Intermediate Body MA-ROP Managing Authority of the Regional Operational Programme MA Managing Authority MC-ROP Monitoring Committee of the Regional Operational Programme MEF Ministry of European Funds MRDPA Ministry of Regional Development and Public Administration RDA Regional Development Agency ROP Regional Operational Programme SMIS Single Management Information System 3 7 The Regional Operational Programme (ROP) is a Relative performance of EU structural programs in Romania (as of November 30, 2013) top performer in terms of 120% 107% 118% absorption rates, in the Romanian context, despite its 103% 100% 85% complex and diverse portfolio of 82% 80% 80% 78% 60% 48% 44% 45% 40% 46% projects. 40% 33% 30% 27% The ROP addresses a wide-ranging set of needs at the level of each of the 22% 22% 20% 19%18% 18% 21% eight regions, covering areas like urban development, transport infrastructure, social infrastructure, business development, and tourism. Some projects are relatively small and easy to implement, while others are large, complex 0% ROP Environment Transport Competitiveness Administrative Human Technical infrastructure works, with cumbersome technical documentation and lengthy capacity resources assistance procedures. As such, it is remarkable that, as of November 30, 2013, the ROP % contracted of total ROP funds available % paid to beneficiaries % paid by EC to Romania remains the country’s top performing operational program, particularly when it comes to payments from the EC, which stood at 44.2% of the total ROP Source: Ministry of European Funds (www.fonduri-ue.ro) allocation. 9 Proportion of beneficiaries who are “highly satisfied” and “very satisfied” with the RDA/MA support across different stages Interview and survey feedback 100% is overwhelmingly positive 90% 82% in terms of beneficiaries’ 80% satisfaction with the ROP, particularly when compared to other 70% 62% 62% 60% 60% operational programs (OPs). 52% 49% 47% 46% 50% 40% Beneficiaries welcome their collaboration with the Intermediate Bodies (IBs) 30% and the Managing Authority (MA). There is a sense of shared partnership and a common understanding that all stakeholders involved genuinely want 20% the program to succeed. The ROP owes much of its relative success to the 10% fact that MA and IB staff: are closer to program applicants and beneficiaries; demonstrate enhanced understanding of what needs to be done on the 0% ground; rely on experienced people who know how to design and implement RDA MA RDA MA RDA MA RDA MA optimal solutions, working in close partnership with beneficiary teams, many 1. Promoting calls for projects 2. Evaluation of application 3. Training sessions 4. Project monitoring of whom they know and trust; have greater accountability for outcomes and results vis-à-vis beneficiaries and the general public; and, finally, they benefit from quicker, more effective feedback loops from beneficiaries. 11 The ROP includes a very diverse portfolio of beneficiaries and projects The first guiding principle of the current analysis remains: “first, do no harm.” The ROP components that are functional and efficient in supporting applicants and beneficiaries should be strengthened to leverage accumulated institutional knowledge and to minimize disruptions – a high level of predictability is valuable, in of itself, as a way to provide a stable environment with clear rules for applicants and beneficiaries. Equally important, it should be clear that this report is about how to assist beneficiaries, taking as a given their current capacity. This is not to say that there is not much to do on beneficiaries’ end to boost their ability to design and implement ROP projects. But the key topic of this analysis remains the support available to ROP applicants and beneficiaries. 13 Beneficiary perceptions regarding the importance of various obstacles related to ROP projects This is neither to say that Excessive bureacracy assistance mechanisms currently available to ROP & documentation Delays in project evaluation applicants and beneficiaries & selection Delays in reimbursements Lack of capital for co-financing & cash flow are sufficient and flawless, Frequent legislative changes nor that a program’s performance is simply a Inadequate public procurement framework Insufficient prefinancing matter of superior absorption rates. Low quality technical documentation Lack of high-quality consulting services A survey was sent out to beneficiaries in all eight regions to inquire about the Lack of transparency main obstacles faced in accessing ROP funding and implementing projects. Of the total 470 responses, the proportion of people who characterized Low quality of applicant guides various obstacles as “extremely important” and “very important” varies from a staggering 81.5% for “excessive bureaucracy and documentation” and 80% Poorly trained AM staff for “frequent legislation changes” to only 31.7% for “poorly trained RDA staff” (see figure on the left). Also, over half of the respondents identified additional Poorly trained RDA staff obstacles as “extremely important” and “very important”, in the following order: 0 50 100 150 200 250 300 350 400 450 500 delays in project evaluation and selection (73%), delays in reimbursements (69%), lack of capital for co-financing and cash-flows (68%), inadequate public Extremely important Very important Relatively Important procurement framework (62%), insufficient prefinancing (56%), and poor- quality technical documentation (52%). Hardly important Unimportant N/A 15 In addition, some important issues fall beyond the immediate scope of this report, although they are worth mentioning as part of the broader context of the ROP’s implementation. For one, a better effort is needed to improve the selectivity and prioritization of ROP investments. It is not always clear that current ROP projects match local needs and are aligned with broader regional and national-level strategies, but the selection process (based on the first-in-first-out rule) has often privileged absorption instead of strategic impact. Moreover, there are concerns that local authorities will not be able to shoulder the operation and maintenance (O&M) costs related to finalized ROP projects. Finally, national budget funding for regional and rural development continues to duplicate ROP resources through the newly established National Program for Local Development, although selection and implementation procedures are generally less rigorous than under the ROP. There is a need to ensure coherence between national-budget projects and EU-financed ones. Such key issues, albeit beyond the scope of this analysis, are addressed in other ongoing studies, including by the pending ROP’s intermediary evaluation, which includes highly useful data on projects’ impact, in many ways offering a complementary analysis to the current work. 17 While there is a long, well-known Beneficiary perceptions of benefits/costs of ROP projects 13% list of issues related to EU-funded Benefits exceed costs projects’ implementation, the good news is that most beneficiaries Costs exceed benefits continue to believe in the net 87% positive value of the program. 87% of respondents to the survey answered that the costs involved are lower than the benefits of ROP financing. As for beneficiaries’ appetite for applying for future ROP funding, 83% of the answers were positive, with only 4% rejecting the idea of seeking this type of financing again – a very encouraging sign for the ROP’s attractiveness in the next period. While the perceived “benefit-cost Beneficiary responses to whether they would apply for future ROP financing ratio” remains greater than 1, the MA and the IBs teams should aim to increase it further, minimizing hurdles for beneficiaries, to the full extent possible. 4% 13% Yes No I don’t know yet 83% 19 Perspectives from other Perspectives from Member States* Romania** “Controls are increasing and we are obliged to take on more advice from “Much of what we do on a daily basis is answer questions and provide In this context, the first other departments. Audit overkill is significant. I have examples of 6 documents to [auditors]. And they check the same things over and over recommendation is to shift the paradigm to a “beneficiaries projects being controlled 6 times by 6 again, they check each other. bodies!” – Director, City Hall – Member State official “We’re all presumed to be fraudulent beneficiaries of EU funds. What first” model, which aligns perfectly with the aim of the current report. “The Court of Auditors is controlling in happened to ‘innocent until proven areas which should be the job of our guilty’? It’s a Kafkian story: they civil servants. We don’t want to spend question everything! If they find too much time on this. The Managing something, the smallest thing, To be sure, there is an inherent tension between the two critical functions Authority spent 2 months responding instead of guiding us to avoid making that the ROP system must fulfill in relation to applicants and beneficiaries: to the Court of Auditor’s requests.” the same mistake in the future, they monitoring and control versus partnership and support. Evidence from field – Member State official impose [financial] corrections.” interviews with beneficiaries and other key stakeholders suggests that the ROP – Public-sector ROP beneficiary system would benefit from balancing the current rigorous focus on control and “The Commission had innovative sanctions with stronger efforts around partnership and support. Institutionally, actions programs 3 years ago where “We’re barely keeping up with the IBs are better placed to support applicants and beneficiaries on the ground, financial management and audit everything going on. And the system while control functions more naturally rest with central authorities (the MA, the rules were not applied so rigidly. This is slanted against its best people. Audit Authority, etc.). allowed a laboratory for implementing Why? Because we struggle to implement these projects, we care, we more risky and innovative projects want to do something good. And we with not so much money. In 2007- get punished in return. As a politician 13, we don’t dare to take on risky it’s better to stay put and wait for the projects.” next election than deal with all the – Member State official headaches of EU money..” – County Council President * Qtd. in Mendez and Bachtler, “Administrative Reform and Unintended Consequences: An Assessment of the EU Cohesion Policy Audit Explosion,” Journal of **Source: field interviews, January- May 2013 European Public Policy, 18:5, 758-760 Striking the right balance between assistance and control of beneficiaries is a common challenge 21 Providing adequate beneficiary support mechanisms at the MA and the IB level depends on two key variables for each applicant/beneficiary: applicable selection model and portfolio value. Regarding the former, dedicated funding for a project or set of projects (as in the case of growth poles) can and should be complemented by customized, close engagement by MA/IB teams, throughout the project cycle. The aim is to ensure technical assistance that is fully aligned with specific beneficiary needs and helps deliver the most impact possible. By contrast, if the selection is done on a competitive basis, support should be standardized. 23 Optimal support mechanisms vary across applicants/ Typology of ROP support mechanisms beneficiaries and corresponding axis/priority area/selection models. Dedicated Extensive, The portfolio value is equivalent to the total number of ROP projects customized multiplied by their average budget, and there are two relatively distinct support project groups: public-sector institutions implementing a large, complex set of projects; and public, private, or non-profit beneficiaries of relatively smaller, simpler investments. Importantly, this approach is fully aligned with Type of selection model the recommendations made by the EC in its October 2012 position paper regarding Romania’s preparations for 2014-2020, as follows: “Whereas tailored assistance could be directly channeled to larger beneficiaries, accompanied by Efficient, sound ownership and appropriate endogenous reinforcement and restructuring, standardized streamlined assistance should be provided to sectors sheltering a broader support scope of smaller and more heterogeneous beneficiaries (tailored guidance, Competitive help desk, dissemination of standardized documents, etc.).”1 Small, simple Value(complexity) of ROP portfolio Large, complex (# projects * average budget) 1 “Position of the Commission Services on the development of Partnership Agreement and Programmes in Romania for the period 2014-2020,” Ref. Ares(2012)1240252 - 19/10/2012, p.37 25 Horizontal Cross-Themes A. Regionalization A. Regionalization Romania’s Planning Regions Most beneficiaries are in favor of the Government’s continued efforts to push forward the plan for devolving increased responsibilities to the regional level, although some elements will not apply before the 2021-2027 programming period. First, one key advantage of a regional-focused ROP was identified as a higher level of responsiveness to the regional-level needs and priorities. Second, beneficiaries note that the MA in Bucharest is inevitably more removed from the situation on the ground, unlike RDAs, which are able to customize their assistance and intervene more quickly, as needed. Beneficiaries would welcome a regional MA as a best-of-both-worlds option: it would be close to the ground, knowledgeable of their specific situation, able to react quickly, and it would also hold the decision-making power to adopt certain needed measures and provide informed answers. Perhaps most importantly, regionalization promises to deliver truly regional projects by ensuring proper coordination of the political and administrative levels, as long as it does not replicate the current dynamics that do not encourage projects across multiple regionalization jurisdictions (e.g., cities, counties, etc.). In particular, one possibility promises to deliver Cross-Themes worth considering further for 2014-2020, based on interview truly regional projects Horizontal feedback and in conjunction by ensuring proper with the project selection models workstream, is a “regionalization- coordination of light” option, with an ROP system adapted to different types of the political and projects, depending on their administrative levels magnitude. 27 9 B. Human Resources positions, with adequate, meritocratic hiring practices that bring much needed skills from key areas (economics, civil engineering, B. Human Resources and legal affairs). One final recommendation for leveraging internal resources is for the MA and the IBs to capitalize on the existing knowledge of experienced beneficiary staff. One possibility is to bring them together under an ROP Corps of Practitioners (ROP- The ROP’s success depends first and foremost on the people CP), which would meet periodically and send suggestions to IBs, who make the system work – particularly those responsible for the MA, and the MC-ROP, as necessary. It is expected that the designing and implementing projects. Corps could quickly become a critical component of the ROP’s Without their passion, engagement, and persistent involvement, even continuous learning and adaptation system, and would ensure the best institutional set-up would be ineffective in reaching the ROP’s an even higher degree of the system’s responsiveness to the fundamental objectives. problems identified in the field. Moreover, members of the ROP- CP could also be selected to become trainers of staff involved in the legal framework Internal human capital EU-funded projects. As such, the highest priority for constraining the the ROP system is to ensure low hiring of public turnover of critical internal staff. First, this requires creating the servants should conditions for adequate pay to project teams in public-sector institutions, be reconsidered throughout an investment’s cycle, and to ensure that units not only during the implementation stage. In addition to expanding the focused on EU- bonus system, options should be explored for increasing the salary financed projects are base in line with performance. Equally adequately staffed important, many of the units working on EU-financed projects remain Cross-Themes understaffed and overburdened with Horizontal responsibilities related to managing large portfolios. This is particularly true of the recent period, when teams have had to both accelerate the implementation of ongoing projects to complete them before the 2015 deadline and also prepare new projects for the 2014-2020 budgeting cycle. In this context, the legal framework constraining the hiring of public servants should be reconsidered to ensure that units focused on EU-financed projects are adequately staffed at all times. All this should be based on a proper system that puts the right people into the right 29 Beneficiary perceptions of possibly requiring FIDIC contract forms for ROP-financed projects B. Human Resources 100% EXternal human capital 90% At the same time, it is unrealistic to require ROP beneficiaries 80% to complete all tasks internally – in fact, outsourcing certain services is both encouraged and viable, given than such 70% expenses are typically eligible to be covered by EU funds. 60% The key challenge in managing 50% publishing the names external resources is that the market is not fully mature or 40% of consulting firms transparent, so selecting the right 30% who prepare and/or consulting company is a daunting task for most beneficiaries of ROP 20% manage projects and funds. Solutions to increase the market’s transparency include 10% enabling user-feedback publishing the names of consulting mechanisms would 0% firms who prepare and/or manage <2 2-5 >5 projects and encouraging peer- improve the quality of Number of ROP projects in beneficiary portfolio feedback mechanisms. In addition, contracts with external resources services provided should be strengthened to protect the interests of applicants All beneficiary types support the idea of an online ratings system for and beneficiaries and hold consultants accountable for the service providers quality of their work, contributing to, among other things, the reduction of errors in filling out financing applications and Public institution 51% 26% 13% reimbursement requests. Cross-Themes Private company 41% 31% 16% Horizontal NGO 32% 26% 32% 0% 20% 40% 60% 80% 100% Extremely useful Very useful Relatively Not useful Hardly useful Not useful N/A 31 C. Access to Finance C. Access to Finance Adequate access to finance throughout all stages of a project is of paramount concern for interviewed beneficiaries, across all sectors (public, private, and non-profit). “connect” Field research observations suggest that several banks in beneficiaries to Romania, including some of the major ones, have specific financing institutions products targeting the needs of and representative project beneficiaries. Recently, the market for such instruments associations has grown in size and complexity, but many beneficiaries remain unaware of available opportunities, while, for their part, a number of banks continue to regard beneficiaries of structural funds with high skepticism. As lending is expected to resume along with the recovering economy, the main anticipated need is to “connect” beneficiaries to financing institutions and representative associations (e.g., the Romanian Association of Banks). This is a role that the MA and the IBs can easily play, acting as a bridge between the demand and supply of financing needs for ROP projects. Banks could also Cross-Themes play a role in assessing the viability of specific proposals (e.g., as part Horizontal of the cost-benefit analysis), based on their accumulated experience and knowledge of market dynamics. 33 An overwhelming majority of beneficiaries welcome the introduction of D. Electronization e-documentation D. Electronization 3% 2% There are several important advantages of an e-based system: speed, cost savings, and accuracy. 5% the MA can consider Recognizing this, the EC has Extremely useful promoted e-cohesion strongly, 19% Very useful developing its own proposing that by December 31, 2014, beneficiaries should 40% Relatively useful electronic system be provided with a way to communicate only electronically to cover the ROP’s Hardly useful with MAs and IBs. The Authority basic needs Not useful for Coordinating Structural 31% N/A Instruments (ACSI) has launched a process for developing the mySMIS application, which would serve as an interface with beneficiaries, complementing the current SMIS backend database. What remains to be determined is the quality of the final product and its implementation, but in any case it is recommended to leave ample time for consultations, The heavy reliance on hard copies has generated serious storage problems beta testing, and piloting. While the need for avoiding duplication is paramount, if the mySMIS launch will be delayed much further, the MA-ROP can consider developing its own simplified electronic system to cover the basic needs of the program. The full project cycle should become e-based. Cross-Themes As a first step, the introduction of electronic forms for the application phase would reduce bureaucracy and associated Horizontal costs, and help prevent beneficiaries from making mistakes (e.g., by only allowing certain answers), ensuring smoother completion of projects further down the line. The evaluation process could also take place on an easily accessible electronic platform. Under this setup, there would be no need to pay the extra costs of transporting evaluators to the RDA headquarters for the desk evaluation. Evaluators could perform a review of submitted 35 Beneficiaries have noted several challenges faced under the status quo: High volume of hard-copy documents Nearly all key exchanges the full project cycle documents remotely and then a single on-the-spot verification, between the MA, the IBs, and beneficiaries are based on hard-copy should become D. Electronization so a larger proportion of the documents. This holds for the entire project documentation (e.g., proposal, budget could be dedicated to annexes, amendments, etc.), as well as for all communications. e-based. covering actual fees and ensuring higher-quality evaluations. During the implementation phase, all Duplication The requirement to print multiple copies of the same document required exchanges of information, including reimbursement increases administrative and transportation costs. It is not uncommon for requests, progress reports, and supporting documentation, beneficiaries to send, literally, a van full of thousands and thousands of would be submitted and verified electronically. This would avoid pages. This is true of the application, as well as of subsequent reimbursement requests, some of which may contain lengthy files related to public procurement duplication (e.g., submitting the public procurement file with procedures. On that note, interviewed beneficiaries express disbelief that it is every reimbursement request). necessary, for instance, to send the entire public procurement documentation for every single reimbursement request. Importantly, an e-based ROP does not mean that all in-person interactions would be eliminated. Storage Field visits offered many opportunities to see the tangible result of excessive bureaucracy. There are rooms filled with project-related documents still, not everything Above all, verification missions in the field will be needed to ensure can be done online and often a single project will take up several shelves, particularly if it required a that documents submitted complex technical project. This system is not only costly, but also risky in terms of electronically are exact copies of losing or misplacing documents and, because everything is in hard copy, some originals – under this system, beneficiaries would be required to beneficiaries do not take the time to save an electronic copy of the documents. keep all original paperwork in an easily accessible location and make it available whenever needed by auditors. At the same time, not all MA-IB-beneficiary interactions can happen electronically. “if we do not see it in writing on paper, with a stamp In-person trainings, for instance, have proved much more useful next to it, it is not to be trusted.” than online seminars. The same goes for field visits by evaluators to verify feasibility studies against the reality on the ground. Mentality The principle enunciated by many beneficiaries and RDA representatives is the following: “if we do not see it in writing on paper, with Cross-Themes a stamp next to it, it is not to be trusted.” This is a direct consequence of the lack of experience with electronic forms, as well as a byproduct of a system Horizontal that has focused disproportionately on audits and control mechanisms. Interviewed beneficiaries have expressed concern that they need to always be insured against potential risks. As a result, they write formal addresses to the IBs and the MA and keep the entire trail of responses to be able to show, if need be, that they did everything in their power at a particular junction in a project’s implementation. This is understandable and often a good, prudent practice, but it continues to add to the volume of paperwork and to the work of all parties along the chain (beneficiary, IB, and MA staff). 37 E. Targeted Support: communities through their ROP CLLD approaches projects. Also, MA/IB teams Engaging Marginalized should develop targeted support are expected for members of marginalized to increase the communities to apply as potential Communities Engaging Marginalized Communities beneficiaries of ROP funds. Along effectiveness the same lines of bottom-up participation, the MA-ROP and A key priority for Romania’s development in 2014-2020, supported and efficiency the IBs should work together to by the EC, other international donors, and the Romanian Government, is the integration of marginalized communities and of projects and evaluate whether the Community- Led Local Development (CLLD) the promotion of inclusive growth. service delivery, and instrument is appropriate for the ROP 2014-2020. The CLLD holds the ROP has In particular, Roma integration has to be a primary focus of any efforts aimed contribute to social great promise for a program like the ROP, which is well positioned not generated at sustaining Romania’s development, inclusion through Support: to take advantage of its closer taking into account the community’s substantial impact strong potential to make a contribution empowerment. presence to beneficiaries on the ground. More generally, CLLD on critical issues to the country’s absorption of EU funds approaches are expected to and long-term growth. Despite all this, increase the effectiveness and efficiency of projects and service like social integration, ROP projects targeting Roma and E. Targeted delivery, and contribute to social inclusion through empowerment. other marginalized communities are labor market few and remain mostly concentrated on rehabilitating infrastructure like participation, and social centers. Without downplaying school enrolment of the importance of these initiatives at the local level, the reality is that the Roma. ROP has not generated substantial impact on critical issues like social Cross-Themes integration, labor market participation, and school enrolment of Roma. Horizontal To address the current situation, there is a need to improve the ROP’s programming and support mechanisms to deliver more impact for marginalized communities in urban areas. This will depend on adopting targeted selection criteria and encouraging beneficiaries to engage Roma and other marginalized 39 Phase-Specific Support Phase I: Pre-Application Phase I: Pre-Application In the pre-application phase, assistance to potential applicants should Actual ROP Eligible ROP target two key areas: programming and pipeline development (i.e., applicants applicants awareness-raising and project development support). On programming, it is recommended that working groups of beneficiaries stay engaged throughout the entire process, which should aim to become increasingly transparent and participatory. Beneficiary preferences should All Successful also be reflected in the program’s implementation framework (e.g., specific potential “graduates” of applicants Actual ROP ROP programs procedures, definition of eligible vs. ineligible expenditures, etc.). It is also for ROP beneficiaries (completed recommended that those decision-makers responsible for drafting the PA, financing projects) the ROP, and related documents experiment with online consultation tools. As for project development efforts, it is recommended that the IBs take a more proactive stance in working with potential beneficiaries to define and operationalize viable, impactful ideas. Additional best practices include Pre-Application Application Implementation Post-Implementation appointing a liaison person and improving preapplication training sessions Phase Phase Phase Phase to ensure the submission of error-free, high-quality financing applications that contribute to a smoother implementation further down the road. Phase-Specific Support 41 Phase II: Satisfaction with RDA support during the application phase is generally Application, Evaluation, and Contracting high Application, Evaluation, 80.00% 60.00% Five areas are most relevant for providing enhanced assistance and Contracting 71.28% to ROP applicants at this point in the process: applicant guides, 58.61% eligible expenditures, helpdesk activities, training, and ongoing 40.00% 47.37% communication. 20.00% A. Applicant Guides 0.00% Beneficiaries generally appreciate the guides as comprehensive, Public institution Private company Non-profit useful, and sufficient for understanding the process of filling out and submitting a financing application, but there are also two main areas of improvement. First, repeated changes should be minimized, particularly on finalized Phase II: Even in the most modern classrooms, some of the old chairs were kept guides, to maintain predictability. This would avoid indirectly penalizing proactive beneficiaries, the “early-risers,” who are forced to make adjustments and spend additional resources to align their proposals to the requirements from the updated guides. Second, two key elements should be added to the guides to improve their transparency: the potential timeline (minimum and maximum duration) of each step in the process (including the contracting phase), based on the MA-IB audit trails (“pistele de audit”); and a full section on “beneficiary rights and obligations,” as included in Phase-Specific the financing contract. Even though the latter is an annex to the applicant guide, to avoid any confusion among applicants, it is recommended to maintain the same content for both documents referenced here. Support B. Eligible vs. Ineligible Expenses For the ROP, the list of eligible expenses is defined through a common order (“the Eligible Expenses Order” – EEO) by the Minister of Regional Development and Public Administration and the Minister of Finance. 43 Interviewed ROP beneficiaries urge decision-makers to either (A) improve their local RDA representative first, in the county or regional office, and the Eligible Expenses Order (EEO) based on their feedback; (B) eliminate the only turn to the MA staff under special circumstances, when they feel the EEO and rely on applicable national level legislation instead; or (C) define need to reach someone in the Ministry. The helpdesk function should be only ineligible expenditures. Given the accumulated experience in the current maintained and strengthened going forward. period, the ROP 2014-2020 would need to take into account extensive inputs Application, Evaluation, from beneficiaries regarding the revision and expansion of EEOs (option A). At the same time, it is doubtful that one can regulate every single project-specific D. Training instance, so options B and C appear more feasible. RDAs and the MA organize post-contracting training sessions with beneficiaries who have signed contracts or have started project and Contracting implementation. What is more, the current EEO establishes a maximum value of 5% for The importance of these sessions for ensuring smooth project completion expenses related to project design and technical assistance (Article 3), is reflected in both survey results and interview feedback from field visits. which has contributed to the low quality of technical documentation and Going forward, the MA and the IBs should continue to organize this training construction work. and improve its delivery, keeping in mind the need for: effective timing International benchmarking suggests that this artificial cap should be raised and coordination; increased frequency; and enhanced customization for to at least approximately 10% of the project’s value. One consequence different types of beneficiaries (e.g., based on their level of experience with of inadequate budgeting relates to the performance of construction site EU funds). supervisors, who often serve in this role for multiple contracts at once, even in different localities. In some cases, they endorse progress and inspection reports without actually verifying the situation on site, which later turns against E. Ongoing Communication Phase II: the beneficiary in the form of unanticipated, ineligible additional expenses. A recurring theme in survey responses and interviews is the lack of Increasing the cap to a reasonable level would help facilitate high-quality transparency and predictability of the evaluation and contracting services by both technical designers and construction site supervisors, leading process. to better, smoother implementation. Beneficiaries understand procedural steps and selection models, but they have less visibility into the timeline and progress of their application C. Helpdesk throughout the process and particularly once their documentation is sent to the MA-ROP. A solution with benefits on all sides would be to increase Each RDA has established a functioning helpdesk that offers assistance the transparency of the flow of documents, developing a simple online Phase-Specific to interested ROP applicants, with positive results to date. tracking system. Applicants would be able to access the platform to For their part, IB-level helpdesks are able to act as a first point-of-contact for determine where their application is at any one point in time, how long the applicants who need guidance around how to fill out an application and how process is expected to take, and who the main point of contact is at each to obtain the required supporting documentation, functions that the MA has Support step in the process. This could increase the timeline’s predictability and delegated to RDAs. The closer this assistance gets to beneficiaries, the better transparency, mitigate concerns among beneficiaries, and free up capacity it can address their needs in a timely manner, which suggests that most of the in the MA and RDAs helpdesks, which could quickly refer applicants to the helpdesk activities should remain concentrated at the IB level. Sure enough, online system. It would also help improve accountability up and down the feedback from interviews confirms this hypothesis: most beneficiaries contact process chain within the MA and the IBs, ensuring visibility around bottlenecks and capacity constraints. 45 Phase III: Implementation Phase III: Implementation A. Public Procurement This is by far the greatest concern expressed by ROP beneficiaries in field interviews centers on issues related to public procurement and associated corrections. Over 70% of public beneficiaries rate the public procurement framework as an “extremely” or “very important” obstacle. Financial corrections related to public procurement procedures are also widespread among local authorities. The key issue is that Romanian legislation on public procurement procedures related to EU-financed projects failed to incorporate until 2011, 4.5 years into the ROP’s implementation, several key elements (e.g., clear definitions of restrictive criteria, applicable corrections, etc.). The total sum imposed in penalties by December 2012 had exceeded EUR 300 million. By far the most common problem is related to “illegal qualifying criteria” – these are also referred to as restrictive/disproportionate criteria and often result from requesting a particular level or type of experience from the bidders. Beneficiaries characterized the vast majority of these errors as purely non-intentional or wrongfully imposed. 80% of sanctioned beneficiaries challenged the corrections and 41% have sought remedy through legal court action, often arguing that retroactive application of Phase-Specific law is unconstitutional. Many of the initial decisions from Appellate Courts are favorable to beneficiaries, though most final definitive verdicts by the Supreme Court are only expected in 2014 or 2015. Support The key issue of public procurement is the suboptimal, inefficient ex-ante and ex-post verification system, with a chronic lack of accountability of all stakeholders involved, except for beneficiaries, who are left to bear the entire burden of potential mistakes. 47 9 Interviewed beneficiaries call attention to the urgent need of publishing an A total of 48% of interviewed subjects consider this factor to be an official collection of cases and their related corrections. Maximum transparency extremely/very important obstacle against effective absorption of ROP is required in order to help beneficiaries avoid making the same mistakes in funds. Simply put, beneficiaries are currently exposed in their relationship the future. A complementary measure is to put together lists of acceptable with both project designers and contractors: the latter seek to point out selection criteria. Equally important, there need to be clear-cut standards for faults in the technical project, hiking the agreed price set through the Phase III: Implementation which level of sanction applies in a particular case, with consistent application public procurement process; the former are reluctant to agree to changes of these rules regardless of which body makes the proposal for corrections. of the project and put the blame on the workers in the field for failing Penalties should be applied only as a last resort and after all verifications have to apply the technical documentation. This eventually leads to significant been carried out by specialized auditors with the right set of skills (including ineligible expenditures when there are differences between the technical technical expertise). Moreover, as far as control institutions are concerned, they project (TP) and the situation on the ground during implementation. EU- should take responsibility for their recommendations and share the costs of level regulations do not accept such discrepancies as “unpredictable.” potential corrections if they failed to prevent beneficiaries from going forward with a faulty public procurement procedure, or have reimbursed them despite The leading causes for this situation revolve around lack of capacity obvious irregularities. and inadequate accountability mechanisms. For their part, beneficiaries rarely have the available resources to properly B. The Design Stage: Technical Documentation check projects submitted by designers. Technical experts are often Beneficiaries often mention the low quality of technical documentation as selected based on the lowest price criterion (a broader issue of the current the leading cause of implementation delays and ineligible expenditures. public procurement framework). The ROP system tried to counteract these weaknesses by including a review of technical projects during the evaluation stage. This has not always worked, as it puts more pressure on beneficiaries to finish studies before the six-month deadline and, also, the An oversized boiler, due to a faulty technical design, creates budgetary pressures for the school and local authorities in Livada, Cluj County technical project evaluators mainly perform an “administrative” check on the documentation to see if all supporting documents are included, without having the capacity to perform in-depth checks on technical solutions, the lists of quantities, etc. MA and IB teams of experts should help beneficiaries avoid Phase-Specific discrepancies between the technical documentation and the situation on the ground by endorsing standard contract forms. These would be part of the public procurement documentation and help Support ensure that qualified, professional bidders enter the competition. Best practices would include the following: a clear stipulation of the designer’s responsibility to deliver a high-quality output; a complete timeline for every deliverable, including scheduled payments, through the implementation stage; a clear requirement that all property rights over the technical 49 9 documentation belong to the beneficiary; and a direct relationship with project who must be held accountable for mistakes resulting from their own work verifiers, who should be contracted and paid by beneficiaries, and should be (e.g., errors in submitting reimbursement requests). For all such actors, accountable for approving deliverables that do not abide by strict standards. increased accountability should be complemented by a requirement to have Another solution for reducing friction among designers, beneficiaries, and professional insurance. This would facilitate the process of recuperating at construction firms during the implementation phase is to allow design-and-build least some of the financial corrections generated by mistakes that make it Phase III: Implementation contracts (and eliminate the requirement of submitting the technical project past these filters and would also help “clean” the market in the long run. during the evaluation stage). The winning bidder takes on the responsibility Further research is needed to determine how this model could work in of developing the technical project and implementing it, so any discrepancies practice in Romania. must be solved without incurring additional costs for the beneficiary, unless particular construction works are determined by truly unpredictable conditions, As for monitoring and reimbursements, most of the room for in which case they would qualify as eligible. improvement lies with measures aimed at procedural simplification and greater decentralization of responsibilities at the regional IB level. C. The Execution Stage: Contracts, Monitoring, and It is important to acknowledge the government’s recent efforts (e.g., GEO Reimbursements 27/2013) aimed at short-circuiting reimbursement processes and enabling In particular, the FIDIC standard contract forms would help beneficiaries beneficiaries to maintain proper cash flow. Interviewed public sector manage complex projects better, while protecting the interests of all beneficiaries were enthusiastic about this measure, while private companies stakeholders involved. and NGOs expressed their hope that they would become eligible for the The main benefits of the FIDIC contracts revolve around a balanced sharing alternative mechanism soon. Such initiatives should be replicated going of risks between the beneficiary and the contractor and a greater degree of forward. familiarity of international investors with the specific conditions and procedures mandated by these contracts. Reintroducing FIDIC in the Romanian legislation (primarily the Red Book with design separate from execution and the Yellow Book for design-and-build contracts) promises important benefits, as long as the risks (e.g., legislative misalignment, lack of capacity) are properly addressed. More broadly, stronger contracts are needed vis-à-vis the whole set Phase-Specific of service providers that play a role during the implementation stage: construction supervisors, auditors, and project managers. For instance, construction supervisors should be required to stay on-site full time, and penalized accordingly if they breach this provision. For their part, Support auditors currently play a limited role, with low accountability: they certify expenditures by checking accounting documents, but do not typically go beyond desk assessments. Instead, they should evaluate reimbursement requests, including those procedures that have led to particular expenses. The same principles of stronger accountability should also apply to project managers, 51 Phase IV: Post-implementation Phase IV: Post-Implementation There are two priority areas for improved beneficiary assistance in the Last but not least, the ROP’s positive features can and should be post-implementation phase: ex-post monitoring and evaluation (M&E) for replicated during the next programming period, which depends durability; and enhanced knowledge exchange. fundamentally on leveraging the resources that already exist in the Beneficiaries note the lack of technical knowledge of some monitoring experts system. and their overly rigid interpretation of rules. Fundamentally, this has to do with This is certainly true of the program’s “graduates” – i.e., beneficiaries who the understanding of what qualifies as “substantial modification” in article 57 of have successfully completed ROP-financed projects. Given available data CE 1083/2006 – flagging an issue as such generates a special procedure that from SMIS and other RDA-level databases, the MA and the IBs would have can result in financial corrections. In practice, there are multiple situations where the possibility to select the top-performing projects and beneficiaries, beneficiaries avoid any adjustments, even though good reasons may exist to seeking to draw lessons learned and best practices. There are multiple ways perform them. A more clear definition of the ROP’s interpretation of Article 57 for leveraging program graduates and facilitating knowledge exchange. For in EC 1083/2006 would be very helpful to beneficiaries, along with a procedure instance, one simple possibility is to add to the final progress report and for allowing changes to projects, when justified by special circumstances. to ex-post monitoring reports submitted by beneficiaries to the IBs a few questions asking for suggestions for future beneficiaries. Another option is to host in-person workshops with successful ROP graduates acting as Clarifications are also needed when it comes to acceptable project-level trainers and mentors for new beneficiaries, within the proposed Corps of indicators and procedures used in assessing them after the completion of Practitioners. In addition, graduates could disseminate their experiences an ROP investment. through a range of information and communication technology solutions First, on the choice of indicators, applicants should be encouraged to select for knowledge management and sharing. from a pre-defined list aligned with program- and axis-level targets, which would be easy to implement particularly through an e-application form. At a minimum, Phase-Specific chosen indicators should fit the “SMART” set of criteria: specific, measurable, attainable, relevant, and time-oriented. Second, interviewed beneficiaries have also requested a detailed methodology applicable to how project-level Support indicators are monitored and evaluated throughout the implementation and postimplementation phases. Third, for projects failing to reach their targets, if a potential penalty is deemed necessary, it should be, at most, equivalent to the difference between the set target (100%) and the level achieved in practice. Still, imposing such penalties will depend only on the MA’s decision. 53 Top Recommendations The figure below summarizes the top challenges and corresponding Top Recommendations recommendations. CHALLENGES RECOMMENDATIONS I There are still significant capacity constraints on beneficiaries’ ability to manage internal and external resources Internal: Improve the compensation system (higher salary base, bonuses covering the entire cycle) and offer more focused trainings. External: Promote standard contract forms to boost providers’ accountability and guarantee high-quality services. Enable user-feedback mechanisms. II Project implementation continues to be plagued by issues related to public procurement and poor quality technical documentation Clarify and complete the public procurement framework, in line with upcoming EU-level regulations for 2014-2020 Move away from the “lowest price” as sole selection criterion Adopt the red and yellow FIDIC and build beneficiary capacity to manage these types of contracts 55 CHALLENGES RECOMMENDATIONS III Bureaucracy continues to be excessive and an electronic interface with beneficiaries is missing Develop an e-ROP for the entire project cycle, allowing beneficiaries to submit all documentation electronically Implement automated verifications to speed up Top Recommendations processing Develop a simple, easy-to-use online tracking platform for beneficiaries to check the status of all their requests I Knowledge sharing remains relatively weak and best practices are not effectively disseminated Establish an ROP Corps of Practitioners bringing together the most experienced beneficiaries and enabling the dissemination of best practices through institutionalized channels Leverage the knowhow of beneficiaries who successfully complete ROP projects (i.e., close the feedback loop) Marginalized communities lack targeted ROP Define targeted selection criteria to encourage ROP- interventions & support financed investments in marginalized communities Develop the capacity of marginalized groups to design and implement ROP projects (e.g., through the CLLD instrument) 57