FOR OFFICIAL USE ONLY Report No: PAD3400 INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT PROJECT APPRAISAL DOCUMENT ON A PROPOSED LOAN IN THE AMOUNT OF EUR 43 MILLION (USD 48 MILLION EQUIVALENT) TO THE REPUBLIC OF SERBIA FOR SERBIA ACCELERATING INNOVATION AND GROWTH ENTREPRENEURSHIP November 20, 2019 Finance, Competitiveness and Innovation Global Practice Europe And Central Asia Region This document is being made publicly available prior to Board consideration. This does not imply a presumed outcome. This document may be updated following Board consideration and the updated document will be made publicly available in accordance with the Bank’s policy on Access to Information. CURRENCY EQUIVALENTS Exchange Rate Effective (October 31, 2019) Currency Unit EUR 0.8965 = USD 1 FISCAL YEAR January 1 - December 31 Regional Vice President: Cyril E Muller Country Director: Linda Van Gelder Global Practice Director: Alfonso Garcia Mora Regional Director: Lalita Moorty Practice Manager: Mario Guadamillas Task Team Leader(s): Maja Andjelkovic, Dusko Vasiljevic ABBREVIATIONS AND ACRONYMS 4S Serbia’s Smart Specialization Strategy C&J Competitiveness and Jobs Project CEE Central and Eastern Europe CFU Central Fiduciary Unit CPF Country Partnership Framework DA Designated Account EFA Economic and Financial Analysis EMP Environmental Management Plan ERR Economic Rate of Return ES Environmental and Social ESMF Environmental and Social Management Framework ESRS Environmental and Social Review Summary EU European Union EUD European Union Delegation to the Republic of Serbia EUR Euro currency FI Financial Intermediaries GDP Gross Domestic Product GM Grants Manual GoS Government of Serbia GRS Grievance Redress Service HEIs Higher Education Institutions HGFs High-Growth Firms HR Human Resources IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results ICT Information and Communication Technology IF Innovation Fund IFC International Finance Corporation IFRs Interim Financial Reports IMGGE Institute for Molecular Genetics and Genetic Engineering IPA Instrument for Pre-Accession Assistance IPB Institute of Physics Belgrade IPF Investment Project Financing ISR Implementation Status and Results M&E Monitoring and Evaluation MFD Maximizing Finance for Development MoESTD Ministry of Education, Science and Technological Development MoF Ministry of Finance MTR Mid-Term Review NBS National Bank of Serbia NERP National Economic Reform Program NPV Net Present Value OECD Organisation for Economic Co-operation and Development PAD Project Appraisal Document PDO Project Development Objective PER Public Expenditure Review PIEs Project Implementation Entities PIU Project Implementation Unit POM Project Operations Manual R&D Research and Development RDIs Research and Development Institutes RSD Serbian Dinar currency SAIGE Serbia Accelerating Innovation and Growth Entrepreneurship (Project) SCD Systematic Country Diagnostic SDF Serbian Diaspora Facility SF Science Fund SIDC Secure Identification Credentials SIP Serbia Innovation Project SMEs Small and Medium Enterprises SOE Statement of Expenditure SRITTP Serbia Research, Innovation and Technology Transfer Project SROs Science and Research Organizations (accredited) TA Technical Assistance ToR Terms of Reference UNESCO United Nations Educational, Scientific and Cultural Organization USD US Dollar currency USAID United States Agency for International Development VC Venture Capital WB The World Bank The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) TABLE OF CONTENTS DATASHEET ........................................................................................................................... 1 I. STRATEGIC CONTEXT ...................................................................................................... 7 A. Country Context................................................................................................................................ 7 B. Sectoral and Institutional Context .................................................................................................... 8 C. Relevance to Higher Level Objectives............................................................................................. 11 II. PROJECT DESCRIPTION.................................................................................................. 12 A. Project Development Objective ..................................................................................................... 12 B. Project Components ....................................................................................................................... 13 C. Project Beneficiaries ....................................................................................................................... 16 D. Results Chain .................................................................................................................................. 16 E. Rationale for Bank Involvement and Role of Partners ................................................................... 17 F. Lessons Learned and Reflected in the Project Design .................................................................... 18 III. IMPLEMENTATION ARRANGEMENTS ............................................................................ 19 A. Institutional and Implementation Arrangements .......................................................................... 19 B. Results Monitoring and Evaluation Arrangements......................................................................... 20 C. Sustainability................................................................................................................................... 20 IV. PROJECT APPRAISAL SUMMARY ................................................................................... 21 A. Technical, Economic and Financial Analysis ................................................................................... 21 B. Fiduciary.......................................................................................................................................... 22 C. Legal Operational Policies ............................................................................................................... 24 D. Environmental and Social ............................................................................................................... 24 I. GRIEVANCE REDRESS SERVICES ..................................................................................... 27 II. KEY RISKS ..................................................................................................................... 27 I. RESULTS FRAMEWORK AND MONITORING ................................................................... 30 ANNEX 1: Implementation Arrangements and Support Plan ................................................. 41 ANNEX 2: Project Description .............................................................................................. 46 ANNEX 3. Overview of Serbia’s R&D, Innovation and Entrepreneurship Reform Agenda ....... 54 ANNEX 4: Economic Analysis................................................................................................ 55 ANNEX 5: Sustainability Action Plan..................................................................................... 58 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) DATASHEET BASIC INFORMATION BASIC_INFO_TABLE Country(ies) Project Name Serbia Serbia Accelerating Innovation and Growth Entrepreneurship Project ID Financing Instrument Environmental and Social Risk Classification Investment Project P170185 Moderate Financing Financing & Implementation Modalities [ ] Multiphase Programmatic Approach (MPA) [ ] Contingent Emergency Response Component (CERC) [ ] Series of Projects (SOP) [ ] Fragile State(s) [ ] Disbursement-linked Indicators (DLIs) [ ] Small State(s) [ ] Financial Intermediaries (FI) [ ] Fragile within a non-fragile Country [ ] Project-Based Guarantee [ ] Conflict [ ] Deferred Drawdown [ ] Responding to Natural or Man-made Disaster [ ] Alternate Procurement Arrangements (APA) Expected Approval Date Expected Closing Date 17-Dec-2019 30-Sep-2024 Bank/IFC Collaboration No Proposed Development Objective(s) The project’s development objective is to improve (i) the relevance and excellence of scientific research; and (ii) innovative entrepreneurship and access to finance for enterprise growth, as a way of contributing to Serbia’s growth and competitiveness. Relevance of research refers to the potential of research results to be commercialized on the market, thus contributing to the economy, while excellence refers to international recognition of the quality of the research. Page 1 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) Components Component Name Cost (USD, millions) Research Sector Reforms 38.00 Enterprise Acceleration 8.00 Project Implementation, Monitoring, Capacity Building 2.00 Organizations Borrower: Republic of Serbia Implementing Agency: Ministry of Education, Science and Technological Development PROJECT FINANCING DATA (USD, Millions) SUMMARY -NewFin1 Total Project Cost 48.00 Total Financing 48.00 of which IBRD/IDA 48.00 Financing Gap 0.00 DETAILS -NewFinEnh1 World Bank Group Financing International Bank for Reconstruction and Development (IBRD) 48.00 Expected Disbursements (in USD, Millions) WB Fiscal Year 2020 2021 2022 2023 2024 2025 Annual 0.10 3.50 8.90 13.51 18.15 3.84 Cumulative 0.10 3.60 12.50 26.01 44.16 48.00 INSTITUTIONAL DATA Page 2 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) Practice Area (Lead) Contributing Practice Areas Finance, Competitiveness and Innovation Climate Change and Disaster Screening This operation has been screened for short and long-term climate change and disaster risks Gender Tag Does the project plan to undertake any of the following? a. Analysis to identify Project-relevant gaps between males and females, especially in light of Yes country gaps identified through SCD and CPF b. Specific action(s) to address the gender gaps identified in (a) and/or to improve women or Yes men's empowerment c. Include Indicators in results framework to monitor outcomes from actions identified in (b) Yes SYSTEMATIC OPERATIONS RISK-RATING TOOL (SORT) Risk Category Rating 1. Political and Governance ⚫ Substantial 2. Macroeconomic ⚫ Moderate 3. Sector Strategies and Policies ⚫ Substantial 4. Technical Design of Project or Program ⚫ Substantial 5. Institutional Capacity for Implementation and Sustainability ⚫ Substantial 6. Fiduciary ⚫ Moderate 7. Environment and Social ⚫ Moderate 8. Stakeholders ⚫ Substantial 9. Other 10. Overall ⚫ Substantial Page 3 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) COMPLIANCE Policy Does the project depart from the CPF in content or in other significant respects? [ ] Yes [✓] No Does the project require any waivers of Bank policies? [ ] Yes [✓] No Environmental and Social Standards Relevance Given its Context at the Time of Appraisal E & S Standards Relevance Assessment and Management of Environmental and Social Risks and Impacts Relevant Stakeholder Engagement and Information Disclosure Relevant Labor and Working Conditions Relevant Resource Efficiency and Pollution Prevention and Management Relevant Community Health and Safety Not Currently Relevant Land Acquisition, Restrictions on Land Use and Involuntary Resettlement Not Currently Relevant Biodiversity Conservation and Sustainable Management of Living Natural Not Currently Relevant Resources Indigenous Peoples/Sub-Saharan African Historically Underserved Traditional Not Currently Relevant Local Communities Cultural Heritage Not Currently Relevant Financial Intermediaries Not Currently Relevant NOTE: For further information regarding the World Bank’s due diligence assessment of the Project’s potential environmental and social risks and impacts, please refer to the Project’s Appraisal Environmental and Social Review Summary (ESRS). Legal Covenants Sections and Description Loan Agreement: Maintain PIU through Project implementation | Description: Loan Agreement Schedule 2, Section Page 4 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) I.A.1 The Borrower shall maintain throughout Project implementation, with composition, resources, terms of reference, and functions acceptable to the Bank: (a) a Project implementation unit within MoESTD (PIU), to be responsible for: (i) the overall coordination of all Project implementation activities; (ii) ensuring that the requirements, criteria, policies, procedures, and organizational arrangements set forth in the Project Operations Manual are applied in carrying out the Project; (iii) preparation of all Project implementation documents, including Project supervision reports; and (iv) monitoring and evaluation of the Project; and (b) a Central Fiduciary Unit (CFU), within MoF, to be responsible for the procurement and financial management of the Project, as detailed in the Project Operations Manual. | Frequency :CONTINUOUS Sections and Description Loan Agreement: Governance of Science Fund and Innovation Fund | Description: Loan Agreement Schedule 2, Section I.A.2 The Borrower shall cause the Science Fund and the Innovation Fund to maintain throughout Project implementation a governance structure acceptable to the Bank, including, inter alia, a managing board, program boards and investment and/or selection committees, as applicable, with composition and rules of operation acceptable to the Bank and in accordance with the provisions set forth in the Project Operations Manual. | Frequency :CONTINUOUS Sections and Description Loan Agreement: RDI Transformation Plans | Description: Loan Agreement Schedule 2, Section I.A.3 3. For purposes of implementing Part 1.2 of the Project, the Borrower, through MoESTD shall submit to the Bank for approval any given Transformation Plan, and upon approval by the Bank of said Transformation Plan, the Borrower, through MoESTD, shall: (a) enter into an agreement with each Selected RDI (RDI Agreement), under terms and conditions acceptable to the Bank, including, inter alia, the Selected RDI’s obligation to implement the respective Transformation Plan with due diligence and efficiency, and in conformity with appropriate administrative, technical, financial, economic, environmental and social standards, and practices, and in accordance with the pertinent provisions of this Agreement; and (b) immediately thereafter, cause each Selected RDI to implement the corresponding Transformation Plan in accordance with its terms and in a manner satisfactory to the Bank. | Frequency :CONTINUOUS Sections and Description Loan Agreement: Science Fund and Innovation Fund Grant Manuals | Description: Loan Agreement Schedule 2, Section I.C.1 The Borrower shall cause the Science Fund and the Innovation Fund to make Science Fund Grants and Matching Grants, respectively, to Beneficiaries in accordance with eligibility criteria and procedures acceptable to the Bank and as set forth in the applicable Science Fund Grant Manual and Innovation Fund Grant Manual. | Frequency :CONTINUOUS Sections and Description Loan Agreement: Safeguards | Description: Loan Agreement Schedule 2, Section I.E.1 The Borrower shall, and shall cause the Science Fund and the Innovation Fund to, ensure that the Project is carried out in accordance with the Environmental and Social Standards, in a manner acceptable to the Bank, including compliance with the ESCP (Section I.E.2 of Schedule 2 of the Loan Agreement). | Frequency :CONTINUOUS Page 5 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) Conditions: Type Description Effectiveness Additional Condition of Effectiveness consists of the following, namely, that the Subsidiary Agreements have been executed in a manner acceptable to the Bank. Page 6 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) I. STRATEGIC CONTEXT A. Country Context 1. Following years of recession and slow growth, the Serbian economy expanded by 1.8 percent on average over the 2015-2017 period, with a stronger growth of 4.2 percent in 2018. Growth started to recover after higher investment (average annual growth of 8.3 percent annually) and strong increases in exports (up 10.7 percent annually in real terms). Consumption recovered, but at a slower pace (1 percent annually in real terms). Growth of the industry and services sectors contributed most to overall growth of the economy between 2015 and 2017, while agriculture had a negative contribution in 2015 and 2017. In 2018, growth was broad-based with all three major sectors rising faster than in the previous year. Despite the generally positive economic outlook in the near-term, challenges remain. Growth will depend on the pace of ongoing structural reforms and progress toward EU accession. 2. Labor market performance has improved. According to the Labor Force Survey data, unemployment again declined in 2018, to an estimated annual average of 13.3 percent, and employment rose by 1.5 percentage points compared to 2017, which suggests that 37,500 more jobs were created in 2018. As a result, employment among the population 15-64 years old reached a record high of 58.8 percent. In 2018, too, average wages went up by 4.4 percent in real terms (and stood at 420 euros a month, net of taxes). 3. Strong revenue performance and spending controls led to budget surpluses in 2017 and 2018. In 2017, Serbia had a surplus of 1.2 percent of GDP, underpinned by strong revenue collection, spending controls (including savings from interest payments), and, to some extent, due to under-execution of public investment. In 2018, the budget recorded a surplus of 0.6 percent of GDP despite some relaxation in spending controls. As a result of prudent fiscal policies, public debt continued to decline and stood at 54.3 percent of GDP at the end of December 2018, with further, albeit slower, declines expected during 2019. 4. As Serbia positions itself for EU membership, increasing competitiveness in the European market remains a priority. A structural shift in Serbia’s growth model, driven by increased productivity and higher-value added production, is needed to boost competitiveness and spur economic growth. This can be achieved, in part, through enterprise innovation facilitated by increased and more efficient investments in applied research and development (R&D), and, support for research commercialization, enterprise formation, and growth of innovative small and medium enterprises (SMEs). 5. Extreme precipitation, extreme temperatures and drought are major climate risks and natural hazards likely to affect Serbia's sustainable development. Serbia has already experienced an increase in the frequency of flooding. In 2014, massive floods across Southeast Europe left over 50 people dead in Serbia alone and resulted in damages of over EUR 1.5 billion. At the same time, mean annual temperatures have risen by 0.3°C per decade. Increase in temperatures could lead to severe water shortages and increased instances of heat waves. Significant additional risk comes from an increased frequency of droughts which, according to government estimates, have caused damages of over EUR 3.5 billion since 2000. As a signatory to the Paris Agreement, Serbia is committed to international efforts to avoid dangerous climate change. In its Intended Nationally Determined Contribution to the United Nations Framework Convention on Climate Change, Serbia pledged to reduce greenhouse gas emissions by 9.8 percent by 2030, from base‐year 1990. The project aims Page 7 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) to contribute to climate change adaptation and mitigation efforts in Serbia through targeted activities in Component 1 and Component 2, as specified in paragraphs 72, 73 and 74. B. Sectoral and Institutional Context 6. Research is the basis of new knowledge and technologies, and innovation is the adoption of invention by the market, often through entrepreneurial action. Innovation and entrepreneurship are important drivers of growth, due to their role in (i) shifting growth patterns to be more productivity based and trade-oriented, and (ii) increasing economic dynamism. Specifically, innovative firms grow faster: by 15 percent in sales, and 8 percent in labor productivity, and firm R&D expenditures significantly increase performance: by 12 percent in sales and 6 percent in labor productivity growth, on average in the Western Balkans. Young companies also introduce competition into markets and create new markets by developing and commercializing new services and products. Young, innovative firms are also more likely to create new employment opportunities than older ones. 1 7. A 2018 World Bank analysis shows that high-growth firms (HGFs) in client countries are also younger than the average. For example, start-ups account for about 40 percent of all HGFs in Brazil, Cote d’Ivoire, Ethiopia, and Hungary, and around 30 percent in Indonesia. The same analysis identifies the following factors as significantly contributing to the probability of high growth of a firm: innovation capability, network economies, managerial capabilities and worker skills, and global linkages.2 8. Innovation in Serbia is currently limited by constraints to achieving research excellence (high quality of outputs) and research relevance (economic and societal applicability), both of which are needed for innovation to contribute to growth. Innovation-related indicators point to market failures as well as inefficient public resource allocations. Serbia has (i) low levels of public and private R&D expenditures, (ii) a science financing model that results in inefficiencies, due to its largely non-competitive structure, and (iii) low levels of public-private research collaboration, which results in low levels of commercialization of research results that could support economic growth. Around 0.9 percent GDP is invested in R&D, well below EU average of 2.03 percent, with most of the funding coming from the Government. This indicates that the private sector is severely underinvesting in R&D and innovation, suggesting market failures (likely due to uncertainty and information asymmetries) and a need for public financing and other structural incentives. 9. To address these challenges, the Government, supported by the World Bank and the European Union Delegation to the Republic of Serbia (EUD), has initiated significant reforms, articulated in the Research for Innovation Strategy, 2016-2020. 1 For instance, see Kauffman Foundation (2015), “The Importance of Young Firms” https://www.kauffman.org/what-we- do/resources/entrepreneurship-policy-digest/the-importance-of-young-firms-for-economic-growth and “Lederman, Daniel; Messina, Julián; Pienknagura, Samuel; Rigolini, Jamele. 2014. Latin American Entrepreneurs: Many Firms but Little Innovation. World Bank Latin American and Caribbean Studies; Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/16457 License: CC BY 3.0 IGO.” 2 World Bank (2018). High-Growth Firms: Facts, Fiction, and Policy Options for Emerging Economies. Page 8 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) Research for Innovation: Strategy on Scientific and Technological Development of the Republic of Serbia, 2016- 2020- Key Features The overall objective of the Strategy is to improve the efficiency and effectiveness of the Serbian scientific research system, through 1. Encouraging excellence and relevance of scientific research in the Republic of Serbia. 2. Strengthening the connection between science, economy and society to encourage innovation. 3. Establishing an effective management system for science and innovation in the Republic of Serbia. 4. Ensuring excellence and availability of human resources for science and economy and social affairs. 5. Improving international cooperation in the field of science and innovation. 6. Increasing investment in R&D through public funding and encouraging investments of the business sector in R&D. 10. Government efforts include a comprehensive set of reforms that includes supply (R&D) and demand (private sector innovation) side interventions, along with connecting the two. These are presented in the reform matrix in Annex 3, which distinguishes between reforms that have already been completed by the Government, those covered under other projects, those supported by the Project and those not yet supported. On the supply side, the reforms focus on: (i) establishing a strong strategic and legislative framework; (ii) improving research excellence and relevance through competitive research funding; (iii) strengthening Scientific Research Organisations (SROs) through performance-based institutional funding and reforms; and (iv) developing and maintaining human capital needed for excellence in research. On the demand side, reforms center around: (i) incentivizing private sector R&D activities; (ii) accelerating enterprise growth; and (iii) improving the regulatory environment for venture capital. Finally, to link the two, the Government is making efforts to improve business-academia collaboration and technology transfer. 11. A key element of the supply side reforms is changing the model of public financing of R&D activities in the country, including a transition to competitive financing through the introduction of a new funding instrument, the Science Fund of Serbia. To this end, a new Law on the Science Fund was adopted by Parliament in December 2018, and a new Law on Science and Research was adopted in July 2019. 12. Currently, research in Serbia is funded by the Ministry of Education, Science and Technological Development (MoESTD), mostly through project-based funding. With success rates of up to 95 percent, practically all applicants receive funding and the limited resources are split evenly across the research community. Such a system does not systematically promote research excellence, nor does it consider its relevance for economic partners. This is evident in the low and declining output of the Serbian research community, witnessed by the decreasing number of publications since 2012. In addition, the quality of these publications remains low (H-index of 149, compared to 205 in Bulgaria or 221 in Croatia). The current system is highly skewed towards higher education institutions (HEIs) and public research and development institutes (RDIs): the Business Enterprise Sector performs less R&D than public RDIs or HEIs, and the extent of this imbalance is higher than in countries such as Croatia or Slovenia. In addition, 40 percent of funding is allocated to basic research, which is very high compared to 18 percent in Slovenia, 11 percent in Bulgaria or 35 percent in Croatia. 13. The current system incentivizes individual researchers to publish. This has resulted in limited success in generating high quality of scientific output; however, reforms are necessary to establish institutional incentives for increasing the quantity of expertise, and increasing the relevance and quality of research. All Page 9 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) these improvements are necessary to increase the absorption capacity of the country’s R&D sector, including accessing European research funding sources, and thus overall competitiveness of the economy. 14. To create a more effective R&D financing model, a new institutional framework is being developed. Based on good international practice, Serbia has established the Science Fund (SF), an autonomous and professional government institution (based on the Science Fund Law adopted by the Parliament) to implement competitive research funding (comparable to Science Funds, Councils or Agencies in EU member states). The SF addresses the above-mentioned barriers, as a vehicle for (i) increasing R&D funding as a percentage of GDP; (ii) improving the effectiveness of public spending on R&D, by fostering competitive funding that sparks excellence; and (iii) improving business-academia collaboration and, in turn, greater commercialization rates of research. In addition, a model for providing base institutional funding is being introduced through the new Law on Science and Research and associated by-laws. 15. The newly established Science Fund will conduct competitive calls for proposals for R&D grants and rely on independent and expert decision-making based on international peer-review. By supporting high quality basic and applied research, the Science Fund will leverage public R&D spending to connect the supply and demand side by forging business-academia linkages and also encourage private sector R&D investments. Based on regular evaluation, Science Fund instruments will be adjusted over time to improve their effectiveness and impact. Further, recent tax breaks for R&D investment could boost private R&D spending. The above-mentioned reforms are being integrated and aligned with Serbia’s Smart Specialisation Strategy (4S), expected to be effective by the end of 2019. In addition, the MoESTD is setting up a working group for the new R&D and Innovation Strategy, to succeed the current one (expiring in 2020). 16. Current financing levels of innovative businesses in Serbia are also low, pointing to a significant market failure. According to USAID’s Serbia Business Survey 2017, 84 percent of small and medium enterprises (SMEs) are financed solely from their own sources. Micro-financing and venture capital investment are impeded by an unclear legal framework, very high costs of deal discovery for investors, and persisting information asymmetries between entrepreneurs and investors about opportunities for investment. Meanwhile, commercial bank loans are accompanied by high requirements for collateral or long history of operations, neither of which young, innovative firms typically possess. Lack of affordable financing, coupled with limited training and mentoring, is preventing Serbian companies from modernizing their production and investing in innovation and commercialization, with young firms and SMEs particularly affected by this. 17. A 2016 Word Bank analysis identified promising potential of the Serbian entrepreneurship ecosystem, but noted the following limitations, which continue to persist: • Constrained supply of IT, managerial, and creative skills, • Lack of “smart money,” i.e., training and mentoring coupled with financing for innovative early - and growth-stage enterprises, with a gap especially prominent in the USD 500,000-USD 1,500,000 range, • Low rates of commercialization of inventions and innovations, and • Challenging business environment for entrepreneurs. 18. While the Government is addressing the first, third and fourth constraints through the above-mentioned reform agenda (see also Annex 3), the second deficiency is proving particularly challenging to resolve. In fact, Page 10 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) Serbia received the lowest amount of VC financing in the CEE region3, despite positive upward trends in business formation generally, and in innovative technology sectors specifically In 2017, the total venture capital investment in Central and Eastern Europe (CEE) was approximately EUR 108 million. In Serbia, the amount was only EUR 1.5 million, compared to EUR 5.6 million in Romania, EUR 4.8 million in Bulgaria, EUR 3.6 million in the Czech Republic, EUR 2.6 million in Croatia, and EUR 28.6 million in Hungary. Thus, adjusting crudely for GDP, it is estimated that, with a better business environment, the average venture capital investment in Serbia could increase by several fold, EUR 7-20 million annually.4 19. The restricted supply of early-stage funding, coupled with limited mentoring and training, are frequently cited as a barrier to innovation and entrepreneurship in most ecosystems around the world; in Serbia, interview data indicates a particularly acute situation: • Consultations with existing and aspiring angel investors conducted by the World Bank in 2016-2019 indicated that the incentives equity and early-stage investments are inadequate, which negatively impacts the supply of private early stage equity capital. This finding was corroborated by interview and survey responses by entrepreneurs, very few of whom reported receiving equity investments. • A small number of experienced angel investor groups is active in the country (approximately five in total, with low levels of activity) and very few startups have received equity investments so far. • Over half of the interviewed entrepreneurs indicated finding investors, especially those with valuable expertise, as a challenge. 20. Finally, a World Bank Public Expenditure Review (PER) on SME Support (forthcoming, 2019) shows that there is a lack of selectivity in which SMEs are provided with support. There is strong global evidence that most SMEs do not grow. As Serbian programs rarely have selection processes based on any qualitative assessment of applicant firms’ growth orientation and strategy (the Innovation Fund programs being a welcome exception), most funding is likely going to firms that will not grow and contribute substantially to the economy. This means that at best, resources are being provided to SMEs that will not grow (although the resources may make them more efficient), and at worst, the funding is not even impacting their productivity. In fact, these resources may be keeping SMEs in business that would otherwise leave the marketplace. Meanwhile, growth-oriented businesses may not be receiving the support that would most benefit Serbia in terms of income growth and job creation. C. Relevance to Higher Level Objectives 21. The proposed operation is fully congruent with the World Bank Group’s Country Partnership Framework (CPF) for FY16-205 focus areas (Focus area 1: “Economic Governance and the Role of the State” and Focus Area 2: “Private Sector Growth and Economic Inclusion”). Both focus areas are aligned with the Government’s Strategy of Scientific and Technological Development of the Republic of Serbia, 2016–2020: Research for Innovation. The operation specifically responds to the second focus area, Objective 2a: Contribute to priority business climate improvements. An important intervention under this objective is 3 With the exception of Ukraine, which received EUR 1.47 million, and other countries where negligible amounts were recorded --Bosnia & Herzegovina, North Macedonia, Moldova, and Montenegro. 4 In addition to limited VC funding, Serbian entrepreneurs also face Interest rates that are significantly higher compared to benchmark countries, (4.25 percent in December 2018 compared to 1.4 percent in Czech Republic and Hungary), making debt financing prohibitively expensive to many early-stage entrepreneurs. 5 Report No. 94687-YF discussed by the Board of Executive Directors in June 2015 Page 11 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) continued support to building Serbia’s innovation and technology transfer system based on promising results yielded from pilot efforts in these two areas and a need to scale them up to create a broader impact on employment generation. All three components of the Project directly and jointly contribute to the Research for Innovation Strategy, 2016-2020, with activities of the Science Fund supporting primarily public R&D, activities of the Innovation Fund supporting primarily enterprise innovation, and the diaspora-related activities supporting innovation generated by both public and private entities. The Project also has links to the first CPF focus area, Economic Governance and the Role of the State, specifically Objective 1b: More Effective Public Administration & Service Delivery. Establishing and maintaining appropriate governance of the Science Fund and improving governance across public RDIs would allow these entities to improve the efficiency and effectiveness of the scientific research system, increase excellence and relevance of scientific research, and better link these institutions with the private sector. 22. The International Finance Corporation (IFC) recognizes that entrepreneurship via digitalization and fintech seems to be lagging in Serbia and is working to advance the agenda jointly with the World Bank. Specifically, the IFC has noted the importance of a new law on alternative financing, as a requirement for increased investment in domestic and regional venture capital, seed investment and crowdfunding vehicles. Further, support for startups, accelerator hubs and incubators is seen as critical for increasing the number of start- ups and financing vehicles available to entrepreneurs. Component 2, Enterprise Acceleration, creates opportunities for synergies with IFC activities on these fronts. 23. The Project is expected to contribute to the Maximizing Finance for Development (MFD) agenda. By matching private sector, equity-based investment into early stage innovative enterprises, the Project will stimulate additional investment. This investment, along with grants dedicated to research activities in public institutions, will, in at least some cases, result in additional R&D activities by the private sector, and in the creation of additional intellectual property. This additional intellectual property creation could, in turn, stimulate additional private financing to bring products and services to the market. Finally, outreach and incentives to the diaspora, planned under the Project, will also serve to leverage capital and knowledge-based resources beyond the direct financing to be made available. II. PROJECT DESCRIPTION A. Project Development Objective PDO Statement The project’s development objective is to improve (i) the relevance and excellence of scientific research; and (ii) innovative entrepreneurship and access to finance for enterprise growth, as a way of contributing to Serbia’s growth and competitiveness. Relevance of research refers to the potential of research results to be commercialized on the market, thus contributing to the economy, while excellence refers to international recognition of the quality of research. PDO Level Indicators (i) Number of international scientific publications in top 10 percent of recognized journals (ii) Number of collaborative applied research projects Page 12 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) (iii) External investment generated by participating companies (iv) Number of new or improved innovative products or services introduced to the market B. Project Components Component 1: Research Sector Reforms (EUR 34 million) 24. Subcomponent 1.1: Serbia Science Fund (EUR 22.5 million) – This subcomponent will: (a) Finance selected competitive programs of the Science Fund, such as basic science grants, applied research grants with incentives for promoting linkages between the private sector and R&D community, incentives for enhancing collaboration with EU (e.g., Horizon 2020, Horizon Europe) and other international programs. The project will fund matching grants for some of the direct grants for public research financing programs designed and developed in phases. This component will be an addition to Government budget resources allocated for this purpose, and any EU IPA 2019 contribution that may be allotted. (b) Provide technical assistance (TA) for the operationalization of the Science Fund, including governance, organizational structure, monitoring and evaluation (M&E) and other key institutional elements. In addition, the TA will support designing the Science Fund programs based on international best practices. (c) Support the design of a SF program focused on skills of researchers to access international financing and collaboration opportunities. The TA will also support the MoESTD and the SF in the prioritization and development of SF’s programs. The Project is estimated to support some 220 competitive research projects involving approximately 600 to 700 researchers from 40-50 institutions. 25. Subcomponent 1.2: RDI Reforms (EUR 8 million) – This subcomponent will support the design and implementation of institutional transformation plans for selected RDIs. It will include reforms of at least four public R&D Institutes (RDIs) by providing incentives for undertaking institutional reforms on a voluntary (opt- in) basis. This will build upon the successful technical assistance experience under the Serbia Innovation Project (SIP) where two public RDIs – namely the Institute of Physics Belgrade (IPB), and Institute for Molecular Genetics and Genetic Engineering (IMGGE) – initiated important institutional reforms consistent with the Action Plan of the Research for Innovation Strategy, 2016-2020 adopted under the Serbia Research, Innovation and Technology Transfer Project (SRITTP) with Bank support. Through a phased approach, public RDIs that apply and are approved by the MOESTD to participate in the program will undergo detailed assessments. In the first phase, all RDIs will undergo a self-assessment done by RDI management. Second, interested RDIs will be invited to apply to undergo a detailed independent external assessment conducted by a team of international experts. These will evaluate key aspects of an RDI, with agreed criteria including: 1) governance practices, 2) staff capabilities, 3) HR and other policies and processes, 4) quality and quantity of R&D outputs, 5) collaboration with other RDIs, private sector and international community, and 6) potential for upgrading excellence of research and its relevance to the private sector and overall Serbian economy. Lastly, based on the results, detailed transformation plans for selected RDIs will be prepared by the external assessment teams in close collaboration with the RDI leadership and MoESTD, and satisfactory to the Bank. These will include detailed action plans with specific milestones with deadlines. The project will provide financing for the implementation and monitoring of the transformation plans (and for RDI assessments, as needed). The project financing support will be clearly linked with the timely achievement of specific milestones and will focus on enhancing the excellence and relevance of participating RDIs. No advance Page 13 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) allocations will be made for any participating RDIs, instead financing will be provided only when they achieve agreed specific milestones in a timely manner. No transfer of Loan proceeds will be made to the selected RDIs; however, eligible expenses will be financed by the MoESTD. The Project is estimated to benefit over 200 researchers and result in 75 collaborative projects with researchers and private sector from ten institutions. 26. Subcomponent 1.3: Serbian Diaspora Facility (EUR 3.5 million) - The subcomponent will support establishment of a Serbian Diaspora Facility (SDF) within the Science Fund to finance technical assistance and provision of grants to scientists, researchers, entrepreneurs and Serbian diaspora to transfer knowledge and skills from the diaspora community back to the country. More specifically, the facility will leverage the strengths and desire of the Serbian diaspora community to contribute to the research, innovation and entrepreneurship ecosystem in Serbia. This technical assistance and matching grants program will aim to attract promising scientists, researchers and entrepreneurs from the Serbian diaspora community to transfer knowledge and skills back to Serbia through a variety of activities (mostly advisory) including participation in policy making, governance, program management and monitoring, advisory bodies and networking; collaboration in scientific and applied research and technology transfer, etc. It will include a number of instruments such as knowledge-exchange vouchers for Serbian researchers, and grants for joint research projects (in Serbia and/or abroad), filing of IP, technology transfer and R&D commercialization, technology development, etc. Safeguards will ensure that (i) proposals are peer-reviewed by a panel of prominent independent scientists and entrepreneurs from both within and outside Serbia who will provide written recommendations to the Selection Committees; (ii) nominations/invitations to the selected candidates include measurable outputs/outcomes. The project will finance 100 percent of the vouchers (grants) provided through the SDF, which would cover up to 80 percent of diaspora projects costs. In order to ensure that diaspora collaborations are objective and committed, sponsoring beneficiary institutions and companies will contribute at least 20 percent of the total project cost; such beneficiary matching levels will be established by the Science Fund based on the nature of the diaspora programs. The Project is estimated to benefit over 150 Serbian researchers and 20 private firms collaborating with over 70 diaspora members involving some 20 global institutions. Expected results include increasing the number and quality of scientific publications, number of IP filings, and rates of commercialization of research results. An appropriate M&E framework will be put in place to evaluate the effectiveness of all diaspora support programs. Component 2: Enterprise Acceleration (EUR 7 million) 27. This component is designed to build on existing programs of the Innovation Fund IF. The enterprise acceleration program will consist of two streams: one for early (idea) stage, and the other for growth (scale-up) stage companies. Each stream will serve up to 20 companies (with a minimum of two founders per company) per year, selected competitively by participating investors (including angel investors, early stage VC funds, etc.). Each stream will consist of a structured, 2-3 month program of intensive training and mentoring, with 1/3 of the time spent on a joint curriculum (one per stream) and 2/3 of the time spent on targeted mentoring with sector-specific mentors (for instance, a cardio-vascular medical devices startup will ideally be paired with founders who have successfully “exited,” i.e. sold or taken to a public market, a company focused on cardio-vascular medical devices). The goal of the program will be to increase the growth of each company, with at least 30% of the participating companies expected to achieve 10% annual growth 6-12 months after the program. If, throughout the program, market feedback on a company’s product, Page 14 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) service, or strategy, does not indicate high growth potential, mentors will work with company founders to “pivot,” or adjust strategy to market demand, if that is possible. Mentors committing to providing over 40 hours of their time to the program will be compensated with modest per diems (in line with Government consulting ceilings), while more casual mentors will serve as volunteers, as is industry practice. Companies participating in the growth-stage stream will have already demonstrated “market traction,” i.e. evidence of demand, through the presence of users or paying customers, and thus, it is expected that most of these companies will be able to advance their growth during and after the program. Companies participating in the early stage stream may also need to reexamine the basic idea and assumptions behind the company, and, in some cases, decide not to pursue the idea further; however, founders will still benefit from an intensive course that will cover project and people management practices, strategy development and execution, market entry, partnership development, fundraising, and so on. The focus will be on businesses based on digital technologies, and specialized sub-groups could be considered (e.g. ICT, food and agriculture, health, biotech, education and gaming, etc.). Companies will receive non-dilutive investment (through a co-financing facility) and extensive technical assistance in negotiating with individual investors or early stage funds, which may be equity-based.6 28. The enterprise acceleration program would be structured as follows: (a) Co-Investment Fund to provide Matching Grants for the financing of investments and technical assistance to Selected Companies (up to EUR 4.5–5.5 million in total over four years). This would include co-investment matching grants ranging in value between EUR 100,000 and EUR 450,000 depending on the maturity and needs of the company. The Co-Investment Fund will finance matching grants, which would cover up to 60 percent of project costs. Private individual investors or funds will contribute at least 40 percent of the total project investment at the time of grant award (previously raised funds will not be eligible). Recipient companies will then benefit from intensive, tailored acceleration (described below under the Technical Assistance Facility), to maximize impact of the investment. Strategic areas to be supported include innovations in software and IT, life sciences, and food and agriculture, among other spheres to be confirmed. Equity ownership will be agreed between company founders (original owners) and investors, and Project matching grants support will be strictly non-dilutive. As part of technical assistance, companies will be familiarized with standard investment term sheets, to facilitate their agreements with investors (whether individuals or funds). (b) Technical Assistance Facility (EUR 2 million over four years); this would include TA support for the: (i) competitive selection of companies, program and curriculum development and implementation, and mentoring; (ii) angel group/network creation and deal flow development, training for existing entrepreneurship hubs on acceleration and growth-financing, and establishing linkages with international angel and VC networks; (iii) regulatory analysis and development of a reforms matrix to enable further development of innovative early and growth stage enterprises in Serbia; and (iv) participation of Serbian diaspora in innovative entrepreneurship mentoring and angel investments. Component 3: Project Implementation, Monitoring, Capacity Building (EUR 2 million) 6 A similar model implemented at the regional level in Africa (XL-Africa.com) resulted in close to 50% of participating firms in the growth (scale-up) stream raising Series A funding (USD0.5-USD2 million), between three weeks and one year after the program. Based on this, the model was awarded the World Bank President’s Award for Excellence. Page 15 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) 29. This component will finance activities related to project implementation and monitoring, including the operations of a Project Implementation Unit (PIU), established at the MoESTD and part of the operating costs of the Central Fiduciary Unit (CFU), already established at the MOF. This will include operational and fiduciary (procurement, environmental and social safeguards), M&E, project audits, studies, policy/program design and capacity building support to the MoESTD, Science Fund, Innovation Fund and selected RDIs. The CFU will be in-charge of fiduciary responsibilities for the project, while the PIU will be responsible for all other project implementation related aspects. C. Project Beneficiaries 30. The Direct Project Beneficiaries are the MoESTD and its associated project implementation entities (PIEs): the Innovation Fund and the newly established Science Fund. 31. End beneficiaries include: 1. Public accredited SROs – in the new model of financing research activities, public accredited SROs will receive performance-based institutional financing and will not entirely depend on project based funding. Further, public RDIs will also benefit from support for their external assessments and transformation plans. This will allow them to develop more long-term strategic planning, and enhance their excellence and relevance of their R&D outputs. Opening new avenues of co-operation with the private sector (and international entities) will generate new revenue streams for RDIs. 2. Researchers – competitive grants offered by the SF will support the researchers by providing them with grants that match their project needs. Grants will be designed to encourage participation of women researchers. Preparing project proposals for the SF will improve their capacity to bid for EU and other international research grants. 3. Private sector enterprises and investors – through the SF subcomponent, new opportunities for business- academia collaboration will open and will complement the existing programs of the Innovation Fund. Through the enterprise acceleration component, entrepreneurs, early stage companies, individual investors, and investment funds, will have increased access to investment, knowledge, and deal flow, respectively. Women entrepreneurs and researchers will be encouraged to take part in these programs, including through one-on-one outreach via peers and leaders, targeted workshops for women entrepreneurs and researchers, as well as through additional opportunities for mentorship by established women leaders (women mentors will be matched to women beneficiaries, at the request of participants). An important target for Science Fund grants will be to award at least 25 percent of funds to projects with women principal investigators. D. Results Chain 32. The project aims to strengthen the excellence and relevance of scientific research by introducing a new funding mechanism that supports excellence and relevance of scientific research (Component 1.1), reforming public RDIs to be able to enhance their R&D excellence and relevance including to better commercialize the results of scientific research (Component 1.2), building the capacity and culture around the importance and demand for scientific research and introducing mechanisms to accelerate innovative ideas for startup/growth stage businesses (Component 2). This should lead to increased relevance and excellence of scientific research and to more accessible financing for innovative entrepreneurship. At the end, the project outcomes Page 16 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) contribute to enhanced firm-level growth and competitiveness. The SAIGE project Theory of Change is presented in Table 1. Each of the PDO-level indicators is dedicated to a specific objective. It is worth mentioning that PDO-level indicator 4, “number of new or improved products,” which relates to the objective of increasing innovative entrepreneurship, focuses on the commercial viability of accelerated firms. Similarly, PDO-level indicator 3, “external investment generated by participating companies,” which relates to the objective of increasing access to finance, focuses on the investability of accelerated firms. E. Rationale for Bank Involvement and Role of Partners 33. The Bank has a track record of decade-long and successful cooperation with Serbia and the EU on research, innovation and entrepreneurship. During 2011-2016, the World Bank administered the first EU Instrument for Pre-Accession (IPA) financed Serbia Innovation Project (SIP, P126229). This Project assisted the Government of Serbia (GoS) in the operationalization and institutional capacity building of the Innovation Fund, piloting financial instruments for technological development and innovation by enterprises and public RDIs, and assisting Government in formulating the RDI sector reform policy. 34. Building upon the success of the SIP, the EU IPA financed in 2014 the Serbia Research, Innovation, and Technology Transfer Project (SRITTP, P145231). The SRITTP leveraged the institutional capacity built at the Innovation Fund and the enterprise innovation programs piloted under the SIP as well as lessons learned from the TA program with public RDIs. This project also builds on the strategic planning activities for the research and innovation sector as well as technology transfer initiatives. 35. The Innovation component (USD 32 million) of the World Bank financed Competitiveness and Jobs Project (USD 100 million loan approved in September 2015, P152104) has provided continuity in financial support Page 17 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) for enterprise innovation and facilitates technology transfer and commercialization piloted by the Innovation Fund, and supports design of public research sector reforms. 36. The institutional set-up of the Innovation Fund, which is mainly a result of technical assistance provided by the World Bank under these projects, has been crucial for ensuring good governance and transparency, and thus, for safeguarding investments into the Fund. Based on this, the GoS has requested the Bank’s continued assistance in ensuring the institutional integrity of a new body for funding scientific research, the Science Fund, and in the application of international good practices in design, evaluation and program management, and in the piloting of new enterprise support instruments by the Innovation Fund. 37. The project will be closely coordinated with the EU. The support from the EU has been vital in providing financing for innovation in Serbia and promoting cooperation between the research community and the private sector. With this support, the Innovation Fund has positioned itself as the leader in facilitating access to EU and global markets and as an important factor in fostering national integration in the EU innovation ecosystem. Thus far, the EU has allocated a total of EUR 14.5 million from the IPA programs to the Innovation Fund, and is now considering almost EUR 48 million of support under IPA 2019 program in parallel to the SAIGE Project. 38. Project implementation will be led and coordinated by the MoESTD, through the Science Fund and the Innovation Fund. Given that the Project is a continuation of existing activities, including the Competitiveness and Jobs Project, and the Serbia Research, Innovation, and Technology Transfer Project, it is expected that the Ministry will continue to engage expert consultants to support implementation of the Project. Assistant Ministers for Science, and Technology and their teams are expected to be directly involved, which will allow the Project to benefit from institutional continuity and existing capacity within the Ministry. However, the fact that the Science Fund has been founded only recently will necessitate activities focused on implementation capacity building. F. Lessons Learned and Reflected in the Project Design 39. The Bank has supported Serbian authorities in enhancing country’s R&D, innovation and entrepreneurship ecosystem for the past decade. This has included studies, technical assistance, policy and program design and execution, as well as institutional capacity building. As a part of this involvement, the Bank has learnt several key lessons: a. institutional good governance and capacity building is crucial; this includes governance of institutions and their programs that minimizes room for conflicts of interest and ensures adequate staffing, b. ensuring gradual approach to implementation is important: this includes introduction of limited number of programs initially which are scaled up and new programs introduced as institutional capacity is built, demand is generated, and successful track record built up, c. sound governance and transparency in the design and execution of policy and support programs are critical, allowing for appropriate oversight, monitoring and evaluation, and reporting, d. reforms work best when they are accompanied by an appropriate policy and regulatory system and supported by proper incentives for stakeholders to reform, e. the contribution of the diaspora community is an important element in the success of these programs. Page 18 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) This Project has been designed to explicitly incorporate these lessons, with its focus on good governance, capacity building, a combination of investment and technical assistance to support reforms, and targeted activities to engage the diaspora. 40. Further, analysis of previous experiences of Bank financing for matching grants for the private sector shows a number of lessons: a. Matching grants must be made at the appropriate level to ensure meaningful private sector participation, which signals market demand and commitment. The level may need to be adjusted during implementation to find the optimum in a given market. Due to extensive experience with co- financing levels by the Innovation Fund, these levels will be somewhat easier to estimate than they would be otherwise. b. Governance arrangements must be transparent and conflicts of interest must be minimized. This requires the use of professional, expert selection committees and peer review systems, which help to avoid political and other influences in the evaluation and selection of grants awards. The Innovation Fund in Serbia is often used as an example of global good practice in this respect, due to its sound governance system for existing grant programs. Science Fund governance is being modeled after the Innovation Fund. c. The financing "match" should not come from a (different) public source. In this project, the project operations manual (POM) will stipulate that public resources will not be an eligible “match.” d. Key performance indicators (KPIs) must be established, against which grant tranches are disbursed to beneficiaries, thus ensuring timely course-correction when needed. The project operational manual will stipulate the continuing use of KPIs by the Innovation Fund, and the Science Fund. e. Grants should be coupled with TA or capacity building, to encourage that funds are spent in strategic ways, and increase the chances of a lasting positive impact on human capacity. There is a growing body of literature supporting the additionality of capacity building (see, for instance, OECD, 2012, Policies for Seed and Early Stage Finance , as well as World Bank, 2016, How to Make Grants a Better Match for Private Sector Development: Review of World Bank Matching Grants Projects). The project specifically couples TA with grants for this reason. III. IMPLEMENTATION ARRANGEMENTS A. Institutional and Implementation Arrangements 41. The Ministry of Education, Science and Technological Development (MoESTD) will be responsible for the overall project coordination and implementation with specific entities responsible for the implementation of their respective components. A single Project Implementation Unit (PIU) will be established at the MoESTD reporting to the MoESTD State Secretary (Research and Technology) who as Project Coordinator will be responsible for coordination of the project with all project implementing entities (PIEs), namely the Science Fund and the Innovation Fund. The PIU will be responsible for all project implementation related activities including technical, operational, environmental and social safeguards, reporting, monitoring and evaluation, audits, studies, and capacity building, etc. The fiduciary activities related to the procurement and financial management aspects will be handled by the Central Fiduciary Unit (CFU) at the Ministry of Finance which is responsible for handling such activities for several Bank financed projects. The CFU will hire additional staff as necessary to handle this responsibility. As of November 2019, the PIU has been established, and the Project Operations Manual (POM) outlining detailed project implementation arrangements including operating, Page 19 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) fiduciary and M&E procedures, staffing, responsibilities, resources, etc. has been developed and deemed satisfactory by the Bank. 42. One PIU Project Officer will be dedicated to coordinating activities of the Science Fund. The Science Fund will be responsible for the implementation of Component 1.1 Science Fund and Component 1.3 Diaspora Facility and will hire/assign appropriate staff/consultant as required. The SF is a new entity and has no experience with the World Bank projects thus will require significant capacity building assistance under the project. The MoESTD will be responsible for the implementation of Component 1.2 RDI Reforms. This will include selection of RDIs to participate in the project based on assessments of RDIs. The MoESTD will also be responsible for the approval of RDI transformation plans and monitoring of their implementation; these plans will be subject to prior review and approval by the World Bank. A second PIU Project Officer will be dedicated to coordinating activities of the Innovation Fund. The Innovation Fund will be responsible for the implementation of Component 2. Enterprise Acceleration. The Innovation Fund has years of successful experience in managing the Bank and EU financed projects. Given that enterprise acceleration is a new activity for the Innovation Fund, the project will include technical assistance for enhancing IF’s capacity to manage this program. All Grants Manuals for Project financed grants outlining detailed policies and procedures including evaluation, procurement, environmental, reporting and M&E procedures will be subject to prior review and approval by the World Bank. B. Results Monitoring and Evaluation Arrangements 43. A Results Framework with project-specific indicators and actionable monitoring arrangements has been agreed with project partners. Result-oriented project implementation will be achieved through regular progress monitoring. M&E expertise will be included in the PIU, SF and IF and a system will be developed for the M&E of project outcomes and outputs. This will include regular reporting to the World Bank twice a year: a progress report will be delivered up to one month after the end of each six-month period. The reports will be discussed during implementation support missions and to improve future programming. 44. A significant portion of the project is devoted to building the capacity, program design and implementation of a recently established government institution (the Science Fund); thus, this project will also develop the institutional M&E framework for the Science Fund and support the long-term monitoring of outcomes. 45. The Project will be subject to regular implementation support missions conducted by the World Bank. The progress assessed during these missions will be reported by the World Bank team to its management through Implementation Status and Results (ISR) Reports which will include a review of key implementation issues and performance indicators. In the third year of project implementation, a detailed Mid-Term Review (MTR) will be conducted. Given the comprehensive nature of these reforms, there are many areas that the Project will likely tackle but which cannot be predicted in advance. The Results Framework will be revisited and updated during the Mid-Term Review. At completion of the Project, an Implementation Completion and Results (ICR) Report will be prepared. C. Sustainability 46. The objectives of the project are closely aligned with the Government’s priorities. Following a period of a difficult but successful macro fiscal consolidation, Government of Serbia is now shifting its economic policy Page 20 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) priorities to focus more on achieving growth by improving competitiveness. Transformation towards a knowledge based economy, including by increasing and improving support for innovation and research, is seen as one of the main tools to achieve this. To this end, the reforms identified in the National Economic Reform Program (NERP) 2019-2021,7 are fully congruent with the project objectives and planned activities. For example, key issues identified in the research and innovation sector are discussed on page 98 of NERP, and they include: insufficiently efficient model of financing support to science, lack of support for business innovations, inadequate connection between business and academic community, and “brain drain”. The project will support the Government in addressing these main issues in the sector. As NERP is the key strategic document outlining medium term economic policy goals, having these reforms as an integral part of NERP contributes to their sustainability. 47. The Government has demonstrated solid commitment to implementing the reforms in the R&D sector. Following the adoption of the Strategy on Scientific and Technological Development of the Republic of Serbia, 2016-2020, the Government has, after initial delays, adopted the accompanying Action Plan setting out a credible roadmap for the reforms of the sector. One of the critical reforms identified in the Strategy and the Action Plan is changing the model of financing R&D activities, including a transition to competitive financing through the introduction of a new funding institution, the Science Fund. The Law on the Science Fund was adopted by Parliament in December 2018, and in the first half of 2019 the Fund become operational. The call for the first program supporting young researchers, financed by the Science Fund, was announced on June 21, 2019. Also, a Law on Science and Research was adopted by the Parliament in July 2019. In parallel, authorities have demonstrated strong commitment to these reforms through budgetary allocations: the funding available from the Budget for the Innovation Fund programs has been more than doubled for 2020 from the 2017 level, and funds have been allocated in the Budget for the initial programs of the Science Fund. It is expected that, as the Science Fund expands its capacity and portfolio of instruments, it will be able to leverage additional public resources (see Sustainability Action Plan in Annex 5 for details). 48. Sustainability of the R&D sector reforms is further strengthened by the commitment of other international partners, most notably the EU. In the 2019-2020 EU IPA funding cycle significant resources are being considered to support research and innovation reforms. This will include funding for capacity building in the Innovation Fund and Science Fund, and for new and existing programs of the two institutions. IV. PROJECT APPRAISAL SUMMARY A. Technical, Economic and Financial Analysis 49. The Project Net Present Value (NPV) is estimated at EUR 8.9 million at a 15 percent discount rate8, and the Economic Rate of Return (ERR) at 21 percent based on the total project investments. Our methodology accounts only for the project’s impact on direct beneficiaries rather than using a broader multiplier approach which would encompass positive externalities and spillover effects, along with overall growth of the innovation ecosystem in Serbia. 7Available at: https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/serbia_erp_2019-2021.pdf; see e.g. page 3. 8We estimate a 15 percent discount rate as the risk-adjusted opportunity cost of capital. Additional NPV estimates at other discount rates have also been provided as a measure of the sensitivity of our analysis. Page 21 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) 50. Due to difficulties in estimating a precise return from public sector reforms and pure technical assistance activities, our economic analysis focuses on the direct beneficiaries of Component 2: Enterprise Acceleration. That said, the investments under Component 1: Research Sector Reforms are likely to amplify the overall impact of the project by enabling growth of the wider research and innovation eco-system. 51. We have estimated the above-noted NPV and ERR based on the assumption that project beneficiaries are likely to have higher revenue growth rates and a lower failure rate than they would without project support. A discussion of these assumptions and their sensitivity analysis are provided in Annex 4. B. Fiduciary 52. All financial management and procurement activities under the SAIGE Project will be serviced by the Central Fiduciary Unit (CFU) within the Ministry of Finance (MoF), complemented by PIU staff, and staff of the project implementing entities – the Science Fund and the Innovation Fund. (i) Financial Management 53. The Project will follow traditional financial management arrangements, with one PIU and three implementing entities. The CFU, already established in the MoF, will be in-charge of fiduciary responsibilities for the project, while a Project Implementation Unit will be within the MoESTD. The Science Fund, the Innovation Fund and MoESTD will be responsible for technical aspects of implementation. 54. Based on the above arrangements and the project design, the overall financial management risk for the project is substantial. To finalize and implement the financial management arrangements (i) the Science Fund will need to be sufficiently resourced; (ii) Project Operations Manual acceptable to the Bank has been developed and approved in November 2019 and Grants Manuals acceptable to the Bank will be prepared prior to first call for each of the programs to be financed by the Project, and (iii) effective communication and reporting lines between the MoESTD, Innovation Fund, Science Fund, PIU and the CFU will need to have been established. 55. The annual audited project financial statements will be provided to the Bank within six months of the end of each fiscal year and at the closing of the project. The audit will be conducted by a private audit firm acceptable to the Bank and in line with ToR agreed with the Bank. The audit ToR will extend the scope in order to assess applied procedures with regard to grants and level of their alignment with Project Operations and Grants Manuals. 56. The CFU will submit a full set of consolidated interim un-audited financial reports (IFRs) consolidated for all entities and components for each calendar quarter throughout the life of the project. Acceptable accounting software will be used for project accounting and reporting, including principal financial reports being quarterly IFRs and annual project financial statements. 57. Internal controls and procedures to be used on the project will be described in Project Operations and Grants Manual, draft of which is expected to be prepared by negotiation. This will minimize risk of an error, safeguard project’s assets and ensure use of funds for intended purposes. Application of the controls and procedures in practice will be verified by the Bank’s supervision. Page 22 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) 58. The Designated Accounts in foreign currency for administering the project funds will be opened by MoESTD/PIU, Science Fund and Innovation Fund, respectively at National Bank of Serbia and commercial banks (acceptable to the Bank). Control environment in the NBS is considered to be acceptable. Allowed methods of disbursement will be advances to the designated account, direct payments, reimbursement and special commitments. Grants to beneficiaries will flow based on advance not to exceed 50 percent of the total grant amount, while following tranches will be disbursed based on verification by the SF, IF and CFU on the use of previous tranche for intended purposes. Such verification will include (i) review of quarterly financial reports prepared by beneficiaries and delivered to the CFU (ii) on site visits performed by the PIU and the external consultants. (ii) Procurement 59. The overall implementation and oversight of procurement will be carried out by the CFU. The initial procurement risk rating is substantial primarily due to limited staff capacity and coordination issues between the stakeholders, i.e., PIU/MoESTD, CFU, Science Fund, and Innovation Fund. Procurement will be conducted in accordance with the World Bank’s Procurement Regulations for IPF Borrowers: Procurement in Investment Project Financing – Goods, Works, Non-Consulting and Consulting Services (July 2016, revised November 2017 and August 2018). The project will also be subject to the World Bank’s Anti-Corruption Guidelines, dated July 1, 2016. 60. The prior review thresholds for substantial risk projects as provided in the ECA Regional Procurement Maximum Thresholds, effective January 2, 2014 (revised November 15, 2017) will apply, unless otherwise noted: Goods and Non-Consulting Services – USD 2,000,000; Consulting Firms – USD 1,000,000; and Individual Consultants – USD 300,000. Direct Selection will be in accordance with paras. 6.8 to 6.10 for Goods, Works and Non-Consulting Services and paras. 7.13-7.15 for Consulting Services of the Procurement Regulations. More details on procurement arrangements are provided in the Project Procurement Strategy for Development (PPSD) for the project. 61. Risks and mitigation measures are summarized in the table below: Risks Mitigation Measures CFU capacity to handle additional projects Hire additional staff (FM and Procurement specialists, ongoing) PIU capacity in preparing TORs and Hire consultants to supplement or increase technical specifications the PIU capacity. PIU capacity in evaluating bids and World Bank to provide training consultants’ proposals opportunities to be attended by PIU staff Selection of individual consultants World Bank to conduct prior review of contentious and long. critical positions and/or conduct due diligence for post review contracts. Coordination challenges between the Regular meetings between the PIU and the MoESTD/PIU and the CFU staff CFU and submission of quarterly progress reports to the Bank. Page 23 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) .C. Legal Operational Policies . Triggered? Projects on International Waterways OP 7.50 No Projects in Disputed Areas OP 7.60 No . . D. Environmental and Social . The project is classified as Moderate Risk taking in account the nature of the project, small size of sub-projects and the experience of the implementing agency in managing similar activities and the application of new and energy efficient technologies. Project will not directly fund civil works and no adverse impacts such as involuntary land acquisition, impacts on biodiversity, on cultural heritage, are expected. Also, the Project will not finance any of the activities listed in the World Bank Group Exclusion List. The environmental risks will be small in magnitude, of temporary nature and directly associated with the listed investments and TA activities under the Project. In few cases, the mitigation activities will need to be designed to deal with disposal of wastewater, communal, industrial or hazardous waste. The project may finance research involving human beings/tissue/embryos and animals if done in accordance with EU directives and procedures (Horizon 2020) and national laws and regulations on ethical research. Project will conduct public awareness raising about the applied research ethics and procedures to avoid misconception about this type of research. Any activities that may have moderate and significant environmental and social impacts, including involuntary impacts on land or assets, and unpredictable risks for the environment, community health and safety will be deemed ineligible through the Project’s Environmental and Social Screening Procedure to be used for defining grant eligibility. Moderate . impacts will be identified by the ESMF and addressed in activity-specific ESMPs.9 E. Corporate Commitments Citizen Engagement 62. The project is based on a series of engagements with the public and private sectors, including civil society organizations and individuals. These have included over 40 interviews, and over 14 focus groups, and roundtable consultations in the period between 2015 and 2019, with individual researchers, representatives of R&D institutes and University faculties, Serbian Academy of Sciences and Arts, Chamber of Commerce, Serbian Association of Managers, Serbian Business Angels Network, Serbian Venture Network, Digital Serbian network, representatives of the early stage investor community (StartLabs, South Central Ventures, ICT Hub Ventures, Serbian Business Angels Network) entrepreneurship support organizations such as incubators and hubs (StartIT, ICT Hub, Potkrovlje, Impact Hub, University of Belgrade Faculty of Engineering ICT Inkubator, Vojvodina ICT Cluster) and individual entrepreneurs, investors, and business leaders. Further, extensive analysis of the R&D sector, which included a series of 10 policy notes involved structured engagement with stakeholders of the R&D system in Serbia, while an entrepreneurship ecosystem assessment gathered stakeholders on a series of general and specific topics (for instance, difficulties of citizens to access services 9The draft ESMF and SEP were disclosed on October 22, 2019 on the MoESTD website, and published in a national daily newspaper on October 25. The public consultation was held on November 1. Page 24 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) provided by entrepreneurs via electronic payments). The project design further benefits from an entrepreneurial survey conducted in 2016 by the Serbia Bank Country Office, which gauged societal-level attitudes toward and opportunities for entrepreneurial activity. 63. Citizen engagement and awareness building will be continued at key stages of project implementation, and will be documented through annual plans and reports. Specifically, interested applicants will be able to ask questions and share comments about application procedures during open door days in advance of each call for proposals for grants to be awarded by the Science Fund and the Innovation Fund. Additional consultation periods will allow potential applicants to seek clarifications on the application procedures in person or via email. Public events will be held to announce each of the calls for proposals as well as the winners of each call. Moreover, two annual surveys will be conducted, one with beneficiaries of Science Fund programs, and another with beneficiaries of Innovation Fund programs, to gauge their satisfaction with and room for improving the programs. The results of these annual surveys and the associated changes for future programming will be captured in a dedicated indicator in the results framework. The data from these surveys will be gender disaggregated where possible (demographic data will be made optional to protect confidentiality). These arrangements are further defined in the Stakeholder Engagement Plan for the project. 64. A grievance redress mechanism will be established in each of the Science Fund and the Innovation Fund and any grievances tracked during preparation and implementation. Gender 65. According to the World Bank “Women, Business and the Law 2019 (WBL): A Decade of Reform” which measures gender inequality in the law and identifies barriers to women's economic participation, Serbia has WBL 2019 Index Score of 96.88 percent and holds 18th place, right after OECD high-income economies that have the highest average score of 100 percent, meaning that women are on equal legal standing with men across all eight indicators measuring how laws affect women throughout their working lives. 66. Moreover, in the science and research area in Serbia, women’s participation is relatively high, at 48.4 percent of researchers as of 2016 (UNESCO, Women in Science, 2018). However, the 2013 Enterprise Surveys data where 360 business owners and top managers were interviewed, shows that 29.8 percent of firms have female participation in ownership and 13.7 percent of firms have majority female ownership. At the same time, there is only 15 percent of firms with a female in top managerial positions. This indicates that women are less likely than men to be self-employed, firm owners, and firm managers 67. In the sphere of early-stage innovative entrepreneurship, the portion of companies with at least one woman founder is estimated to be 17 percent, while the portion of companies with all-female founding teams estimated at 2 percent (Digital Serbia Initiative, 2018). The share of women working in technology startups in the area of software development is approaching 15 percent, which, while above international standards, is still significantly below 50 percent. Of the previous (2011-2018) Innovation Fund matching grants recipients, 18% were led by women, which approximately reflects the proportion of women in the general population of innovative entrepreneurs. The percentage of women investors is likely to be considerably lower: while a small group (fewer than ten) of women investors has formed around the agenda of impact investing, no women are listed on the international AngelList directory of business angels active in Serbia. Page 25 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) 68. Access to finance for women managers and women owners in Serbia is lacking, despite gender-neutral legislation, particularly facing constraints in access to assets like immovable property, traditionally the key collateral requirement by financial institutions. Women are perceived to be disadvantaged in their ability to raise start-up capital. In recent years Serbia has made significant steps in improving access to property as it would enhance women´s opportunities to access finance and engage in entrepreneurship. 69. The project, through Component 2--enterprise acceleration, will encourage the participation of women entrepreneurs and women-owned businesses engaged in innovative activities in Serbia. The project will therefore track gender-disaggregated results for each indicator where this is possible (data for related PDO indicator and intermediate results indicator, see results framework). The project will track the number of female-led business that benefit from the enterprise acceleration programs and that upgrade/develop innovative products/services/practices because of business support provided by the project. Further, specific programming will be considered during design stages to encourage participation of female scientists and entrepreneurs in the Science Fund programs and diaspora activities. 70. Specifically, the project will seek to collaborate with local partner organizations, such as the Impact Hub, ICT Hub, Digital Srbija Initiative, StartIT, Serbian Association of Managers, Association of Business Women in Serbia, and others, to deliver programming specific to women entrepreneurs identified as missing from the ecosystem. This will include: • Mentoring by experienced women founders • Workshops on topics of specific interest to women founders and managers • Peer networking • Introductions to female angel investors 71. In addition, the Project will increase visibility of support for women entrepreneurs through customized communications campaigns executed through the press, television and radio, as well as social media. Visibility of initiatives targeting women entrepreneurs in Serbia was identified to be virtually non-existent in popular media, and recognition of existing programs was very low, according to a study by UN Women. Climate and Disaster Risk 72. Mean annual temperature in Serbia is projected to increase by 1.9℃ by 2100 with the highest increase during summer. Annual accumulated precipitation over the territory of Serbia in total is expected to be higher by between 3.5 percent and 6.8 percent, but with decreases during the summer by 3.2 percent and 4.5 percent. Serbia is a land-locked country, but increased frequency of flooding around major rivers is already being experienced. By addressing climate change issues, the project will contribute to the efforts of Government of Serbia to fulfill their international commitments established by the Paris Agreement and Serbia’s Nationally Determined Contribution to the United Nations Framework Convention on Climate Change. The project will primarily finance grants to firms and researchers, and through these the project will include activities targeted at (i) raising awareness of opportunities and risks related to climate change, and (ii) building mitigation and adaptation capacity in public and private sector through supporting projects in sectors that could potentially contribute to mitigation and adaptation. 73. More specifically Component 2 Enterprise Acceleration Program is expected to, among others, support enterprises from the sectors that contribute to climate change mitigation and adaption. This is based on the Page 26 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) experience with existing programs supported by the Innovation Fund. For example, a number of companies participating in the Mini- and Matching Grants programs and the collaborative grant schemes operate in the following sectors: energy efficiency (including both physical products, and software for energy management), advanced agriculture (including innovative seed technologies, use of resilient seed treatment, software for simulating environmental impact of different agricultural technologies) and recycling. On average, such companies represent 10 percent of the overall number of companies awarded Innovation Fund grants in the past. It is reasonable to expect that a similar percentage of companies from these sectors will also be the beneficiaries of the Enterprise Acceleration Program. Further, trainings and mentoring programs within Component 2 Enterprise Acceleration Program will include climate change topics, to point out business opportunities and raise awareness of risks related to climate change. In addition, through Subcomponent 1.1 Serbia Science Fund support will be provided to various research areas. Since the Science Fund is a new institution, it currently does not have a track record in supporting project from relevant climate change mitigation and adaption areas. Based on the broader research landscape in Serbia, it is highly likely that projects related to climate change will be supported by the Science Fund. Applications related to energy efficiency, energy management, renewable energy, climate risk management, innovative seed technologies, etc. are anticipated.10 At the open door days and workshops for researches these types of projects will be promoted and encouraged. 74. The project will also generate climate co-benefits by having all of the processing for grants under Subcomponents 1.1, 1.3 and Component 2 performed on-line. Currently, applications for all of the existing IF programs are being processed using on-line paperless procedure. This will be the case for the new program supported by Component 2. Similarly, SF grants supported with Subcomponents 1.1 and 1.3 will also be processed fully online. I. GRIEVANCE REDRESS SERVICES 75. Communities and individuals who believe that they are adversely affected by a World Bank (WB) supported project may submit complaints to existing project-level grievance redress mechanisms or the WB’s Grievance Redress Service (GRS). The GRS ensures that complaints received are promptly reviewed in order to address project-related concerns. Project affected communities and individuals may submit their complaint to the WB’s independent Inspection Panel which determines whether harm occurred, or could occur, as a result of WB non-compliance with its policies and procedures. Complaints may be submitted at any time after concerns have been brought directly to the World Bank's attention, and Bank Management has been given an opportunity to respond. For information on how to submit complaints to the World Bank’s corporate Grievance Redress Service (GRS), please visit http://www.worldbank.org/en/projects- operations/products-and-services/grievance-redress-service. For information on how to submit complaints to the World Bank Inspection Panel, please visit www.inspectionpanel.org. II. KEY RISKS . 10Recently adopted Smart Specialization Strategy for Serbia includes Energy Efficient and Eco-Smart Solutions as the horizontal priority topic; further, one of the “vertical” priority areas is Food for Future, which includes Sustainable Agrifood Production as a sub-priority. Page 27 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) 76. The overall risk for the project is rated substantial considering the individual risk ratings discussed below. All of these will require significant attention during the preparation and implementation stages. 77. Political and governance risk is substantial given the political processes in the country which may affect the project. Upcoming parliamentary elections in March 2020 may disrupt the continuity of the dialogue in the sector. This risk will be mitigated through early engagement with the new government and by the fact that much of the project implementation will be with the Innovation Fund and Science Fund, both of which are entities that are technical and should not be immediately impacted by possible political realignments after the elections. Further, close coordination with other development partners involved in the reforms in the sector, in particular the EU, will help mitigate this risk. 78. Sector Strategies and Policies risk is substantial given the complex nature of reforms in the research and development sphere. Given the decades long period of non-competitive funding for science, existing stakeholders, used to the current system, may question whether the proposed reforms and new instruments (including the Science Fund) will result in a more efficient science funding system overall, and may resist any changes. To mitigate these risks, the client project team will ensure regular and detailed progress reporting to the scientific community. This risk is also mitigated by the increase in the overall envelope for funding science, supported by the project. Other development partners (e.g. EU) are contemplating contributing resources to further increase the available funding. With more funding available, it is anticipated that the community will be able to have a relatively easier adjustment to transition to competitive funding. On the enterprise side, acceleration is an increasingly popular concept and the demand seems to be high; however, the success rate of innovative enterprises is relatively low worldwide (around 10-20 percent), and this will likely be at the lower side in Serbia. The Project team will ensure that participating companies and support organizations have realistic expectations through regular outreach. 79. Technical Design of Project risk is substantial given that the Project supports various institutions and programs of the research, innovation and entrepreneurship ecosystem. Risks related to technical design will be mitigated by ensuring that experienced advisors, hired by the MoESTD, Science Fund and Innovation Fund, provide advice and support to these institutions. To support this, the project design includes substantive technical assistance component and the team will work closely with counterparts in selecting and guiding the work of external consultants to support implementation of the project. 80. Institutional capacity for implementation and sustainability risk is substantial, given the diversity of actors in the RDI eco-system which are all individually relevant for different components of the project. Many of the actors have been involved in previous projects and have proven to be reliable implementation partners. Lessons learned from the decade of Bank’s engagement in the country related to this agenda will provide a solid basis for mitigating the risk. That said, the recency of the establishment of the Science Fund raises the risk. Substantive technical assistance and intensive support by the Bank team will help mitigate this risk. This will include reviewing, providing inputs to and issuing no objections to key governance documents of the new Science Fund, in order to ensure that its institutional foundations are sound. 81. Grants instruments carry particular risks if grant program design, selection criteria and procedures, and institutional oversight do not follow global good practice. These risks have been encountered and addressed through previous Bank projects in this sector, and are thus discussed in more detail under the Lessons Learned section above. A best practice to mitigate against many of these is the careful sequencing and phasing in of Page 28 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) programs (as the Innovation Fund has done to date), and scaling them up as institutional capacity is confirmed and demand becomes clear (as the Innovation Fund has done with programs piloted under projects supported by the Bank). A similar approach will be used with the Science Fund, so that the design can be adjusted based on experience and as its capacity grows. 82. Stakeholders Risk is substantial due to coordination of multiple stakeholders from government, private sector, research sector and donor organizations involved in the innovation and entrepreneurship ecosystem in Serbia relevant to the Project. The project design is based on the WB team’s ten -year engagement with local stakeholders providing technical assistance to the MoESTD and RDI sector. This process has also provided insights and lessons learned that will be built into the design and implementation of this project as mitigation measures, including how to ensure good coordination of actors. The government has committed to implementing significant reforms of the R&D sector and their continued buy-in will be crucial. Their commitment is already reflected in the adoption of relevant legislations and allocation of budgetary funds for implementation as of 2019 (and planned for 2020). In addition, stakeholder engagement, as described earlier, will further enhance ownership of the agenda. Page 29 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) I. RESULTS FRAMEWORK AND MONITORING Results Framework COUNTRY: Serbia Serbia Accelerating Innovation and Growth Entrepreneurship Project Development Objectives(s) The project’s development objective is to improve (i) relevance and excellence of scientific research; and (ii) innovative entrepreneurship and access to finance for enterprise growth, as a way of contributing to Serbia’s growth and competitiveness. Relevance of research refers to the potential of research results to be commercialized on the market, thus contributing to the economy, while excellence refers to international recognition of the quality of the research. Project Development Objective Indicators RESULT_FRAME_TBL_ PD O Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 Improving relevance and excellence of scientific research Number of scientific publications in top 10 percent of internationally 1,000.00 1,000.00 1,030.00 1,060.00 1,100.00 recognized journals (annual, not cumulative) (Number) Number of collaborative applied 0.00 20.00 50.00 100.00 150.00 research projects (Number) Improve innovative entrepreneurship and access to finance for enterprise growth External investment generated by participating companies (USD 0.00 0.40 1.40 2.60 4.00 million) Page 30 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) RESULT_FRAME_TBL_ PD O Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 Number of new or improved innovative products or services 0.00 0.00 20.00 35.00 50.00 introduced to the market (Number) PDO Table SPACE Intermediate Results Indicators by Components RESULT_FRAME_TBL_ IO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 Component 1: Research Sector Reforms Subcomponent 1.1 Serbia Science Fund Amount of grant awards by SF, 0.00 5.00 11.00 18.00 25.00 cumulative (USD million) Percentage of which going to projects with women principal 0.00 10.00 15.00 20.00 25.00 investigators (Percentage) Amount of international funds attracted by SF supported projects 0.00 0.00 1.00 3.00 5.00 (USD million) Number of publications supported 0.00 0.00 0.00 20.00 80.00 by SF programs (Number) Number of SF staff trained (Number) 0.00 5.00 10.00 15.00 20.00 Subcomponent 1.2. RDI Reforms Establishment of Institutional Law on Science and Research New financing model New financing model New financing model By-laws adopted Funding of SROs (Text) adopted implemented implemented implemented Number of internal assessments of RDIs (Number) 0.00 5.00 10.00 15.00 20.00 Page 31 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) RESULT_FRAME_TBL_ IO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 Number of external assessments of RDIs (Number) 0.00 0.00 3.00 6.00 6.00 Number of RDIs executing 0.00 0.00 1.00 2.00 4.00 transformation plans (Number) Number of collaborations between participating RDIs and private sector/international entities 0.00 0.00 25.00 50.00 75.00 (Number) Subcomponent 1.3. Sebian Diaspora Facility Number of diaspora members participating in the program 0.00 10.00 20.00 50.00 70.00 (Number) Number of Serbian researchers participating in the program 0.00 20.00 40.00 80.00 150.00 (Number) Percent of which are women 0.00 10.00 15.00 20.00 25.00 (Percentage) Number of joint publications by local and diaspora researchers 0.00 0.00 0.00 10.00 20.00 (Number) Number of international R&D institutions participating in the 0.00 2.00 5.00 10.00 20.00 Program (Number) Component 2. Enterprise Acceleration Number of enterprises completing acceleration program (Number) 0.00 0.00 40.00 80.00 100.00 Percentage of which have at least one woman founder 0.00 0.00 10.00 20.00 20.00 (Percentage) Number of accelerated enterprises achieving at least 10% annual 0.00 0.00 10.00 20.00 30.00 Page 32 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) RESULT_FRAME_TBL_ IO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 growth one year after the program (Number) Percentage of SF and IF survey respondents who report in annual surveys that effective engagement 0.00 0.00 50.00 60.00 65.00 processes have been established (Percentage) IO Table SPACE UL Table SPACE Monitoring & Evaluation Plan: PDO Indicators Methodology for Data Responsibility for Data Indicator Name Definition/Description Frequency Datasource Collection Collection MoESTD uses access to This indicator measures bibliometrics data to both the quantity and identify all publications quality of the scientific Bibliometrics affiliated with Serbian output of the Serbian data institutions in all Number of scientific publications in top 10 research community. A time measured by Annual scientific journals and MoESTD percent of internationally recognized lag can be expected Journal categorizes them journals (annual, not cumulative) between project activities Citation according to journal and their impact on Report - JCR impacts (measured by publication numbers and Journal Citation Report quality. - JCR) Number of collaborative applied research This indicator measures the Annual Project M&E MoESTD will assemble MoESTD projects improvement of relevance reporting data from both SF Page 33 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) of Serbian scientific research system projects and annual by looking at the number of reports of RDIs. projects which include collaborations with international researchers, diaspora, private sector and others. The indicator will include all such collaborations supported through component 1 of the project. This indicator measures the IF internal impact of the accelerator records and Data will be obtained External investment generated by program by looking at the Annual IF project M&E from IF. participating companies (USD million) ability of participating firms reporting to generate external system investments. This indicator measures the impact of the accelerator Data will be obtained program through looking at from IF. The IF will, for the final output of the verification purposes, participating companies in seek proof in the form the form of new or IF internal of transaction records, Number of new or improved innovative improved innovative records and Annual anonymized lists of IF products or services introduced to the products and services project M&E users of a new product market introduced to the market. reporting or screenshots of e- Products and services will system commerce websites be considered to be new or where the product is improved if they meet the sold. criteria specified in the Oslo Manual for measuring innovation. Thus, in addition Page 34 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) to products or services marketed for the first time, products and services will qualify if they achieve “improvements in technical specifications, components and materials, software used as part of the product or service, user friendliness, or other functional characteristics.” Further, multiple improvements will be counted only once per company, during the period of the project. ME PDO Table SPACE Monitoring & Evaluation Plan: Intermediate Results Indicators Methodology for Data Responsibility for Data Indicator Name Definition/Description Frequency Datasource Collection Collection This indicator will look at the level of SF disbursement SF internal of grants from loan funding. records and Data will be obtained Amount of grant awards by SF (USD It will also measure which Annual SF project M&E from SF million) percentage of this funding is reporting allocated to projects with system women as principal investigators. Percentage of which going to projects This sub indicator measures Annual SF internal Data will be obtained SF with women principal investigators percentage of SF grants records and from SF Page 35 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) (from loan funding) project M&E allocated to projects with reporting women as principal system investigators. This indicator will measure how much research teams supported by the SF will be SF internal able to generate additional records and research funding from Data will be obtained Amount of international funds attracted Annual project M&E SF international sources (EU from SF by SF supported projects (USD million) reporting and others). A time-lag can system be expected between the beginning of the their SF funding and obtaining additional grants. his indicator will measure Annual the scientific output of SF starting SF internal programs by looking at the from year 3 records and Data will be obtained Number of publications supported by SF SF quantity of publications of project project M&E from SF programs resulting from SF supported implement reporting projects. A time-lag of two ation system years can be expected. This indicator is measuring the intermediate output of SF internal the capacity-building of the records and Data will be obtained Science Fund supported Annual SF Number of SF staff trained project M&E from SF through this project by reporting looking at the total number system of SF staff trained through the project. Establishment of Institutional Funding of This indicator looks at the Annual Official Data will be obtained MoESTD SROs establishment of the proper Gazette and from MoESTD Page 36 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) legal framework that internal supports RDI reforms records of through establishing MOESTD institutional funding for public SROs. MoESTD internal This indicator will measure records, the number of internal implementati Data will be obtained assessments of RDIs Annual. on reports MoESTD Number of internal assessment of RDIs from MoESTD. performed under the from RDIs, project as a basis for and project external assessments. M&E reporting system MoESTD internal This indicator will measure records, the appetite for RDI reforms implementati Data will be obtained by looking at the number of Annual on reports MoESTD Number of external assessments of RDIs from MoESTD RDIs volunteering to from RDIs, undergo external and project assessments. M&E reporting system MoESTD internal This indicator will measure records, Data will be obtained Number of RDIs executing transformation the number of RDIs which Annual MoESTD implementati from MoESTD. plans undergo reforms supported on reports by the project. from RDIs, and project Page 37 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) M&E reporting system This indicator will look at MoESTD the output of the RDI internal Data will be obtained Number of collaborations between reforms by measuring the Annual records and from MoESTD from RDI MoESTD participating RDIs and private number of collaborations project M&E annual reports. sector/international entities established by RDIs with reporting private sector/international system entities. Number of Diaspora SF internal members which have records and Data will be obtained Number of diaspora members Annual SF initiated collaborations with project M&E from SF participating in the program Serbian researchers through reporting the Diaspora Program. system Number of Serbian SF internal researchers who have records and Data will be obtained Number of Serbian researchers established collaborations Annual SF project M&E from SF participating in the program with Diaspora and what reporting percentage of these system researchers are women. SF internal Percentage of researchers records and Data will be obtained Annual SF Percent of which are women participating in the program project M&E from SF. that are women. reporting system This indicator measures the Annual SF internal scientific output of the starting records and Data will be obtained Number of joint publications by local and Diaspora Program through from year 3 SF project M&E from SF diaspora researchers the number of joint of project reporting scientific publications. A implement system time-lag of two years can be ation Page 38 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) expected. This indicator measures the SF internal interest generated in records and Data will be obtained Number of international R&D institutions Annual SF international R&D project M&E from SF participating in the Program institutions for participating reporting in the program. system This intermediate indicator measures the number of IF internal enterprises completing the records and Data will be obtained Number of enterprises completing Annual IF acceleration program. It also project M&E from IF acceleration program measures which percent of reporting these are founded by at system least one woman. IF internal Percentage of enterprises records and Data will be obtained Percentage of which have at least one completing acceleration Annual IF project M&E from IF. woman founder program that have at least reporting one female founder. system This indicator measures the output of the acceleration program by looking at the IF internal number of enterprises records and Number of accelerated enterprises Data will be obtained which have, after Annual project M&E IF achieving at least 10% annual growth one from IF completing the program, reporting year after the program been able to generate at system least 10% annual growth, one year after program completion. Percentage of SF and IF survey Each year the SF and IF will Online The results of the Annually IF and SF respondents who report in annual surveys conduct one online survey surveys of surveys and how they that effective engagement processes have each for all of their grant have impacted the Page 39 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) been established programs and this indicator applicants for design of programs will measure the satisfaction SF and IF under SAIGE and future of grant applicants with the programs. programs will be institutions' engagement published on the processes. websites of the SF and IF. ME IO Table SPACE Page 40 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) ANNEX 1: Implementation Arrangements and Support Plan COUNTRY: Serbia Serbia Accelerating Innovation and Growth Entrepreneurship 1. The Ministry of Education, Science and Technological Development (MoESTD) will be responsible for the overall project coordination and implementation with specific agencies responsible for the implementation of their respective components. A Single Project Implementation Unit (PIU) will be established at the MoESTD reporting to the MoESTD State Secretary (Research and Technology) who as Project Coordinator will be responsible for coordination of the project with all project implementing entities (PIEs). The PIU will be responsible for all project implementation related activities including technical, operational, environmental and social safeguards, reporting, monitoring and evaluation, audits, studies, and capacity building, etc. The fiduciary activities related to the procurement and financial management aspects will be handled by the Central Fiduciary Unit (CFU) at the Ministry of Finance which is responsible to handle such activities for several Bank financed projects. The CFU will engage additional staff as necessary to handle this responsibility. Operations of the PIU including its policies/resources, responsibilities, as well as the Project Operations Manual (POM) outlining detailed project implementation arrangements including operating, fiduciary and M&E procedures, staffing, responsibilities, resources, etc. will need to be satisfactory to the Bank. 2. The Science Fund will be responsible for the implementation of Component 1.1 Science Fund and Component 1.3 Diaspora Facility and will hire/assign appropriate staff/consultant as required. The SF is a new entity and has no experience with the World Bank projects thus will require significant capacity building assistance under the project. The SF has set up a Governing Board and a Scientific Council. It will establish Program Boards, as and when needed, made up of professional experts which will be responsible for the design and evaluation of R&D projects. The following will be subject to World Bank review and approval: • Selection procedures of the SF Governing Board and Programs Board members. • Procedures for the selection and operations of the peer reviewers and selection committees for all SF programs. • Grant Manuals for all Bank supported SF programs outlining detailed policies and procedures including evaluation, procurement, environmental, reporting and M&E procedures for grants. 3. The MoESTD will be responsible for the implementation of Component 1.2 RDI Reforms. This will include selection of RDIs to participate in the project based on assessments of RDIs, both self-assessment to be done by each RDI management, and a detailed independent external assessment, to be conducted by a team of international experts (being initiated under the Competitiveness & Jobs Project). The MoESTD will also be responsible for the approval of RDI transformation plans and monitoring of their implementation; these plans will be subject to prior review and approval by the World Bank. The Project will finance technical assistance to MoESTD to undertake these activities as well as for policy and capacity building for the design and implementation of R&D sector reforms, preparation of future R&D and Innovation Strategy, laws, rules and regulations, etc. 4. The Innovation Fund will be responsible for the implementation of Component 2. Enterprise Acceleration. The Innovation Fund has years of successful experience in managing Bank and EU financed projects. Given that Enterprise Innovation is a new activity for the Innovation Fund, the project will include relevant technical assistance for enhancing IF’s capacity to manage this program. All Grants Manuals, for Project financed grants, outlining detailed Page 41 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) policies and procedures including evaluation, procurement, environmental, reporting and M&E procedures will be subject to prior review and approval by the World Bank. Financial Management Implementing Entities and Staffing 5. The Central Fiduciary Unit (CFU) will be in charge of fiduciary responsibilities for the project, while a Project Implementation Unit within the MoESTD, the Science Fund and the Innovation Fund will be responsible for technical aspects of the implementation. The SF will be responsible for the implementation of Component 1.1 Science Fund and Component 1.3 Diaspora Facility. The MoESTD/PIU will be responsible for the implementation of Component 1.2 RDI Reforms. The IF will be responsible for the implementation of Component 2. Enterprise Acceleration. 6. The CFU has undertaken fiduciary responsibility for all upcoming projects supported by the World Bank and other international donors. Currently, the CFU team consists of Director, Head of Operations, Procurement Specialist and Financial Management Specialist (FMS), and the hiring process for another FMS is under way. The cost of CFU staff will be shared across all the portfolio of projects supported by them. Planning and Budgeting 7. The project’s budget will be prepared by MoESTD/PIU, Science Fund and Innovation Fund for their respective allocations under project components, with overall support from the CFU. There is sufficient capacity for planning and budgeting within CFU in order to manage project funds in terms of optimal allocation, liquidity and overall performance. The Science Fund is a new entity and has no experience with the World Bank projects; thus, it will require significant capacity building. Variances of actual versus budgeted figures should be monitored on a regular basis, appropriately analyzed, and corrective actions taken. The CFU will prepare in-year financial plans and cash forecasts based on the project’s budget, thus ensuring adequate liquidity management and withdrawal of funds. Accounting System 8. Acceptable accounting software, administered by the CFU, will be used for project accounting and reporting. Accounting records should include appropriate analytics of expenditures per contracts and each specific payment. 9. The project will follow cash basis of accounting (cash based International Public Sector Accounting Standards), recording transactions when actual payment is done, rather than when they are incurred. Transactions should be accounted for within 8 days of incurring. There should be appropriate back up of accounting records on external drives, as well as appropriate security regulation with regard to access and editing rights of the financial information. Internal controls 10. Procedures and controls to be applied on the project will be detailed in the Project Operations Manual (POM) and, given the substantial amount of project allocated for grants, Grants Manual (GM). The POM should detail procedures and processes regarding planning and budgeting, accounting, financial reporting, internal controls, flow of funds and external audit for the project. It should also describe roles and responsibilities and communication channels and modes between the implementing entities and the CFU. This will minimize risk of an error, safeguard project’s assets and ensure use of funds for intended purposes. Application of the controls and procedures will be verified by the Page 42 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) Bank’s supervision. Some of the key internal controls to be applied for the project should include: (i) appropriate authorizations and approvals of all purchases, relevant documentation, transactions of payments etc.; (ii) segregation of duties as different persons are responsible for different phases of a transaction; (iii) reconciliations between project accounting records and other relevant sources of information (Client Connection, bank account statements, etc.) performed at least monthly by senior finance staff; and (iv) original documentation supporting all project transactions properly filed. 11. Component 1.1 Science Fund and Component 2 Enterprise Acceleration will finance grants to diverse group of beneficiaries so there is a requirement of the adoption of detailed Grant Manuals by PIEs, detailing procedures for eligibility, evaluation, selection and implementation of grants (flow of funds, reporting back). The manuals will be subject to the Bank review and no objection. Contract management 12. Contract implementation will be monitored by the implementing entities. Checks and controls of the total contract amount and payments which are due will be checked before each payment under contracts by CFU as well. Respective technical staff and CFU will review and approve invoices and accompanying documentation against contracts provisions for ceilings, dynamics of payments and quality of deliverables. Financial Reporting 13. CFU will submit a full set of interim un-audited financial reports (IFRs) consolidated for all implementing entities and project components and sub-components for each calendar quarter throughout the life of the project. The IFRs will be due 45 days after the end of each quarter. The format of the IFRs will be agreed between the GoS and Bank and attached to the minutes of negotiation and the POM. The following financial reports will be submitted to the Bank: (a) The Statement of Cash Receipts and Payments, (b) The Statement of Expenditure by Activity (c) Designated Accounts Statements (d) Statement of Grants Breakdown by Beneficiaries (e) Notes to the Statements. External audit 14. The annual audit of the project financial statements will be conducted by a private audit firm acceptable to the Bank, in line with agreed Terms of Reference (ToR). The ToR (and POM) have been agreed between the GoS and World Bank and attached to the minutes of negotiation and the POM. In addition, the audit will include extension of scope to grants and level of their alignment with the grant manuals. It should include verification of the adequacy of financial reports delivered by the grant beneficiaries in relation to the accompanying documentation (contracts, invoices) and performance review at least on a sample basis to ensure that agreed outputs are delivered in an efficient manner with respect to grant program. The audit report will be submitted to the Bank no later than six months after the end of the audited period. Audited project financial statements will be posted to the MoESTD website within two weeks upon the audit report being accepted by the World Bank. Page 43 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) Financial management covenants 15. The financial management legal covenants for the project, included in the Disbursement and Financial Information Letter (DFIL) is as follows: (i) CFU to maintain an adequate financial management system. (ii) CFU to prepare interim un-audited financial reports (IFRs) for each calendar quarter and deliver to the Bank no later than 45 days after the end of the reporting quarter. (iii) Annual project financial statements audited by a private audit firm acceptable to the Bank and such audit to be delivered to the Bank not later than six months after the end of the audited period. Action plan 16. The implementation of the following actions has been agreed with the GoS to ensure acceptable financial management arrangements. Table 1. Financial Management Actions Deadline Responsible Action Project Operations Manual POM prepared by negotiations MoESTD, IF, SF, CFU prepared describing controls and procedures for the project PIU, IF, SF with key functions Completed by negotiations MoESTD, IF, SF staffed established Funds Flow and Disbursement Arrangements 17. Project funds will flow from the World Bank Loan Account to the separate Designated Accounts (DA) opened by MoESTD/PIU, Science Fund and Innovation Fund, respectively at National Bank of Serbia and commercial banks (acceptable to the Bank). These will be foreign currency (USD or EUR) accounts from which the funds will be withdrawn and will be used only for the purpose of inflows and outflows under respective project’s components. Payments in foreign currency to contractors based abroad will be executed directly from DA. Funds needed for payments in local currency will be transferred to a corresponding RSD accounts opened with Treasury for the same purpose. 18. Each project implementing entity i.e. MoESTD/PIU, Science Fund and Innovation Fund will prepare separate withdrawal applications for the replenishment of their respective DA. Project funds will flow from the Bank to the respective DA either as an advance or based on the Statement of Expenditure (SOE) for already incurred eligible expenses. The World Bank can also execute direct payments to the beneficiaries on the basis of direct payment withdrawal applications. 19. The Ceilings for DAs are defined in the Disbursement Letter that accompanies the Loan Agreement. Applications for replenishment of the DA will be submitted monthly or when one-third of the amount has been withdrawn, whichever Page 44 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) occurs earlier. Documentation requirements for replenishment will follow standard Bank procedures as described in the Disbursement Handbook. Before funds from the Loan Account may be withdrawn or committed, the authorized representatives of the implementing entities, as designated in the Loan Agreement, must furnish to the World Bank, electronically through the Client Connection website (http://clientconnection.workdbank.org), or through an authorized signatory designation letter, the names of the officials authorized (a) to sign and submit applications for withdrawal and (b) to receive Secure Identification Credentials (SIDC) from the World Bank. Since there will be three separate DA, each implementing entity will have separate authorized representatives and will be obliged to furnish the names of the authorized officials separately. Chart 1: SAIGE Project Flow of Funds and Documents Page 45 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) ANNEX 2: Project Description The Project will consist of the following components and subcomponents: Component 1: Research Sector Reforms (EUR 34 million) 1. Subcomponent 1.1: Serbia Science Fund (EUR 22.5 million) – This subcomponent will finance certain grant programs of the Science Fund, likely to include a) basic science grants, b) applied research grants with incentives for promoting linkages between the private sector and the R&D community, c) grants for young researchers, d) incentives for enhancing collaboration with EU (e.g., Horizon 2020, Horizon Europe) and other international programs, and e) grants contributing to Serbia’s Smart Specialization Strategy (4S). TA under this component will support the operationalization of the Science Fund, including its governance, organizational structure, monitoring and evaluation (M&E) and other key aspects. The TA will also support designing SF programs based on international best practices. An activity focused on skills of researchers to access international financing and collaboration opportunities will be designed and integrated into the programs of the Science Fund. A Serbian diaspora program, described in Component 1.3, will be closely connected to the Science Fund. 2. Impact and sustainability of the Science Fund depend on the gradual development of its capacity and sequential introduction of new programs, based on lessons from early evaluation. This project will support the MoESTD and the SF in prioritizing and developing programs of the SF. To date, all research activities have been funded by the MoESTD. Thus, careful consideration will be given as to how and when to transfer responsibilities for each program from the MoESTD to the SF, as the SF gains experience and builds its institutional capacity including in dealing with the diversity of interests among the members of the scientific community and other stakeholders. 3. To ensure governance and decision-making structures modelled on international best practices, support will be provided in the following areas: • Governance: Besides formal management and governing bodies as defined by the SF law, it will be crucial for the Science Fund to include program selection boards composed of successful researchers and other relevant professionals from different fields. These boards will be primarily made up of international experts. As in the case of the Innovation Fund, the Serbian diaspora would be an important source of international experts who have strong ties to the country and who are willing to take part. • Evaluation procedures: The evaluation criteria and processes for each call for proposals need to be clear and transparent, and based on both qualitative and quantitative criteria. The first step in each evaluation should be a carefully designed international peer-review process. The SF will work with a broad database of international peer reviewers who will be paid per project proposal. • Decision-making: Financing decisions should be made by expert bodies, that is selection boards with reputable, mostly international, members. As these bodies cannot cover all fields of science, intermediary advisory groups for each major scientific field may be considered. • Program design: each program needs to reflect best practices and also the specificities of the Serbian research system. Funding allocations should be determined based on real needs instead of being equally distributed amongst all researchers. Page 46 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) • IP and know-how: the SF will need to develop clear rulebooks and guidelines on intellectual property developed through SF funded projects, to both protect the public investment and incentivize institutions and researchers to commercialize their results. • Monitoring and evaluation: all programs will need to be closely monitored and regularly evaluated, with robust M&E systems deeply imbedded in the SF’s work. This project will support developing the M&E and impact assessment framework. 4. The project will provide support to the SF in developing the above-mentioned key elements in several steps. Key activities for each step are highlighted below: Step 1: Preparation and Design • Preparation of Science Fund’s operating policy and procedures manual (rule book) • Training, study tours and mentoring of management, staff and selection boards • Design of support programs including preparation of grant manuals • Establishment of selection boards(s) and peer-reviewed system • Design of communication and promotional activities Step 2: Ongoing Technical Assistance • Operation of selection boards(s) and peer-reviewer system • Execution of communication and promotional activities • Design of monitoring and evaluation and impact assessment framework • Conducting relevant feasibility and foresight studies and assessments • Support to ministry, R&D organizations, researchers and SMEs for enhancing regional and EU collaboration (e.g., Horizon 2020, Horizon Europe++) and other international programs • Identification of opportunities, roster of skilled persons for educating and assistance in writing and following up of proposals. • Promotion of regional cooperation activities (e.g. joint evaluation of projects, joint regional proof of concept, etc.). • Enhancing skills of researchers to access international financing for scientific research, through a dedicated program of the Science Fund (related to above activities). Step 3: Direct research grants The project will provide financing for direct grants for public research financing under certain programs (to be agreed between the Bank and SF, during implementation based on POM criteria) designed and developed in phases 1 and 2 as described above. This component will be an addition to Government budget (and possibly from EU IPA 2019 Program) resources allocated for this purpose. Indicators: 1. Amount of grant awards by SF (USD million), and percentage of grant awards going to projects with women principal investigators (percentage) 2. Amount of international funds attracted by SF supported projects (USD million) 3. Number of publications supported by SF programs 4. Number of SF staff trained Subcomponent 1.2: RDI Reforms (EUR 8 million) Page 47 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) 5. This subcomponent will support public RDIs sector reforms by providing incentives for undertaking institutional reforms by RDIs on a voluntary (opt-in) basis. This will include providing financing to a select group of RDIs (4) who are willing and able to reform. 6. There are 184 accredited SROs in Serbia out of which 62 are RDIs and 121 are higher education institutions (HEIs). A majority of the RDIs are public (50) while there are 8 RDIs that are established as part of the Serbian Academy of Sciences and Arts, four (4) military RDIs (which will be ineligible for assistance under this Project), and five (5) privately owned. RDIs can apply for accreditation with the MoESTD under two categories: scientific institute (focus on basic research) and research and development institute (focus on applied research). Scientific institutes represent 29 out of the 62 accredited RDIs. 7. Overall, public RDIs represent roughly one third of the entire public research sector in Serbia, both in terms human resources and scientific outputs. A total of 5,644 staff are employed by public RDIs, out of which 3,659 are researchers. In comparison, a total of 12,069 researchers are funded by the MoESTD, in all RDIs and HEIs in Serbia. Public RDIs have accounted for roughly 23,000 scientific publications in the period 2009-2018 out of the approximately 61,000 in the whole country. Close to 60 percent of the MoESTD budget for researcher salaries is allocated to RDIs, however, this is due to the fact that university researchers receive their basic salaries from the education portion of the MoESTD budget. 8. While a number of public RDIs have achieved notable success, both scientific and market-oriented, there are still serious issues in the sector. A majority of the public RDIs are small institutions, lacking in critical mass, capacities and resources needed to tackle bigger research projects and to partner with international researchers or the private sector. At the same time, there are a number of small institutes with overlapping areas of expertise that could potentially be consolidated. 9. The legal framework for research financing in Serbia, until this year, incentivized primarily scientific publications as outputs, and provided only project-based funding which did not allow RDIs to develop long- term planning and research agendas at the institutional level. Further, governance of RDIs until recently gave very little power to the management, governing boards have been populated by researchers and government officials and science councils, comprised of staff, set the research agenda and made HR decisions. The new Law on Science and Research will help establish merit-based institutional funding of RDIs based on regular evaluations and will support long-term strategic planning at the institutional level. The newly established Science Fund will complement activities of the Innovation Fund in providing well-funded competitive basic and applied research projects. Resources developed under the IF’s Technology Transfer Facility can support RDIs’ efforts to commercialize their research. 10. This Component will build upon the successful TA experience under the Serbia Innovation Project where two public RDIs namely Institute of Physics Belgrade (IPB), and Institute for Molecular Genetics and Genetic Engineering (IMGGE) initiated important institutional reforms. A phased approach will be used. In the first phase, all RDIs will undergo detailed self-assessments. Second, RDIs will be invited to apply to undergo detailed independent external assessments. These will be conducted using criteria agreed with the Bank and will be conducted by a team of international experts. These assessments will evaluate key aspects of an RDI, including governance practices, staff capabilities, HR and other policies and processes, quality and quantity of R&D outputs, collaboration with other RDIs, private sector and international community engagement, as Page 48 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) well as potential for upgrading excellence and economic and social relevance of their R&D outputs. Based on these detailed assessment, a robust transformation plans for select RDIs will be prepared by the external assessment teams in close collaboration with RDI leadership and MoESTD, and satisfactory to the Bank. these will include detailed action plans with specific time bound milestones related to various aspects of RDI transformation. The project will provide financing for the implementation and monitoring of the transformation plans (and for RDI external assessments, as needed). The project support will be clearly linked with the timely achievement of specific milestones and will focus on enhancing the excellence and relevance of participating RDIs. No advance allocations will be made for any participating RDIs; instead, support will be provided only when they achieve agreed specific milestones in a timely manner. No transfer of Loan proceeds will be made to the selected RDIs; however, eligible expenses will be financed by the MoESTD. 11. Selected RDIs will be supported through assistance in: (i) upgrading governance systems, operational and financial structure; (ii) strengthening strategic planning and organizational management; (iii) human resource development through adopting appropriate policies, as well as staff training; (iv) introducing modern budgetary, accounting, Management Information System (MIS) and audit systems; (v) establishing, where applicable, profit centers; (vi) strengthening marketing, contract and intellectual property management and commercialization capacity; (vii) introducing transparent criteria for initiating and supporting research programs and individual projects; (viii) enhancing collaboration with HEIs, other RDIs, private sector and international community; and (ix) upgrading select laboratory infrastructure and facilities. This program is expected to have important demonstration effects and provide impetus for more public RDIs to undertake similar reforms in the future. Indicators 1. Establishment of institutional funding of SROs 2. Number of internal assessments of RDIs 3. Number of external assessments of RDIs 4. Number of RDIs executing transformation plans 5. Number of collaborations between participating RDIs and private sector/international entities Subcomponent 1.3: Serbian Diaspora Facility (EUR 3.5 million) – This subcomponent will support engagement of Serbia diaspora community in the development of research and innovation ecosystem in Serbia. 12. Serbia has a population of approximately 7 million and according to some estimates, almost 4-5 million diaspora members, including approximately 1.2 million in North America and 2-3 million in Europe. In 2018, they remitted approximately USD 3 billion to the country. Many Serbian diaspora members are well educated, networked and successful in their new countries. This diaspora community can contribute to the strengthening of the Serbian research, innovation and entrepreneurship ecosystem by providing scientific excellence, financial networks, knowledge, investments, new market opportunities, etc. During the last decade or so, Serbia has experienced a significant amount of brain drain as many young educated people (including researchers) have left the country due to limited opportunities at home. Recognizing this problem, Serbia has introduced some initiatives to benefit from diaspora (this includes involvement of diaspora in the Innovation Fund); however, these programs are small and have limited impact. Serbia should view its diaspora community as a ‘brain bank’ opportunity and not solely as a result and source of ‘brain drain,’ since many of diaspora members are well placed in their new countries and have a strong desire to assist their homeland. They can contribute in various ways, including participation in policy and program design and Page 49 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) implementation, membership in management and advisory boards, access to technology, finance and market networks, joint research, technology transfer and commercialization, problem solving for industry, investments and training and skills development, etc. 13. The project will support the launch of a Serbian Diaspora Facility (SDF), through TA and matching grants, to use the strengths and desire of its diaspora community to support the immense potential of innovation and entrepreneurship in Serbia. This program will aim to attract promising scientists, researchers and entrepreneurs from the Serbian diaspora to transfer knowledge and skills back to Serbia through a variety of activities. The following will be considered: (i) Collaboration in scientific and applied research and technology transfer. Scientific institutions within Serbia (or abroad) could host the potential recipients of the awards and extend necessary infrastructure and facilities in areas of high economic priority for Serbia. This could include undertaking joint R&D projects, filing of IP, R&D commercialization, visits of eminent expatriate scientists (and/or Serbian scientists abroad), or short-term placements of expatriate specialists with Serbian research institutions, HEIs and industry, and joint appointment of expatriate scholars in Serbian scientific institutions. Assistance may cover the cost of specific lab equipment, studentships/fellowships for the accompanying scholars, etc. (ii) Participation in policy making, governance, program management, advisory bodies and networking. This could include involvement of Serbian diaspora members in the design and implementation of various policy and programs, and membership in governance boards, selection committees and advisory bodies. The program could also support participation of diaspora researchers and entrepreneurs in the peer review, evaluation, and monitoring of R&D and innovation projects. This could also include support for MoESTD in strengthening Serbian scientific diaspora networks, in collaboration with the Serbian Academy of Sciences and Arts 14. This Program will be managed by the Science Fund. Safeguards will ensure that: (i) proposals/applications are peer-reviewed by a panel of prominent independent scientists and entrepreneurs from both within and outside of Serbia, who will provide written recommendations to selection committees; (ii) the nominations/invitations to the selected candidates are prepared with measurable outputs/outcomes; (iii) reviews of the financed initiatives are carried out by persons nominated by the Committees to ensure that objectives are (being) achieved and the funds are used as designated; and (iv) activities are audited annually by an independent auditor (together with other activities supported through the project). An appropriate M&E framework will be put in place to evaluate the effectiveness of all diaspora support programs. 15. The project will finance 100 percent of the grants provided by the Science Fund, which would cover up to 80 percent of projects costs. In order to ensure that diaspora collaborations are objective and demand-driven, sponsoring beneficiary institutions and companies will contribute at least 20 percent of the total project cost; such beneficiary matching levels will be established by the SF based on the objectives of each diaspora program. Financing may be limited to one project per one Serbian diaspora member (some exceptions may be allowed based on outstanding results and performance, but these will require specific approval by MoESTD). Strategic areas to be supported through this program would include life sciences, food and agriculture, information technology, nanotechnology, new materials and structures, and narrow and specific areas of special priority will periodically be announced by the MoESTD based on national priorities. Page 50 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) 16. The SF will set up the Program Approval Boards(s) in agreement with the Bank. Operating procedures and concepts will be reviewed one year after commencement of the Program and adjusted where needed based on the experience. 17. This Program builds on the Bank’s experience in implementing a similar initiative in Croatia. Through a Bank project, the Unity for Knowledge Fund (UKF) was established in Croatia to leverage the knowledge and expertise of Croatia’s Diaspora and link it to local researchers, businesses and policy makers. The UKF was successful in both increasing the quality of scientific publications of Croatian researchers, as well as enabling increased commercialization of research results. Indicators 1. Number of diaspora members participating in the program 2. Number of Serbian researchers participating in the program, and percent of which are women 3. Number of joint publications by local and diaspora researchers 4. Number of international R&D institutions participating in the Program Component 2: Enterprise Acceleration (EUR 7 million) 18. The enterprise acceleration program will consist of two streams: one for early (idea) stage, and the other for growth (scale-up) stage companies. Each stream will serve some 20 companies (teams of at least two founders) per year, selected competitively by participating investors (including angel investors, early stage VC funds, etc.). Each stream will consist of a structured, 2-3 month program of intensive training and mentoring, with 1/3 of the time spent on a joint curriculum (one per stream) and 2/3 of the time spent on targeted mentoring with sector-specific mentors (for instance, a cardio-vascular medical devices startup will ideally be paired with founders who have successfully “exited,” i.e., sold or taken to a public market, a company focused on cardio-vascular medical devices). The goal of the program will be to triple the valuation of the company, i.e., to rapidly advance its development. If, throughout the program, market feedback on a company’s product, service, or strategy, does not indicate high growth potential, mentors will work with company founders to “pivot,” or adjust strategy to market demand, if that is possible. Companies participating in the growth-stage stream will have already demonstrated “market traction,” i.e., evidence of demand, through the presence of users or paying customers, and thus, it is expected that most of these companies will be able to advance their growth during and after the program. Companies participating in the early stage stream may also need to reexamine the basic idea and assumptions behind the company, and, in some cases, decide not to pursue the idea further; however, founders will still benefit from an intensive course that will cover project and people management practices, strategy development and execution, market entry, partnership development, fundraising, and so on. The focus will be on businesses based on digital technologies, and specialized sub-groups could be considered (e.g. ICT, food and agriculture, health, biotech, education and gaming, etc.). Companies would receive non-dilutive investment (through a co- financing facility) and extensive technical assistance in negotiating with individual investors or early stage funds, which may be equity-based.11 19. The enterprise acceleration program would be structured as follows: 11 A similar model implemented at the regional level in Africa (XL-Africa.com) resulted in close to 50% of participating firms in the growth (scale-up) stream raising Series A funding (USD0.5-USD2.0 million), between three weeks and one year after the program. Based on this, the model was awarded the World Bank President’s Award for Excellence. Page 51 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) (a) Co-Investment Fund (up to EUR 4.5–5.5 million in total over four years): this would include co- investment grants ranging in value between EUR 100,000 and EUR 450,000 depending on the maturity and needs of the company. The Co-Investment Fund will finance matching grants, which would cover up to 60 percent of project costs. Private individual investors or funds will contribute at least 40 percent of the total project investment at the time of grant award (previously raised funds will not be eligible). Recipient companies will then benefit from intensive, tailored acceleration (described below under the Technical Assistance Facility), to maximize impact of the investment. Strategic areas to be supported include innovations in software and IT, life sciences, and food and agriculture, among other spheres to be confirmed. (b) Technical Assistance Facility (EUR 2 million over four years); this would include support for: ▪ the competitive selection of companies, program and curriculum development and implementation, and mentoring (approximately EUR 450,000 over four years); ▪ angel group/network creation and deal flow development through two channels, (i) cultivating and equipping an organized angel investment community, and (ii) working with science and technology parks, incubators, accelerators, hubs, and other entrepreneurship support organizations (for instance, through grants) to identify promising entrepreneurs and to improve their investment readiness; the program will include a knowledge transfer component, equipping local entrepreneurship support organizations with the ability to implement acceleration programs and strengthen partnerships with investors (approximately EUR 0.8 million over four years). ▪ regulatory and legal analysis and development of a policy reforms matrix to enable further development of innovative early and growth stage enterprises in Serbia (approximately EUR 300,000); ▪ participation of Serbian diaspora in mentoring innovative enterprises and making angel investments (approximately EUR 450,000); This This could also include support for the engaging diaspora network. A fledgling diaspora initiative, Serbian Entrepreneurs, has been launched by successful company founders in San Francisco and New York. 20. A number of Serbian and regional partners have already been identified in two categories (investors and entrepreneurship support organizations), including SC Ventures, ICT Hub Ventures, and StartLabs (as a source of financing and mentoring), StartIT, Digital Serbia Initiative, ICT Hub, Serbian Association of Managers, and Science-Technology Park (as a source of entrepreneurship support services and mentors), among others (for instance, the Chamber of Commerce is being considered but not yet confirmed). Program coordination is expected to be led by the Innovation Fund, with close technical implementation support by the World Bank. Indicators: 1. Number of enterprises completing acceleration program, and percentage of which have at least one woman founder 2. Number of accelerated enterprises achieving at least 10% annual growth one year after the program 3. Percentage of SF and IF survey respondents who report in annual surveys that effective engagement processes have been established Component 3: Project Implementation, Monitoring, and Capacity Building (EUR 2 million) Page 52 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) This component will finance project implementation and monitoring, including the operations of a Project Implementation Unit established at the MoESTD. This will include operational and fiduciary (procurement, environmental and social safeguards), M&E, project audits, studies, policy/program design and capacity building support to the MoESTD, SF, IF and selected RDIs. MoESTD. The project will finance consultants and non- consulting services, equipment, training and study tours, studies, audits, and operating costs, etc. Procurement and financial management will be handled by the CFU at the Ministry of Finance (project will finance relevant project related operating costs of the CFU). Page 53 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) ANNEX 3. Overview of Serbia’s R&D, Innovation and Entrepreneurship Reform Agenda Page 54 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) ANNEX 4: Economic Analysis 1. An economic and financial analysis (EFA) has been conducted across the project components to determine the value of the anticipated benefits relative to the costs associated with this project. The Project Net Present Value (NPV) is estimated at EUR 8.9 million at a 15 percent discount rate12, and the Economic Rate of Return (ERR) at 21 percent based on the total project investments. 2. In reference to this project valuation, several points are worth noting. First, this calculation is indicative, using expected average parameters for the project beneficiaries and the resulting impact of the project components. While these assumptions are based on the project team’s discussions with potential beneficiaries along with results of other projects and economic literature, they are not precise representations of the impact that the project will have. The exercise of financially modelling the project helps to ensure that project funds are being allocated to investments and activities that will provide a return to the local population, along with helping to identify key risks and thresholds for the project to achieve its desired impact. Additionally, international development projects often target regions and sectors where the risk is high since these types of projects can specifically take on higher risk investments because of the social good that could result from them—as such, financial valuations of these investments may not be as high as those in other regions, but the social value of such investments could be considerably higher. 3. That said, our methodology (detailed below) accounts only for the project’s impact on direct beneficiaries rather than using a broader multiplier approach which would encompass positive externalities and spill over effects, along with overall growth of the innovation ecosystem in Serbia. The value of such externalities is difficult to estimate; however, the social rate of return based on growth of the innovation ecosystem can be upwards of 60 percent. While this is not an exact estimate of the project’s value, this high value of broader social benefits from the project is worth underscoring. 4. Due to difficulties in estimating a precise return from public sector reforms and pure technical assistance activities, our economic analysis focuses on the direct beneficiaries of Component 2: Enterprise Acceleration. That said, the investments under Component 1: Research Sector Reforms are likely to amplify the overall impact of the project by enabling growth of the wider research and innovation eco- system via the Serbia Science Fund, R&D institute reforms, and the Serbian Diaspora Facility. The structure of these investments builds upon lessons learned from other projects in Serbia and the broader region, including those supported by other donors. 5. The economic analysis of this project is based on increased revenues for companies receiving financial and technical support under Component 2. We calculate our estimates based on the following parameters: - Number of beneficiaries: 100 recipients under the Co-Investment fund, evenly split between the start- up and scale up stages. Recipients will contribute 40 percent of the grant size delivered under the fund. 12We estimate a 15 percent discount rate as the risk-adjusted opportunity cost of capital. Additional NPV estimates at other discount rates have also been provided as a measure of the sensitivity of our analysis. Page 55 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) - Channels of impact: The primary assumptions in this analysis is that project beneficiaries will demonstrate additional growth above that without the project investments. Additionally, for the startup beneficiaries, we assume that project investments will result in a lower failure rate amongst beneficiaries. o Additional growth rate: We estimate a steady state revenue growth rate of 5 percent for firms without the project investments, with an additional revenue growth totalling at: an additional 30 percent for the scale-up beneficiaries, and an additional 50 percent for start-up beneficiaries. We assume a higher additional growth rate for start-ups due to their low initial revenue base. Depending on the sector, high-potential start-ups can achieve 5 percent revenue growth on a monthly basis off of their lower revenue base. The above assumptions are supported by a wide range of studies in different regions on the impact of different types of support services for SMEs. While specific data on this type of innovation support isn’t available, we’ve provided a summary of the different supporting studies below: ▪ Sarder, et al. (1997) study “The Importance of Support Services to Small Enterprises in Bangladesh” find 5-16 percent increase in employment, sales and productivity with technology adoption support provided to SMEs. ▪ Tan & Lopez-Acevedo (2005) look at the impact of SME programs in Mexico using panel firm data and find that 9-14 percent improvement in training and 9 percent improvement in technology absorption have been achieved. Various SME programs – Business Advisory Services, Technology Development, Credit, Supplier Development (1992 – 2000) in Chile presented similar results. ▪ Lopez-Acevedo & Tan (2010), “Impact Evaluation of SME Programs in Latin America and Caribbean” found 8 percent increase in wages and 9 percent increase in productivity as a result of these programs. ▪ A Harvard Business Review Study conducted by Anne Marie Knott calculates the impact of R&D investments in the US by estimating a Research Quotient (RQ) which defines a relationship between firm inputs (capital, labor, and R&D investments) and firm output (revenues) based on regression analysis of American firms. Their analysis estimates that a 10 percent increase in RQ results in an increase in market value of 1.1 percent, which can translate to a 10-20x multiple in firm revenues. Since this analysis is based on R&D investments in established firms rather than start-ups, we have increased revenue growth assumptions for the project’s start-up beneficiaries. o Reduction in failure rate: With the project investments we assume a 50 percent reduction in the failure rate due to the assistance provided to beneficiaries. For scale-up beneficiaries, we assume a 30 percent failure rate without project investments and a 15 percent failure rate with the project investments. For start-up beneficiaries, we assume an 80 percent failure rate without project investments and a 40 percent failure rate with the project investments. While these assumptions are broadly consistent with research on the impact of start-up accelerators and innovation Page 56 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) programs, these are extremely difficult to predict; as such we have specifically tested this for sensitivity. - Timeframe of impact: Due to the timeframe required for R&D, we estimate that impact of the project investments will occur in the years following the actual R&D support. We estimate a three-year delay for impact on start-ups, and an eight-year delay for impact on scale-up beneficiaries. 6. Additional assumptions are provided below: Scale up Startup beneficiary beneficiary Number of entrepreneurs 50 50 Failure rate w/o project intervention 30% 80% Falure rate w/ project intervention 15% 40% Average annual income (Euros) 1,000,000 50,000 Cost as percentage of income 80% 80% Job creation rate (per $ revenue) 0.000005 0.000005 Average annual salary (Euros) 20,000 20,000 Annual revenue growth (w/o project) 5% 5% Additionality to growth rate 30% 50% Number of years that additionality applies 3 3 7. With the assumptions noted above, the project valuation and other key metrics are estimated as follows: - NPV at a 15 percent discount rate: USD 8.9 million - ERR: 21 percent - NPV at a 12 percent discount rate: USD 15.9 million - Total jobs created over 15-year period: 1,900 8. Sensitivity analysis: - Reducing the estimated growth rate additionality for all project beneficiaries by 20 percent reduces the project ERR to 19 percent. - Increasing the estimated growth rate additionality for all project beneficiaries by 20 percent increases the project ERR to 24 percent. - Increasing the assumed failure rates with the project for scale-up and start-up beneficiaries from 15 percent and 40 percent, respectively, by 20 percent (to 18 percent and 48 percent, respectively) reduces the project ERR to 19 percent. 9. Reducing the assumed failure rates with the project for scale-up and start-up beneficiaries from 15 percent and 40 percent, respectively, by 20 percent (to 12 percent and 32 percent, respectively) increases the project ERR to 23 percent. Page 57 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) ANNEX 5: Sustainability Action Plan I. Summary 1. The SAIGE Project builds on past reforms, including those introduced through the “Research for Innovation Strategy, 2016-2020” and changes to the overall model for financing research through the Law on the Science Fund of the Republic of Serbia (adopted in 2018) and the Law on Science and Research (adopted in 2019). The Government’s commitment to the reforms is further affirmed in the National Economic Reform Programme (NERP) for 2019–202113 and the accompanying Budget allocations (discussed in more detail below). 2. The SAIGE Project will, inter alia, provide technical assistance for institution building of the newly established Science Fund, introduce a new enterprise acceleration program at the Innovation Fund, and support the MoESTD in undertaking further reforms, including those of public research and development institutes (RDIs) in Serbia. In addition to technical assistance, the SAIGE Project will also finance some of the grant programs managed by the SF and IF, to complement the GoS Budget funds. 3. Targeted technical assistance to these two institutions is already being provided through the ongoing Bank supported project Competitiveness and Jobs (C&J). As of July 2019, with funding from the C&J, the MoESTD is hiring three international consultants: - An Institutional Advisor to the Science Fund; - An R&D Sector Advisor to the MoESTD; - An Enterprise Acceleration Advisor to the Innovation Fund. 4. These consultants are expected to provide strategic guidance and capacity building to the MoESTD, SF and IF. They will be retained once the implementation of the SAIGE Project start, ensuring continuity of support. 5. Further, initially through the Serbia Research, Innovation and Technology Transfer Project (SRITTP), and then through C&J project, the Bank has been providing extensive support to the IF, and later also SF, in preparing grant manuals, relevant by-laws and other key documents. As part of the SAIGE project, all similar key documents will need to obtain Bank approval, ensuring their alignment with international good practice. 6. The Government’s commitment to the success of the IF and the SF is demonstrated in Budget allocations. The Government has allocated EUR 5 million for SF programs in its first year of operations (2019) and pledged another EUR 10 million in 2020. The Budget allocation for the Innovation Fund was approximately EUR 9 million in 2019 (and EUR 11 million expected for 2020), nearly double in value compared to 2016. 7. In terms of capacity, the Government has approved hiring 20 additional staff for the SF and seven for the 13 ERP is a key document outlining Governm ent’s medium-term economic policy priorities. It is prepared annually, as a part of the EU accession process. It builds on the macro framework set out in the Fiscal Strategy and goes into greater detail on sectoral reform priorities, based on sectoral strategies and action plans. Latest ERP for the 2019-2021 period is available at the Ministry of Finance website: https://mfin.gov.rs/pages/article.php?id=10923 as well at the EC website: https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/serbia_erp_2019-2021.pdf. For example, on page 64 of ERP 2019-2021 noted: “In the following period, the priorities include the establishment of an efficient model for science funding, greater support to business innovation, as well as stronger links between the business and academic community, resulting in higher private sector investment.” Page 58 of 59 The World Bank Serbia Accelerating Innovation and Growth Entrepreneurship (P170185) IF. Recruiting process is underway in both funds. II. Sustainability of the Science Fund 8. The SF was established by the Law on the Science Fund, adopted by Parliament in December 2018. The Government is fully committed to the new entity. In addition to the committed Budgetary resources, the Government requested additional support from the EU through the IPA 2019 program and the World Bank through the SAIGE Project. This international support represents more than just additional funding for the SF. The Government has learned from its experience in collaborating with the EU and WB on the establishment of the Innovation Fund and has requested to replicate this model to ensure that the SF is established on a sound basis, modelled following international good practices, with independent and transparent decision making – thereby building trust in the community while at the same time ensuring that the best researchers will be supported through competitive research funding. Even though there has been no contractual commitment, the MoESTD, SF and PMO have consulted significantly with the World Bank on the establishment of the Science Fund and design of its first instruments. 9. Additional provisions will be made in the SAIGE Project to help the institution gain momentum and reach sustainability as a critical institution for the Serbian R&D sector. This will include provisions for the Government to commit to securing full operational capacity of the SF for the duration of the projects and no-objections for all SF governing documents and programs, as well as key personnel appointments, grant evaluation boards, and grant manuals. III. Sustainability of the Innovation Fund 10. The Innovation Fund was established in 2011 with the adoption of the Innovation Law in 2010 (which has since been amended in 2013). As an autonomous Government institution, it reports to the MoESTD which secures its operational and program funding. Since then, it was supported through the EU IPA funded Serbia Innovation Project, and the Serbia Research, Innovation and Technology Transfer Project, executed by the World Bank. This project provided significant funding for technical assistance and capacity building of the IF, as well as funding for IF programs. Further, Innovation Fund’s governance and programs are supported under the ongoing World Bank’s C&J Project which includes Government obligation to provide certain level of funding to IF as well as prior review and approval of grant manuals for all IF programs. The Bank remains fully engaged through reviewing new program designs and providing no objections. 11. Since 2011, the IF has shown institutional resilience and has developed project management capacity; it is well acquainted with both EU and World Bank procedures and score very high on client satisfaction. As the number of programs increase, including the SAIGE supported business accelerator instrument, the IF will plan to build additional capacity. In addition, the World Bank has advised that the IF needs to build internal M&E capacities which has been agreed with IF management. The EU has also advised the IF to build appropriate capacities to directly implement IPA projects, which will in the long-run be essential for managing Structural funds. Page 59 of 59