Report No. 59207-LAC e Practice of Policy-Making in the OECD: Ideas for Latin America Design: Alejandro Espinosa | sonideas.com The Practice of Policy-Making in the OECD: Ideas for Latin America October, 2010 Public Sector and Governance Unit Poverty Reduction and Economic Management Department Latin America and the Caribbean Document of The World Bank Vice President: Pamela Cox Sector Director: Marcelo Giugale Sector Manager: Verónica Zavala Task Managers: Fernando Rojas Mariano Lafuente iv THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Contents List of Acronyms ························································································································································································································································································································ vii Team Members for this Report and Acknowledgments ······························································································································································································ viii Executive summary ····················································································································································································································································································································· 1 The Latin American pursuit of more technically sound policy-making. ······································································································································································································· 1 The Practice of Policy-Making in the OECD: Ideas for Latin America ············································································································································································································· 3 1 Introduction: A Framework for Describing the Policy Process ············································································································································································ 7 2 The OECD Policy Process: Characteristics and Challenges ·····················································································································································································13 Common Challenges················································································································································································································································································································································ 13 Common Elements in the Policy Process ························································································································································································································································································ 14 Differences among OECD Systems ······································································································································································································································································································· 19 Systems and Agencies dedicated to Policy Assessment and Evaluation in OECD Countries········································································································································· 22 3 Some Considerations for Institutionalizing the Policy Process in Latin America··················································································································25 Extracting ideas from the OECD experience applicable in the Latin American context ···················································································································································· 26 Ideas for Better Policy-Making in Latin America ···································································································································································································································································· 29 Annex 1: Government-Wide Evaluation Systems and Agencies: Common Features and Differences Across OECD Countries·····································································································································································································································································································································33 Annex 2: The United Kingdom ··············································································································································································································································································37 Public Management··················································································································································································································································································································································· 37 Main Routines of the Policy Process ···································································································································································································································································································· 37 Changes under the 2010 coalition government···································································································································································································································································· 41 Subsidiary Routines of the Policy Process ····················································································································································································································································································· 41 Concluding Comments ········································································································································································································································································································································· 42 Annex 3: New Zealand ······································································································································································································································································································44 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA v The Formal Public-Management Model ························································································································································································································································································· 44 Recent Changes ···························································································································································································································································································································································· 48 Concluding Comments ········································································································································································································································································································································· 49 Annex 4: Canada ························································································································································································································································································································50 Political Routines at the Center of Government····································································································································································································································································· 50 Budget and Expenditure Management Routines at the Center of Government ·········································································································································································· 50 Technical (Administrative) Routines at the Center of Government··············································································································································································································· 51 Concluding Comments ········································································································································································································································································································································· 53 Annex 5: The United States ·························································································································································································································································································54 Policy Routines at the Center of Government·········································································································································································································································································· 54 Policy (and Program) Assessment in the Executive ····························································································································································································································································· 56 Policy Assessment in Congress····················································································································································································································································································································· 57 Concluding Comments ········································································································································································································································································································································· 58 Annex 6: France ···························································································································································································································································································································59 Politics and Public Management ·············································································································································································································································································································· 59 The Policy Process in the Prime Minister's Office·································································································································································································································································· 59 The Economic Social and Environmental Council································································································································································································································································· 61 The Budget and the Policy Process········································································································································································································································································································· 61 The Policy Process in the Ministries········································································································································································································································································································ 62 The Particular Case of Project Evaluation ······················································································································································································································································································ 62 Parliamentary Vetting of Policies ··············································································································································································································································································································· 64 The Court of Accounts ·········································································································································································································································································································································· 65 Alternative Policy-Advice Organizations ························································································································································································································································································ 65 Concluding Comments ········································································································································································································································································································································ 65 Annex 7: Spain ·····························································································································································································································································································································67 The Policy-Making Framework····················································································································································································································································································································· 67 Policy and Program Evaluation in Spain ··························································································································································································································································································· 68 Recent Reforms to Improve Public Policy Evaluation: the Agency for Public Policy Evaluation and Service Quality (AEVAL) ··········································· 70 Concluding Remarks ················································································································································································································································································································································ 71 Bibliography ······································································································································································································································································································································72 vi THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Boxes Box 1. Policy Expertise ­ the Issues ········································································································································································································································································································· 11 Box 2. The Technical-Political Balance in the Royal Commissions ····················································································································································································································· 17 Box 3. Spending Reviews in OECD Countries ·········································································································································································································································································· 18 Box 4. Involving Social Actors: The Scandinavian Model ············································································································································································································································ 19 Box 5. Chile's Pension Reform: Political Consensus Based upon Commonly Shared Technical Information ······························································································ 20 Box 6. Public Acceptance of Large Projects: the French CNDP ··························································································································································································································· 21 Box 7. Coordinating Policy-Making in Chile: The Budget Directorate and the Second Floor ······································································································································· 27 Box 8. The Challenge of Coordinating Policy-Making in Mexico ······················································································································································································································ 27 Figures Figure 1. The Policy Cycle······································································································································································································································································································································· 8 Tables Table 1. The Policy Cycle ········································································································································································································································································································································ 8 Table 2. Alternative Sources of Institutionalized Policy Advice and Expertise for Government ······························································································································· 10 Table 3. The Policy Cycle and the Main Policy Instruments ····································································································································································································································· 15 Table 4. A Classification of Government-Wide Evaluation Systems and Agencies in Selected OECD countries ·················································································· 23 Table 5. OECD Policy Assessment Framework········································································································································································································································································· 24 Table 6. Technical Strengths and Weaknesses of Government Actors in the Policy Process ········································································································································· 28 Annexes Boxes Box 2-1. A Methodology for "Good" Policy-Making························································································································································································································································ 39 Box 2-2. Current Activities of the Strategy Unit (UK) ······················································································································································································································································ 40 Box 3-1. Two Examples of Policy Development in New Zealand ­ Economic Transformation and Skills········································································································ 45 Box 6-1. The General Inspectorate of Social Affairs (France)·································································································································································································································· 63 Box 6-2. Breaking the Monopoly of Specialized Knowledge in France····································································································································································································· 63 Box 7-1. Evaluations undertaken by Spain's Quality and Evaluation Agency in 2007 ····························································································································································· 71 Tables Table 1-1. United Kingdom: Formalized Evaluation Routines······························································································································································································································· 33 Table 1-2. New Zealand: Formalized Evaluation Routines ········································································································································································································································· 34 Table 1-3. Canada: Formalized Evaluation Routines ························································································································································································································································· 34 Table 1-4. United States: Formalized Evaluation Routines········································································································································································································································· 35 Table 1-5. France: Formalized Evaluation Routines····························································································································································································································································· 36 Table 1-6. Spain: Formalized Evaluation Routines ······························································································································································································································································· 36 Table 2-1. The Policy Cycle and Public Policy Instruments in the UK ········································································································································································································· 38 Table 3-1. The Policy Cycle and Public Policy Instruments in New Zealand ······················································································································································································ 45 Table 4-1. The Policy Cycle and Public Policy Instruments in Canada ······································································································································································································· 51 Table 5-1. The Policy Cycle and Public Policy Instruments in the United States ··········································································································································································· 55 Table 6-1. The Policy Cycle and Public Policy Instruments in France ·········································································································································································································· 60 Table 7-1. The Policy Cycle and Public Policy Instruments in Spain ············································································································································································································· 69 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA vii List of Acronyms Agencia de Evaluación de Políticas Públicas AEVAL Agency for Public Policy Evaluation and Quality of Services y Calidad de Servicios (Spain) CBO Congressional Budget Office (US) CNDP Commission Nationale du Débat Public (France) National Commission for Public Debate DPMC Department of Prime Minister and Cabinet (New Zealand) EOP Executive Office of the President (US) EU European Union GAO Government Accountability Office (US) Dirección General de Inspección, Evaluación y IEPSQD Inspection, Evaluation and Public Service Quality Directorate Calidad de los Servicios Públicos (Spain) IGAE Intervención General de la Administración del Estado (Spain) State Administration General Intervention IGAS Inspection Générale des Affaires Sociales (France) General Inspectorate of Social Affairs MAF Management Accountability Framework (Canada) NAO National Audit Office (UK) NGO Non-governmental organization OAG Office of the Auditor-General (New Zealand) OECD Organization for Economic Cooperation and Development OMB Office of Management and Budget (US) PART Program Assessment Rating Tool (US) PCO Privy Council Office (Canada) PMDU Prime Minister's Delivery Unit (UK) PMO Prime Minister's Office (Canada) PMSU Prime Minister's Strategy Unit (UK) PSA Public Service Agreement (UK) RGPP Révision Générale des Politiques Publiques (France) General Review of Public Policies (GRPP) SNCF Société Nationale des Chemins de Fer Français French National Railway Company SOI Statement of Intent (New Zealand) SPEAR Social Policy Evaluation and Research Committee (New Zealand) TBS Treasury Board Secretariat (Canada) viii THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Team Members for this Report and Acknowledgments T he research team was jointly led by Fernando Rojas, former Lead Public Sector Specialist (LCSPS and co-task team leader until retirement from the World Bank in April 2010) and Mariano Lafuente, Public Sector Analyst (LCSPS and co-task team leader). Geoffrey Shepherd (Consultant, LCSPS) shared in the drafting of this report, with participation from Mariano Lafuente, Fernando Rojas and Daniela Felcman (Consultant, LCSPS). The document partially builds on a previous report prepared by a larger team which also included Nicolás Bertello, Remy Prud'homme, Juan Manuel Quesada, Gisela Scaglia and Fanny Weiner. Karl Salgo (Director of Strategic Policy, Privy Council Office, Canada) commented extensively on the Canada Annex, David Shand (consultant) reviewed the New Zealand Annex, Professor Kenneth Apfel (School of Public Policy, University of Maryland) and Professor Phil Joyce (School of Public Policy, George Washington University) provided inputs for the United States Annex, and Jose Manuel Rodríguez (Senior Public Sector Specialist, LCSPS) provided inputs for the annex on Spain. Osvaldo Feinstein (Adviser, AEVAL), Harry Hatry (Director, Public Management Program, Urban Institute), Pierre Kopp (Professor of Economics at Pantheon-Sorbonne University, Paris I), Keith Mackay (Consultant, LCSPS), Nick Manning (Adviser, PRMPS), Kathryn Newcomer (Director of George Washington University's Trachtenberg School of Public Policy and Public Administration) and Yasuhiko Matsuda (Sr. Public Sector Specialist, EASPR) provided valuable advice for this report. Peer reviewers Francisco Gaetani (Executive Secretary, Ministry of Planning, Brazil), Ignacio Irarrazaval (Director, Center for Public Policy, Chile's Catholic University) and Verena Fritz (Public Sector Specialist, PRMPS) provided very valuable comments. Participants in a Seminar on "Strengthening Policy-Making Capacity in Latin America: What can be learned from the OECD?" ( June 1-2, 2010, World Bank, Washington, D.C.), which included practitioners and policy-makers from Latin American and OECD countries commented on a previous draft. Jonathan Cavanagh and Chris Humphrey edited the report. The team was assisted by Patricia Méndez, Sharon Spriggs and Claudia Nin. The team would particularly like to thank Marcelo Giugale (Sector Director, LCSPR) and Denis Robitaille (Manager, LCSDE) for the support in the planning and implementation stages of this study. Finally, the team would also like to thank the authorities of the Ministry of Finance of Spain for co-financing this activity through the Spanish Trust Fund for Latin America and the Caribbean. THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 1 Executive summary 1. Different studies show that trust in public institu- 3. While some of these challenges are strongly af- tions in Latin America tends to be, on average, the fected by the political reality of Latin American lowest in the world.1 While in some Latin Ameri- countries, such as their concentrated presidential can countries the introduction of performance systems, or are perceived to be shaped by cultural management appears to be the prevailing lever to aspects, which may take a very long time to change, reconstitute this public trust, increasing the quality there is clearly room for improvement in the prac- of public policies is a key stepping-stone which tice of policy-making. In this context, an analysis contributes to the same objective. This study sum- of the experience of some OECD countries, which marizes the practice of policy-making in selected are known for having achieved relatively better OECD countries and provides ideas for improving results from their public policies, may be a good the quality of public policies in Latin America. starting point, not only to get ideas on what road to choose but also alerts on which options to avoid. 2. Most Latin American governments are eager to These pointers can help Latin American govern- improve their policy-making process, but they ments in the process of improving policy-making. typically face, at least, five important challenges: (i) insufficient technical capacity at the center, or summit, of government to formulate policies or The Latin American pursuit of more evaluate them ex-ante or ex-post - by center of technically sound policy-making. government we refer to the political authority at the summit of the executive power, usually located 4. Developing countries tend to be weak in promot- in the prime ministerial or presidential office, but ing policy processes that are technically driven and central secretariats (for example, cabinet offices) technically sound. Latin America appears to be and ministries (for example, Finance) provide at a turning point in the balance between a very vital administrative support to the center; (ii) poor political and a more technically determined policy- coordination across policies coming from the sec- making process. Citizens are ever more demanding tors, leading to inconsistent and incoherent poli- of clear and relevant government priorities, precise cies; (iii) low levels of contestability during policy goals, outputs and outcomes that give value for formulation, whether within the executive, from money, reliable monitoring, disclosure of perfor- the legislature, or from outside the public sector; mance information, adequate financing to secure (iv) a disconnect between policy-making and the policy implementation, and objective evaluations. implementation of these policies; and (v) while Governments and congresses are increasingly politics shapes policy-making in all countries, aware of the need to substitute former input-based Latin American countries tend to have a much clientelistic political strategies with fresh, citizen- greater distance between proposals from different focused results. political parties as compared to the OECD . 5. In response to these demands, Latin American governments are under pressure to inform the 1 See for example Blind (2006) and Arizti et al. (2010) population of policy options, systematically discuss policy alternatives at a technical level, and justify America, those systems have largely remained at their priorities on the basis of a technically sound the program and project level. However, program cost-benefit analysis. Sophisticated universities, and project performance indicators are predomi- sharp private sector think-tanks, and articulate nantly based on process and output information NGOs are better equipped than in the past to and they are, in any case, only superficially linked hold governments accountable. These local actors, with higher government goals or the new priorities together with a more engaged international com- of an incoming administration. Program evaluation munity, have come to expand the traditional inter- is usually circumscribed to program goals; as such, national interest in financial sustainability broadly it has little or no say in relation to policy intent. defined. Today such an interest is at the forefront in analyzing value for money and interpreting the 9. Though improvement is noticeable, presidential of- perceptions of the citizen-client. fices often do not do a good job in technically spec- ifying core government performance indicators or 6. As part of their movement towards performance indicating the way executing agencies are expected management, Latin American governments are to contribute to those goals. As a result, the policy increasingly resorting to a more rational policy monitoring and evaluation functions located at cycle ­ and enhancing technical capacity to this the apex of government-wide performance-based effect ­ although the process remains largely systems are not yet as credible as they need to be. weak. Governments have not usually been strong Neither do they consistently contribute in practice in designing, analyzing, or coordinating policy to fighting budget inertia or securing the overall proposals. While policies are normally driven by an consistency of government programs. On paper, incoming President's agenda or a national devel- ministries and independent agencies align their ac- opment plan, concrete policy formulation may not tions to overall government goals or standardized follow or may not be based on a technical process. capacity strengthening programs, but in practice Policy proposals may be so far apart from what they observe their own performance targets and other strong parties would agree to, or not techni- tailor organizational development measures to cally convincing, that public or Congress support their own perceived needs. This discrepancy is may not be obtained. Analytical capabilities may exacerbated in the case of inter-government pro- reside in planning agencies, but these agencies do grams for joint action at sub-national levels. In the not currently have strong influence on either presi- end, the entire performance management system is dential offices or budget policy. Cabinets play a weakened by multiple sets of performance targets subordinate role and presidential offices tend to be and an array of different incentives that are not more political than analytical. Few agencies have consistent with each other or with government policy design expertise. priorities. 7. Up till the arrival of performance budgeting, 10. The current challenge is to strengthen control of budget processes used to be typically inertial, the quality of public policy through a better de- rather than attuned to accommodating policy pri- lineation of the way ministries and agencies are to orities. Where program monitoring and evaluation contribute to the country's priority development (M&E) capacity has developed, it has usually been goals (and an accompanying specification of indi- circumscribed to specific projects and programs; cators of expected results). Further integration of therefore, it has not fed very effectively into the the entire policy cycle is needed to adjust budget process of making new policies or correcting old priorities and regularly create fiscal space for the ones. Parliaments are usually technically weak, government's core program. It is also necessary though they too are becoming more transparent to fine-tune the relation between ministries, po- and more responsive to voters' demands in terms litically responsible for sector contributions to the of products and outcomes. Civil society initiatives, core goals of each administration, and the executive both institutionalized and spontaneous, to monitor agencies that are technically responsible for ex- and assess the governments' major policy programs ecuting government policies. Further clarification are growing in numbers ­though they may not yet of individual responsibilities is most important be sufficient to importantly influence policy mak- when it is necessary to coordinate actions between ing and adequately monitor policy implementation. central and sub-national levels of government. 8. Technical weaknesses at the policy level impact 11. In view of these challenges, the core administration negatively the quality of the performance manage- of the central government (typically presidency, ment systems that Latin American governments planning, finance) often over-reacts, directly im- are warmly embracing. As practiced in Latin posing its own goals and intruding with micro- 2 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA management that violates agency specialization 15. Evidently, the reference to Latin America as a whole and hinders the autonomy and creativity of execut- conceals a huge variety of institutional arrange- ing agencies. Attempts to over-extend the scope of ments, technical capacity and base-line develop- policy decisions to detailed implementation end ments in the policy and performance-management up by confusing the mandates and roles of govern- areas. The report's recommendations will likely fit ment units, further blurring accountability. There- better those Latin American countries that have a fore, protecting the essential boundaries between higher technical capacity and are more advanced government units has become a parallel challenge in building a performance management system at the policy level in Latin America. throughout the administration. Still, the listing of Latin American challenges and the study's 12. It is against that background of problems and recommendations are so broadly formulated that challenges of Latin American governments that they might fit, to a greater or lesser degree, most this study has been prepared. It is the expectation countries in the region. of this study that ideas drawn from the OECD countries' drive towards higher consistency and ef- 16. Having said that, the report is not about Latin fectiveness in government actions can shed light on America. It is about the ideas that can be identified the pros and cons of a given approach to enhancing from the policy making, policy implementation, the quality of policy making and its connection and policy evaluation cycle in six OECD countries. with implementation. Reducing the risk of poor There is no attempt to present the policy process policies and lack of government cohesion is more in a single Latin American country. That exercise likely when governments compare their own vision lies ahead, as part of a future research agenda. If against the mirror of governments that have had and when that comes, the analytical framework a long and serious experience with performance proposed in chapter 1 may prove valuable to study management. country cases and draw lessons from the Latin American experience. The next section summarizes 13. Clearly, this is an area where there are no standard the main findings from the analysis of the selected recipes. The scope and quality of the policy process OECD countries and the application of those are highly dependent on each country's idiosyn- findings as pointers for improving public policy cratic bridge between politics and administration making in Latin America. or the particular national balance between political calculus and technical assessment. The process of controlling the quality of public policies will vary The Practice of Policy-Making in the according to each country's institutional arrange- OECD: Ideas for Latin America2 ments. Therefore, this report cannot emphatically recommend adoption of this or that institution, 17. Policies are blueprints for public action. These process, or instrument. Nevertheless, controlling blueprints then need be implemented, evaluated the quality of policy making appears to demand and, where necessary, corrected. Such an ideal a certain technical and political empowerment policy process ­ or cycle ­ of "planning-doing- at the central level and among key government checking-acting" is, of course, subject to strong and non-government players without which no political influences, some good, some bad. Politics country seems to reach effectiveness and cohe- needs to be balanced with technical process and sion in policy making. It is those basic ­ yet not technical expertise. There is a gap in the public- sufficient-- features that this report selectively management literature on drawing lessons on identifies in OECD countries. Chapter 1 provides good practice in policy making.3 This paper tries the framework for identification of key features to start filling this gap by providing a summary and chapter 2 summarizes the findings from six description of what the policy process looks like in selected OECD countries. six OECD countries ­ to what extent it is orga- nized, technically rational, performance-focused 14. Once those key common traits among OECD and expert-based. But generalizing is difficult: the countries, and the reasons behind them, have been policy process is not highly routinized; patterns identified, the report moves on to quickly compare vary from country to country because political the basic common features of quality control of public policy with the problems and challenges for 2 "OECD countries" generally refers to those countries having acceded to mem- Latin American countries that have been summa- bership before the 1980s. Six OECD countries were studied for this report: rized above. This leads to some broad recommen- Canada, France, New Zealand, Spain, the UK, and the US. Two Latin American countries, Mexico and Chile are members of the OECD club. In this report, dations that are directly targeted to present-day these two countries are grouped as Latin American countries. Latin America. This is the task of chapter 3. 3 But see an IDB study on the Politics of Policies (Stein et al., 2005). 3 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA regimes and institutional arrangements differ; and 20. But there are also important differences among policy-making practices change continuously. A OECD countries. Most notably, in parliamentary principal reason for change has been the growing regimes, political authority is, de facto, monopo- size and complexity of government. This has made lized by the executive. As a result, cabinets and it increasingly difficult for the center of govern- their secretariats dominate policy coordination. ment to get the agencies to comply with its policy In the US, policy-making authority is, in practice, goals. And attempts to improve performance have shared by Congress and the executive branch; the sometimes unintendedly worsened the cohesive- cabinet and government agencies, though placed ness of policy. within the executive branch, are also influenced by Congress. For this reason, the President has built 18. Notwithstanding the differences among OECD up a large powerful secretariat, the Executive Office countries, there are common features to the policy of the President, to oversee policy. To mirror this process that are of long standing. First, a strong power, Congress has also built up its own strong political coordinating authority at the center of policy-process bodies. (In comparison, the policy- government is responsible for: making policies process bodies in the legislatures of other OECD consistent across sectors (and ensuring that pub- countries are weak.) There are also differences lic agencies cooperate to this end); and ensuring among the Westminster parliamentary regimes: that agencies implement policies as planned. This the UK has developed a policy-design and -as- authority is a cabinet (in the case of many parlia- sessment capacity at the center, while Canada and mentary regimes), a prime-ministerial office, or a New Zealand are more inclined to use the center to presidential office. Second, a strong (but not neces- implement processes that encourage the agencies sarily large) technical secretariat provides technical to follow good policy-management practices. advice and support to the central political author- ity. Third, a complex organizational system has 21. Two concluding observations about OECD ex- emerged to deal with complex and large govern- perience are relevant for other countries that need ment: traditional hierarchical structures are now to improve policy management. First, the policy complemented or partially replaced by devolved process in OECD countries does not and cannot instances (semi-autonomous and autonomous constitute a tight process like the budget system. agencies, local governments) and special arrange- This reflects the heterogeneous nature of "policy" ments to coordinate policy and implementation and the political nature of policy making (and the (cross-agency commissions, information and audit somewhat abrupt break between the sphere of systems, and so on). Fourth, strong cadres of senior politicians and the sphere of administrators). One public servants ­ most of them career profession- sign of this is that policy-management initiatives als ­ provide a crucial bridge between politics and can often have short and unsuccessful lives. Sec- technical expertise and between policy making and ond, the policy process is complex. Not only does it policy implementation. Fifth, the agencies them- depend on the overall architecture of government ­ selves generally have a strong policy-design and lines of authority, agency structure and incentives, -assessment capacity, though this can vary between civil-service arrangements. There are also multiple agencies. policy-related instances and routines ­ research and planning instances, M&E systems, spending 19. In addition, two important common trends in reviews, etc. This may seem like redundancy, but in policy management are emerging. First, a growing fact it reflects the challenge function inherent in performance culture is helping clarify the link be- the market-for-advice model. tween policy and implementation. Related to this, a number of instruments are strengthening the links 22. Any of the six country cases presented in the between policy making and the budget. These in- annexes will describe policy instruments and clude spending reviews, performance contracting, processes that may prove useful to Latin Ameri- and M&E systems. Second, along with the growth can governments wishing to improve their policy in government, alternative and contesting sources cycle. Practitioners and analysts may therefore of policy discussion and advice (advisory commis- review the annexes of the report to provoke ideas sions, think tanks, interest groups, and so on) have or draw their own inference from individual cases become important. While this movement is more and comparative analysis. Those individual cases important in some countries than others (the US notwithstanding, this report focuses on common more than France, for example), it betokens a long- ideas arising from the OECD experience. It is in term ­ and pretty much universal ­ shift from a the common lessons, not in country-by-country "monopoly-of-advice" model of policy making to a variations, that the most solid substance can be "market-for-advice" model. found. As seen from the previous paragraphs, 4 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA those common ideas lie in the realm of core secretariat to support this function. First, the of- policy actors, their responsibilities and their fice of the President is typically strong in political capabilities. matters, including managing relations with other branches and levels of government and coordi- 23. A voice of caution is needed before summarizing nating political support. And it is increasingly common lessons from the OECD experience. committed to demanding results from ministries Obviously, those lessons need to be tailored to the and executing agencies. But this office typically particular institutional framework of each country possesses little policy expertise. Obviously, it is before they are transplanted and adopted in Latin not appropriate for this report to propose political America. In addition, policy makers need to be solutions ­ for instance, it is simplistic and unre- aware of the timing and the political risks involved. alistic to mandate strong cabinets in presidential The recommendations listed below cannot be regimes. On the other hand, presidents can (and completed in a short period and they will require already do in some countries) convene effective leadership and political will as much as a good dos- political bodies for sector coordination, akin to age of expertise and reform continuity. However, cabinet committees, for economic and social af- taking heed of the lessons from OECD countries fairs, security issues, and so on. in these areas is needed if trust in the Latin Ameri- can state is to be rebuilt. Enhancing public sector 26. Second, technical secretariats in the presidential credibility requires building the proper institu- office are generally weak or non-existent. To take tional and instrumental bridges between political charge of policy, the President would need such a announcements, government priorities, flexible secretariat. Almost any of the OECD cabinet and financing, execution capacity, delivery of quality presidential offices covered in this report provides services, and citizen satisfaction. At this point in useful information for the design of a technical the region's drive towards a results-oriented rela- secretariat. The Executive Office of the President, tionship between the public administration and the provides perhaps the most relevant example for citizen-client (voter and taxpayer), these bridges presidential regimes, though the countries of the are essentially needed to strengthen transparency, region would be expected to start out on a much accountability and performance. The following six smaller scale. The UK's Cabinet Office or France's ideas summarize the application of those common Prime Ministerial Office are also interesting ex- elements of OECD country experience to the amples because they are pro-active in looking at challenges that Latin American countries often policy problems and proposing solutions. currently face. 27. (ii) An organizational system that coordinates policy 24. (i) Political authority and technical expertise at the making. Policy-making in the OECD is anchored center-of-government. The OECD country exam- in organizational structures, and in coordination ples indicate that good policy making begins with and control mechanisms. Organizational structures the combination of effective political authority and determine who does what tasks and organizational technical expertise at the center of government. hierarchies and boundaries set up particular sets of The political authority ­ a prime ministerial, cabi- incentives. Coordination mechanisms, such as cab- net, or presidential office ­ is responsible for the inets or more ad-hoc committee formats (or "lead government's program as a whole, overseeing the ministries" in the New Zealand case and "super- conception of policy ideas, deciding which policies ministries" in Australia) are designed to counter are to go forward, coordinating between sectors, the effects of public agencies pursuing their own and overseeing that policies are implemented as agendas, and instead make them focus on the goals intended. In a complex, modern economy, the po- cascading down from the center of government litical authority could not possibly carry out these into a particular policy area. Control mechanisms functions without the support of a strong technical are the means by which the principal at the cen- secretariat. This secretariat is expert in administer- ter seeks to have his/her priorities carried out by ing the operating processes that promote or vet the agent in the line ministry. There is an evident policy ideas, ensure their coordination and consis- problem of policy vulnerability in Latin-American tency, and attend to their implementation. countries, risking cohesion and effectiveness in the uncertain transition from government priorities to 25. The single most important step to improve the expected results. This makes this broad topic clearly policy-making process in the countries of the central for an agenda of better policy-making in region would be to strengthen the center of the region which involves such issues as: setting government, in terms, first, of a political capacity up a technical secretariat at the center of govern- for managing policy and, second, of a technical ment (and the implications of this for the roles of 5 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Ministries of Finance which, until now, have been strategic planning. More ambitious organizational expanding); opening up, where this does not ex- initiatives (such as ministerial research units) ist, a process of stronger contestability within the might be considered later. executive by establishing adequate standard proce- dures to test policies, politically and technically, at 30. (v) Strong capacity of the legislature. The US Con- cabinet meetings; creating a super-ministry or lead gress ­ the closest example to Latin American agency to coordinate policy-making in a particular presidential systems ­ plays a key role in shaping policy area; working through inter-ministerial policies and has a strong budget office and techni- committees; and other techniques for "joining-up" cal committees to do so. The technical capacity the distinct parts of the administration. of Latin-American parliaments to develop policy ideas and shape and evaluate policy proposals com- 28. (iii) Senior public servants who provide expert policy ing from the executive is not very strong. Despite advice and coordinate between policy making and im- not being as technically and politically empowered plementation. Senior public servants are appointed as the US Congress, they do have a mandate to according to both political and "hybrid" (political shape policy and are politically stronger than the mixed with technical) criteria in OECD countries, legislative branch of most parliamentary systems. depending on the country. Their role in bridging the Therefore, giving congresses a more solid technical world of politics and policy making and the world basis for making policies or assessing proposals of implementation is crucial. They play this role could reinforce contestability and improve policy- both at the center (in center-of-government sec- making in Latin America. The Congressional retariats, notably) and at the top of the ministries. Budget Office and the Committees of the US With a small number of exceptions ­ particularly Congress provide possible models that go in this Brazil and Chile ­ these professional cadres are direction. Many Latin American congresses are more notable for their absence than their presence already part of a worldwide movement that is driv- in Latin America. This is a broad reform, in many ing for results, and this will push them to demand ways beyond the scope of this paper, because it is a greater policy-making and -evaluation capacity one of the keys to better public management more of their own. generally. But it also has a particular importance for better policy making, principally because senior 31. (vi) Active alternative channels of policy advice exist. public servants provide the link between policy Finally, the development of alternative channels making and implementation and many of them are of policy advice, when capacity is available, clearly policy experts. represents another option for strengthening policy- making in Latin America. Following the review of 29. (iv) Strong capacity of the line ministries. Line min- OECD countries, there is a range of possible al- istries in the OECD tend to have strong analytical ternative institutional formats that could reinforce capacity in specialized units or departments. In ad- technical analysis by opening up the policy-making dition to their own financial, human resource and process to expertise in academia and, more broadly, asset management units, which are also common the non-government sector. Such initiatives can among line ministries in Latin America, OECD also encourage the participation of stakehold- ministries also sometimes have management in- ers--not as experts or evaluators, but rather as part formation systems at the sector level to inform the of the debate--in the consideration of different policy-making process. Many ministries in Latin policy alternatives. This process can bring viable America have a weak capacity to manage their policy alternatives to light, and it can minimize own policy-making activities and/or coordinate resistance from interested parties. the policies of the agencies they oversee. To im- prove this capacity, in a first stage, the ministries might (with help from the center) concentrate on improving the level (amount and quality) of information available to the minister to inform policy-making. This would enable ministries to better harmonize the broad presidential priorities they are responsible for with the specific programs that they and the agencies that are attached to them are designing and, later, implementing. To this end, strengthening the ministerial planning offices would give ministries a greater capacity to gather and analyze information and to use it for 6 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 7 1 Introduction: A Framework for Describing the Policy Process 32. This section suggests a simple, yet not perfect frame- objectives and methods of a further elaborated work on policy processes. It has the intention to policy design). Policies are largely identified and provide the reader with theoretical background and decided on in the political sphere. analytical focus related to the information on the OECD country examples that will be presented.4 35. Policies can cover "larger" or "smaller" areas, e.g. education policy versus sports policy. Narrower 33. In summary, the framework proposes to: policies can nestle within broader policies, e.g. military procurement within defense policy. While conceive policies as ideas and plans for in general, it is difficult and (for present purposes action, i.e. something that occurs before at least) not very profitable to distinguish between implementation and is (conceptually at different policy types, the focus of this report's least) distinct from implementation; interest will be on the policy process involving, typically, "larger" policies that target higher level see policies as sitting within a policy process ­ government goals, require greater public spending a cycle of activities from plan and design to and evoke greater political interest. An important decision, to implementation and evaluation; distinction is between policies that achieve their ends through public services (typical education, argue that there are three drivers of the health, security policies, and so on), hence require policy process: institutional arrangements, substantial bureaucratic resources of money and competition (contestability) among ideas, people, and those that achieve their ends through and policy expertise; and regulation (macroeconomic, fiscal, and trade, notably) or taxation, hence require fewer public provide some tentative suggestions about resources.5 This report covers both policy types. how good outcomes of the policy process might look like. 36. The policy process. This is the set of routines whereby policies are identified, decided on, implemented, and 34. What are policies? A policy is a plan to reach evaluated. For clarity of presentation ­ to provide a public-interest objective. The term is used some conceptual order in a complex world ­ this somewhat loosely: policy can range from an idea report starts out by positing a particular "ideal type" (broadly-stated intentions and, sometimes, meth- of policy process (see Table 1). Of course, the reality ods to achieve them) to a blueprint (more precise is far messier: this ideal type is not to be observed in any OECD country, but it is an aspiration, in some form or other, in these countries. In a first phase 4 Our framework has been influenced by Althaus, Bridgman and Davis (2007), who use a three-pronged framework for discussing the Australian policy pro- cess: political arrangements ­ how the distribution of power affects the locus 5 Some policies mix these different modes: for instance, a policy of CO2 reduc- of policy making; functional responsibilities within government, in particular tion might include a program of railway investments, but also a carbon tax and between politicians and civil servants; and the policy cycle. modified building codes. Table 1. The Policy Cycle Phase Stage * Policy identification: Identification of issues Work on policy alternatives also occurs here, inside and outside the government (see Table 2). Policy analysis Policy instruments 1. Policy-making Policy decisions: Consultation and coordination Assessment and Design This is the main area where policy choices are assessed and new policies are planned, the principal focus of this report. Decision Program formulation Program implementation: Execution Policy outcomes can also be monitored here. 2. Program Monitoring administration Program evaluation: Evaluation Policy outcomes can also be evaluated here. Correction of programs or policies * Loosely based on Figure 13.1 of Bridgman, and Davis (2007). Figure 1. The Policy Cycle Policy Policy Program Program Program Assessment Decision Formulation Implementation Evaluation and Design Source: Authors of ideal-type policy-making, policies are identified idea of a policy process is partly based and itself a (inside and outside the government) and the gov- particular sub-set of the policy process. It is also an ernment then agrees, through intra-governmental idealized concept. In the real world, as described routines, on which policies to design, adopt and by political scientists, government policy-making is implement. In a second phase of program admin- not, or not only, a linear set of technical routines, but istration, the public sector executes these policies a considerably more untidy process subject to many through programs, then evaluates whether the exogenous influences.6 First, this process has inputs programs achieved what was intended. Evaluation coming from all directions, including private inter- is not only an important part of the linear process est groups. Second, the nature and the complexity cycle, but it also contributes through its findings to of subject matters differ widely and the method- the design of new policies or corrects policy imple- ologies for identifying and designing policies are mentation (Figure 1). In this report, we use "policy process" and "policy cycle" interchangeably. Clearly, specialization and role differentiation are critical to 6 There is a substantial political-science literature on policy making (see, for instance, Allison and Zelikow, 1999; Fischer, Miller, and Sidney (2007). The protect transparency and accountability, particularly thrust of this literature is to understand who influences policy making and how. during the second phase of the policy cycle. There is abundant evidence that policy making is not dominated by neutral expertise. The views in the literature of how policy making actually works emphasize policy networks (communities that discuss and advocate policies), 37. The policy process is a loose concept, not strict in political processes, and political discourse. Different views of how policy mak- ing might best work emphasize pluralism (broad consultation, information shar- content, sequence, or timing. This stands in contrast ing, participation, consensus, "extended peer review" of citizens) or competing to the highly-defined budget cycle, on which the advocacy (policy contests), thus downgrading (not eliminating) expertise. 8 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA broad, ranging from technical analysis to imitation mostly undertaken in the earlier part of the policy of policies adopted elsewhere to postulates that process, both when policies are being identified build on ideology. Third, the precise dividing line and designed and when it is time to make decisions between policy and implementation is hard to de- on them. Policies can also be evaluated when les- fine. Fourth, many governments have adopted little sons are drawn (through M&E activities) during in the way of formal policy cycle routines, other or after the implementation of these policies. These than the traditional differentiation of roles between evaluations may then affect future policies. the executive, legislature and external auditing or control units, or the requirement of presidential 40. Drivers of the policy process: institutions. At its review and approval of those policies that need to simplest, the policy process is a statement of an ob- be submitted to the legislature. Programs and pro- vious linear progression: decision follows idea and gram evaluation do not always feed back into policy design, implementation follows decision; lesson- decisions. Therefore, the notion of a rational policy learning (sometimes) follows implementation. At a cycle is much disputed. Nonetheless, the policy more complex level, the policy process is an institu- cycle, however messy, does exist, and well-governed tional construct involving formal decision-making countries aspire to make it as effective as possible organizations and procedures (cabinet, budget, for their particular circumstances. This long stand- etc.), a formal division of labor (e.g. politicians lead ing aim usually intensifies as governments advance in identifying policies, civil servants in implement- towards performance-based management. ing them), and a set of institutions that help deter- mine the behavior of actors in the process. There 38. The two big phases of this idealized process are are institutional similarities and differences among policy making and program administration. This OECD countries. This diversity is the result of distinction between policies and programs is vital. A important variations in political arrangements and program is a detailed set of targets and arrange- paths of historical evolution. ments to implement a policy. Programs are mostly implemented in the administrative sphere of gov- 41. Drivers of the policy process: contestability. Contest- ernment. Obviously, policies shade into programs, ability is an additional institutional feature of the but the important distinction for present purposes policy process. A competition among ideas occurs is that policy making is ex-ante and program imple- when alternative sources of policy advice and ideas, mentation ex-post.7 By the same token, we use the whether inside or outside the government, have terms policy analysis (see below) to refer to an ex- gained, or can gain, the ear of the government in ante analytical activity within the policy-making the policy process. The policy process can go on phase and evaluation to refer to an ex-post activity outside the government--through the work of within the program-administration phase. independent think tanks, for example--or in a grey area between the government and the non-govern- 39. Policy analysis is the systematic analysis (or as- ment world--in commissions, advisory councils, or sessment or scrutiny) of alternative policies to public research institutes, to name a few (Table 2). provide information to public officials (mostly to In OECD countries, the channels of policy making politicians, but also to senior civil servants) to help have multiplied, thus sharpening the competition them make better policies and program choices. of ideas. Public policies require public scrutiny for three reasons. First, given the prevalent influence of 42. Drivers of the policy process: expertise. This study private interests in government, policy proposals concentrates primarily on institutions and sec- are not always in the public interest and must be ondarily on contestability. It was able to gather subject to testing. Policy analysis can help to chal- relatively little information on expertise. Exper- lenge private interests. Second, public sectors have tise is vital, but strong differences do not seem to grown substantially in size and complexity, so poli- exist between countries. This is an important area cies require analysis on account of their complexity. where more work is urgently needed. Box 1 sug- Third, there are always alternatives, and politicians gests that policy expertise can be viewed in terms and the public need to be informed as to which of outcomes, methods, inputs, and institutional alternatives have been considered. Policy analysis is context. 43. Outcomes of the policy-process. To judge the out- 7 Another formulation ­ not the one we have adopted ­ is to see programs comes of particular institutional arrangements, as smaller units (and projects as even smaller units) nested within larger units contestability attributes, and the availability and which are policies (and, similarly projects nested within programs), for instance: a public health policy, a national child immunization program, a school-district organization of policy expertise, there must be a immunization project. sense of what "good" policy-making looks like. 9 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Table 2. Alternative Sources of Institutionalized Policy Advice and Expertise for Government Government Intermediate Civil Society Central policy review and advisory organizations within the executive Temporary blue-ribbon government commissions Legislative support agencies Academic think tanks Permanent independent advisory bodies Policy advice central Legislative committee staff Advocacy think tanks to mission Contract research and Independent government audit agencies ministerial think tanks Central banks Political party think tanks Research-oriented NGOs Treasury and Finance ministries and research bureaus Corporate think tanks Interest groups Consulting firms to government Policy advice University research centers Government statistical agencies International lending agencies peripheral to mission Action- and service-oriented NGOs Supra-national organizations Professional associations Personal staff of legislators Source: Weaver and Stares (2001), page 9. The following criteria might provide the basis for 44. Within the scope of this report, the assumption is a discussion.8 Good policies are those that are: made (but with no intention to demonstrate it) that the policy processes of OECD countries that have i. Consensus-built among key stakeholders; been surveyed tend to have generally reasonable outcomes. This is clearly not true in every particu- ii. economically sound, i.e. they respond to a lar case, but this report assumes that these OECD public need in a cost-efficient manner; countries, however imperfect their policy processes and some of their results, have generally produced iii. implementable politically, i.e. the likely veto outcomes in terms of growth, equity, sustainability, points in congress or parliament can be and responsiveness to citizens that other countries managed; would wish to emulate. iv. implementable technically, i.e. not over- 45. Topics outside of the scope of this report. There are ambitious given capacity in the line some important aspects of the policy-making departments, etc. Policy design should process which were excluded from this report. be consulted with executing agencies and First, the analysis of how policies are passed - i.e. include broad guidelines as to the way the legislative cycle - is obviously relevant but is a these agencies are expected to contribute specialized topic ancillary to the main focus of this to higher level performance indicators or study work. The report does make reference to the otherwise align their own actions with legislative branch, but it does so when analyzing government priorities,; the actors who may be involved in the development or formulation of policies as opposed to their ap- v. technically responsive, i.e. when imple- proval. Secondly, the report does not touch upon mented they will address the political/tech- the role of the judiciary ­ e.g. how broad policy nical concern that led to their preparation; laws are then interpreted by courts - which can also have significant effects on the cohesiveness vi. sustainable, i.e. not leading to exponential of government's policies and actions. The latter costs and adjustable in the light of experi- is illustrated, for instance, by the growing role of ence; and the courts, particularly the constitutional courts or highest judicial authority in the definition of social vii. stable, i.e. not easily abandoned. rights that are protected by some Latin American constitutions. Third, the report makes reference to but does not analyze exhaustively aspects related 8 These are loosely based on the IDB study by Stein et al (2005). to the influence of political structure of OECD or 10 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 1. Policy Expertise ­ the Issues What do the attributes of good policy analysis and advice look like? Does good policy analysis and advice have specific attributes? In the early 1990s, for instance, the Australian govern- ment gave weight to the following attributes of analysis: accuracy, comprehensiveness, timeliness, responsiveness, clarity, practicality, appropriateness, fairness, cost-effectiveness, and consultation. The government also envisaged processes that: canvassed a full and comprehensive range of realistic options; provided options to support the Government's clearly-expressed policy priorities; outlined the pros and cons of each option; ensured ministers were provided with a whole-of-government perspective; and struck an appropriate balance between brevity and comprehensive detail. - The approach of economists would be to look at other attributes. Is there a public-goods issue to solve? Is regulation or public production the more appropriate response? Is the proposed solution cost- effective? - In any case, policy problems are not homogeneous: they may be large or small, cases of emergency or non-emergency; they will respond to different public-interest issues. Clearly no single standard of policy analysis will suffice. What methods of policy analysis work best? At the more general level there is a number of desirable steps in policy analysis, from identifying the issue through to presenting trade-offs. Here is a possible list of these steps: identi- fication of the issue to be address; identification of alternative ways to address the issue; estimation of the future environment within which the policy or program will be used; estimation of the future financial costs of each alternative (including both investment and operating costs); estimation of the future results of each alternative (including both intended and possible unintended effects) and corresponding monitoring indicators; estimation of the feasibility of alternative ways to implementing each alternative (such as potential political or technological barriers); presentation of the tradeoffs among the alternatives to public officials. There are also many different tools and instruments. Examples of tools that can be useful include: extrapolation of recent historical data, including time trend statistical analysis; simulation modeling; establishing standard sets of financial cost elements; use of surveys to provide insights into likely citizen behavior; use of expert judgment; use of pilot testing of at least elements of the policies or programs being considered. Examples of instruments are: cost-benefit analysis, cost-effectiveness analysis, and the Logical Framework. There are differ- ent ways of handling uncertainty and risk. Because policy problems are not homogeneous, the sophistication (and data-intensity) of the analysis will vary according to the time cycle in which the problem must be solved and the size of the problem. What are the resources required? Policy analysis requires resources that must be paid for. These resources include skills (including evaluation, statistics, drafting, and social, economic, administrative, engineering, legal, and communication disciplines). In addition, the quality (accuracy and timing) and availability of information feed directly into the accuracy and comprehensiveness of policy advice, so the quality of program monitoring and, more broadly, statistical services is vital. What are the institutional conditions (within the policy process) that best support effective policy analysis? This might depend on several factors: - How are policy issues identified? For instance, the issues might enter through the country's national develop- ment plan, the cabinet or the budget process, or might simply be the emergency product of crisis. - What constraints might specific attributes of the policy process put on the demand for policy analysis? This might depend, for instance, on the quality of political decision makers or political incentives (incentives for clientelism, for instance) and on transparency and accountability requirements. - What are the appropriate organizational formats for policy analysis? There are many possibilities: specialized units or departments within ministries, ad-hoc studies, financing of external research, central resource centers to help the ministries, cabinet level or presidential reviews. 11 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Latin American countries in policy formulation. faced by Latin American governments committed For example, when proposing ideas for improving to strengthening the quality of public policies. policy-making, the report does take into account the fact that most Latin American countries have 46. This introductory chapter discussed definitions and strong Presidential systems and that these differ concepts related to the public policy process and de- notoriously from Parliamentary systems, but we do scribes the scope of this report. Chapter 2 examines not focus on the differences in the policy-making the policy processes of six OECD countries, includ- process when a country has a coalition government ing federal countries, such as Canada and the US, vis- à-vis a one-party majority in a Parliamentary and unitary countries. The United Kingdom, New system. Fourth, and notwithstanding the increas- Zealand, and Canada have "Westminster" forms ingly important context of decentralization, this of government and were among the earliest of the report confines itself to the policy-making process reformers within the "New Public Management" in central governments. Finally, the report does school. The United States is of particular interest not explore political and institutional variations because of the way the presidential system of gov- among Latin American countries. Clearly, there is ernment affects options in the policy process. France visible asymmetry when it comes to institutional and Spain are civil code countries where policy- development, technical capabilities or political making is more constrained by legal fiat. These six arrangements of Latin American countries. How- country experiences are analyzed in more detail in ever, this report is not about policy making in Latin Annexes 2 to 7. Chapter 3 takes the OECD context America. Rather, it is focused on extracting ideas ­ as background to offer a number of possible direc- especially common ideas--from OECD countries tions that Latin-American countries might consider that appear to be relevant to common challenges as a means of strengthening the public policy cycle. 12 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 13 2 The OECD Policy Process: Characteristics and Challenges 47. This chapter summarizes the policy process in six and perhaps as a result of these developments--the OECD countries--Canada, France, New Zealand, tensions between politicians and technocrats have Spain, United Kingdom and United States--that sharpened. Democracy, too, is forcing the policy might provide useful policy pointers for improv- process to become much more complex and plural- ing the formal policy process in other countries. ist. Because the policy process is itself ill defined and because comparative study of the policy process is 49. Of course, different countries have also faced dif- sparse, the information on OECD country-specific ferent challenges. In particular, the policy process experiences is, inevitably, partial and incomplete. in Westminster-system countries seems to have Nonetheless, the pattern of policy processes--and deteriorated more than in other countries. In in particular policy assessment--described in this Westminster-system countries, the concentration chapter (and, in more detail, in Annexes 2 to 7) of political power in the executive and the looser offers a range of experiences and techniques that legal framework permitted more daring public other government may find illuminating and rel- management reforms. Indeed, in the UK and New evant. Zealand in particular, reforms separating policy- making from implementation (notably, through the creation of executive agencies) had the unintended Common Challenges consequence of undermining policy coherence across government. Thus, in the name of efficiency, 48. The policy process in OECD countries faces organizational reforms undermined policy coher- common challenges. In many OECD countries, ence and eroded the skills and the standing of the a century ago, policy-making was a closed sys- civil service. Efforts in the United States since the tem centering on the political center (cabinet or middle of the last century to build up a corps of President) and senior civil servants. Then experts neutral expertise at the center of the executive have recruited into the civil service, and thereafter been partially frustrated by the felt need of modern external experts and organizations, became more presidents to maintain political control through important. This "thickening" of the policy-advice political appointments. function was driven by the growing complexity and volume of tasks in government (and the cor- 50. By the 1990s the countries that had led the mana- responding growth of a technocracy and formal re- gerial revolution were sensing a crisis of policy search routines). Increasing complexity, the growth capacity. "Concerns about declining policy capacity in the size of government, the accompanying fiscal have been expressed by ministers, senior public ser- pressures (to "do more with less"), globalization vants and scholars. Prime ministers from Australia's and integration, and a growing public demand for John Howard, to New Zealand's Helen Clark to more open and results-accountable government, Britain's Tony Blair have stated publicly that they among other factors, are combining to make the have been underwhelmed, and at times let down, policy process more difficult. At the same time-- by advice from their bureaucratic advisers. The Australian Wheat Board (AWB), Iraq weapons sory organizations (including contracted research intelligence and `children overboard' controversies and think tanks) play an increasing role; and there are recent cases in point. In Queensland, Premier is a growing role for citizens' consultation. Mul- Peter Beattie, has complained bitterly about the tiple internal routines and external policy advisory quality of advice provided to him by public service organizations create contestability in policy ideas departments and agencies. Public service leaders and allow conventional thinking to be challenged have also expressed concern about declining policy (Weaver and Stares, 2001). Related to this, client advising skills and competencies ...." (Tiernan, or user consultation is being increasingly built 2007). into the process. (See OECD, 2003, on engaging citizens online.) The market-for-advice model has 51. No-one has measured policy capacity, but there is developed more quickly in some OECD countries a perception of a systematic decline in this. There (such as the US and the UK) than others (France, is a number of reasons (see Peters, 1996; Tiernan, for example). 2007). Public affairs have become increasingly dif- Common Elements in the Policy Process ficult to manage on several accounts. Their growing complexity not only demands high 53. The policy process is political and heteroge- skill levels, but also requires "integrating" neous. Policy ideas come from many directions policy-making across sectoral boundaries. and these ideas may be debated and tested in Globalization ­ and for European countries many ways. So it is very difficult to identify a the integration dimension ­ adds to the policy-making system, i.e. a set of formalized complexity. The public is growing more plu- routines. And what routines there are vary from ralist and more demanding. Inputs-focused country to country. At the same time, there is administration has been replaced by outputs what might be called a quasi-system with typical and outcomes-based accountability. political and economic routines at the center of government and, to a lesser extent, typical and There is said to have been a general erosion largely technical routines in the line agencies, of the skills and standing of the public ser- all within the common policy cycle of policy vice and, along with it, policy capacity. This identification, policy planning and design, policy is the result of not only fiscal limits, but also decision, program formulation and implementa- of politicians in some countries moving tion, and program evaluation. to curb the power of senior civil servants. These issues have been salient in the UK 54. How functional responsibilities in the policy pro- and Australia in recent decades. cess are divided between the center of government and the line agencies is crucial in characterizing Structural reforms, including marked sepa- national approaches to policy-making. Table 3 ration between policy planning and execu- distinguishes between three types of routine at tion, have also contributed: governments the center of government (political, technical- have become better at implementing, but administrative, and budget) and routines in the line often at the cost of strategic coherence. This agencies (largely technical-administrative). Insofar was the case with the early New-Public- as available information permits, the country-by- Management reforms of the UK and New country presentation of OECD experience with Zealand. policy assessment will broadly follow the charac- terization of the policy process included in Table 3 52. The challenges to the policy process, and the way (see Annexes 2-7). some governments are responding to them, make it possible to talk in terms of the partial shift from 55. Not all routines are carried out by all govern- one policy process paradigm to another. The old ments. But most governments, of both the par- "monopoly-of-advice" model, a closed and some- liamentary and presidential variety, have consti- what centralized quasi-system dominated by the tutional arrangements that subscribe to a broadly cabinet (or presidency) and technocrats, is gradu- similar set of political-authority structures and ally giving way to a "market-for-advice" model. In administrative structures that drive, respectively, this new model, policy responsibilities have become the policy-making and program-administration more decentralized to the line agencies; there are parts of the policy cycle. These are the core com- multiple possible overlapping administrative rou- mon elements of the policy process in OECD tines within the government; external policy advi- countries: 14 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Table 3. The Policy Cycle and the Main Policy Instruments The array of instruments assessing and coordinating policy and implementation within the government The policy cycle 2.1 Politics: 2.2 Technical: 2.3 Budget: 2.4 Technical: at the center of at the center of at the center of in the ministries government government government Policy identification: Proposals by the executive - Identification of issues Center-of-government (President, Prime - Policy analysis Minister, or cabinet) secretariats - Policy design, objectives Proposals by the legislature Specialized research/ Specialized research/ and instruments policy agency policy agency Ad-hoc studies Policy making Ad-hoc studies Policy decisions: Financing of The budget Financing of - Consultation and coordination Cabinet deliberation external research decision cycle external research - Assessment Parliamentary deliberation Technical support to Program/ project ministries (information, appraisal - Decision Public consultation, surveys methodology) Expenditure reviews - Program formulation Program implementation: Performance monitoring Ministry-level "Political" management - Execution information systems Value-for-money management ("early warning") reviews information systems - Monitoring Program administration Program evaluation: Performance audit Performance Evaluation system - Evaluation and evaluation evaluation systems or ad-hoc studies - Corrective actions Policy-making at the center is dominated by The annual (now, in many OECD countries, political considerations. The chief author- multi-annual) budget cycle, is also part of ity of government (column 2.1 of Table 3) the policy cycle at the center of government takes in, assesses, and sounds out ideas; then (column 2.3). Funding those programs that it decides on main policy design: the objec- require public money, the budget cycle is tives and instruments. This chief authority a mix of political decision-making and could be an individual (President or Prime technical routines. In OECD countries, Minister) or a collective (cabinet). In presi- and many others, the cycle is extending dential systems the legislature also plays from the policy-making phase into program a role. Given the primacy of politics, the implementation and performance (through process can be ad-hoc, especially for larger results-informed budgeting, spending re- decisions and those involving newer issues. views, monitoring, and, often, evaluation). Nonetheless, technical support plays an The ministries (and other government important role in political decisions at the agencies), at the periphery, are very im- center (column 2.2). Over time, center-of- portant, quasi-independent policy makers government technical secretariats ­ typi- and, of course, they bear the main re- cally cabinet secretariats in many parlia- sponsibility for program implementation mentary countries, the Prime Minister's (column 2.4). The ministries are connected office in France or the White House staff in to policy making at the center largely the US ­ have become more important. So through cabinet or budget processes. too has specialized technical support in the While they are politically-led and polit- form of dedicated research or advice bodies, ically-instructed, the ministries typically or ad-hoc commissions and studies. (France have a mixed political-technical role in is a country with a number of these bodies policy making and a more technical role in at the center ­ see Annex 6) implementation. Like the center, the min- 15 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA istries have also, over time, supported their between the center of government and the line policy-cycle routines through specialized agencies. The center uses its political authority to research functions, management-informa- coordinate the policy process. The center also tends tion systems, and evaluation systems. to the "big picture"--the broad lines of national strategy--or else it becomes directly involved in 56. Comparing Table 3 with similar matrix tables the more important or more contentious policies. prepared for each of the countries in annexes 2 In rarer cases, it may play a role in the line agency to 7, broad similarities emerge. Of the countries policy process by providing advice, information, or analyzed, all except for the United States have second opinions. Even rarer is a central interven- parliamentary systems (though in practice some tion in managerial decisions at the agency level: in of them, like France, have elements of presidential OECD countries the adage of "let the managers systems). Thus, it is not surprising that the United manage" serves as a rule of thumb for the division States looks quite different from the others. of labor between the center of government and the line agencies. 57. Common among the elements of this quasi-system in OECD countries is a notable break between 60. At the heart of the coordination process is the the policy-making and program implementation cabinet (or its equivalent such as the council of phases of the process. In the policy-making phase, ministers), a collective body for managing the politics and technical analysis are mixed together, policy process. The cabinet is supported by stand- organizations external to government (political ing sub-committees. These political organs are parties, think tanks, etc.) may be important, and supported by a cabinet or prime-minister's office there is comparative lack of routinization. The im- that manages the administrative routines of the plementation phase is more dominated by techni- policy process, but also provides a permanent body cal considerations, and routinization may, depend- of neutral technical expertise for the coordination ing on the country, be quite advanced.9 Notably, and assessment of policy proposals. Additional ex-post evaluation results tend not to feed back, methods of coordinating among line agencies as they are supposed to according to the "linear include inter-ministerial committees or organiza- model", into policy-making--they feed back into tional modifications (for instance, the creation of better implementation and, more rarely, budget mega-ministries in Australia or the designation of decisions. An obvious reason can be adduced why lead agencies in New Zealand). there is little feedback from program implementa- tion into policy-making: such feedback implies 61. The line agencies identify detailed policies and design an encroachment of the technical-administrative their implementation. Recognizing that the line sphere of civil servants on political prerogatives.10 agencies are the repositories of technical expertise, the center leaves the more detailed shaping of poli- 58. Current policy processes in the countries analyzed cies and their implementation to the line agencies. can be characterized by several common features, This study's review has not uncovered any domi- some long established and others developing. nant policy routines at the agency level. In some These features, seen most clearly in parliamentary cases, planning/strategy departments sit as the systems but also seen in the United States presi- "brains" at the top of the ministry and may serve as dential system, are summarized as follows. bridges between national strategies and managerial autonomy at the agency level, including monitor- 59. The center of government coordinates and leads the ing and alignment of agency performance goals policy process. First, there is a clear division of labor with overall government priorities. Elsewhere, line ministries may utilize policy-oriented commis- sions and councils appended to ministries, research 9 Australia had a centrally-directed evaluation system in the period 1988-96. departments, or external policy advice. Chile's monitoring and evaluation system, as led by the Budget Direction, is a prominent example of routinization in the implementation phase. Various OECD countries have established periodic spending reviews (United King- 62. Institutional mechanisms manage the supply of dom, France, Italy, and others). These routines are the exception rather than the rule and most of them are, in practice, geared more to better implementation expertise and the interface between politicians and than to better policies. As examples of routinization of ex-ante policy instru- the bureaucracy. Politicians in the government ments, France and Chile's Mideplan have institutionalized cost-benefit analysis for projects. dominate the policy-making process and are voted 10 There may also be a less obvious, if related, reason: it is difficult to match in and out of office. The civil service is a perma- the "tidy" categories of implementation with the "untidy" processes of policy-making. For instance, politicians have to package policy objectives and nent, professional bureaucracy that dominates the pronouncements in terms of what sells to electors; these are not necessarily program implementation process and provides categories that are convenient for administrators. Yet the current drive towards results-based accountability is demanding an alignment of policy and project politically neutral expertise to the government, results, from the top level of government to implementing agencies. even at the policy-making stage. Politicians are 16 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 2. The Technical-Political Balance in the Royal Commissions In Commonwealth states, a royal commission is established by the government as an advisory committee to in- quire into a matter of public concern on an ad-hoc basis. Royal commissions are called upon to look into matters of great importance and controversy. Royal commissions have at least an educative impact, and may contribute to policy proposals that are considered by the cabinet. At worst they are used as vehicles for diffusing political problems or are overtaken by the need to respond to events more rapidly. They have been used in countries such as the United Kingdom, Canada, Australia, New Zealand and most other Commonwealth countries. Royal commissions are set up by the head of state. In practice, once a commission has been established, the government cannot end it. Commissions may last many years and even outlast the government that created them. Royal commissioners have considerable power; however, they must stand by the terms of reference of the commission. Therefore, governments are usually very careful when defining the scope of the commission's work and generally include an ending date in the terms of reference. The results of royal commissions are published in reports of findings containing policy recommendations. Some of these are quite influential, eventually becoming law. However, many of them have been completely ignored by governments. Australia's attorney general recently commissioned a revision of its Royal Commissions Act of 1923, which was to be completed by the end of 2009. Among the reasons for initiating this review it was stated that "Royal Commissions have been an important means of inquiry and source of advice to government since Federation. But they have tended to be highly technical, time consuming and expensive [...] While there will always be a need to be able to establish a Royal Commission, it is timely to consider whether faster and more flexible inquiries may be appropriate in some cases". Some of the latest commissions in the OECD Commonwealth states are: the Auckland Governance Commis- sion (New Zealand, 2007-2009), the Victoria Bushfires Commission (Australia, 2009), Royal Commission on the Future of Health Care in Canada (Canada, 2000-2002), and the Royal Commission on Long Term Care for the Elderly (United Kingdom, 1998). Sources: Australian Attorney General Department www.ag.gov.au ; UK Parliament www.parliament.uk either elected or serve as delegates for elected service through various forms of "hybrid" appoint- officials, subordinated to government and politi- ments in which senior civil servants are selected to cal platforms. Civil servants respond primarily to serve in the most senior posts under ministers. In bureaucratic incentives. Therefore, ensuring the the last three decades, OECD governments have presence of high-level civil service expertise, as well tended to become increasingly concerned with get- as a smooth interface between the two groups, is ting a civil service more immediately responsive to central to a policy process which is able to align their political programs (Manning and Shepherd, government priorities and implementation. 2009). As a result, an increasing (though still modest) numbers of appointees to senior posts 63. Expertise has traditionally been ensured through are from outside the civil service, and there is an a merit system emphasizing competitive entry increasing tendency to base senior appointments into the civil service, reasonable compensation, job on fixed-term contracts that define the expected tenure, and so on. The career system, as practiced performance of the incumbent. in France and Spain (and Brazil), represents a par- ticular approach to anchoring policy in expertise. 65. Other instruments in the policy process: panels, royal Traditionally, Westminster systems have relied commissions, program evaluation and audit institu- more on generalists, though this has been chang- tions. Other long-standing elements of the policy ing. The US, in contrast, relies heavily on a mix process fairly common to the countries looked at of political appointments (in large numbers) and include ad-hoc "blue ribbon" panels to assess and civil-service appointments to secure this expertise. propose policies (see Box 2 on Royal Commissions in Commonwealth countries); parliamentary com- 64. Countries with parliamentary systems have man- missions to review policy proposals; external audit aged the interface between government and civil institutions attached to parliaments to carry out 17 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA performance reviews of programs that have been childhood development reform might be taken as an implemented; and ex-post program evaluation advanced example of external advice for implemen- carried out by the government (usually by the line tation of a fundamental policy. agencies). Perhaps with the exception of the "blue- ribbon" panels, these additional instruments have 67. In addition, many OECD countries are promoting not proven very important in the policy process. systematic spending reviews as part of the budget- Parliaments have lacked resources to do a better preparation process (Box 3). These reviews are job. External evaluators have typically looked at ef- largely meant to address efficiency and effectiveness ficiency and effectiveness in implementation and are in implementation, and the findings about "good" typically enjoined not to question the original policy and "bad" programs can then be used to adjust new intention. Ex-post evaluation within the executive budget appropriations. This tool does not appear, has, de facto, had a greater impact on implementation so far at least, to have provided a forum for discuss- than on policy design. The exception is the United ing the substance of policies, and it remains to be States, where Congress has played an active role in seen how far links between the budget and policy policy-making, mainly through its committees and processes will develop in OECD countries. the Congressional Budget Office (see Annex 5). 68. A short mention should also be made of policy- 66. Market-for-advice and spending reviews as new fea- making in Scandinavian countries (Box 4), which tures in the OECD policy process. As part of the shift has, for decades, required the active involvement towards a "market-for-advice" model, there has been of social actors (Rokkan, 1967; Hoefer, 1994). The a proliferation of alternative policy advice organiza- logic for involving social actors is both political tions, dedicated to providing more specific technical and to mobilize expertise. Even though the Scan- inputs into the policy process, inside and outside dinavian approach to policy-making may not be government (Weaver and Stares, 2001). The United appropriate for all societies, the basic concepts of States has long had such organizations outside the concerted involvement of social actors with their government, but their growing importance in the expertise in policy-making is an important model United Kingdom and Germany is also noteworthy. of governing and assessing policy. Meanwhile, some governments are creating new in- struments at the center to generate policy ideas and 69. This model has in recent times occasionally been provide information and methodologies for the line practiced in Latin American countries. In Chile, agencies. The United Kingdom is a leading example for example, the pension reform of 2008 involved of this (see Annex 2). Mexico's extended discussions an inclusionary engagement of actors with differ- for the recent reform of that country's financial sec- ent political views (Box 5). tor demonstrated the volume and sophistication of external agents able and willing to provide external 70. Participation of civil society has also been used at advice in this particular area of policy reform. Chile's the phase of ex-ante policy assessment by countries committee of experts responsible for providing criti- such as France. For example, this type of civil society cal calculations for the application of that country's engagement has the potential to deal with public fiscal rule, the social security reform or the early resistance to large investment projects (Box 6). Box 3. Spending Reviews in OECD Countries Line ministries conduct program reviews (some of these formalized as ex-post evaluations) as part of their normal business. Some countries have systematized program reviews at the center of government through so- called "spending reviews". These reviews ask whether given outcomes could be achieved with a different mix of programs or working across different sectors. Spending reviews are special forms of program evaluation in the context of the budget process under names such as "Strategic Policy Reviews" (Australia), "Expenditure Reviews" (Canada), "General Review of Public Policies" (France), "Interdepartmental Policy Reviews" (the Netherlands), "Spending Reviews" (United Kingdom), and the "Program Assessment Rating Tool" (United States). These procedures are devices to support the allocative (priority setting) function of the budget. There are three main differences with the program reviews conducted by line ministries. First, spending reviews look not only at ef- fectiveness and efficiency of programs under current funding levels, but also at the consequences for outputs and outcomes of alternative funding levels. Second, the Ministry of Finance or Planning holds final responsibility for the spending review procedure. Third, the follow up to spending reviews is decided in the budget process. 18 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 4. Involving Social Actors: The Scandinavian Model In the Scandinavian countries, involvement of social actors may occur in a variety of ways. All ministries have powerful advisory boards with broad membership of social actors and other government organizations that provide advice and monitor the decisions of the ministries. Most major legislation in Sweden and Norway is prepared by commissions composed of experts and political representatives, which do research to support the proposed legislation. These commissions often take months or years to complete their work, hence this model may not be appropriate for many political systems, but it does provide a model for broad deliberation and expert involvement. This OECD precedent was particularly relevant for the analysis of policy design and implementation of the 2008 pension reform in Chile, and its replication to policy reform and evaluation in similar areas. For less significant decisions, civil society groups are consulted through petitions, allowing them to provide their views and expertise to inform decisions. These petitions are circulated widely among interest groups so that a broader range of views is included in the process. The separation of policy-making--by the ministries--and implementation--through the agencies--allows the ministries to focus on policy. This is the case of Sweden in particular and to some extent of the other Scandina- vian countries. This model has, of course, been copied in other systems but has not been institutionalized with the same degree of expertise. Again, this precedent is widely discussed in the core section of the report since Chile is the Latin American country that has most faithfully adopted the separation between policy making/ coordination (ministries and presidency) and execution (services). Finally, in Denmark and to some extent in the other Scandinavian cases, a large number of policy decisions are made through networks composed of public and (mostly) private actors (Sørenson and Torfing, 2007). Again, the logic is to involve the expertise of the social actors directly in the process, but this also means that a number of decisions are delegated to these structures. Differences among OECD Systems political party and its leader (the Prime Minister). The cabinet then becomes a powerful coordinating 71. The most fundamental difference we have observed mechanism. Proportional representation voting is between parliamentary democracies and the systems tend to result in governing coalitions. United States system of checks-and-balances. The These systems make policy-making a slower pro- United States policy process is thus the exception: cess because coalitions have to be built and the both the President and Congress make policy, and ongoing potential for disagreement has to be man- the struggle between the Congress and the Presi- aged. This became evident in New Zealand when it dent for control of the agenda of the line agen- adopted proportional representation in 1994. The cies means that the President does not exercise system also tends to reproduce some of the inter- full control over his cabinet (see Annex 5). As a party competition, normally seen in elections, as result, the President has had to build up his own potential political divisions within the cabinet, office--the Executive Office of the President--to somewhat akin to what United States presidents strengthen his influence over the policy process. face. For instance, ministries that fall into the hands of junior coalition partners tend to become 72. The core of the policy process in the parliamentary policy fiefdoms that the Prime Minister and the systems is the cabinet, which ensures a collective cabinet have difficulty controlling. approach to vetting policies, resolving differences, and making sure that policies are implemented. 73. Additional differences in the practice of parlia- One of the most important variants to the policy mentary governments that have emerged from the process is a function of voting systems. First-past- study are: the-post voting systems (such as in the United Kingdom) tend to give considerable power, for the In the United Kingdom (see Annex 2), duration of the political mandate, to one particular the center of government (especially the 19 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 5. Chile's Pension Reform: Political Consensus Based upon Commonly Shared Technical Information One of the first steps of President Bachelet after taking office in March 2006 was to establish a high-level techni- cal council, the Advisory Council for Social Security Reform, to draft proposed adjustments to the Pensions System adopted in 1980 that would prevent shortfalls that could affect current and future generations of workers. The 15 members of the Council, led by former Budget Office Director Mario Marcel, represented the entire political spectrum, including technical staff close to the Workers' Confederation (Central Única de Trabajadores, CUT) and one former minister from the Pinochet government. They had been chosen by the President because of their unquestionable technical qualifications and because of their readiness to engage in dialogue. Many of them had studied the Chilean social security system since the early 1990s and had managed over time to absorb a large stock of studies and evaluations pointing to the need to adjust the pensions system. Moreover, none of them had put forward a reform proposal of his or her own, a fact that would supposedly render them more receptive to ideas emerging from the series of hearings. After six weeks of hearings with civil society organizations, business organizations, representatives of the financial sector, research institutes, international organizations and experts, the Council drafted its proposals before the three-month deadline imposed by the President. This in itself strengthened the message that the reform was urgent and set a precedent for compliance with the technical and political deadlines needed to achieve it. The proposal prepared by the Advisory Council was reviewed by a committee of ministers established by the President. That committee, which included the Minister of Finance and the Minister of Labor, and the Budget Director as technical secretary, analyzed the report from both a technical and political standpoint. The Budget Directorate set the technical agenda and presented issues to the political team for them to decide on. Those issues were related to implementation and also to very specific cases that the Advisory Council had not resolved in its proposal. The technical team behind the committee was in charge of crafting the law and leading the negotiations with parliamentarians as well as with the Comptroller General's Office for clearance of the regulations. After about five months of intense work, the bill was submitted to Congress in December 2006. The process by which the reform was crafted helped generate the technical-political consensus needed to ensure a prompt and unanimous passage of the bill in the House and a majority approval in the Senate, considering the controversial topic and the adverse political climate at the time. During legislative proceedings, which lasted a little more than one year, the executive actively participated in parliamentary debate, explaining aspects of the law to Congress. Led by the ministers of finance and labor, with a key role of the Budget Directorate, it helped build consensus and engaged in the political negotiations to achieve the adoption of the law in March 2008. Cabinet Office) has sought to comple- grants the line agencies broad flexibility ment the policy process in the line to perform evaluations, while the Treasury agencies and align government goals at Board Secretariat at the center of govern- different levels by promoting methodolo- ment provides the general framework (see gies, providing information, and filling Annex 4). gaps (for instance, developing policies in some areas that cut across line-agency Operating in a less malleable administrative responsibilities). environment, the center of government in France and Spain has less ability than in The New Zealand and Canadian govern- Westminster-system countries to steer or ments have proven less "interventionist": the influence the line agencies. Cabinets of center has largely stayed out of the substance ministers do not play such a powerful role. of detailed policy-making, instead seeking to In France, the center concentrates on the big ensure that departments have good incen- picture, creating consensus, and influencing tives to manage themselves. New Zealand public opinion (see Annex 6). But the line is noteworthy for seeking to clarify which agencies are not left to their own devices: agency is in charge of policies that overlap the technocratic networks (the Corps) sup- departments (the lead agency concept ­ see port an orderly policy process and, to Annex 3). Canada, in its evaluation system, an extent, the inspectorates police it. In 20 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 6. Public Acceptance of Large Projects: the French CNDP The National Commission for Public Debate (Commission Nationale du Débat Public--CNDP) was created in 2002 as an independent administrative agency at the national level to organize "public debate" for all public investment projects of more than 300 million (about US$450 million at present exchange rates). For each project, it creates (or decides not to create) a specific, ad-hoc committee in a decentralized process. Some 40 such committees have been created in the past six or seven years. They consist of independent members, often retired senior civil servants, who have a quasi magistrate status. Most of the projects concerned have been transport infrastructure projects. The committee organizes a "debate" on the project open to whoever wants to participate; in practice, that often means interested (and opposing) NGOs. The debate is related to the justification and existence of the project, as well as to its details and implementation. Project promoters present their analysis. But participants can--and do--ask for counter-evaluations or expertise by independent experts, selected by the committee, and paid either by the promoter or by the CNDP. The debate cannot last more than four months (exceptions are possible, but there have been very few of them). The cost of a debate is estimated to range from 0.5 million to 2 million (US$0.7 to 3 million), much less than 1 percent of the infrastructure cost. The committee is not a decision-making body: it merely organizes and structures the debate, then summarizes it without taking sides and positions. The outcome of the debate is not binding on the project promoter. However, the project promoter must, within three months, make public the conclusions they draw from the debate. This soft and ad-hoc procedure has proven rather effective. In some cases, the project promoter abandoned the project. A case in point is a 400,000-volt aerial power line between France and Spain; it was later decided to bury the line (at a much higher cost). In other cases, the project was very significantly modified. The proposed rail link between Paris city and Paris airport was initially to be a 12 km long tunnel, and was transformed by the debate into a re-use of existing rail lines plus a 2 km long tunnel, a much cheaper solution. In nearly all cases, it had a positive "upstream" impact on the initial project. It induced or forced the project promoter to better examine and formulate the project, to make the effort of considering possible or likely bones of conten- tion, and to modify the project in order to minimize opposition or maximize acceptance. In most cases, the misunderstandings and fears that had motivated opponents were substantially reduced, if not eliminated. The practice has shown that public or private infrastructure project promoters do not have a monopoly of good ideas, and that a fair and well organized debate does enlarge the choice of solutions, helps find a better one, and facilitates public acceptance. Spain, the "President of the Government" In the United Kingdom, the Strategy Unit, (equivalent to the Prime Minister in other attached to the Cabinet Office, uses small countries) has significant powers that make teams of analysts to undertake a broad him the key player in the policy-making range of policy-oriented studies at the re- process (see Annex 7). However, the larger quest of the Prime Minister or ministries, line ministries have significant autonomy and it also provides methodological support for policy-making in their areas, which to the policy process in the line agencies limits the central government's reach and (see Annex 2). The studies are made public, requires a larger infrastructure for policy though they are predominantly addressed coordination. to the government. 74. Some of the routines and organizations that sup- France's prestigious Council for Economic port the policy process at the center of govern- Analysis, under the Prime Minister, follows ment come and go: the politicized nature of the a different model: a standing body of promi- policy process at the center means that these rou- nent economists oversees studies under- tines and organizations do not have a guaranteed taken by individual (and equally prominent) long life. With this caveat, the country cases have economists on particular economic topics produced some examples of instruments worth (see Annex 6). These studies are predomi- further study: nantly intended to inform the public. 21 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA France's Center for Strategic Analysis, also differences in patterns across the six countries are under the Prime Minister, bears a greater summarized below. resemblance to the United Kingdom's Strat- egy Unit. It uses in-house expertise as well 76. Ex-ante policy assessment (column 3 of Table 4): This as commissioned research to assess policies tends to be about policy choices and not necessarily and policy proposals and to help build public about the details of program implementation. There consensus on directions for policy (see An- is no example in the six countries of an evaluative nex 6). system for ex-ante policy evaluation.12 But there are some cases of center-of-government agencies dedi- The United States President has a very strong cated to selective ex-ante policy evaluation. These source of policy advice from the Executive include the United Kingdom, the United States, Office of the President (EOP ­ see Annex and France. New Zealand, Canada, and Spain, on 5). Divided into 13 specific policy areas, this the other hand, have no formal processes for ex- is a larger structure than any of the cabinet ante evaluation of policies. offices in parliamentary systems. Within the EOP, the Office of Management and Budget 77. Ex-post program evaluation: Ex-post program eval- is particularly important as an example of a uations can be carried out systematically through budget office charged with program and government-wide systems (column 1) or through management review. Notwithstanding con- dedicated agencies (column 4). Evaluation under siderable advances in the use of neutral tech- either of these arrangements tends to emphasize nical expertise in the EOP in the last half- effectiveness, efficiency, and management, but does century, modern presidents have preferred to not usually centrally address the appropriateness of use it partly as a political instrument. the policy itself. All the countries, with the possible exception of Spain, have established some elements The United States Congressional Budget of a systematic approach, but in different ways and Office also has functions that transcend to different degrees. In all cases, the systems are sub- those of the usual budget offices in other stantially decentralized: the center sets the frame- countries, and provides significant input work, coordinates, and provides support, while the on policies from a resource-management agencies do the actual evaluations. In some cases, perspective (see Annex 5). 11 the center might advise on methodologies, but in none of these countries are specific methodologies In Spain, the newly created Evaluation and imposed. In several cases, the evaluation function Quality Agency (Agencia de Evaluación y is a part (often modest) of a broader management- Calidad--AEVAL) could at some point gain control or management-promotion tool. more influence in the policy assessment pro- cess-- its formal functions allow for this (see In Canada's well-articulated system, the Annex 7). Its evaluations adopt a very flexible center actively ensures that the agencies approach. However, it is still too early to have an evaluation function. It provides a ascertain the impact of the agency, especially framework for evaluation, and it provides considering that, in its first year, it has focused support to the agencies. on the ex-post evaluation of programs. In the United States, as a modest part of a broader management-control system (the Systems and Agencies dedicated to Policy Program Assessment Rating Tool ­ PART), Assessment and Evaluation in OECD Countries the center ensured that the agencies had an evaluation function, and it worked with the 75. The different formal systems (rules and routines) agencies to rate that function and, if neces- and agencies with a significant government-wide sary, to identify a program to improve it13. mandate for ex-ante or ex-post policy and/or program evaluation identified in the six OECD countries can be subsumed, almost without excep- 12 All countries use the coordination and vetting functions of cabinets. This is certainly a system, but since it does not systematically use formal analytical tion, under a five-way classification (see Table 4, techniques, it does not figure as evaluation. France has an articulated system and Annex 1 for more details). The similarities and for ex-ante evaluation of projects, but project evaluation was not considered in the cases presented in Annex 1. 13 The PART program has been discontinued. Some doubts were created, in Congress (which had not participated in PART reviews) and elsewhere, about 11 Anderson (2009) describes the growth in recent years of specialized budget the rigor and independence of the evaluations and the quality of efficiency research offices in various other countries, notably Belgium, Finland, Italy, Korea, measurements. In November 2009, the Obama administration announced Mexico, Netherlands, Poland, Portugal, and the UK. plans to implement a new performance management framework. 22 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Table 4. A Classification of Government-Wide Evaluation Systems and Agencies in Selected OECD countries Government-wide system Center-of-government agency External agency Ex-post program Ex-post program Ex-ante policy Ex-post program Ex-post program evaluation spending reviews assessment evaluation evaluation 1 2 3 4 5 Elements of a United Kingdom Spending Reviews PMSU NAO decentralized system Limited elements of a Spending Reviews New Zealand SPEaR (social policy) SPEaR (social policy) OAG (limited role) decentralized system (planned) Canada Decentralized system Elements of a system Elements of a semi- CBO (economic and GAO (also some ex- United States centralized system PART (until 2009) budgetary issues) ante policy evaluation) (PART until 2009) Center for Strategic Analysis Limited elements of a Revision Generale des France Council for Economic Court of Accounts decentralized system Politiques Publiques Analysis (Economic Policy) AEVAL (nascent Spain IGAE system) Source: Distilled from Annex 1. See List of Acronyms on page vii. In New Zealand, the center ensures that the some of these carry out substantial evaluation func- agencies have an evaluation function as part tions beyond their traditional tasks of conducting of an annual strategic planning exercise legal and financial audits. In this report's sample, the (though evaluation is not a high priority in supreme audit institutions of the United Kingdom this exercise). and the United States do undertake substantial eval- uation exercises. The United States Government Ac- In the United Kingdom, the center lays countability Office (GAO) carries out the broadest down guidelines and provides support for range of evaluative activities (including some ex-ante evaluation, but does not otherwise control policy assessment). This reflects the particular role or supervise the evaluation activities of the that the United States Congress plays (see Annex 5). agencies. 79. Ex-post spending reviews (column 2): As already France does not even have the rudimentary observed, OECD countries have tended to elements of a program evaluation system move towards this specialized form of program as seen in the United States, New Zealand, evaluation. and the United Kingdom14. On the other hand, it does have ministry inspectorates, 80. Another way to read the OECD experience with one of whose tasks is evaluation. institutionalization of policy assessment is to di- vide the field, beginning with broader, multi-sector Spain does not have a program evaluation policies, moving on to more narrowly defined sec- system. Instead, it is the only country in tor policies and ending up with the level of policy the OECD sample with a dedicated central implementation (see Table 5). Policies concerned agency that actively evaluates (and which with the "Whole of Government" level or Higher might become the nucleus of a nascent Political Issues --such as, for example, country government-wide evaluation system). strategy in respect to global warming or other ma- jor multi-sector issues-- need to be dealt with in a 78. External ex-post program evaluation (column 5): All broad way, usually by the Legislature or the central countries have supreme audit institutions, but only units of the Executive, including the Ministry of Finance. The center's role in policy making is to: (i) coordinate, i.e. ensure consistency with govern- 14 France has an evaluation agency in the Prime Minister's office, but it appears to ment's goals, provide second judgments, give ad- be virtually defunct. vice, and decide/bring policy proposals to congress 23 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Table 5. OECD Policy Assessment Framework Type Organization Objective Executive/Cabinet Congress/Parliament Outcomes - high level political concerns Whole of Government Central unit (multisectoral and global issues) Treasury Congress Ministries and agencies Sectoral Policy Program definition Budget offices Advisory bodies to departments and agencies Service Delivery Implementing agencies Performance improvement when legislative approval is required; and (ii) mar- ket and seek citizens' approval for reforms of major impact on daily life of ordinary citizens (i.e. health reform). The "Sector Policy" level includes deci- sions typically taken by actors such as ministries and agencies, where programs are designed. The third level, "Service Delivery", is where programs are implemented by agency staff. It is the agencies, not the center, that carry out the detailed design of policy and its implementation, but there is no set pattern ­ no "best practice" ­ to how they do this. 81. The proposed diagram is a first approach to select the level of government unit primarily responsible for policy making and policy assessment as a func- tion of the level of participation ("competence") of each government unit. As such, it is a useful refer- ence for the proposals of chapter 3. Still, this refer- ence diagram should be supplemented with other lessons from OECD experience. These particular features include, inter alia, the complementary roles of the Legislature and the Executive in policy assessment and the need to produce some evalua- tions led by non-government experts. 24 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 25 3 Some Considerations for Institutionalizing the Policy Process in Latin America 82. This is a study about OECD practices. It is not a recent years. Similarly, the Budget Office/Ministry study about Latin American practices. It does not of Finance in Chile now appears to be sharing with attempt to present the policy process in a single the Ministry of the Presidency some of the extraor- Latin American country. That exercise is a research dinary influence it has had in the last two decades. agenda that lies ahead. For such an exercise, the In Peru, the Presidency of the Cabinet (Presidencia analytical framework proposed in chapter 1 may del Consejo de Ministros ­ equivalent to the Prime prove valuable to analyze country cases and draw Minister's office) may have more or less influence lessons from the Latin American experience. depending on presidential and prime-ministerial styles. The balance of influence and the coordina- 83. Still, the study is intended to offer ideas taken tion of policy making between Uruguay's Office from the OECD experience as references for of Planning and Budget (Oficina de Planeación y Latin American countries. The more those refer- Presupuesto), which is attached to the Presidency, ences fill in the gaps or fit the challenges currently and the Ministry of Economics and Finance also faced by Latin American governments, the more appears to change as a function of presidential relevant the study will be. For this reason, the study style, politics and personalities. needs to provide a summary presentation of some of the weaknesses of current policy making and 85. The recommendations summarized at the end evaluation in Latin America. Such a presentation of this chapter are likely to better fit those Latin will serve as the background against which Latin American countries that are more advanced in American governments may connect with and as- technical capacity and in building a performance sess the relevance of the recommendations of the management system that permeates the entire study. Admittedly, the documentation and analysis administration. It is within this group of countries of OECD countries was selectively focused on that the issues related to controlling the quality of those issues that appear to have the most relevance major public policies are being raised. For example, for Latin America. Chile has been actively debating institutional and instrumental options for policy evaluation since 84. It is evident that, in Latin America as a whole, President Bachelet first took office. The situation there is a huge variation of institutional arrange- is similar for Mexico under Presidents Fox and ments, technical capacity and base-line conditions Calderón. Having said that, it is nonetheless true in the policy and performance-management areas. that the listing of Latin American challenges and In addition, the roles and the capacity of institu- the study's ideas for better policy-making are so tions that are involved in the policy cycle change broadly formulated that they might fit, to a greater over time. The Colombian National Department of or lesser degree, most countries in the region. Planning (DNP), for instance, played a pivotal role in policy making in the 1960s, 1970s, and 1980s. 86. The following paragraphs propose a list of policy Though the DNP is still a major player in policy and institutional gaps that appear to jeopardize the making, the Presidency itself has gained ground in quality of public policy in Latin America. Immedi- ately afterwards, the study will present conclusions decision, as they are in many OECD countries. and recommendations from the OECD experience Latin American presidents tend to have secretariats that may help fill in those gaps. for coordination and support that are strong in po- litical matters and are just beginning to strengthen their technical capacity. Chile's previous attempt to Extracting ideas from the OECD experience establish a technically strong group of experts on applicable in the Latin American context the "second floor" of the Presidency or its current drive to create a "Delivery Unit" and an ex-ante 87. This brief tour of policy-making experience in se- policy evaluation unit at the Secretaría General de la lected OECD countries is not the last word: rather, Presidencia (SEGPRES) is a case in point (Box 7). it is an initial attempt to look at the policy cycle. Another example is former Colombian President There does not appear to be a single "best prac- Uribe's decision to exercise tighter control over tice"; institutionalization of policy making is and the system for monitoring the administration's will continue to be idiosyncratic, i.e. it will remain priority programs (SIGOB). So is the drive of largely dependent on each country's own institu- two successive Mexican administrations, those of tions and political culture. Moreover, the practice Fox and Calderón, to strengthen the Presidency's of six countries does not exhaust the possibilities capacity to identify and monitor indicators of their of defining different formats and instruments in key presidential programs (Box 8). the policy process. But the review does suggest some common features that may serve as reference 90. Ministries of Finance are often relatively strong points for Latin America's current drive towards on policy matters, but they tend to have a pre- strengthening the quality control of public policies. dominantly fiscal viewpoint (as compared to Policy-making tends to be a creative and messy a broader public-interest viewpoint). In some process using multiple sources and modes of infor- cases, Ministries of Planning make up for this, but mation, while implementation is a tighter process they often lack political authority. Still, as Latin governed by the (quasi-) science of management. American countries move from first-generation to No strong formal feedback links were found from second- and third-generation reforms, and from implementation (M&E) to policy-making. Policy input-based to results-informed administration, making is a deeply political process and the OECD Ministries of Finance and/or Ministries of Plan- experience suggests that there are limitations on ning are expanding their range of interests to the extent to which the policy-making process can include effectiveness and efficiency of government be routinized in any democracy. programs, particularly budget programs. As those ministries, as well as sector ministries and agencies 88. Nonetheless, there are broad common insti- and parliaments, strengthen their technical capac- tutional patterns. These include: coordinating ity, government goals will be better aligned and mechanisms at the center of government that are evaluation results will be seeds sown in fertile soil. supported by technically (and politically) strong It will also be necessary to strengthen ministerial secretariats; adequate policy-making expertise control of the quality of results of executive agen- in ministries and agencies; a clear separation of cies, as a means to stimulate policy and program policy making and program implementation; and alignment as much as linkages. various other support institutions, including ad- hoc policy advice. And there are changing features 91. If Latin America is to look to OECD experience of the landscape that are important. In particular, on the policy cycle, should it pay more attention OECD countries are moving from a form of pol- to parliamentary or to presidential systems? Latin icy making which was politically monopolized to American political arrangements in some ways one where competition between ideas, from both lie between the parliamentary and United States inside and outside the government, is becoming systems and, in other ways, may lie outside. Latin more important. Table 6 seeks to summarize, American constitutions most commonly give sig- both for the different OECD countries surveyed nificant power to the President. Cabinets as such in this study and for Latin American countries, generally do not play an important role in the the nodal points in national policy processes and policy process. Rather, it is a few influential minis- to give a sense of where country types are institu- ters, agency directors or presidential advisers who tionally strong or weak. have privileged participation in the selection of policy priorities and/or are in charge of monitor- 89. A first impression is that policy-making institutions ing their results. Formal or informal attempts to do not work as well in Latin America. Cabinets are create ministerial committees for policy coordina- not arenas of policy discussion, coordination, and tion at the sector level, or the social, infrastructure, 26 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 7. Coordinating Policy-Making in Chile: The Budget Directorate and the Second Floor Chile did develop a powerful machinery for committing each government agency (the "Servicios") to precise targets, including policy alignment with the sector, with other sectors and with government priorities. In the case of Chile, the whole performance machinery has been linked to powerful salary incentives that are in turn reflected in budget decisions. Since this machinery is primarily routinized and tied to the Budget Directorate, some recent Chilean presidents felt the need to strengthen their own capacity to influence policy and implementation. In addition to monitoring presidential goals, these presidents strengthened the advisory role of the so called "second-floor experts" (referring to their location in the presidential palace). This group was generally composed of technically solid professionals with some political expertise. In the end, the second floor proved to be too informal for systematic improvement of policy making. Also, it was not clearly linked to (and did not make much use of ) the performance information produced by the Budget Directorate. In turn, the performance information was increasingly undermined by sectors and agencies misrepresenting their priority goals or the way they effectively served inter-sector or presidential priorities. As a matter of fact, govern- ment agencies felt the need to accommodate goals and results to the strict areas and indicators approved by the Budget Directorate while in practice developing their own individual strategic programs. In view of nearly a decade of learning the skills of performance control, Chile is now advancing towards (i) strengthening ex-ante policy evaluation and monitoring capacity at the Secretariat of the Presidency and the projected Ministry of Social Development, and ex-post policy evaluation at an independent agency; (ii) also strengthening sector coordination, broadly defined, through inter-ministerial committees; (iii) controlling the quality of agency goals and performance through more decentralized means at the sector level; (iv) using new, primarily non-monetary incentives to secure policy alignment, control, and rewards at the executive level. Box 8. The Challenge of Coordinating Policy-Making in Mexico The Fox administration in Mexico developed a sophisticated system for monitoring presidential as well as sector (ministerial) goals. The Secretaría de la Función Pública was technically responsible for monitoring a large number of indicators and periodically reporting progress to the President and his cabinet. In addition, the President had commissioners in charge of each of the areas of presidential priorities. These experiments were discredited over time and finally abandoned. Sector secretaries did not find their own challenges well represented in selected performance goals and presidential priorities did not provide enough guidance as to each sector's contribution to the objectives of the administration. Autonomous or semi-autonomous executing agencies were either ruled by inertia or by their own particular goals. Priority goals were not necessarily reflected in the budget. In the end, the whole exercise did not seem to have contributed to strategic cohesion of government actions. Rather, it appears to have led to frustration with performance management ­ and more fragmentation than integration. As a result, the Calderón administration is attempting to develop a lower-profile monitoring of presidential goals that is linked to monitoring and evaluation of budget programs (Secretary of Finance) and social programs (Secretary of Social Development). or finance committees that have been practiced Chile's social policy --will prove to be effective and elsewhere in the region, have generally failed and sustainable in the medium term. have been short-lived. It seems that, without direct presidential involvement, ministers do not make an 92. Congresses typically have greater powers than par- effort to coordinate among their peers. It remains liaments (such as powers of veto), yet in most cases to be seen whether the ongoing attempts by the not the powers to initiate policies or appropriate Chile's current administration to recreate such sec- budgets that the United States Congress enjoys. tor ministerial committees ­or the new Ministry Political parties tend to be neither strong nor of Social Development in charge of coordinating durable, power is often monopolized by their lead- 27 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Table 6. Technical Strengths and Weaknesses of Government Actors in the Policy Process Executive branch Center of Ministry of Legislative branch Country Cabinet Line Ministries Evaluation Government Finance/ Planning OECD MoF: STRONG, Decentralized STRONG (supported by Westminster but subordinate STRONG systems, with varying WEAK technical secretariat) countries to Cabinet central control MoF: STRONG, Decentralized STRONG WEAK, subordinate STRONG United States but subordinate STRONG systems, with some (supported by OMB) to President (supported by CBO) to President central control Decentralized STRONG MoF: STRONG, OECD European systems, with efforts (supported by Somewhat WEAK but subordinate STRONG Somewhat WEAK civil-law countries to establish strong technical secretariat) to Center central agency STRONG in political MoF and/or MP: Latin American coordination, mostly WEAK, subordinate WEAK in Generally WEAK, often STRONG in WEAK countries WEAK in policy to President many cases but varies by country policy coordination coordination ers, and there can be many parties. These features mechanisms, notably coordination through the reduce the parties' ability to represent the public cabinet, central secretariats, and senior public interest and discipline presidents and legislatures, servants. but congresses are often left with substantial veto powers that can create powerful divisions of pur- 95. In Latin-American countries, on the other hand, pose between the legislature and the executive. there is a deeper sense that execution is not con- sistent with policies. Even in the more advanced 93. In most (but not all) Latin American countries, countries of the region, policy proposals may be cadres of senior civil servants are too weak in quoted in the media and may be influential inside technical skills and/or too politically dependent on and outside government, but unless they are ex- governing politicians to play a role as a politically plicitly brought down to the program level (and, in neutral, management-driven source of expertise. many cases, introduced as a program in the budget), It is only in a few countries of the region that they remain a dead letter. As a result, government programs exist to encourage senior professional agencies often feel that the center (i.e. the office cadres.15 of the President) does not give implementable instructions. Interestingly, Ministries of Finance 94. Latin American countries have proven to be a have partially filled this technical policy void in fertile ground for policy ideas. After all, Latin recent years, but like their equivalent ministries America has seen a ferment of policy innovations in the OECD, they have a narrower mandate of in the last two-to-three decades. But these charac- budget control and program implementation that teristics do seem to have led to a significant breach is not well-suited to the broader policy role of the between ideas and implementation. We have noted center of government. Nor are finance ministries the institutional gap between policy making and sufficiently informed by management experience implementation in the OECD countries, but this to facilitate alignment between agencies' reachable gap is effectively dealt with through organizational goals and higher level government goals. 96. There is also the opposite case where the center 15 "Sistema de Alta Dirección" in Chile; "Sistema de Direçao e Assessoramento of government feels that the agencies are not Superiores (DAS) in Brazil", among other regimes complying with the mandates given them. Hence, 28 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA apparently good policies (such as cash transfers) technical secretariat, typically a cabinet office (in can often lead to poor results on the ground. An- Westminster-system countries), a prime-minis- other aspect of this breach between policy making terial office (France and Spain), or a presidential and implementation is the failure of the results of office (the Executive Office of the President in monitoring & evaluation to feed back into policy the US). These offices are expert in administering making, though this phenomenon is little different the operating processes that promote or vet policy in OECD countries. As Latin America advances ideas, ensure their coordination and consistency, towards more definite separation of policy plan- and attend to their implementation. This requires ning and implementation, those top-down imple- a minimum of sector-specific policy expertise, but mentable instructions will necessarily expand and some technical secretariats (the UK, but above all become stronger. The latter move is illustrated, the US) incorporate greater technical expertise among others, by the ongoing drive of the Piñera than others (such as Canada and New Zealand). administration to consolidate the monitoring and coordination role of ministries vis-a-vis executing 100. The center of government in Latin-America ­ i.e. agencies in Chile. the office of the President ­ is typically strong in political matters, including managing relations with other branches and levels of government Ideas for Better Policy-Making in Latin America and coordinating political support. In addition, it is increasingly committed to demanding results 97. The above features of the region undoubtedly from ministries and executing agencies. However, provide obstacles to "rational", knowledge-based the political authority possesses little policy ex- policy making and implementation that appear to pertise ­ and, very often, has a weak commitment be initially greater in Latin America than in the to technically sound policy making. Cabinets are OECD countries. Still, as also indicated above, not used as deliberative or decision-making bodies. Latin America is advancing towards more open- And technical secretariats in the presidential office ness to technical expertise and more consistent in are virtually non-existent. (To the extent there the coordination of policy and implementation. In is expertise near the center, it is in Ministries of this spirit, this study can propose six main ideas Finance and planning, but this is at some remove that might be considered if the goal is to improve from the political authority of the President.) the efficiency of the policy cycle in Latin America. These six ideas have been selected because, first, 101. The single most important step to improve the they clearly draw upon some of the lessons learned policy-making process in the countries of the re- from the evolving picture of OECD countries and, gion would be to strengthen the center, in terms second, they appear to be implementable under the both of a political capacity for managing policy and main political and institutional arrangements that of a technical secretariat to support this function. prevail in the region. If performance management is to come of age in Latin America, a presidential office capable of ef- 98. (i) Political authority and technical expertise at the fectively providing strategic directions as well as center-of-government. OECD country examples securing internal coherence, corporate discipline, indicate that good policy making begins with the and alignment of outcomes is essential in the re- combination of effective political authority and gion. Obviously, it is not appropriate for this report technical expertise at the center of government. The to propose political solutions ­ for instance, it sim- political authority is responsible for the government's plistic and unrealistic to mandate strong cabinets in program as a whole, overseeing the conception of presidential regimes. On the other hand, presidents policy ideas, deciding which policies are to go for- can (and already do in some countries) convene ward, coordinating between sectors, and overseeing effective political bodies for sector coordination that policies are implemented as intended. In many (akin to cabinet committees) for economic affairs, countries with parliamentary systems, the political for social affairs, for security issues, and so on. authority is constituted by the cabinet. In the US system of presidential government, the President 102. The President and any such political authorities incorporates much of that political authority within under the presidency would need to be supported his own office, the White House, but also shares by a presidential policy secretariat. Almost any of some of the authority with Congress. the OECD cabinet and presidential offices covered in this report would serve as a useful starting point 99. In a complex, modern economy, the political au- for the design of a technical secretariat. The Execu- thority could not possibly carry out these policy- tive Office of the President provides perhaps the making functions without the support of a strong most relevant example for Latin America, though 29 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA the countries of the region would be expected to limited to sector coordination than goal-alignment start out on a much smaller scale. The UK's Cabi- or channeling agency programs towards higher net Office or France's Prime Ministerial Office, government priorities. Otherwise, experiments also provide interesting examples because they with super-ministries have been short lived in are pro-active in looking at policy problems and Latin America.16 This has also been the case for proposing solutions. Within the UK Cabinet Of- ad-hoc sector committees at the ministerial level, fice, for instance, a Strategy Unit studies specific where a particular challenge is to maintain the policy issues and an Implementation Unit (jointly motivation to participate for those ministers who overseen by H.M. Treasury) attends to problems of are not leading. Innovative governments have tried policy implementation. a wide array of incentives (institutional, collective, individual) to enhance agency performance; but 103. (ii) An organizational system that coordinates policy those incentives have had little effect on the qual- making. Policy-making in the OECD is anchored ity of agency goals, including goal alignment or in organizational structures, coordination mecha- performance-enhancing goals. nisms, and control mechanisms. Organizational structures determine who does what tasks and 105. (iii) Senior public servants who provide expert organizational hierarchies and boundaries set up policy advice and coordinate between policy making particular sets of incentives. Coordination mecha- and implementation. Senior public servants are nisms, such as cabinets or more ad-hoc committee appointed according to both political and "hybrid" formats (or "lead ministries" in the New Zealand (political mixed with technical) criteria in OECD case and "super-ministries" in Australia) are de- countries, depending on the country. Their role in signed to counter the effects of public agencies bridging the world of politics and policy making pursuing their own agendas, and instead make and the world of implementation is crucial. They them focus on the goals cascading down from the play this role both at the center (in center-of- center of government into a particular policy area. government secretariats, notably) and at the top of In the Westminster countries, coordination by the ministries. With a small number of exceptions cabinet can be fairly "tight" because of the doctrine ­ particularly in Brazil and Chile ­ these profes- of collective responsibility. Control mechanisms sional cadres are more notable for their absence are the means by which the principal at the center than their presence in Latin America. This is a seeks to have his/her priorities carried out by the broad reform, in many ways beyond the scope of agent in the line ministry. Traditionally, the mode this paper because it is one of the keys to better of control in the public sector was to command, public management more generally. But it also has but command has increasingly become mixed, to a particular importance for better policy making, the extent feasible, with contractual formats (per- principally because senior public servants provide formance agreements) and incentives. the link between policy making and implementa- tion and many of them are policy experts. 104. There is an evident problem in Latin-American countries that the uncertain transition from gov- 106. (iv) Strong capacity of the line ministries. Line ernment priorities to expected results puts the ministries in the OECD tend to have strong cohesion and effectiveness of policy making at risk. analytical capacity in specialized units or depart- Among other things, an agenda for improving the ments. In addition to their own financial, human transition from policy to implementation involves resource and asset management units, which are such issues as: setting up a technical secretariat at also common among line ministries in Latin the center of government (and, perhaps changing America, OECD ministries also sometimes have the roles of Ministries of Finance which, until management information systems at the sector now, have been expanding); opening up, where this level to inform the policy-making process. Many does not exist, a process of stronger contestabil- ministries in Latin America have a weak capac- ity within the executive by establishing adequate ity to manage their own policy-making activities standard procedures to test policies, politically and and/or coordinate the policies of the agencies technically, through collective deliberation (cabinet they oversee. To improve this capacity, in a first meetings, for example); creating a super-ministry stage, the ministries might (with help from the or lead agency to coordinate policy-making in a particular policy area; working through inter- 16 Chile is currently trying to create a Ministry of Social Development that leads ministerial committees; and other techniques for and coordinates all social ministries, evaluates their performance, and plays "joining-up" the distinct parts of the administra- a prominent role in priority and programmatic alignment of social policies, including health and education. The government recently submitted a bill to tion. Super-ministries have been temporarily Congress, and discussion within the Legislative was still pending at the time this tried in the region, but their role has been more report was written. 30 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA center) concentrate on improving the level and the Chilean pension reform (Box 5). The range (amount and quality) of information available of alternatives includes the following: to the minister to inform policy-making. This would enable ministries to better harmonize the An institution setting up panels of non- broad presidential priorities they are responsible government experts on a case-by-case basis for with the specific programs that they and the (for example, the US National Academy of agencies that are attached to them are designing Sciences or France's Council for Economic and, later, implementing. To this end, strengthen- Analysis). ing the ministerial planning offices would give ministries a greater capacity to gather and analyze An institution to contract out policy-ori- information and to use this information for stra- ented research to non-government entities tegic planning. More ambitious organizational on a case-by-case basis (e.g. the National initiatives (such as ministerial research units) Science Foundation in the US or, at some might be considered later. point, Colombia's Colciencias). 107. (v) Strong capacity of the legislature. The US The creation of a more permanent corps of Congress ­ the closest example to Latin American expertise within an autonomous govern- presidential systems ­ plays a key role in shaping ment agency to carry out policy studies on policies and has a strong budget office and techni- a case-by-case basis. (A version of Spain's cal committees to do so. The technical capacity AEVAL, but one more explicitly devoted to of Latin-American parliaments to develop policy ex-ante policy problems, might be a model ideas and shape and evaluate policy proposals com- for this.) ing from the executive is not very strong. Despite not being as technically and politically empowered The appointment of commissions of experts as the US Congress, they do have a mandate to to carry out policy studies on a case-by-case shape policy and are politically stronger than the basis. This could institutionalize a practice legislative branch of most parliamentary systems. somewhat like the Royal Commissions in Therefore, giving congresses a more solid technical Commonwealth countries (Box 2) or repli- basis for making policies or assessing proposals cate, to the extent possible, Chile's pension- could reinforce contestability and improve policy- reform model (Box 5). making in Latin America. The Congressional Budget Office and the Committees of the US 109. Neither the review of the organizational structure Congress provide possible models that go in this for coordination of policy goals nor improving direction. Many Latin American congresses are technical capacity for policy making are steps that already part of a worldwide movement that is driv- can be completed in the very short term. They ing for results, and this will push them to demand require leadership and political will, as well as a greater policy-making and -evaluation capacity good dosage of expertise and reform continuity. of their own. However, further consolidation of Latin America's achievements, as well as selective importation and 108. (vi) Active alternative channels of policy advice adjustment of the lessons from OECD countries exist. There is a range of possible alternative in- in those two reform areas are badly needed if trust stitutional formats that could reinforce technical in the Latin-American state is to be rebuilt. En- analysis by opening up the policy-making process hancing public sector credibility requires building to expertise in academia and, more broadly, the the proper institutional and instrumental bridges non-government sector. Such initiatives can also between political announcements, government encourage the participation of stakeholders--not priorities, flexible financing, execution capacity, as experts or evaluators, but rather as part of the delivery of quality services and citizen satisfaction. debate--in the consideration of different policy At this point in the region's drive towards a results- alternatives. This process can bring viable policy oriented relationship between the public adminis- alternatives to light, and it can minimize resistance tration and the citizen-client (voter and taxpayer), from interested parts. These initiatives are common those bridges are essentially needed to strengthen in many OECD countries and are being adopted transparency, accountability and performance. in Latin America as the critical mass of experts ex- pands and civil society organizations become more vocal and more technically capable. Examples of successful stakeholder involvement are the French Commission Nationale du Débat Public (Box 6) 31 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 33 Annex 1: Government-Wide Evaluation Systems and Agencies: Common Features and Differences Across OECD Countries Table 1-1. United Kingdom: Formalized Evaluation Routines Internal Evaluation: Internal Evaluation: The Budget Cycle: External Evaluation: Prime Minister's Ministerial Evaluations Spending Reviews National Audit Office Strategy Unit Elements of a system: guidance System for ex-post Agency for ex-ante Agency for ex-post Type from the center for ex-post program evaluation policy evaluation program evaluation and ex-ante evaluation (spending reviews) The Treasury carries out, currently on a three-year The NAO is an The PMSU is a central Policy formulation and evaluation cycle Comprehensive independent Institutional unit (i.e. not a system) are largely decentralized and Spending Reviews supreme-audit arrangements attached to the not part of a system. (CSRs). Related to this, it institution reporting Cabinet Office. oversees departmental to Parliament. spending reviews. The NAO carries out a range of audit and evaluation functions. As part of a broader CSRs analyze spending Evaluations are of task of support to the across all ministries, as well as implementation Prime Minister and the the allocation of spending. (efficiency, In addition to the coordinating and ministries in policy- Departmental spending effectiveness, etc. vetting role of Cabinet mechanisms making (and associated reviews, undertaken for (not of the policy Purpose, (including a large and powerful Cabinet methodologies), the each ministry, review itself). "Value-for- scope, and Office), the role of the center is to Unit carries out policy- the results achieved and money" audits impact provide some methodological guidance oriented studies (i.e. ex- their cost. Spending (produced at the (from the Treasury) and analytical ante policy evaluation) reviews are the basis for rate of about 60 a support (from several sources). in a relatively modest three-year (medium-term) year) feed into the number of areas. forward expenditure Spending Reviews. estimates (covering some The NAO also 59% of all spending). regularly validates agencies' data systems as part of the CSR cycle. The PMSU and the Expertise is Cabinet Office select largely in-house. Analytical methods and sources policy areas for study. Evaluation studies, of expertise vary widely. Quality Expertise is largely plus occasional Methods assurance is the responsibility of each in-house. There is no compendia of department. Departments publish many dominant analytical data-validation of their studies on their websites. methodology. Studies reports, are are published on published on the website. the website. Table 1-2. New Zealand: Formalized Evaluation Routines Internal Evaluation: Internal Evaluation: The Budget Cycle: External Evaluation: Ministerial evaluations SPEaR Spending Reviews Office of the Auditor General System for ex-post Elements of a system for Local system for ex-ante Agency for ex-post Type program evaluation ex-post program evaluation and ex-post evaluation program evaluation (spending reviews) While Cabinet vets and coordinates policy-making, policy formulation and evaluation are largely decentralized and not part of The Social Policy Evaluation a system. However, the center and Research Committee The government is planning of government requires and (SPEaR), led by the Ministry to institute a system of The OAG plays no role in Institutional coordinates a departmental of Social Development, is a spending reviews overseen policy evaluation and a limited arrangements routine for strategic planning: small group that attempts by the center and carried role in program evaluation. agencies' strategic plans are to coordinate and promote out by the departments. contained in a Statement social-policy analysis of Intent (SOI), submitted across relevant agencies. annually with the budget. The center also designates "lead agencies" for cross- departmental policy issues. SPEaR provides support for In addition to its traditional The SOI sets out the government researchers, audit function, the OAG department's goals and undertakes its own audits the performance data Purpose, evaluation strategy (though, analyses, and seeks to that departments provide in scope, and within this, the evaluation incorporate research results their annual reports. It also impact function remains somewhat into the policy-making carries out a limited number of underdeveloped). process. Its impact has performance audits (but the so far been limited. policy itself is taken as given). SPEaR commissions some OAG has its own staff Methods The SOIs are available online. reports from experts. These and also uses the services are published online. of accounting firms. Table 1-3. Canada: Formalized Evaluation Routines Internal Evaluation: The Budget Cycle: The Treasury Board Secretariat Evaluation System Spending Reviews Elements of a system for Types System for ex-post program evaluation ex-post program evaluation (spending reviews) The Treasury Board Secretariat (the unit responsible for public administration) The Cabinet has an Expenditure Institutional oversees a decentralized system for ex-post evaluation. There is no such system for Review Committee, but arrangements policy formulation ­ beyond the normal coordinating and vetting function of the this monitors, rather than Cabinet and of the Privy Council Office, a powerful secretariat to the Cabinet. undertaking formal analyses. The system is concerned to evaluate the implementation of programs (or larger sets of Purpose, programs), taking the policy intent as given. The TBS defines policy and program evaluation scope, and standards, supports the activities of the departments, oversees their work, and feeds impact evaluation results into the policymaking and spending-review processes. The departments are free to identify the programs to be evaluated and define evaluation processes. The TBS maintains an Audit and Evaluation Database containing and linking information Methods on a wide range of analytical results (not online). Outside technical advise is sought mainly by each agency according to their needs. The TBS does not use outside experts. 34 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Table 1-4. United States: Formalized Evaluation Routines Evaluation by the Executive: Evaluation by Congress: External Evaluation: Program Assessment Congressional Budget Office Government Accountability Office Rating Tool, PART Elements of a system for Agency for ex-ante and Type Agency for ex-ante policy evaluation ex-post program evaluation ex-post evaluation PART was a management and results evaluation tool administered by OMB, involving the collaboration of the agencies but with no participation from Congress or other actors outside Institutional the executive branch. In addition, CBO is an independent, non-partisan GAO is an independent, non-partisan arrangements the Government Performance office working for Congress. office working for Congress. and Results Act (GPRA) defines government policy on evaluation and sets the evaluation methodology. But there is no government-wide evaluation system as such. PART was a tool to help identify a program's strengths and weaknesses in order to inform funding and management decisions. The adequacy CBO provides analysis, information, of evaluation is one of several topics and estimates that help Congress GAO performs a wide range of tasks Purpose, covered. OMB and the agency develop make economic and budgetary including legal and performance scope, and a plan to improve performance and decisions. This includes studies audits and, at the request of impact management and then track the and briefs on policy issues (ex-ante Congress, ex-post evaluations, and progress that is made. The process can evaluation), with particular reference analyses of policy options. occasionally result in steps to terminate a to economic and budgetary issues. program. Between 2004-2008 PART has evaluated 98% of government programs, accounting for 90% of the budget. PART was based on a standardized questionnaire covering the purpose and design of the program, strategic planning, program management, and CBO relies mostly on its own staff. GAO relies mostly on its own staff. Methods results. Responses were prepared by Its reports are published online. Its reports are published online. each agency and approved by OMB, The results, available online, consisted of numerical ratings, as well as a brief commentary for each question. 35 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Table 1-5. France: Formalized Evaluation Routines Internal Evaluation: Internal Evaluation: The Budget Cycle: Internal Evaluation: External Evaluation: Center for Strategic Council for General Review of Ministerial evaluations Court of Accounts Analysis Economic Analysis Public Policies Elements of a Agency for ex-ante System for ex-post Agency for ex-ante Agency for ex-post Type system for ex-post and ex-post policy program evaluation policy evaluation program evaluation program evaluation evaluation (spending reviews) Most policy and program evaluation takes place at the This is a new spending level of the ministries review process and the agencies they The CSA works on under the authority This is a Council of control. Sometimes behalf of the Prime of the Council for The Court is an Institutional prominent economists the ministries have Minister, but also Modernizing Public independent, arrangements chosen by the specialized bodies provides expertise Policies (chaired by quasi-judicial body. Prime Minister. (planning departments, to ministries. the President). The inspectorates, reviews are carried commissions) to do this. out by the ministries. There is no national evaluation system. The Council's mission is The dominant work of The CSA provides to provide a diversity The Review examines the Court is traditional expert advice to the of expert views on the justification for audits, for which it has government on public economic policy and cost of public Purpose, a high reputation. The policies, specifically on choices. Its studies programs and scope, and Court has recently the implementation provide a mixture of proposes changes, impact entered the field of of government ex-post and ex-ante where justified. ex-post evaluation of strategies (ex-ante policy evaluation. selected programs policy evaluation). The studies are and policies. often influential. The Council selects topics for study The CSA uses and commissions internal and external studies (from Council The Court uses experts and a broad The Review uses members or other internal expertise. Its Methods range of evaluation internal and external experts). These are reports are published methodologies. auditing expertise. then published under on its website. All its studies are the names of the published online. authors (and made available online). Table 1-6. Spain: Formalized Evaluation Routines Internal Evaluation: The Budget Cycle: Evaluation and Quality Agency (AEVAL) IGAE System for ex-post program Type Agency (and nascent system) for ex-post program evaluation evaluation (spending reviews) Evaluation in Spain has hitherto been mostly an ad-hoc activity of public agencies, without any formal framework or regularity. AEVAL is a newly-formed agency Institutional IGAE is an agency under the Ministry linked to the Ministry of the Presidency and directed by a Governing Council arrangements of Economics and Finance. (which has government, union, and expert members). AEVAL is intended to initiate the process of institutionalizing evaluation in the public sector. AEVAL's objectives are to support monitoring and evaluation in the public IGAE carries out ex-post evaluations Purpose, sector, to support the improvement of public services, and to promote of programs (i.e. spending scope, and transparency and a better use of public resources. Its main activities are reviews) with a view to analyzing impact policy evaluation and the promotion of quality management. effectiveness and performance. The Governing Council defines the evaluation program, and AEVAL coordinates evaluations with the relevant ministries. AEVAL takes a pragmatic and eclectic Methods approach to methodologies. Evaluations are mostly conducted by AEVAL's internal staff. AEVAL undertook its first evaluations in 2007 and publishes the results online. 36 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 37 Annex 2: The United Kingdom 2.1 In the last century, policy-making in the UK17 has 2.5 Reforms under a radical Conservative government become a much more complex business as govern- (1979-97) largely emphasized efficiency: at first, ment has grown bigger and more complex. (This quick value-for-money managerial changes (the is a common phenomenon across the OECD; see Efficiency Unit); then the separation of govern- Box 2.1.) The construction and vetting of policies, ment functions into clear cost centers, with man- formerly the preserve of the Cabinet and senior civil agers held accountable for outputs (the Financial servants, has now itself become a complex business. Management Initiative, established in 1982 and The government has been gradually opening up overseen by the Treasury). The most important to outside ideas and expertise in the last century reform, Next Steps (initiated in 1989), was to sepa- (Stone, 2001), and in the last four decades, more rate policy-making from policy-implementation by or less, formal public institutions have been built hiving off most of central government into separate outside the Cabinet to support the policy process. executive agencies and contractualizing the rela- tionship of these agencies and their sponsoring 2.2 By the later 1990s there was a sense of crisis about departments (ministries). the UK's policy capacity. (This crisis was felt in other Westminster-system countries; see Box 2.) 2.6 The Labour government, in power from 1997 to This was especially evident in the flurry of activity 2010 generally maintained and developed the sys- (for a few years at least) to improve the process tem of performance-oriented management, while under the Blair (Labour) Government of 1997. introducing a more explicit system of performance planning and budgeting. From 1998, departments 2.3 The UK's system of parliamentary government and were given medium-term budgets and allowed its lack of a written constitution vest considerable to carry forward unspent money to the next year. political powers in the government. There is a sense Public Service Agreements between departments that these arrangements have made the UK "the and the Treasury set out the departments' objectives fastest gun in the West" in the introduction of new and targets. From 2000, there was a shift from set- policies, and it is this that has helped lead to policy ting a large number of outcome and process targets fiascos such as hastily-introduced legislation in to a smaller number of mainly outcome targets. 1991 to control the breeding of dangerous dogs. From 2002 departments had to identify their most ineffective areas of spending, where spending would then be wound down. The reforms since 1997 were Public Management also intended to counterbalance the unintended consequences of Next Steps by emphasizing a more 2.4 The UK was part of the vanguard of New Public strategic vision and "joined-up" government. Management reforms, first emphasizing the use of performance-oriented-management tools to improve efficiency, then, more recently, seeking to place more Main Routines of the Policy Process emphasis on policy consistency across government. 2.7 The UK government has a dense web of activities that are designed to contribute, directly or indirectly, 17 See: Davies (2004); House of Commons, Public Administration Select Commit- to the policy process. For ease of presentation these tee (2007); National Audit Office (2001); Stilgoe, Irwin, and Jones (2006). activities are divided into "main" and "subsidiary" Table 2-1. The Policy Cycle and Public Policy Instruments in the UK Politics & policy: Budget: Technical: Technical: The policy cycle at the center at the center at the center in the departments Policy identification Prime Minister's Strategy Unit Ad-hoc policy Strategy directorates reviews, enquiries Research function Cabinet, Cabinet committees, The Government Social (some Departments) Cabinet Office, Research Unit Independent inspectorates Prime Minister's Office, The National School or regulators attached Policy decisions to some Departments including Policy Directorate of Government Parliamentary Select The Government Ad-hoc policy reviews Committees Office for Science Non-Departmental Government-funded Public Bodies Research Councils Public Service Agreements monitored via: (a) H.M. Program implementation Treasury Spending Reviews; (b) Prime Minister's Delivery Unit National Audit Office "Value-for-Money" audits (feed into Spending Reviews); Program evaluation Ad-hoc evaluations ex-post evaluation of selected programs and processes (but not of policies) cc routines. The main routines are related to the Cabi- 2.10 The Prime Minister's Strategy Unit was formed net, the government's performance-management from parts of previously-existing policy-ori- system, or the line agencies own arrangements. ented units in 2002 18. Located in the Cabinet They form the core of a web of activities that sup- Office, it provides strategic advice to the Prime port the policy-making process. (See also Table 2.1 Minister, supports Departments in develop- which presents the main routines according to the ing policies (including strategic capacity), and schema of Table 3.) identifies emerging issues. Thus it provides broad methodology guidelines and training 2.8 Presided by the Prime Minister and bound by on policy-making, but also carries out policy- collective responsibility, the Cabinet remains the oriented studies in specific areas (often where ultimate arbiter of policy. The Prime Minister's issues are cross-departmental). The Unit has Office includes a Policy Directorate, whose job is few day-to-day responsibilities and its work is primarily one of political coordination. The Cabi- organized around clusters of domestic policy net Office supports the work of the Cabinet. such as education and health. Small teams are formed to meet the needs of each project and 2.9 It was a Conservative government in 1970 that first then redeployed to new work upon its comple- introduced a technical policy capacity at the center tion. Recruitment focuses on those with out- of government with the creation of an internal standing generic analytical skills, and staff is a think tank (the Central Policy Review Staff ) at- mix of permanent civil servants on loan from tached to the Cabinet. This unit lost its focus when government departments and others from out- it was forced to move from longer-term strategy side the government on fixed term contracts or into crisis management later in the decade, and it secondments. The Unit works across the range was closed in 1983. A number of different policy of public-policy issues from the big picture ­ for units were born and died from 1998 onwards. instance, "future strategic challenges to Britain" These have been responsible for proselytizing the ­ to the little picture ­ for instance, helping the departments about better ways to make policy and have also themselves been engaged in some policy work. (Box 2.1 summarizes guidelines for good 18 The Unit was set up bringing together the Performance and Innovation Unit policy-making produced by one of these units.) (PIU) and the Prime Minister's Forward Strategy Unit (FSU). 38 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 2-1. A Methodology for "Good" Policy-Making Professional policy-making for the twenty first century, (Cabinet Office, 1999) was produced by the Centre for Management and Policy Studies, a forerunner of the Prime Minister's Strategy Unit. The document identified nine features of good policy-making: Forward Looking - Defining policy outcomes and taking a long term view Outward Looking - Taking account of the national, European and international situation; learning from the experience of other countries; recognizing regional variations. Innovative, Flexible - Questioning established ways of dealing with things, encouraging new and creative ideas, identifying and managing risk. Joined Up - Looking beyond institutional boundaries; setting cross-cutting objectives; defining and communicat- ing joint working arrangements across departments; ensuring that implementation is part of the policy process. Inclusive - Consulting those responsible for implementation and those affected by the policy; carrying out an impact assessment Evidence based - Basing policy decisions and advice upon the best available evidence from a wide range of sources; ensuring that evidence is available in an accessible and meaningful form. Evaluated - Systematic evaluation of the effectiveness of policy is built into the policy-making process. Reviews - Existing/established policy is constantly reviewed to ensure it is really dealing with problems it was designed to solve. Lessons learned - Learning from experience of what works and what does not Department for Children, Schools and Families ported by hard information, including "Value- to develop programs to train teachers. (Box 2.2 for-Money" audits conducted by the National provides some examples of the recent work pro- Audit Office (NAO). Second, but on a less rou- gram.) The Strategy Unit does not have close tine basis, the Prime Minister's Delivery Unit equivalents in other governments, though it (PMDU, a cooperative endeavor of the Cabinet shares some of the functions of the Executive Office and the Treasury) "trouble-shoots" in the Office of the US President. education, crime, health, and transport sectors. These monitoring activities relate more to ser- 2.11 Budget-related routines are also, though more vice delivery (implementation) than to policy tangentially, part of the policy process. Public design, but, of course, implementation problems Service Agreements (PSAs) were the Labour can lead back to design problems. government's performance contracts between departments and the center of government 2.12 Each PSA identifies who is responsible for (Her Majesty's Treasury, the UK's Ministry of monitoring compliance. This is usually the most Finance). PSAs set out clear targets for achieve- relevant Secretary of State (Minister) while a ments in terms of public service improvements senior official is nominated to be responsible for and a Delivery Agreement19. Performance is are managing the Delivery Agreement and chair- systematically monitored in two ways. First, the ing the PSA Delivery Board, comprising all the Treasury does this routinely through regular lead Departments. A Cabinet level Committee Spending Reviews. These Reviews are also sup- will also usually monitor progress and hold Departments and programs to account for their deliverables. There are no automatic sanctions 19 The Deliver Agreements explain how the government aims to deliver its high- for failing to meet the PSA targets, although level targets under the PSAs. this will likely trigger additional scrutiny--by 39 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 2-2. Current Activities of the Strategy Unit (UK) Education, children and families: ongoing work with the Department for Children, Schools and Families Health: ongoing work with the Department of Health Care: ongoing work with the Department of Health on the care and support system Home Affairs: ongoing work with the Home Office and the Ministry of Justice on youth crime and other issues Public Service Reform - the Unit has a small team taking forward work on cross-cutting public service reform issues. The Government's overall approach to improving public services was recently set out in "Excellence and Fairness: Achieving world class public services" Welfare reform, skills and life chances: ongoing work with the Department for Work and Pensions and the Department for Innovation, Universities and Skills A cross-cutting project on food and food policy An ongoing seminar program Source: http://www.cabinetoffice.gov.uk/strategy/ HM Treasury, the PMDU, Cabinet Office and The NAO periodically produces a compendium perhaps even a relevant parliamentary commit- of its validation reports 20 tee--and discussion. 2.14 The bulk of the work on policy development is 2.13 Departmental Spending Reviews are at the largely decentralized, going on at the level of the heart of the UK's performance management line agency. Every department has a strategy di- system and are probably the most comprehen- rectorate; some have a research function;21 some sive type of evaluation. SRs take place every have attached to them independent inspectorates three years and are agreed between the UK or regulators. The departments often commission Treasury and each spending ministry, with the policy studies from outside experts. Ministers often aim of reviewing current government priori- create Non-Departmental Public Bodies (more ties, the outcomes being achieved and at what colloquially known as "Quangos" ­ quasi-non- cost. One important outcome of the SRs is an government organizations) to perform various agreed set of budget forward estimates for the functions, including providing policy advice (for next three years. While the review of spending instance, the Sustainable Development Commis- covers the entire budget envelope, the nominal sion, English Heritage). forward estimates cover around 59 percent of expenditure, known as "Departmental Expen- 2.15 There is no single system of evaluation, nor type of diture Limits", and excludes the remaining evaluation, for government activities and therefore demand-led spending (social security, debt no simple metric for measuring coverage. The type service, tax credits) that is managed annually. of evaluation and organizational unit to be evalu- Since 1998, three Comprehensive Spending ated can vary enormously depending on the pur- Reviews (CSRs) have analyzed spending, costs, pose of the evaluation. The high level evaluation of and results across all ministries and have taken PSAs can involve a number of different organiza- a significant look at the allocation between tions (i.e. across Departmental boundaries) while programs. During each CSR cycle the NAO aims to validate each data system that measures progress toward a PSA's target over its life cycle. 20 For more information on the 2007 spending review process see: http://www. hm-treasury.gov.uk/sr_csr07_index.htm Over the 2005-08 period, the NAO examined 21 See for instance the Department for Environment, Food and Rural Affairs: 237 data systems operated by 17 Departments. http://www.defra.gov.uk/evidence/index.htm 40 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA program evaluations may be more narrowly focused Whether stringent cuts and bold reforms, once on individual organizational units. Evaluations and designed in detail, can be implemented together, reviews may be coordinated by the central budget especially under a coalition government, remains to agencies (e.g. HM Treasury or the PMDU), the be seen. Nonetheless, there has been a sense in the line ministries, the National Audit Office (NAO) UK that, particularly in a climate of deep economic or by specially convened Commissions or Reviews. crisis, the coalition is ushering in a new style of (See below for more on the NAO.) government. 2.16 The departments carry out ex-ante appraisals and 2.19 How the instruments of public management, in- ex-post evaluations according to the (broad) pre- cluding those for policy making, will change is so cepts of a centrally established methodology. The far not clear. PSAs are to be replaced by Structural main "bible" for this is the "Green Book" produced Reform Plans, but the substance of this change is by the Treasury, but other (online) material from not yet evident. There may, however, be a move- the center also provides support22. The center of ment towards moving some of the policy expertise government can provide support for the policy out from the Cabinet Unit to the departments (a work of the departments, but does not vet it. And move, perhaps, towards the more decentralized no doubt there is a substantial variance in the qual- policy making models of Canada and New Zea- ity of policy work ­ both policy development and land?). By July, 52 Quangos (including nine of the evaluation ­ among departments.23 Given the wide 10 Regional Development Agencies) had been variety of evaluation practices in proportion to the abolished or lost their funding. scale of the impact of a policy, program or project, and to some extent on the level of public interest, 2.20 On the budget side, the spending reviews remain the use of external experts and stakeholder con- a central policy tool. In addition, the new govern- sultation varies considerably. For example, guid- ment created in May 2010 an Office for Budget ance for the 2007 CSR encouraged stakeholder Responsibility. Three independent experts, sup- consultation in the review and development of all ported by a small secretariat drawn from the PSAs. Also, the departments will often contract Treasury, will assess the public finances and the out impact evaluations of politically significant economy for each budget and pre-budget report. programs that are led by a prominent "expert" in The OBR is an example (somewhat akin to an the particular field. independent central bank) of using a body inde- pendent of the government to oversee or regulate a 2.17 There is no systematic quality assurance process particular area of government policy. But the OBR from the center of government for evaluations; has had a shaky start, its independence having been although most departments and agencies have questioned on account of its reliance on Treasury developed their own quality assurance processes personnel and methods. (e.g. independent review committees etc). Most evaluation reports are published on the govern- ment's websites. Subsidiary Routines of the Policy Process 2.21 A host of other institutional organizations and Changes under the 2010 coalition government systems adds to the density of the UK policy pro- cess. Some examples of evaluation and other policy 2.18 In May 2010, a coalition of Conservatives and Lib- processes are provided below: eral Democrats came to power, the country's first coalition government in over 60 years. Faced with 2.22 The National Audit Office is responsible for the a deep fiscal crisis, the government announced a financial audit of all government departments and program of unprecedentedly deep public-spending agencies as well as a wide range of other public cuts, but it also outlined a bold program of reforms, bodies, in addition to reporting to Parliament on many of them designed to limit the role of the state the economy, efficiency and effectiveness with and devolve power to front-line service providers which these bodies use public money. The NAO and citizens (in health and education, notably). carries out ex-post evaluations of selected programs and processes, but it is not mandated to evaluate the policies themselves. The NAO also carries out 22 See http://www.hm-treasury.gov.uk/d/green_book_complete.pdf and has produced guidelines for Value-for-Money 23 Examples of evaluations can be seen on all Departmental websites, e.g. the audits. The subjects to be audited range from entire Department of Health produced a report on the National Health Service in 2008: http://www.dh.gov.uk/en/AdvanceSearchResult/index.htm?searchTerms= program areas to specific components. The NAO Final+report+of+the+NHS+next+Stage+review. takes a risk management approach to select the 41 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA areas to be audited, along with consideration of the The National School of Government is views of Parliament's Public Accounts Committee a (non-ministerial) Department) that (PAC), the Departments and other stakeholders or provides training on policy-making and interested parties. The NAO presents around 60 strategy (and supports the Policy Hub). reports a year to the PAC on the Value-for-Money with which Government departments and other The Government Office for Science (in the public bodies have spent their resources. Department for Innovation, Universities, and Skills) conducts various "futures" activi- Performance Reports. Government De- ties through its "Foresight Programme" and partments are required to publish two its "Horizon Scanning Centre".27 performance reports a year ­ the Annual Performance Report and the Autumn Per- Government-funded Research Councils in formance Report. These reports should the economic and social (ESRC), medical clearly set out the level of performance (MRC), and engineering and physics (ES- achieved and the Departments' progress PRC) areas fund non-government research. towards its PSA and other objectives. Parliamentary Select Committees are cross- Evaluations at Ministries. Ministries also party bodies that evaluate and advise on produce their own evaluations, which can government actions. They can take evidence cover individual programs or reviews of stra- from ministers and the public and require tegic areas. Evaluations can be conducted information to be given. But they tend to internally, sometimes using separate evalua- be poorly staffed, a reflection of political tion units, or contracted out. The evaluation centralization. method is expected to be proportional to the importance (financial or strategic) of 2.23 Substantial policy-related activities go on outside the area to be evaluated. Guidance for ap- the government, in think tanks, interest groups, praisal and evaluation is provided in the UK NGOs, professional associations, and consult- Government Green Book. ing firms. And a lot of research occurs outside academia ­ in health authorities, market research Ad-hoc expert policy reviews by experts firms, the voluntary sector, and local government, or by prominent citizens are also commis- for instance. According to Stone (2001), these al- sioned by the Treasury and other parts of ternative policy-advice organizations create, along the Government.24 Such external reviews with the institutions inside government, a vibrant are favored where the topic is highly techni- "market for advice" in UK. And it may also be that, cal or where it is controversial (and where to an extent, these alternative organizations have the Government may want to keep itself at made up for the apparent secular decline in policy a distance). capacity within the government. Ad-hoc within-government policy reviews: in 2006, for instance, the government un- Concluding Comments dertook a series of "futures" reviews related to broad social and economic issues.25 2.24 It is difficult to know how well the different gov- ernment agencies and processes work, but they The Government Social Research Unit generally have a good reputation for effectiveness. (Cabinet Office and the Treasury) provides It also seems that various agencies, notably the strategic leadership and runs a research net- Cabinet Office and the Treasury work well across work for government social researchers. It organizational boundaries. To the extent that runs, with the National School of Govern- present practices constitute a system, these are the ment, a portal on policy-making called the main elements of that system: Policy Hub.26 The policy process remains considerably decentralized and un-routinized, as it must, 24 From 1999 58 reviews were carried out (and published on the Web Site). A given the characteristics of policy-making. prominent recent example is the Stern review on the economics of climate change: http://www.hm-treasury.gov.uk/Independent_Reviews/indepen- (Monitoring of PSAs is routinized, but the dent_reviews_index.cfm 25 See http://www.cabinetoffice.gov.uk/secretariats/economic_and_domestic/ policy_review/ 26 See http://www.nationalschool.gov.uk/policyhub/ 27 See http://www.foresight.gov.uk/index.asp 42 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA impact of monitoring on the policy process is subsidiary.) The center (Cabinet Office and the Trea- sury) suggests methodologies that range from the specifics of statistical modeling to the generalities of how to make good poli- cies. These methodologies are broad, and the departments are free to choose what they want to do. (While there are standardized routines ­ short of detailed methodologies ­ for ex-post evaluation and monitoring, there are no standardized routines ­ only broad methodologies ­ for ex-ante policy- making/evaluation.) The departments apply these methodologies (if unevenly), in part because the performance-management sys- tem demands it, but more generally because there is a growing managerial culture. But diverse though the policy-making instruments may seem, they are sometimes woven into each other (for instance, Value- for-Money audits feed into the monitoring of PSAs28). The center also fills in with explicit policy work where it sees policy-making gaps, typically on whole-of-government issues. The diversity of routines permits more of a "challenge" function ­ the ability to question conventional thinking or provide alternative explanations to fit the same facts ­ within government. Policy institutions can often have a lim- ited life: in particular agencies inside the Cabinet Office have changed regularly. Institutional diversity and impermanence may well reflect the prevalence of political considerations, or else the need for recently invented institutions to evolve (especially when solving one problem creates another). The UK government has been careful, albeit under intense public pressure, to conduct formal post-mortems on major policy mis- takes. 28 Following a Value-for-Money audit the Public Accounts Committee publishes its own report and the Government responds formally to say what it will do to implement the Committee's recommendations. Where appropriate, the recommendations maybe incorporated in to PSA and Service Delivery Agree- ments. 43 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 44 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Annex 3: New Zealand 3.1 New Zealand is a small country (population of 4.1 3.3 The original model has been significantly trans- million). It has a Westminster system of government formed since then. A number of reviews commis- in which parliament chooses the Prime Minister. In sioned by successive governments had identified a general terms, this system tends to lead to powerful significant degrading of the policy development governments with relatively limited checks and bal- and evaluation function. There was perceived to ances and the exclusion of permanent minorities. be a lack of focus on government wide objectives, However, the system was substantially modified with each minister approaching outcomes in in 1993 when the "first-past-the-post" electoral relative isolation to the outcomes of other public system was replaced by proportional representation, service organizations or desired government wide a voting system which favors coalition government outcomes. and greater representation of minorities. The change requires constant negotiating with the minority 3.4 Since 2003 the formal system has changed to parties, and any policy changes involving legislation one of "budgeting for outputs and managing for receive very detailed scrutiny from parliamentary outcomes" as provided for in the Public Manage- committees, with minorities often extracting a lot ment Act of 2004. This new approach recognizes for their support. that chief executives (the equivalent of the per- manent secretary or general secretary in other governments) are responsible for managing their The Formal Public-Management Model organization so as to contribute to the achievement of outcomes. They also advise ministers on the pos- 3.2 New Zealand's public sector reforms commenced sible outcomes to be sought, their relative priority with a series of major reforms in the late 1980s. and which outputs may be most appropriate to (The model and its subsequent evolution were simi- achieve those outcomes. Of course, some ministers lar to the UK's, but more consistently applied.) The have shown no interest in policy development and reforms were based on a model separating outcomes evaluation. from outputs, with the former perceived as the prime responsibility of ministers and with limited involve- 3.5 This new management approach is reflected ment of civil servants in policy issues. The budget in a Statement of Intent (SOI), which is a for individual government organizations became a public planning document prepared by each "purchase agreement" between the Chief Executive organization setting out its goals over the next and the Minister for the provision of outputs of an 3-5 years and its evaluation strategy. The SOI agreed quantity, quality, timing and cost. It was for is presented to Parliament at the same time as ministers to determine what outcomes were desired. the budget and is intended to form part of the Departments and ministries were reduced in size by budget dialogue. It must be endorsed by the re- separating service delivery activities into separate sponsible minister and is reviewed by the three agencies, leaving only a small residual core to advise central agencies ­ the Department of Prime ministers on policy issues. With policy development Minister and Cabinet (DPMC), the Treasury and evaluation left to ministers this area became (equivalent to the Ministry of Finance), and degraded and unsystematic, despite attempts to the State Services Commission (the central link outputs and outcomes to government policies personnel agency) ­ in terms of consistency through a system of key results indicators. with government policy. Table 3-1. The Policy Cycle and Public Policy Instruments in New Zealand Politics & policy: Budget: Technical: Technical: The policy cycle at the center at the center at the center in the departments Policy identification Cabinet, Cabinet committees, DPMC, Lead agency concept Policy Advisory Group State Services Treasury: selective Commission (civil Advisory bodies to departments Mandated system of involvement in development service regulator): and other agencies Policy decisions system of and evaluation of policies minor role. inter-departmental The Statement of Intent. consultation. The Statement of Intent. Parliamentary Select Committees Program implementation Committee of Office of the Advisory Bodies Program evaluation Expenditure Control Auditor-General (OAG) Box 3-1. Two Examples of Policy Development in New Zealand ­ Economic Transformation and Skills The Government has adopted an economic transformation agenda which comprises various elements and is led by the Ministry of Economic Development with major input also from the Treasury. Within such a broad policy a number of discrete policies have also been developed. For example a sub-agenda in this area is a draft strategy on research, science and technology, which is the prime responsibility of the Ministry of Science and Technology, but again with major input from other ministries such as Agriculture (much research is focused on the agricultural sector) and Economic Development. A "skills strategy" to improve workforce skills has recently been adopted by the Government. The Ministry of Labour has been the lead agency in developing this strategy, but it has involved other ministries such as Economic Development and Education, as well as major inputs from business and trade union representatives. For a number of other sectors discrete policies or strategies exist. For example in the education sector there are explicit policies which have been adopted by cabinet on the following issues Tertiary education, aimed at improving quality of and access to tertiary education, and to fund tertiary institutions on their role and performance, rather than as previously to provides funds based on number of enrolments The Maori education strategy, based on improving participation and achievements of the indigenous Maori community Both of these policies were initiated by ministers but detailed work on them involved the Ministry of Education and Treasury, with input also from DPMC and other agencies such as the Tertiary Education Commission. 3.6 The Policy Process. New Zealand does not have 3.7 The Articulation of Government Policies. Policies a highly centralized system of policy develop- are set out in a general way in the governing party's ment and evaluation. Policy development and election manifesto. However given the existence of evaluation is relatively decentralized but with coalition governments in New Zealand since the individual ministries and agencies playing introduction of proportional representation sys- a key role as the "lead agency" in their area. tems in 1993, these may be modified by a formal (Table 3.1 provides a guide to the main rou- published agreement signed with coalition parties. tines.) Typically, policies are made and evaluated in a 45 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA decentralized manner, with fluid forms of insti- 3.13 Because it manages the government's budget and tutional participation (see two sector examples in has a key role in advising on structural economic Box 3.1). issues, the Treasury plays a major role in the de- velopment and evaluation of policies. However, 3.8 The policy development and evaluation process given staffing shortages it has in recent years been in New Zealand thus involves a system of coor- necessarily selective in the depth of its involvement dination through joint working groups involving in particular policy areas. a range of departments, ministries and agencies, depending on the sector or topic area. But major 3.14 Although it is not a policy agency per se, the State initiatives or strategies are led by the relevant Services Commission has a major role to ensure department, ministry or agency based on a "lead the civil service is functioning effectively and is agency" concept. Policies or strategies are set out able to "deliver" in the area of policy development in the SOI of each lead agency and other partici- and evaluation.29 It has an indirect "policy" involve- pating organizations in terms of their contribu- ment in three major areas: tion, by way of outputs, to the common desired outcomes. Thus coordination mechanisms are Managing the performance contracts of therefore important. public service chief executives and evaluat- ing their performance. This performance 3.9 Advisory Bodies. In development and evaluating includes the provision of policy advice to policies the government also makes use of an the government. extensive network of advisory bodies, comprising individuals and representatives of key business Reviewing public satisfaction with the qual- and commercial organizations. For example the ity of public services through surveys. Growth and Innovation Advisory Board provides independent advice to the government on the Reviewing the operation of the "machinery implementation of its economic transformation of government", in particular the extent to agenda. which different state agencies are work- ing in an integrated and coordinated way 3.10 The Central Agencies. As in other Westminster in managing their outcomes and shared systems, the Cabinet is the primary forum for accountabilities across "clusters" of public policy decision. Eight Cabinet committees, coor- sector organizations. dinating policy development and management at the highest levels, meet regularly to provide the 3.15 Monitoring and Ex-Post Evaluation. This is pri- "engine-rooms" of the Cabinet (State Services marily the responsibility of each agency, as reflected Commission, 1999). in its SOI. In other words self evaluation is the basis of evaluation. There is no formal requirement 3.11 Three central agencies overview the policy process for evaluation, only an expectation that this will be to varying degrees - the DPMC, the Treasury, and done as the SOI is developed each year. The extent the State Services Commission. But their policy to which the central agencies review the SOIs is role is limited (and this is reflected in their small limited by their small staffing numbers. Overall it staffing numbers). appears that the SOI has yet to fully develop as a vehicle for evaluation and that progress towards an 3.12 The Department of Prime Minister and Cabinet outcomes focus has been slow. provides policy advice and analysis to the Prime Minister, although this tends to operate on a 3.16 There is a small internal cross agency evaluation reactive rather than proactive basis. The Depart- group, the Social Policy Evaluation and Research ment also services the Cabinet Policy Commit- Committee (SPEAR), led by the Ministry of tee which reviews policy proposals before they Social Development, which attempts to coordi- are considered by Cabinet. The DPMC has a nate and promote evaluation across social policy Policy Advisory Group providing strategic ad- issues. vice to the Prime Minister and other ministers, coordinating advice from other departments, and contributing to policy development across 29 The State Services Commission evaluates public officials' performance and the range of government issues. But this Group carry out surveys to evaluate clients' satisfaction with public services. It showed is small ­ a director and 13 advisors ­ and plays a flurry of interest in the policy process in the late 1990s and it created a policy network providing information and guidance to policy staff, but in practice this a limited role in generating policies or assessing has remained a limited effort. See http://www.ssc.govt.nz/display/document. policy proposals. asp?NavID=176 46 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 3.17 The key roles of SPEAR are to: 3.22 The Auditor-General is appointed by and reports to the Parliament and has considerable stature and Identify and disseminate information needs in authority in its review of government operations. the social policy area, In addition to its traditional financial and compli- ance auditing role, it audits reported information Develop a research and evaluation agenda to on outputs in public service organization's annual meet these needs, reports. The Auditor-General employs staff in two business units - the Office of the Auditor-General Provide quality assurance for government social and Audit New Zealand - and contracts with policy research and evaluation, private sector accounting firms. The total staff of OAG numbers around 300. Assist in integrating the results of such research and evaluation into government social policy 3.23 It also has a broad performance auditing role decisions. under which it may report on the efficiency and effectiveness of operations of government 3.18 The SPEAR, established in 2001, is a technical organizations. However this mandate does rather than a policy-making institution. The Min- not extend to reviewing "the merits of policy istry of Social Development provides the full-time objectives". The Auditor-General may evaluate secretariat and reporting is through the chief execu- the extent to which policy objectives are being tive of that same ministry. There is also close liaison achieved, but does not comment on the policy with the Strategic Social Policy Group of senior objectives themselves. In addition, the perfor- officials working in the social policy area. There is mance auditing role comprises a relatively small an independent or external chair, from outside the part of the audit activities. Thus, the National civil service. Close collaboration with Universities Audit Office does not play a significant role in and other research institutions is also an important policy evaluation. part of the SPEAR's work. The arrangements for its work to input into decision making on evalua- 3.24 The performance auditing role of the OAG is tions is not yet clear. based on a mandate to audit the performance of all public entities. The proposed program of per- 3.19 SPEAR's focus is on reviewing existing policies formance audits is published in the OAG Annual and its use is to decide where new policies may Plan. Around 15 performance audits are completed need to be recommended. As a technical group every year, making this activity a relatively small of social policy evaluation officials, it has issued part of OAG work compared with its financial good practice guidelines. It also seeks to develop auditing role. All performance audit reports are evaluation capacity in social policy evaluation and publicly available and may be discussed by the Fi- research. However, its impact so far appears to have nance and Expenditure Committee of Parliament been limited. For instance, no evaluations have yet or other parliamentary committees relevant to the been carried out. subject matter of the report. 3.20 Parliamentary oversight. The New Zealand Par- 3.25 A performance audit can examine: liament has up to 13 subject-area select commit- tees which report on policy issues to the House.30 How effectively and efficiently a public en- In principle Parliament, which receives the SOIs, tity is working, could also undertake ex-post evaluation, but in practice it has not done so. Whether it is complying with its statutory obligations, 3.21 Office of the Auditor-General (OAG). The OAG is established by the Public Audit Act 2001. Any act or omission that might waste public The general role of the OAG is to provide inde- resources, pendent assurance that public sector organizations are operating and accounting for their performance Any act or omission which might show or in accordance with what Parliament intended. It appear to show lack of probity or financial operates on the basis of its own professional audit- prudence. ing standards. 3.26 This broad mandate is limited by the OAGs' in- ability to question or comment on policy (which 30 http://www.parliament.nz/en-NZ/SC/ includes questioning the objectives of government 47 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA programs). The focus of performance audit reports tives also reviewed the completed reviews to is on identifying opportunities for performance ensure that they met the requirements laid improvement. down by Cabinet. These were then reviewed by the Minister of Finance and considered 3.27 This focus of performance improvement is also by the Cabinet Committee on Expenditure reflected in the OAG issuing a number of good Control. practice guides. So far, 19 good practice guides have been issued covering topics such as report- 3.32 Second, there is an intention to develop a system ing on performance, procurement, setting of fees of in-depth spending reviews to examine the ef- and charges, the role of audit committees within ficiency and effectiveness of expenditures in par- government organizations, codes of conduct and ticular sectors. It is recognized that such reviews severance payments. are resource intensive and time consuming so they will be carried out as rolling reviews over a 3.28 In addition to the performance auditing role and as 3-4 year time period. Initially a limited number part of its audit of the annual report and financial will be undertaken. The government intends to statements, OAG comments on the adequacy and proceed cautiously and adapt procedures as lessons reliability of performance measures. These com- are learned. Each review is expected to take 3-6 ments are reported in the annual statement of months to complete. service performance, which sets out the outputs delivered by the organization as set out in the out- 3.33 Terms of reference for each review will be ap- put based budget appropriation. These are part of proved by the Cabinet Committee on Expenditure the annual financial statements. Control31 on the recommendation of a Committee of senior officials. Each review will be managed by the chief executive of the agency under the direc- Recent Changes tion of a steering group, chaired by an independent external person. The Treasury and other outside ex- 3.29 There are three major initiatives among the latest perts will also be part of this steering group, which changes undertaken by New Zealand in the area of will report to the agency minister, the Minister of policy process and policy assessment: Finance and the Minister of State Services. These ministers will then submit the completed reviews to 3.30 First, line by line expenditure reviews have been the Cabinet Committee on Expenditure Control. carried out by chief executives, feeding into the May 2009 budget. These reviews were aimed at 3.34 The line-by-line expenditure reviews referred to identifying: above will assist in identifying topics for these reviews. However, no decisions have yet been made savings that could be freed up for the 2009 on the timing, number and subject matter of these budget reviews, nor has a formal methodology been devel- oped. These will be ex post reviews. programs that are inconsistent with the new government's priorities and should be 3.35 Third, the Government has initiated efforts for discontinued improving the quality of performance information. Reports by the Auditor-General on the perfor- programs and expenditures that are not ef- mance information contained in the annual reports fective or efficient of each agency (specifically in their State of Service Performance, which is audited by the Auditor- areas where performance information is General) have identified a number of deficiencies insufficient to make judgments about ef- including a lack of clarity on the relationship be- ficiency and effectiveness tween output indicators and the overall outcomes sought by government. As a result, Treasury and 3.31 As a result, the 2009 budget saw some significant the State Services Commission have looked closely reallocations of expenditure particularly to major at the performance indicators contained in the infrastructure expenditure and the health and 2009/10 Statements of Intent (which include fore- justice sectors, and away from the education sector. cast statements of service performance) as part of These reviews were undertaken by chief ex- 31 The Cabinet Committee on Expenditure Control is formed by ministers. See: ecutives and then provided to their minister http://cabguide.cabinetoffice.govt.nz/context/definitions/cabinet-committees/ for consideration. A panel of chief execu- ecc 48 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA an ongoing program of improvement. The govern- implementation appear to have ab- ment has indicated a particular concern for greater sorbed most of the attention and time of clarity on the impact of government expenditures Ministers and officials. Other processes on broad social indicators such as life expectancy have suffered as a result. Particular gaps and child poverty. exist in areas such as evaluation, issues identification (including anticipation of emerging problems), the notion Concluding Comments of long-term and forward-looking research-based policy analysis, public 3.36 It is generally agreed that New Zealand's public- consultation, and strategic analysis and management model, as originally construed in the management." (State Services Commis- reforms of the 1980s, emphasized efficiency in sion, 1999) reaching outputs at the agency level, at the expense of outcomes and of the coherence of policies across 3.37 In responding to perceived failings in the policy government. There has also been a separate criti- process, New Zealand has followed its own par- cism of policy capacity, though this has not been at ticular path. It has deliberately kept the center of the center of reformers' concerns. This criticism ­ government small and the policy-making process that the agencies themselves are not well equipped decentralized. Unlike the UK, for instance, New to make good policy ­ was expressed by the State Zealand has not sought to create its own policy-as- Services Commission in a 1999 report and can be sessment capacity in the center or to promote, from summarized as follows: the center, a network of support to policy-making in the agencies. Instead, the center has worried "Policy analysis and design of delivery about better processes for coordinating policies. instruments - based largely on theoreti- And even here, the principal answer has not been cal frameworks - process coordination, better center-led processes, but the concept of the and the design and management of "lead agency". 49 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 50 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Annex 4: Canada Political Routines at the Center of Government policy, consistent with their minister's agenda and the agenda of the Government as a whole. This will 4.1 As in most Westminster-style governments, the require a base of support in central agencies, part of Cabinet is the supreme policy-making body in whose function is to ensure such consistency. In this Canada. All major government policy matters are regard, it is worth noting that deputies are not ap- forwarded to it for decision. Cabinet ministers make pointed directly by the ministers for whom they work, decisions together, and bear collective responsibility but by the Prime Minister on the advice of the Clerk for them. The Prime Minister has broad discretion of the Privy Council, which reflects the fact that they as to the specific structure of the Cabinet decision- are accountable not only to their own ministers but to making process, including the committee system. As the Government as a whole. a rule, discussion about policy proposals takes place in committees, whose mandates most often pertain to a 4.4 In general, Canadian policy-making styles have particular policy sphere (e.g., economic, social, foreign varied over time. Prime ministers' choices of affairs and defence) and whose recommendations are decision-making styles and systems have reflected ratified by Cabinet. Ratification authority may also both their personal styles and the broader political be delegated to a committee; for example, under the environment. The temper of the times has played a current system, committee recommendations may be role too. Budget constraints or public support for ratified by the Priorities and Planning Committee, Government play a major role in shaping the Prime which is chaired by the Prime Minister. Cabinet may Minister's style of policy-making. (Schacter 1999). also engage in broader strategic discussion. 4.2 The formal decision document in the Canadian Budget and Expenditure Management system is the Memorandum to Cabinet. This docu- Routines at the Center of Government ment is the basis for discussion in Cabinet com- mittees and normally, committee recommenda- 4.5 Traditionally the budget process has been coordi- tions are placed on the full Cabinet's agenda for nated by the Department of Finance, particularly decision. A committee recommendation may be in a context of fiscal constraint. The Minister of Fi- placed on Cabinet's agenda as an annex item which nance must work closely with the Prime Minister, is usually affirmed without discussion. whose approval is required for all budget measures. 4.3 The link between policy and implementation is a 4.6 A significant step, although not strictly a budget responsibility of the deputy ministers, who are the function was the creation of the Expenditure Re- pre-eminent source of public service advice to their view Committee within the Cabinet in 2003, which ministers on all matters and have day-to-day manage- was set up "to conduct a fundamental review of all ment of ministerial departments. Deputy ministers are programs and expenditures".32 This committee was appointed by the Prime Minister on the advice of the chaired by the President of the Treasury Board and Clerk of the Privy Council (who is Head of the Public included other senior ministers. The current compo- Service) and are generally drawn from the ranks of the senior public service. Their responsibilities include helping to build the horizontal mechanisms needed 32 See Department of Finance of Canada http://www.fin.gc.ca/toc/2005/ to support the development and implementation of sdsplan05-06-eng.asp Table 4-1. The Policy Cycle and Public Policy Instruments in Canada Politics & policy: Budget: Technical: Technical: The policy cycle at the center at the center at the center in the departments Policy identification Privy Council Office Prime Minister and Cabinet Treasury Board Secretariat Deputies Prime Minister; Policy decisions Committees Department of Finance Minister of Finance Program implementation Departmental public service Evaluation functions Program evaluation Treasury Board Secretariat within departments sition of Cabinet no longer includes an Expenditure Treasury Board Secretariat (TBS) and the Depart- Review Committee. However, its establishment is ment of Finance (Finance). a reflection of the committee system's flexibility, as specialized and ad-hoc committees can be set up to 4.9 The Privy Council Office. While the Prime Min- deal with specific issues or priorities as they arise. ister has his own staff in the Prime Minister's Office (PMO) to provide policy advice and other 4.7 Since that time, the Government of Canada has in- support from a partisan orientation the PCO is the troduced a new expenditure management system de- Prime Minister's public service department and his signed to ensure value-for-money for all government leading source of non-partisan advice, including spending. A key pillar of this system is the ongoing professional, technical policy assessment. assessment of all direct program spending, known as strategic reviews. Strategic reviews are assessments 4.10 The PCO's role is, broadly, two-fold. As the Prime of all direct program spending to ensure programs Minister's public service department it provides ad- are managed effectively and efficiently. These reviews vice to the Prime Minister, through the Clerk of the support a rigorous results-based approach to man- Privy Council, in all policy areas ­ e.g., economic, so- aging taxpayer dollars responsibly and delivering cial, intergovernmental ­ including a number of areas effective and efficient programs that can better meet that are distinctive prime ministerial responsibilities the priorities of Canadians. As part of the strategic such as the organization of government, certain types review process, departments review their direct pro- of appointments and constitutional advice provided gram spending and the operating costs of their major by the Prime Minister to the Governor General. statutory programs on a four-year cycle to assess how and whether these programs: (i) are effective and ef- 4.11 Additionally, PCO serves as the secretariat to the ficient; (ii) meet the priorities of Canadians; and (iii) Cabinet. In this capacity, it assists the Ministry as a are aligned with federal responsibilities. Through the whole in managing the Cabinet agenda. It reviews all strategic review process, departments also determine policy initiatives forwarded to the Cabinet, ensuring whether there are any lower priority, lower perform- the quality of policy proposals that are put before ing programs for possible reallocation of funding to ministers as well as their consistency with the gov- higher priority, higher performing programs within ernment's broader agenda (the "challenge function"), the department or government. and ensuring that committee chairs are appropriately briefed on policy proposals. Once decisions are made, PCO communicates them to Departments. PCO Technical (Administrative) Routines is staffed by politically neutral public servants. The at the Center of Government years have firmly established PCO's role as a major actor in the policy development and decision-making 4.8 Currently three central agencies provide public process in Canada. This office has also developed a service support at the center of government in reputation for being home to some of Canada's "best Canada: the Privy Council Office (PCO), the and brightest" public servants (Schacter 1999). 51 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 4.12 Another critical function of PCO is to ensure Schacter's (1999) findings, "neither the cabinet nor that all Ministers and their officials interested PCO concerns itself greatly with implementation. in a policy proposal are given an opportunity to There is a strong and well-founded assumption consider it in advance and to express their views, in cabinet that a department sponsoring a policy and all Ministers and their officials interested in a initiative will carry through and implement it, and policy proposal are operating from a common base that no form of close regular oversight is required." of sound information. A significant dimension of the Cabinet process is to ensure that appropriate 4.15 However, Canada's policy and program evaluation interdepartmental consultation has taken place, process has gone through significant changes since including meetings among the central agencies as it began in the 1970s. Among the most significant well as among implicated departments. changes are the conceptual separation between policy and program evaluation, and the develop- 4.13 The Treasury Board Secretariat is a committee of ment of more flexible tools to allow for wider range the Cabinet whose existence has been mandated in of evaluation targets. statute since the early days of Canadian Confed- eration. The Treasury Board Secretariat, provides 4.16 Program evaluation was introduced in Canada in the secretariat support for the Board in a manner 1970s to help improve management practices and analogous to PCO's support for other Cabinet controls. Since then many changes have taken place. committees. Treasury Board is the government's The original approach involved creating evaluation management board and as such sets guidelines and units in the ministries to provide deputies with rec- policies for administrative practices across govern- ommendations to improve the decision ­ making ment. Subsequent to approval of initiatives, Trea- process. The results were uneven. "The definition of sury Board approval is usually required to access program was imprecise and Ministers were largely funds to ensure that implementation is consistent outside the process, often viewing evaluation and with policy and funding approvals. Among other review as an internal bureaucratic process. For lack responsibilities, TBS sets standards for policy and of direction or interest, evaluation units languished. program evaluation and provides support and (Treasury Board Secretariat, 2004). guidance in this matter. It is also responsible for making evaluation information available and to 4.17 According to Schacter (1999), highly formalized assist the Treasury Board in using this information systems for "policy management" and decision- for decision-making. The Treasury Board plays a making have not succeeded in Canada because they lead role in the Strategic Review Process, although were perceived as inefficient means for transmitting final decisions are made by the Prime Minister and information and ideas to cabinet. The paperwork Minister of Finance as part of the budget process. and meetings they generated were rejected as an unproductive use of ministers' time. 4.14 Policy and Program Evaluation in Canada. As in the UK, Canada's evaluation system is decentral- 4.18 Latter, there was an attempt to combine program ized, the agencies being responsible for selecting evaluation with traditional audits. The emphasis was which programs to evaluate, and developing the re- put on having managers use performance informa- quired capabilities for this task, according to TBS tion in decision making (and to demonstrate this). standards. One important aspect that sets Canada However, in 2000 a study revealed that this approach apart from general Latin American experience (or, didn't have enough of an impact, and that evaluation for example, Spain) in evaluation is that outside and audit functions needed to be separated. technical advice is sought mainly by each agency according to their needs. The TBS does not use out- 4.19 Based on these past experiences, in 2001 the Trea- side experts.33 On the other hand, program evalua- sury Board approved the Evaluation Policy and tion is implemented mostly ex-post. According to Standards for the Government of Canada. With a renewed focus on performance management, this policy separated internal audit from evaluations, 33 The TBS maintains an Audit and Evaluation Database where it keeps, highlights, and extended the scope of evaluations to differenti- and/or provides internet references for completed audits, evaluations and ate evaluations of policies, programs, and initiatives. similar studies: (i) Departmental and government performance reports are electronically linked to each other and to more detailed information on audits and evaluations and their findings.; (ii) Several hundred records of findings 4.20 However, even though the new evaluation frame- are added to the database each year, including information from evaluations, audits, manager-led reviews, self- assessments and continuous performance work is intended to separate criteria for policies measurement systems. This information is always being updated; and (iii) Re- and program evaluation, the approach to policy cords on studies are linked, where useful, to departmental strategic outcomes, to government administrative policy interests and to horizontal issues. TBS staff evaluation is different from the one proposed in this is notified regularly of studies relevant to their work. document. The new approach in Canada focuses on 52 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA program evaluation, including its consistency with 4.24 For each area, the MAF suggests ways to measure government policies, and also ex-post evaluations progress and provides indicators. It uses and sup- of policies conceptualized as a series of related pro- ports the information generated by the evalua- grams (or "bigger" programs). It does not consider tion units in each ministry as an input. Agencies assessment for policy formulation, considering the conduct self-assessments using the framework that intent (problem to be solved, objective), in con- TBS has distributed to all federal departments and trast to implementation, which is effectively done agencies. In each department/agency, the deputy is through programs. responsible for establishing an appropriate evalu- ation capacity tailored to the needs and resources 4.21 In short, evaluation capacity is being rebuilt in the of their organization, and evaluating policies, departments. The establishment of "program activ- programs and initiatives, including those of an ity architecture" has assisted evaluation by helping inter-organizational nature. In each department departments identify exactly what programs they or agency, the deputy appoints a senior head of actually have as well as how much they actu- evaluation, forms an evaluation committee, and is ally spend. The strategic review process mentioned responsible for giving the TBS access to evaluation above has increased the relevance of evaluation to plans, and communicating early warnings based individual departments, since the process forces on the evaluations findings. During MAF Round them to prioritize program spending. VI - from September 2008 to April 2009- over 52 departments were to be assessed, including the 4.22 The Management Accountability Framework Privy Council Office and the TBS.35 (MAF) is a tool developed by the Treasury Board Secretariat (TBS) in 2003 to assess management capacity in federal departments and agencies and Concluding Comments provide them with guidance in order to attain a high level of management performance.34 It brings 4.25 Canada has gone through different stages for pro- together different TBS management improvement gram and policy evaluation. From more rigid and initiatives in a wide reaching, but simple frame- process-driven program evaluation, it has moved work. The MAF is also used to help assess the to a more flexible, more holistic approach. A tool performance of deputy ministers. like the MAF could be considered as a reference point for implementing policy evaluation accord- 4.23 The MAF consists of ten areas for which depart- ing to general criteria. Canada is also character- ments must examine, report, and improve: ized by a rather decentralized approach, where a lot of the responsibility for evaluation lies in the Public Service Values departments, which even select the programs to be evaluated. Governance and Strategic Directions 4.26 Also, in Canada, as in other Westminster type Policy and Programs of governments, the office that supports Cabinet policy-making decisions can also be considered a Results and Performance source of experience in ex-ante policy assessment. In this case it is the PCO, which performs a filter- Learning, Innovation and Change Manage- ing of policy initiatives information, manages dis- ment putes over policies and provides policy assessment assistance to Cabinet. Risk Management People Stewardship Citizen-focused Service Accountability. 35 See http://www.tbs-sct.gc.ca/maf-crg/assessments-evaluations/2008/depart- 34 See http://www.tbs-sct.gc.ca/maf-crg/index-eng.asp and Dantzer (2009). ments-ministeres-eng.asp 53 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 54 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Annex 5: The United States 5.1 The OECD is dominated by parliamentary sys- in large part at least, of a cabinet mechanism for tems of government. In these systems, a hierarchi- developing, assessing, and coordinating policies, cal power structure leaves the Prime Minister and the President has had to build up his own staff to cabinet with substantial power in the policy process do this, or else find more ad-hoc ways of doing it. as long as the parliamentary majority continues to support the government. Cabinet government rep- 5.3 Presidents and cabinet ministers, in turn, have not resents a collective approach to managing the policy served the same kind of fairly standard political process: policy alternatives are vetted through this apprenticeship as their parliamentary equivalents. process, and the discipline of collective responsibil- Presidents come to office as individuals, rather than ity is strong enough to resolve differences. Cabinet part of a team, and they wield considerable powers members have typically served a political appren- of patronage to install their own confidants into the ticeship as members of parliament, then junior most senior positions in the executive. There is a members of the government. Supporting cabinet professional and permanent civil service in the US government is an army of permanent senior civil but it does not have the standing of its parliamen- servants with technical and administrative skills. tary equivalent; and, though non-partisan, it can And there are relatively few political appointees. sometimes be eyed with mistrust by an incoming administration. The Senior Executive Service, SES, 5.2 The United States is an outlier.36 American gov- a group that sits at the top of the civilian ranks, is ernment is one of checks and balances ­ of many part of the professional and permanent civil service.37 veto points. The executive and legislative branches of government share policy-making powers (and often the judicial branch also gets into the act). The Policy Routines at the Center of Government legislature legislates, but the President can veto legislation. The President makes a large number of 5.4 Given that the US cabinet is a fairly weak body, political appointments, but the Senate must vote most decisions are made in the Executive either to confirm the appointment of the most senior of bilaterally between the President and his ministers these and Congress can impeach the President. The (secretaries) or in trilateral arrangements between President has a cabinet to advise and assist him, but the President, the secretary concerned, and senior it is not held under the same convention of col- congressional figures. lective responsibility as in parliamentary govern- ments. Congress, with its power of the budget and 5.5 The largest and most powerful central body is the its power to monitor management in the Executive Executive Office of the President (EOP), an or- (by vetting senior appointments and calling senior ganization with approaching 2,000 employees that officials to account), exercises considerable coun- provides an umbrella for more than a dozen staff tervailing power over the agencies in the Execu- agencies.38 White House staff is responsible for tive. Thus cabinet members are somewhat strung between the President and Congress. Deprived, 37 According to the Office of Personnel Management, OPM, in 2006 there were less than 7,000 SES out of a total of 2.7 million civil service employees. 38 The EOP is the functional equivalent of parts of the cabinet system in 36 For a useful comparison of the US style of policy process with the parliamen- parliamentary regimes, containing the equivalent of both the cabinet/Prime tary style see Rose (2006). Minister's office and of some cabinet committees. Table 5-1. The Policy Cycle and Public Policy Instruments in the United States Policy cycle Politics: at the center Budget: at the center Technical: at the center Technical: in the ministries Policy identification: - Identification Executive Office (13 of issues departments) - Policy analysis Specialized policy - Policy instruments Think-Tanks and consulting firms research agencies Office of Management Executive Office of the and Budget Ad-hoc studies Policy decisions: President (specially the Special task forces Financing of external research - Consultation and Chief of Staff and National coordination Security Adviser) Congressional Budget Office - Assessment Congressional Committees staff Decision Congress / Congressional Committees Program implementation: Office of Management - Execution and Budget - Monitoring Program evaluation: Office of Management and Budget (OMB) - Evaluation Ad-hoc evaluations - Correction of Government Accountability Office (GAO) programs or policies some of the senior appointments down to several lature in the US, the analytical strengths of the US levels in the bureaucracy, depending on the agency. Congress constitutes a significant contrast with The Office of Management and Budget, an execu- Latin American congresses, which, generally, have tive office within the Executive consolidates budget considerably less of the technical expertise required proposals from the line departments and presents for assessment of policies. them to Congress, which includes setting spending priorities and assessing resource disputes among 5.8 Furthermore, the powers given to Congress by the agencies. In this way the White House has signifi- Constitution in the areas of budget and of approv- cant control over both personnel and the budget. ing senior appointments provide further contest- ability. The majority of policy initiatives come from 5.6 The individual departments are responsible for the Executive, and Congress plays a reactive role policy-making in their areas of competence, and in assessing and voting on them. However, the US depending on the will and focus of the White Congress is also characterized by a strong control House in specific areas, present a significant level function for which it has different resources than of flexibility to do so. A great deal of policy-making does the executive. authority can be centralized in the White House (Beschel and Manning, 2001). 5.9 On the subject of technical support for policy as- sessment, the United States presents a wide and 5.7 Nevertheless, while the Executive typically con- complex net of policy assessment sources, also centrates significant amounts of power, the US probably bigger than anywhere else. According to Congress has enough policy assessment capacity to Weaver and Stares (2001), there are more than a fulfill its role as check and balance of the Executive. dozen government research agencies, around 1,000 The analytic infrastructure of Congress--including government advisory committees, more than 300 individual legislative staffs, committee staffs, the independent public policy think tanks, more than Congressional Budget Office, and the Congres- 500 university-affiliated research institutes, and sional Research Service, among others--is far dozens of interest groups and consulting firms. larger than any comparable legislative support system elsewhere. Though there is generally more 5.10 These institutions support government policymak- technical capacity in the executive than the legis- ing in different ways and with different level of 55 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA involvement. Some of them are permanent like the composed of four Resource Management Offices Council of Economic Advisors, or the Advisory organized by agency and program area, four statu- Committees, while others are created for specific tory offices, and seven offices that provide OMB- policy issues, like the task forces for healthcare re- wide support. The vast majority of OMB staff re- form during Clinton's administration or federally- main from administration to administration; only funded R&D. The Legislature receives policy ad- about 10% of officials ­mostly at the top--turn vice from different sources, from more traditional over and thus are direct Presidential appointments. and grounded institutions like the Government Accountability Office (GAO), the Congressional 5.14 The OMB also assists the Presidency in assess- Budget Office (CBO) and Congressional Com- ing the effectiveness of agency procedures and mittee staff, to more ad-hoc advice, for example in improving public management by developing private sector specialists when it conducts hearings better performance measures and coordinating on a particular policy issue. 39 mechanisms with the agencies on this matter. The OMB´s activities also involve setting funding priorities, evaluating competing funding demands Policy (and Program) Assessment among agencies, and ensuring that agency reports, in the Executive rules, testimony, and proposed legislation are consistent with the President's Budget and with 5.11 The Executive Office of the President includes 13 administration policies. departments40 that provide policy advice in subjects that range from economic policy (Council of Eco- 5.15 The OMB has a Legislative Reference Division nomic Advisers) and domestic policy (Domestic which articulates the Administration's position on Policy Office) to the environment (Council of En- legislation. This division coordinates the review vironmental Quality) and drug control (Office of and clearance of the Administration's legislative Drug Control Policy). One very important source proposals and statements on bills progressing of policy advice related to the federal budget, as- through Congress. This responsibility frequently sessment of policy legislation initiatives, regulation, requires resolution of conflicting agency views on and management improvement throughout the legislation and the negotiation of policy positions Federal Government is the Office of Management that respect the President's legislative priorities and Budget. and program. 5.12 The OMB, initially called the Bureau of the Budget, 5.16 Other areas in which the OMB has influence in was created in 1921.41 In the first four decades, the assessing policy are regulation and e-government. emphasis of the agency was on budget expertise. The Office of Information and Regulatory Affairs However, in 1970, it was renamed and got more oversees and coordinates Federal regulatory policy, political control from Executive and Congress and information collection and quality, government now plays both a political and analytical role. statistical policies and practices, and Federal infor- mation policy. The Office of E-Government and 5.13 As normally happens with most senior government Information Technology develops and provides officials in the US, OMB authorities are appointed direction in the use of information technology and directly by the President. The OMB´s reach goes the E-Government management initiative.42 further than that of more traditional program/bud- get assessment agencies. OMB's formal mission is 5.17 While the OMB provides routinized technical and to assist the President in overseeing the activities political assistance in key areas of policy, there is no of the Federal Government, specifically, in meet- specific framework for policy evaluation. This is not ing his policy, budget, management and regulatory the case for program and management evaluations, objectives. It has 480 full time employees, and is which are indeed formalized in specific initiatives. One example was the Program Assessment Rating Tool (PART), a collaborative process involving 39 A good example is the National Academy of Science, an honorific society of distinguished scholars engaged in scientific and engineering research that both agencies and OMB participants. Though now works outside the framework of government to ensure independent advice defunct, it is good to include PART in this annex on matters of science, technology, and medicine. US governments have often turned to the NAS for advice on the scientific and technological issues that for referential purposes. Used over the George W. frequently pervade policy decisions. The NAS is a highly prestigious indepen- Bush administration, the objective of the PART dent body, established by law. More details at: http://www.nasonline.org/site/ PageServer 40 For a complete list: http://www.whitehouse.gov/administration/eop/ 41 Before 1921, cabinets played a larger role than they do now. The creation of OMB strengthened the President vis-à-vis the agencies and Congress, but 42 See OMB organization chart: http://www.whitehouse.gov/omb/assets/ weakened the cabinet (Apfel, 2010). about_omb/omb_org_chart.pdf 56 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA was to help identify a program's strengths and sessed 1,015 programs or about 98% of the federal weaknesses in order to inform funding and man- programs, accounting for about 90 percent of the agement decisions aimed at making the program 2008 federal budget. OMB received funding for more effective. PART consisted of a series of ques- 528 full-time-equivalent positions for fiscal year tions in four categories, including program purpose 2010 (the same as in fiscal year 2009).46 All PART and design; performance measurement, evalua- evaluations are published on the website Expect- tions, and strategic planning; program manage- More.gov. Internally, the OMB worked with each ment; and program results. Each agency prepared agency to prepare improvement plans to follow the PARTs related to their programs in coordina- up on findings identified during the assessment tion with a policy analyst from OMB assigned to process with the objective of improving program that particular agency. The primary OMB contact performance. These plans were updated twice a was the program examiner, who received the first year, in the spring, to be used during the budget draft from the agency and worked with it to per- formulation process, and in the fall, for release with form consistency checks in different iterations. If the President's budget (budget discussion approval disagreements remained, agencies could request an in Congress). appeal process to a high level appeals board. Only OMB had the ability to edit the final assessment 5.20 During the Bush administration, the OMB also results. Agencies relied on a variety of personnel implemented a scorecard to measure advances in both from the programs being reviewed and from the President's management agenda throughout central Department offices to complete the PART the Departments. It rated Departments on five cat- reviews. egories: human capital, management of commercial services, financial performance, e-government, and 5.18 OMB, through the PART, evaluated and looked performance improvement. In all cases, the OMB at factors that affected program performance in- provided the guidelines for evaluation, and the cluding program purpose and design; performance Departments were responsible for implementation. measurement, evaluations, and strategic planning; program management; and program results. These 5.21 As of November 2009, the Obama administra- assessments of existing evidence were not impact tion called into question the use of the PART evaluations. In terms of PARTs quality control, instrument and announced plans to implement a OMB's internal management and the agency's performance management framework of its own management checked all assessments. Agencies to examine a selected set of programs each year. could appeal the rating.43 However, there was no The reasons for discontinuing PART point to the review of these ratings by Congress or other actors main weaknesses of the program mentioned above outside of the executive branch. This factor created and, in particular, the lack of interaction with some doubts regarding the rigor and independence Congress47. In the 2011 budget (February 2010), of the evaluations, something that did not help the administration unveiled part of its own man- to promote the use of PART during the budget agement agenda, which included a requirement for discussions in Congress44. In addition, PART's high-priority performance goals and an emphasis methodology to measure efficiency was also ques- on rigorous and independent program evaluations. tioned.45 5.19 There was no particular percentage of the budget Policy Assessment in Congress covered each year by number of programs. PART evaluated on average close to 200 programs per 5.22 Between the Senate and the House of Representa- year, representing approximately 20% of the num- tives the US Congress has a total of 38 Committees ber of federal programs. PART was introduced (four of them joint), with their respective Subcom- in fiscal year 2004 and as of July 2008 it had as- mittees. Both chambers are supported in the area of evaluation by two main non-partisan technical institutions, the Government Accountability Of- 43 See OMB's chart at the Annex 44 "By routinely incorporating agency performance issues into its deliberations fice (GAO) and the Congressional Budget Office and oversight, Congress can send an unmistakable message to agencies that they are expected to manage for results. As we have noted in our earlier work, however, Congress needs to be actively involved in early conversations about what to measure and how to present this information. We previously reported that the PART process used by the prior administration did not systematically 46 For more information, see OMB Fiscal Year 2010 Budget at: http://www. incorporate a congressional perspective and promote a dialogue between whitehouse.gov/omb/assets/about_omb/fy2010_omb_budget.pdf Congress and the President. As a result, most congressional committee staff we 47 "It's crucially important we revamp and revise [performance measurement sys- spoke to did not use the PART results to inform their deliberations" Source: GAO. tems] in consultation with agencies and Congress." ­Testimony of Peter Orszag, http://www.gao.gov/new.items/d091011t.pdf OMB Director, before the Senate Homeland Security and Governmental 45 See, for example, GAO http://www.gao.gov/new.items/d10394.pdf Affairs Committee, January 2009. 57 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA (CBO).48 The Government Accountability Office provides advice to further improve the CBO's work. (GAO) is an independent, nonpartisan agency that A Panel of Health advisers works in a similar way in works for Congress with the mission to provide matters related to healthcare policy issues. support in meeting its accountability function of the federal government. The head of GAO, the Comptroller General of the United States, is ap- Concluding Comments pointed to a 15-year term by the President from a slate of candidates Congress proposes. Other senior 5.25 As in most other cases, in the US, while program leaders are appointed and serve at the pleasure of evaluation processes are more formalized and the Comptroller General. clearly associated with specific Executive and Congress offices (OMB and CBO), policy evalua- 5.23 At the request of congressional committees or tion, especially ex-ante policy evaluation is a more subcommittees or as mandated by law, the GAO informal, varied, and complex process that seems to performs audits on the legality, effectiveness, and depend mostly on the initiatives of the actors in- efficiency of the use of federal funds, and reports volved. Both OMB and GAO conduct qualitative on how well government programs and policies evaluations. No impact evaluations are undertaken are meeting their objectives. It also performs by any of these agencies. The CBO does baseline policy analysis and outlines options for congres- budget projections and cost estimates of the legis- sional consideration and issues legal decisions and lation. The majority of CBO expertise is vested in opinions, such as bid protest rulings and reports on the staff itself. agency rules. GAO evaluates ex-post as well, fol- lowing requests by Congressmen. GAO conducts 5.26 One noteworthy feature is the importance that performance audits, which may look at the impact this country gives to technical support (within that federal programs are having on a particular government and from outside sources) for policy problem, and traditional financial audits. GAO implementation and evaluation. It probably has does not undertake impact evaluations. Although the most extensive network for technical policy evaluations are conducted by GAO staff, GAO is advice, including long standing Executive and authorized to bring in outside experts to consult Congress offices, technical staff in councils and and advise. External consultants, though, are not committees, and outside support from think tanks allowed to do actual audit work. The Comptrol- and consulting firms. As a presidential country, the ler General can be removed only by a vote of the US is characterized by a strong control of policy Congress; the President has no authority to remove within the Executive. But in contrast to most the head of GAO. Latin American experiences, it is better balanced by countervailing powers of Congress, a situation 5.24 The Congressional Budget Office (CBO) began op- in which the latter's significant technical capacities erating in 1975 with the specific mission to provide for policy assessment play a major role. technical advice to Congress on subjects related to the budget and the economy, including the elabora- 5.27 The US has, in summary, many points at which tion of economic estimates to assess the Executive policy contributions are made. This partly reflects budget proposal. The CBO has a panel of advisors, the countervailing nature of US government, with composed of some of its previous directors and its institutional pressures and counter-pressures. eminent economists, who serve two-year terms. The In some ways there is a free market of ideas, but panel meets twice a year to review and comment on in another sense the market is less free because CBO's preliminary forecasts for the economy and institutional roles matter. Technical competence matters a lot, but the degree of neutrality in the advice provided depends on institutional setting 48 Until 1995 there was also an Office for Technology Assessment. (Apfel, 2010). 58 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 59 Annex 6: France 6.1 With more than 50 percent of GDP, France's pub- and endows public servants with a strong role vis- lic sector is one of the largest in the world. The way à-vis politicians as guardians of the public interest. that the government decides on policies and then (This too has its parallels with Latin America.) makes sure that, once implemented, these do the job efficiently and effectively becomes all the more 6.4 Third, senior public servants constitute a techno- important: it is by now widely realized that policy cratic elite ­ a "mandarinate". The arrangements evaluation in the public sector plays the same role that provide the bridge between this technocracy as the market in the private sector. and elected politicians ­ essentially the 600 politi- cal appointments (largely of technocrats) to the top posts below the rank of minister ­ create a blurred Politics and Public Management dividing line between politicians and technocrats. Indeed, there is a considerable integration of the 6.2 Among the national political characteristics that public-service elite and politicians (to the point influence the particular policy process in France where many politicians are recruited from this are the following. First, the President and the elite).50 Prime Minister provide the focus of political policy-making. (This is the equivalent to the role 6.5 Table 6-1 provides a summary "map" of the prin- of the cabinet in Westminster-system countries.49) cipal institutions involved in the policy process France's political system has been characterized as inside the French government. a unique one where President and Parliament share power. On those rare occasions of "cohabitation" when President and the parliamentary majority The Policy Process in the Prime Minister's Office are of a different political hue, the President and the Prime Minister (elected by the National As- 6.6 The Prime Minister plays a central role in policy- sembly) share power. But in the more normal case, making, coordinating the different parts of the the President has control of both Assembly and government, and maintaining a centrally-defined Prime Minister and is therefore the political prime line of policy. His political arm is his cabinet (in mover. But it is the Prime Minister's office that has the French usage of the word, his political staff ­ control over the formal instruments of government all political appointments) and his administrative coordination, hence policy-making (something like arm is the Secrétariat Général du Gouvernement a ministry of the presidency in a Latin-American (mostly civil servants). Neither arm is large: they country). coordinate rather than get significantly involved in the technicalities of details of policy-making. The 6.3 Second, France follows the Napoleonic tradition of two arms work together, but to an extent also work rule-of-law. A somewhat rigid set of rules central- in parallel. (This parallel structure is repeated in all izes political power, protects the structure and role the ministries.) of the state (thus making changes more difficult), 49 Correspondingly, the Council of Ministers (Conseil des Ministres), over which the President presides, is more an instrument of coordination, rather than one 50 Here, perhaps, the parallel with Latin America ends, though Brazil has some of collective decision-making. similarities. Table 6-1. The Policy Cycle and Public Policy Instruments in France Politics & policy: Budget: Technical: Technical: The policy cycle at the center at the center at the center in the departments Conseil économique Centre d'Analyse Stratégique Policy identification et social Conseil d'Analyse Ministries and Ad-hoc Commissions Economique Inspectorates: project Other Councils cost-benefit evaluations; Prime Minister: cabinet* policy assessments and Secrétariat Général Conseil National Policy decisions du Gouvernement de l'Evaluation Ad-hoc Commissions Commissions Parlementaires Ad-hoc Commissions Révision Générale des Program implementation Politiques Publiques Conseil National Ministry Inspectorates: Program evaluation de l'Evaluation project and policy Cour des Comptes evaluations * Cabinet in the French sense: a private political staff. 6.7 As for the more substantive side of policy-making, researchers whose work is organized in five areas: the Prime Minister draws on the expertise and institutions and society; social questions; economic proposals of a number of bodies, some under his and financial affairs; labor, employment, and train- direct control. ing; and technologies. It also creates commissions or committees that prepare reports on various 6.8 In 1948 France created a Planning Commission issues, evaluating current and possible policies. To (Commissariat Général du Plan), answerable to take an example, it recently published an important the Prime Minister. The Commission was, for some report on energy policies. time, a leading practitioner of indicative planning. The five-year plans that were issued were not bind- 6.10 The Center publishes its works in the form of ing, not even for the government. They were the written documents (points of view, notes, re- support, if not the pretext, for policy-making and ports). It draws up an annual report which sums evaluation and consensus building. Over the course up the work carried out on the main strategic is- of time, the prestige and influence of the Planning sues facing French public policy. The Center also Commission declined. In 2005, it was abolished, conducts seminars and study days53. Its research but immediately replaced by a Center for Strategic includes quantitative, qualitative and impact Analysis (Centre d'Analyse Stratégique) that fulfils evaluations. similar functions.51 6.11 The Council for Economic Analysis (Conseil 6.9 The Center assesses, at the request of the Prime d'Analyse Économique) was created in 1997.54 Its Minister, the principal proposed government mission is "to illuminate, by confronting different reforms. Its three main functions are: monitoring, points of view and analyses, the government's eco- providing expertise and assisting decision-makers nomic-policy choices". It has about 30 members. in implementing and carrying out public policies. All prominent economists, they are chosen, on a It works directly under the direction of the Prime non-partisan basis, by the Prime Minister for their Minister52, but its advisory councils and bodies competence. The chairman, who is appointed by the cooperate with line ministries and within the net- Prime Minister, selects policy issues and commis- works of national and European policy-research sions reports from members or non-members. The organizations. Research is performed both by Council reads and comments on draft reports, and the staff and by external researchers funded by also designates a referee. Final drafts are presented the organization. Its staff consists of experts and to the Prime Minister and published, together with 51 See http://www.strategie.gouv.fr/. 52 See organization chart at http://www.strategie.gouv.fr/IMG/pdf/Organi- 53 See CSA Disclosure: http://www.strategie.gouv.fr/article.php3?id_article=167 gramme_CAS_septembre_2008.pdf 54 See www.cae.gouv.fr 60 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA the referee's comments. They are published under ation Council (Conseil National de l'Evaluation) the name of their authors and do not commit the which functions in a very similar fashion.55 It is also Council, though they bear the imprimatur of the housed at the Planning Commission, has no full- Council. Council members and report authors are time staff, responds to the Prime Minister's office, not paid. and creates committees to investigate policies at the request of various ministries. It is discrete and 6.12 The council has been very active. In its ten years, has not achieved visibility and authority. it has produced 75 book-length reports. These include such themes as the sub-prime crisis, 6.16 The above policy-oriented bodies under the Prime competition policy, employment of older people, Minister tend to dwell on the "big-picture" policy and the international financial system. Many of issues, rather than going to the detail necessary the reports look at past policies and discuss future for implementing specific programs. These bodies policies. These reports have generally been well provide a mix of expert advice, partisan advice, and received, widely quoted in the media and read in consensus-building. Being at the heart of politics, the ministries concerned. Because they are writ- they also tend not to have permanent lives. ten by a single author (or at most three), they are clearer, franker, and more readable than reports produced by a Commission (whose findings must The Economic Social and Environmental Council to an extent reflect diverse, possibly conflicting views). Because they are written by recognized 6.17 The Economic and Social Council ­ now retitled and experienced specialists, they have a gravity the Economic, Social and Environmental Council that an evaluation report prepared by a rapporteur (Conseil Economique, Social et Environnemental) who is only specialized in evaluation might lack. ­ is a constitutionally mandated and independent The seal of the Council gives the reports added consultative body.56 Its somewhat corporatist ob- credibility and authority in the eyes of the media jective is to promote a collaborative approach of and the public. different economic interests to economic and social policy. About two thirds of the 231 Council mem- 6.13 Other policy-advisory bodies under the Prime bers are designated by professional organizations, Minister have perhaps played a less prestigious cooperative associations, family associations, and role. Such bodies include: the Council for the the like. The rest are designated by the government Analysis of Society (Conseil d'Analyse de la So- to represent public enterprises, professional inter- ciété), drawing its 28 members from the world of ests from the overseas territories, and prominent research, artists and sports people; the Council for representatives of society. The work of the Council Retirement (Conseil d'Orientation des Retraites), is carried out through designated commissions, concerned with the viability of pension regimes; the whose work is open to the public and supported Employment Council (Conseil d'Orientation pour by designated experts. The commissions prepare l'Emploi), studying issues and making policy pro- reports or opinions, and the plenary of the Council posals in the areas of unemployment, professional votes on the opinions. training, and employment creation and staffed by labor experts and with 50 members representing different economic interests. The Budget and the Policy Process 6.14 In 1990, the government created an Interministe- 6.18 The budget is an important, if partial, expression of rial Committee for Evaluation (Comité Intermin- the government's policies under implementation. As istériel de l'Evaluation), assisted by a scientific in most countries, the French budget process has al- committee. This was housed in the Planning Com- ways been largely political, with the Prime Minister mission and, similarly, worked through ad-hoc acting as arbiter between the competing demands commissions created to evaluate particular policies. of the budget ministry and the spending ministries. Its interministerial character was supposed to In 2007, the government launched a public-expen- ensure that the selection and treatment of topics diture-review process, under the direct authority of went beyond the concerns of a single ministry, and its scientific committee was supposed to guarantee that appropriate and reliable methodologies would 55 See: http://www.evaluation.gouv.fr/cgp/. The Council is formally composed of 14 members, who are assigned for three years and are either representatives be applied. The Committee was not successful and of national and local bodies or chosen for their expertise in the field. Every was abolished in 1998. year this Council proposes the evaluation program for the coming year and provides for methodological assistance and training. It also assesses the quality of evaluation reports and makes them public (biblio: Atlas). 6.15 It was immediately replaced by a National Evalu- 56 See http://www.conseil-economique-et-social.fr/ 61 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA the President. Entitled the General Review of Public often at their own initiative. Such evaluations are Policies (Révision Générale des Politiques Publiques usually not made public. ­ RGPP), this is an effort to promote criteria of ef- ficiency and effectiveness more formally and thereby 6.23 Ministry Inspectorates are somewhat more in- to drive the process of public-sector modernization.57 dependent. Their functions include evaluation of The RGPP will underpin the first multi-year budget past policies, and occasionally assessment of future exercise (2009-2011). The main intention is to effect policies. They also function as organs of internal a long-lasting fiscal adjustment (and to return to a control. The importance, sophistication, and in- balanced budget by 2012). dependence of these various in-house evaluation bodies vary greatly from one ministry to another. 6.19 Government activities will be systematically They are particularly strong in the Ministry of analyzed through a list of questions: what is being Economy and Finance, and the Ministry of Equip- done? for what reason? should it be continued? if ment. But in recent years, ministries like Education so, by whom? and who will pay for it? how can it be or Social Affairs, where evaluation capabilities used done more cheaply? and how will the changes be to be low, have greatly improved their analytical made? External audit teams are working with the skills (for an example, see Box 6-1). In many cases, ministries to carry out the analysis. Reforms are be- the development of data bases and statistics related ing prepared by a staff of some 200 public and pri- to the policy area of each ministry has played a key vate auditors supervised by a committee co-chaired role in the development of evaluation practices.59 by the Secretary General of the President and the Director of the Cabinet of the Prime Minister and 6.24 In addition, the French government, like many including all ministers and the heads of the finance other governments, has often created ad-hoc com- commissions of both assemblies. All final decisions missions or committees to evaluate policies and are taken by the Council for Modernizing Public produce recommendations. These commissions Policies (Conseil de Modernisation des Politiques usually include a mix of senior civil servants, Publiques), chaired by the President.58 businessmen, trade union leaders, politicians (often including mayors), academics, and journalists. They 6.20 So far, the Council has approved 250 measures, pre- are always non-partisan and their reports are pub- dominantly in the areas of reorganization and process lished. Their recommendations are non-binding modernization. Thus, so far at least (and consistent for the government, but they carry some social and with spending reviews in other countries), the actual political weight and cannot be ignored completely. content of policies appears to be a secondary issue. Examples are: the commission on nuclear electric- ity in the 1970s which proposed the French civil nuclear program; and the 2007-08 commission, The Policy Process in the Ministries chaired by Jacques Attali, that evaluated the ob- stacles to growth and how they could be removed.60 6.21 As in other OECD countries, most of the process of designing, implementing, and evaluating poli- cies takes place at the level of the ministries and The Particular Case of Project Evaluation agencies associated with them. And in France as elsewhere, a number of bodies are involved in as- 6.25 Project appraisal (ex ante project evaluation) in sessing (ex ante) and evaluating (ex post) policies. France has a long history.61 France's traditional In many, not to say most, cases, analysis and studies technical strengths in public investments are mix the ex post and ex ante dimensions: they start closely associated with the role of two historical, with an evaluation of past policies and conclude and prestigious State Corps (civil-service Ca- with an evaluation of future policies. reers): the bridge and road engineers (Ingénieurs des Ponts et Chaussées) and mining engineers 6.22 Most French ministries have developed in-house policy capabilities. They take the form of planning departments and ministry Inspectorates. Planning 59 Although there can be statistics without evaluations, there cannot be evalu- departments routinely undertake studies of current ations without statistics. It is interesting to note the role played here by the powerful and highly qualified National Statistical Office (INSEE): the chief policies and of proposed, envisaged or planned statistical officer of each ministry is always seconded by the National Statistical policies, at the request of the minister, but also Office, to ensure quality and uniformity of statistics produced. 60 See: http://www.liberationdelacroissance.fr/files/rapports/rapportCLCF.pdf 61 This report pays particular attention to the case of project appraisal in France because this country carries it out substantially more systematically than other 57 See http://www.fonction-publique.gouv.fr/rubrique231.html and http://blog- OECD countries (and somewhat like Chile) and, by this token, French project pfm.imf.org/pfmblog/2008/05/frances-rvision.html evaluation represents the nearest thing that exists in the OECD to a system for 58 http://www.rgpp.modernisation.gouv.fr/ (ex-ante) policy assessment. 62 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 6-1. The General Inspectorate of Social Affairs (France) The General Inspectorate of Social Affairs (Inspection Générale des Affaires Sociales ­ IGAS) is generally charged, by law, with evaluating public policies implemented by the social services ministries. It monitors public or private bodies responsible for providing extended social protection in the widest sense of the word. It also caries out surveys which may enlighten the government as to the current situation and enables proposals to be made concerning changes in public activities. As a general inspectorate, IGAS enjoys a particular independence within the organizational chart of the social services ministries, being placed immediately next to the Minister. IGAS staff consist of around 80 senior civil servants belonging to a body specializing in evaluation and monitoring assignments, the General Inspectorate of Social Affairs which was given independent status, under the minister in each social-services ministry, in a decree published in 1990. Half of the senior staff are graduates of the Ecole Nationale d'Administration. IGAS generally reports only to the Government and the organizations being monitored. Nevertheless, certain in-depth investigations have had considerable public repercussions. For example, a 1994 study of the merging of the two organizations responsible for employment and employment compensation gave rise, in 1997, to the establishment of a "single window" for the compensation of job seekers who were previously obliged to carry out the same formalities with both organizations. Source: http://www.ilo.org/public/english/dialogue/ifpdial/la/gp/france3.htm Box 6-2. Breaking the Monopoly of Specialized Knowledge in France Should policy evaluations be conducted within the agencies responsible for a policy or an investment, or by institutions external to such agencies? It is true that, in some cases, knowledge and information on an area are concentrated in the agency in charge. France faced such a problem when it tried to evaluate rail track maintenance, a US$4.5 billion per year issue. It turned out that the only people knowledgeable about rail track maintenance in France were people from the French National Railway Company (SNCF), the former rail monopoly which is in charge of rail track maintenance on behalf of Réseau Ferré de France (RFF), the owner of the tracks. Their evalu- ation was bound to be highly favorable to SNCF practices. The solution found was to ask two highly respected professors from École Polytechnique Fédérale de Lausanne, in Switzerland, to undertake the evaluation. (Ingénieurs des Mines). The two Schools (Ecoles) able part of public infrastructure investments. The from which these engineers graduate have at- powerful Ministry of Equipment (now renamed tracted the best students in the country, and have Sustainable Development), a stronghold of In- enjoyed a virtual monopoly of top positions in the génieurs des Ponts, issues methodological direc- public agencies, state-owned enterprises, and pri- tives. These indicate the values of the parameters vate enterprises involved in infrastructure, energy (discount rates for lives, environmental factors, and industry. These Corps were responsible for and so on) to be used. Deciding on these values major decisions in transportation, urbanization, is essential but contentious, and from time to housing, energy and industry, at a time when time, various commissions are created to discuss elected politicians had less weight. The engineers or revise the value of some parameters. All is in were pioneers in developing evaluation techniques place therefore for a widespread use of project to help select investments in the public interest. evaluation in France. These techniques have subsequently been formal- ized and refined. 6.27 Yet, to all appearances the role of project evaluation has declined in France in the past two decades. A 6.26 Cost-benefit analysis is, in principle, required first reason for this is decentralization. Many proj- for all major public investment projects. This is ects that used to be decided by the central govern- particularly so for transportation projects, a siz- ment are now decided by regional or local govern- 63 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA ments. Project evaluation is legally compulsory and these ex post evaluations are to be made. For about widely practiced at the central level, but not at the two decades, this prescription was largely ignored, regional or local level. At this level, the technical and only a handful of evaluations were conducted. skills and/or the political will to undertake invest- Then the General Council for Bridges and Roads ment appraisals are often absent. (Conseil Général des Ponts et Chaussées), the In- spectorate of the Ministry of Equipment, decided 6.28 Second, there is a so-called "optimism bias" in that the law had to prevail, and started to under- many evaluations. Of course, there are great uncer- take these ex-post evaluations. Presently, four or five tainties about the future, but these do not explain evaluations per year are produced and published by the fact that costs are usually underestimated and the Ministry of Equipment. benefits overestimated. This "optimism bias" prob- ably occurs less in France than in other countries, 6.31 The evaluations have two important features. First, but it might be on the increase. In France, it is the they are produced by the Agencies in charge of the agency promoting the project that analyzes the project, under the surveillance of their inspectorate. cost and benefits. These agencies tend to enjoy a As with the ex ante appraisals, these Agencies are growing autonomy, they have their own agendas, tempted to embellish the record. The surveillance and they function like lobbies. They do not always of the inspectorate is effective and imposes a rea- resist the temptation to "massage" their evaluations. sonable degree of fairness, but a sharper distinction In principle, these evaluations are overseen by civil between those evaluating and those evaluated may servants, but in practice the asymmetry of informa- be desirable. Second, the evaluations are primarily tion is blatant. The problem is compounded by the aimed at finding out whether the ex ante appraisal growing complexity of megaprojects. (Project com- was well done: were cost estimates correct? Were plexity argues for a new, more adversarial evalua- traffic forecasts correct, and when not why? This tion process, as Box 6.2 suggests. This is consistent may improve appraisal practices, but it does not with strengthening the challenge function within address broader policy questions. the policy process.) 6.29 Third, decisions have become more political. Tech- Parliamentary Vetting of Policies nocrats no longer have the substantive monopoly over decisions, and elected politicians are less in- 6.32 The two chambers that make up the Parliament, clined to listen to the outcomes of project evalua- the Senate and the House of Deputies, have always tions. It is, of course, normal that elected politicians been, in principle, involved in evaluating policies have the final word on major public investments. embodied in draft laws. In practice, however, However, when evaluations are systematically their activities in this domain have for long been overruled by political considerations, they become modest. As in many other OECD countries, this useless. In 2002, a new government came to office. situation reflects the dominance of the executive in There were 23 major transport investment projects the law-making process and, correspondingly, the in the pipeline, considerably more than could be lack of appropriate skills and time on the part of financed. The new Prime Minister asked a group legislators. of six senior civil servants to appraise these proj- ects. Some projects, the experts showed, had high 6.33 But recent decades saw some attempts at change. internal rates of return, others much lower ones. At The members of both chambers want to be more in- the end of 2003, the Prime Minister convened an volved in policy formulation and to exercise greater inter-ministerial meeting at which all but one of control and influence on the executive. In 1982, they these 23 projects were approved. created a Parliamentary Office for the Evaluation of Scientific and Technological Choices (Office 6.30 The ex-post evaluation of big public investments Parlementaire d'Evaluation des Choix Scientifiques was mandated in 1982, but was not enforced till et Techniques), then in 1986 a Parliamentary Office more recently. Like ex-ante appraisal, these evalu- of Public Policy Evaluation (Office Parlementaire ations suffer partially from self-evaluation (though d'Evaluation des Politiques Publiques). These of- they are also supervised). The evaluations also only fices never took off, probably because they were ask whether the objective was effectively carried not sufficiently funded and/or because they failed out, not whether it was right. Things are beginning to attract staff of a sufficient caliber. France does to change in this respect, in part thanks to the law. not have a non-partisan, well-staffed institution at The same 1982 Law on Transport also prescribes ex the service of its Members of Parliament, like the post evaluation of large transport projects, though it Congressional Budget Office in the US. does not specify when (i.e. after how many years) 64 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 6.34 Special reports prepared by Parliamentary com- reports (on airport policy, or on ports policies, for missions on a large number of specific topics are instance) seem much better informed. becoming an increasingly important tool of policy assessment. Both chambers have permanent com- 6.37 The Court is well placed to make an important missions (Finance, Family and Social Affairs, etc.) contribution, based on its reputation for indepen- that examine draft laws, including the budget law. dence and seriousness and the quality of its staff. To this effect, the Commission often entrusts one On the other hand, the Court seems keen to evalu- or several of its members to prepare a report on ate any policy ­ from rail policy to cancer policies a special topic. This can involve hearings. Senior to university library policies, to take the subjects Senate or House staff members prepare these of recent reports ­ and it is not clear that it can reports, under the guidance of the commission muster the expertise needed to evaluate in depth member. The parliamentary reports, which are such diverse topics. published, usually examine, in detail and with a critical eye, past and present policies and discuss 6.38 The Court conducts ex-post procedure and man- policy options for the future. The views expressed agement audits, and it also deals with cost-effec- are not politically oriented or biased: their authors tiveness analysis. Its mission is to ensure a proper know that this is a condition for the report to be use of public funds. It also controls the manage- used by their fellow members of Parliament. ment of all public agencies as well as all the public or semi-public entities and some private entities benefiting from public transfers. It also advises The Court of Accounts both the Parliament and the Government on issues related to the implementation of budget and social 6.35 For 200 years, the Court of Accounts (Cour des security laws. Audits are performed by magistrates Comptes) has been auditing the accounts of the and staff from the Court. The list of publications of public administration, including those of local the Court of Auditors is available on its website.63 governments, and of public enterprises. It is an important and respected institution. Its yearly report on public accounts is widely commented Alternative Policy-Advice Organizations by the media and feared by administrators. Well staffed and attracting some of the brightest prod- 6.39 In many countries, particularly in the US, but also ucts of the French academic system, the Court is the UK and Germany, a lot of policy assessment a quasi-judicial body that is entirely independent and evaluation is carried out by think tanks and from the government: its members have the status academics. There is very little of that in France. The of magistrate with life-time tenure. A significant only two properly funded think tanks are not in number of them are seconded to ministries or the policy area, but in the business of short-term public enterprises in high-level positions or go macro-economic forecasting (Office Français de into politics, but they can always come back to the Conjoncture Economique, funded by the govern- Court when they want.62 ment, and COE-REXECODE, a research center funded by business). Other institutions with the 6.36 For a long time, the Court saw itself as an accoun- vocation to be think tanks are under-funded tant and guardian of legality, not as an evaluator. and usually commission outside experts to write In recent years, the Court has been entering the reports. The picture is not much better with uni- field of policy evaluation. But it did this with versities. Most academics are, and want to remain, prudence and moderation, not sure whether quite distant from the real world of policy, and few it was exceeding its mandate. It also lacked the are involved in actual policy evaluations. Things necessary skills. Some of its first evaluation-type might be beginning to change. For instance, the reports were rather weak. In a report of the late Toulouse School of Economics and the Paris 1990s, for instance, the Court expressed concern School of Economics are much closer to the world about whether the highway companies, still at that of business and of public policy and are beginning time largely public, could ever repay their debts. to contribute to the policy process. Six years later, these highway companies were privatized, and the market valued them ­ with their debt ­ at about US$30 billion. More recent 63 See: http://www.ccomptes.fr It can also be purchased in bookstores, all reports being published by La Documentation Française. Some reports are published 62 See: http://www.ccomptes.fr/fr/CC/documents/div- by the Parliament as an appendix to the Reports of committees that have re- ers/OrganigrammeQP19Senat2007v2.pdf quested to the Court, and they are available on the website of the Assemblies. 65 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Concluding Comments of changing the ways the agencies went about their business have been to undermine the policy 6.40 The arrangements for collective decision-making process. about policies at the center of government are limited. This may be typical in regimes that are 6.44 Nonetheless, this review suggests other challenges presidential, but without a strongly independent to the policy process in France. There is a sense legislature. In any case, government agencies are that politics are undermining the technocratic ap- substantially separate entities with substantial proach. This is happening in the traditional areas of discretion ­ within broad policy directions set and technocratic strengths, notably in the application coordinated by the center ­ to make their own of cost-benefit analysis to infrastructure projects. policies. The center makes little effort to control But it may also be the case that as public services the policy process (including evaluation practices) move increasingly into other, technically less ame- in the ministries. Instead it is the function of the nable areas such as social services and as public- inspectorates to oversee, ministry by ministry, the service provision generally becomes more complex, policy process. this challenge of retaining technical criteria in the policy process will rise. 6.41 The center of government does somewhat invest in "big-picture" policy futures. This prospective 6.45 If so, France may need to make its policy process work is not done, as it is partly in the UK, through more amenable to handling complexity, as well as in-house expertise, but through consultative the increasing demands for public consultation. bodies and contracting known experts. The main In this respect, France might look more to the intent may be to build political consensus ­ a US model (and, to an increasing degree, the UK), continuation, to an extent, of the tradition of in- where the multitude of alternative policy-advice dicative planning. Some of the consensus-building organizations has moved the government away has a corporatist tinge (in the Economic and Social from the traditional policy-process model of a Council notably); some is partisan (in some ad-hoc "monopoly of advice" and towards the newer model Commissions); and some is addressed more to of a "market for advice". public-opinion formation than to the ministries. The Council for Economic Analysis is a notable success, the result of the high quality of its experts, the readability of papers, and the political standing of the Council. The Center for Strategic Analysis provides an example of partially in-house policy work that can be compared to the UK's Strategy Unit. 6.42 The technocracy, constituted by elite Corps of Se- nior Civil Servants, a dominant players in a series of "policy networks". These networks constitute a system of communication and a community of practice. And they provide something of an alter- native to the more political networks of ministers and their (political) cabinets. The technocratic net- works have promoted various technical approaches: formerly indicative planning, now cost-benefit analysis and policy assessment and evaluation. This policy work is partly done in private (i.e. not addressed to the public); it sometimes tends to be compromised by defensiveness (permitted, notably, by self-evaluation); and it may not be strongly ad- dressed to learning from mistakes. 6.43 In all, there is less of a sense of crisis in policy- making than in the Westminster-system countries. This may be because France's political system has not permitted the same rapid ("shoot-from-the- hip") reforms, where the unintended consequences 66 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 67 Annex 7: Spain The Policy-Making Framework the third one deals mostly with territorial issues), is in charge of providing political and technical advice 7.1 Spain is a Constitutional Monarchy, with a bi- to the President, coordinates the agenda of the cameral parliament (the Cortes Generales). The Council of Ministers, and presides over the meet- executive branch consists of a Council of Ministers ing of secretaries and undersecretaries in which the and is run by a President selected by the National matters to be taken to the Council are decided. The Assembly following legislative elections. Ministry also functions as a liaison with Congress. 7.2 Spain is one of the most decentralized countries in 7.7 The Economic Office of the President (Oficina Europe, formed by subnational entities (Comuni- Económica del Presidente) provides the President dades Autónomas) with their own authorities and with information on the economic situation and Parliament, and endowed with resources. related advice on policy proposals. Among periodic reports on Spain's economic situation, the Office 7.3 The President is the equivalent of the Prime also works on the Government's Economic De- Minister in other parliamentary systems. He has velopment Strategy and the President Economic significant faculties that contribute to make him Report. the major policy maker: he selects the Presidents of the House of Representatives and Senate, and 7.8 Lastly, Spain's Government has also an advisory in- proposes nomination of heads of important state stitution in the State Council (Consejo de Estado), institutions. In relation to the Legislative, the Pres- created in the XVI century. It provides advice on ident can propose the dissolution of Parliament, general policies to the President, Ministers, and the and request a parliamentary vote of confidence. Presidents of the Autonomous Communities, with emphasis on compliance with legality. The advice 7.4 The Council of Ministers (equivalent to cabinets in of the State Council is not mandatory, except when Westminster-system countries) is the other main a particular law indicates. The State Council has a body involved in policy-making at the center of variety of members that provide advice on differ- Government. Its main function is to formulate ent matters, such as health, credit entities, public and approve national policies over the whole area teachers' career, regional budget structures, and of the ministries, especially preparing draft bills private insurance, among others. However, since which have normally been drawn up in ministries the Council's advice is centered on legality and or departmental committees, and are then sent to administrative issues of proposed new legislation, Parliament. its role on policy-making is mostly a formal one. 7.5 To support policy formulation at the center of gov- 7.9 Regarding the different areas of the Executive, ernment, the President and Council of Ministers in practice, the real capabilities of intervention count on a series of entities that provide different from different areas of government depend more levels of technical advice. on elements such as budget availability, capacity for political mobilization, access to information, 7.6 The Ministry of the Presidency, run by the First or technical resources available, than on formal Vice-President (there are two other Vice-Presidents, attributions granted by the normative framework. the second one being the Finance Minister whereas The cooperation and coordination dynamics end up being more the result of complex processes of 7.14 The other concern that has brought policy evalu- interaction and bargaining within the Government ation to the center, involves the particular set of (Comisión para el estudio y evaluación de la cre- policies related to regulation. This growing concern ación de la AEVAL 2004). is shared among EU countries and seems to be a defining factor in the demand for a better quality 7.10 Within the ministries, an important position in re- of ex-ante policy evaluation in order to implement lation to specific area policies is the General Tech- better regulations, and ex-post analysis to assess the nical Secretariat (Secretario General Técnico), in cost-benefit relations of these. charge of drafting the Ministry's general plans and programs, providing technical and administrative 7.15 This growing need for a more ambitious kind of assistance (including statistics), and recommend- evaluation has produced a situation in which more ing reforms to improve services and organizational traditional expenditure and management perfor- reforms. Although the Minister has his or her own mance reviews are contrasted with more innovative cabinet of political appointments, the technical approaches. In some cases these new approaches secretariat is quite likely to have a heavier weight develop as new assignments for existing agencies, in ministerial decisions. However, within the or as is the case of the creation of the Evaluation ministries an explicit source for policy formulation and Quality Agency, involve the development of advice does not appear to exist. new institutions. 7.16 Table 7-1 provides an overview of instruments Policy and Program Evaluation in Spain in Spain's policy process. Currently, Spain's most important program and policy evaluation entities 7.11 Since the late 1980s the Government of Spain has include the following. implemented different reforms to improve public management and to develop evaluation frame- 7.17 The Court of Accounts: this agency answers works. The management improvement initiatives to Parliament, and has the function of audit- have involved Total Quality implementation and, ing financial management through the Central more recently, two standardized management as- Government for legality, economy, and efficiency sessments frameworks: European Foundation for and overseeing compliance with accounting Quality Management (EFQM) and Common standards. It also coordinates auditing functions Assessment Framework (CAF), in line with other in autonomous municipalities by defining general European Union (EU) countries initiatives. criteria and standards. Its twelve members ­ Ac- count Counselors, designated by the Congress 7.12 From an evaluation perspective, Spain has made and the Parliament ­ have the same independence various attempts focused mostly on program evalu- and stability as the judges. They are elected for a ation, and specifically on financial management nine-year period and their profiles include: law- and civil service performance. Even though policy yers, court of account auditors, juries of accounts, evaluations have been commissioned in the central academics and public officials, among others. Rec- government and agencies, these evaluations have ognized professional experience of no less than 15 been mostly ad-hoc, without a general framework years is required to be elected. The President of the or regularity. In many cases the notions of policy Court of Account is appointed by the King for a and program evaluation has overlapped when the period of three years. mission of units in charge of evaluation has been defined, generally leaning towards more focused 7.18 State Administration General Intervention (Inter- program assessment. vención General de la Administración del Estado ­ IGAE): this agency was created in 1988 under 7.13 However, there have lately been at least two initia- the Ministry of Economics and Finance with the tives that show government interest in expanding objective of "ending with the existing incremental the scope of evaluation to include policies, and both routine for budget formulation throughout the pub- are related. The first is the creation of the Evalu- lic sector" (Comisión para el estudio y evaluación ation and Quality Agency (AEVAL) which spe- de la creación de la AEVAL, 2004). It performs cifically has policy evaluation among its objectives, program evaluation with emphasis on effective- while at the same time it explicitly removes itself ness and efficiency from a financial point of view. from processes or other kinds of micro evaluation However, the recently sanctioned General Budget of agencies. This agency also favors a non-dogmatic Law widened IGAE's scope of action, which had methodology, with multiple approaches according traditionally been associated mostly with financial to the specific needs of each particular evaluation. aspects, to supervising the fulfillment of program 68 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Table 7-1. The Policy Cycle and Public Policy Instruments in Spain Politics & policy: Budget: Technical: Technical: The policy cycle at the center at the center at the center in the departments Policy identification Sectorial Commissions: - ANECA and INECSE (education) Ministry of the Presidency - Agencia de Calidad The President's del Sistema Nacional Economic Office de Salud (health) President and Council of Ministers Ministry of the Economy State Council (legality and - Sistema Nacional de Policy decisions Empleo (labour) and Finance administrative advice) Ministry of the Presidency - Subdirección General de Planificación y Eval. de Políticas de Desarrollo (development and integration) Program implementation Agencies staff State Administration General Intervention Agency of Evaluation Program evaluation Ad ­ hoc evaluations and Quality Budget Directorate objectives and management performance reports. dards for public health institutions mandated by This puts IGAE more in line with the expenditure law and monitoring its evaluation. The National reviews approach used in countries like the UK. Employment System on the other hand is tasked with the evaluation of labor policy in Spain. Other 7.19 The Budget Directorate: is involved with the detailed examples of commissions in specific policy areas monitoring of budget execution and evaluation. are the National Institute for the Evaluation and It has also the task of strengthening the objec- Quality of the Education System (INECSE) and tives­based budget formulation mandated by the the National Agency for Quality Evaluation and General Budget Law, in order to support budget Accreditation (ANECA) for Education. relocation decisions. Lastly, the Budget Director- ate also performs an ex-ante financial cost-benefit 7.22 Inspection, Evaluation and Public Service Quality assessment of proposed legislation. Directorate (Dirección General de Inspección, Evalu- ación y Calidad de los Servicios Públicos ­ IEPSQD): 7.20 Observatory for the Quality of Public Services: this is IEPSQD was mapped the former Ministry of a not-for-profit association established by several Public Administrations and was dissolved. Its staff public managers. It has among its objectives the were transferred to AEVAL. During its existence, periodic analysis of the quality of public services, as among its most relevant functions were to formu- well as the coordination of information and citizen late, develop, and monitor quality-improvement participation. It is responsible for informing peri- programs throughout the public sector. It applied odically on the quality of the provision of public methodologies, certified, and incorporated optimal services, for which it presents and disseminates an public management practices. It also developed Annual Report on the Quality of Public Services. specific policy or institutional evaluations for any agency that demanded these. These evaluations 7.21 Sector Commissions: Ministries in specific areas of included outcome and impact assessments, con- policy are supported by different commissions, sistency with the objectives set for a program or which provide technical assessments of policies agency and public service performance. and programs. The activities of these commis- sions range from very specific tasks mandated by 7.23 These evaluations later evolved, starting with evalua- law to more open policy-evaluation processes. For tions focused on efficiency, followed by senior-man- example, the National Health System Quality agement improvement and performance-evaluation Agency (Agencia de Calidad del Sistema Nacional programs, and then citizen involvement. IEPSQD de Salud) is responsible for issuing quality stan- focused on policy evaluation, assessment of the im- 69 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA pact of public regulations, and the quality of public To promote increased rationality in the use services. The 2004 Commission that studied the of public resources. creation of the new policy evaluation agency (AE- VAL) described the IEPSQD as a basic platform for To favor economic productivity and com- developing the activities of the new agency. petitiveness by reducing excessive levels of government bureaucracy. Recent Reforms to Improve Public Policy 7.26 AEVAL is linked to the First Vice-President and Evaluation: the Agency for Public Policy Minister of the Presidency. The governing body of Evaluation and Service Quality (AEVAL)64 AEVAL is formed by a Governing Council (Consejo Rector) with representatives from the Ministries of 7.24 The creation of AEVAL in 2007 was part of a wider Economics and Finance (Budget's Office Secretary set of reforms to modernize public management in General), Foreign Affairs, the Presidency, and from Spain. It was created within the framework of the representatives of unions, and recognized experts Agencies Law passed in 2006 which established in specific policy areas. The agency is structured "agencies" as new models of public organization, in three departments, of which the Department with greater flexibility in organization and man- of Evaluation is in charge of performing program agement, similar to other OECD countries. The and policy evaluations.66 Neither the Ministry of creation of the agency was included in the current Finance nor the Budget Directorate have a role in administration's campaign program in 2004, and the evaluation of programs and policies, however, was publicly supported afterwards in speeches from they can request evaluations from AEVAL. the President and the Ministers of Economics and Finance and Public Administrations. Initially, AE- 7.27 For its evaluations AEVAL has defined very flex- VAL's evaluation universe will be the municipalities ible criteria. There is no structured framework for under the common regime. However, it is foreseen the evaluations regarding methodology or scope of that it will establish evaluation agreements with evaluation. In fact, the agency has explicitly stated autonomous communities. In addition, it should be its intent of not using doctrinaire methodologi- noticed that the Ministry of Public Administrations cal approaches. It will search for quantitative and (Ministerio de Administraciones Públicas) does no qualitative tools from the social sciences, using a longer exist. It was eliminated in 2009 and its func- pragmatic approach. In this way the methodology tions were distributed between a new ministry of employed will acknowledge the underlying com- Territorial Policy (Política Territorial) for matters plexities of policy evaluation. pertaining to cooperation with local governments and Comunidades Autónomas, and the Ministry 7.28 The policies and programs to be evaluated and the of the Presidency for matters pertaining to public scope of the evaluation are mainly defined by the administration and the Función Pública. Agency's Governing Council, based on a perfor- mance contract signed every year with the First 7.25 According to AEVAL´s website, its objectives are:65 Vice-Presidency, the Ministry of the Presidency, and the Ministry of Economics and Finance. The To promote a culture of evaluation and specific terms of the evaluations are coordinated service quality within the public sector. with the corresponding ministry or public entity, with regard to initial work plans, timeframe, pro- To develop and propose methodologies for cedures, the recipients of the evaluation, etc. The the implementation of quality indicators agency conducted nine policy assessments in 2007 and standards for public services. and four in 2008. These evaluations are published online at AEVAL's website.67 To assess agencies´ activities related to their commitment to improve public services and 7.29 In all cases, the evaluations are performed by the report annually to Congress on this. AEVAL staff, relying on occasional outside techni- cal support when needed. Lastly, while the AEVAL To improve the transparency of the impact regulations allow for ex-ante evaluations, until now of public services on society and increase the it has focused on ex-post evaluations and monitor- accountability of public servants to citizens. ing. AEVAL does not perform auditing. 66 See http://www.aeval.es/es/la_agencia/organigrama/ 64 See http://www.aeval.es/es/index.html 67 http://www.aeval.es/es/evaluacion_de_politicas_publicas/evaluaciones_de_ 65 See: www.aeval.es la_agencia/ 70 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Box 7-1. Evaluations undertaken by Spain's Quality and Evaluation Agency in 2007 Effect of measures taken to rationalize pharmaceutical spending (evaluation of the Strategic Plan for Pharma- ceutical Policy designed to contain the increase of drug spending growth). Efficiency of energy security policies (evaluation of reforms to secure energy supply through improving coordi- nation, reforming regulated tariffs, and improving infrastructure). Programmes to foster research, development and innovation ­ Ingenio 2010 (evaluation of this program designed to reduce the technological gap between Spain and the rest of Europe). Financial facilities to foster entrepreneurial activities (evaluation of government subsidies to new business initia- tives). Concluding Remarks istration and constitutes the first entity under the new management approach of the Agencies Law 7.30 Spain's policy-making process is characterized by of 2006. the central role of the President. However, many different actors are involved to different degrees, 7.34 The main risks for AEVAL are probably related to and the larger ministries have significant autonomy its capacity to become a relevant source of policy to propose policies for specific area. The existence assessment information, which depends in part of of autonomous communes with significant in- the technical quality of its evaluations, but also on dependence to govern themselves adds to this the degree in which the agency can establish its complexity. place within the policy-making process. Addition- ally, this more open approach to evaluation will 7.31 Although Spain's Government seems to be quite be tested in a public sector generally used to more structured, with different layers that mix traditional structured financial evaluations and legality audits. and more modern institutions, it has since the 1980s undergone a series of reform initiatives to 7.35 Until now AEVAL has focused on evaluating improve public management. These reforms seem specific programs or policies (defined as a series of to have also created a series of layers that influence interrelated programs). This is different from the policy capacity and evaluation, the newest impulse conceptualization of policies used in this docu- being the Agencies Law ­ a new model for public ment, which sees policy as a set of intentions which management similar to that widely used in other are implemented through programs. Also, although countries like New Zealand ­ and the creation of AEVAL´s flexible approach allows it to perform AEVAL within this framework. ex-ante evaluations, the focus until now has been on evaluations during implementation. 7.32 In this sense, special attention should be placed on Spain's AEVAL, given that in contrast to the 7.36 Only time will tell the extent to which this agency other country experiences which rely on existing is actually institutionalized within Spain's complex agencies, this is a new agency explicitly created policy-making process, and accomplishes its goal to implement a new view of program and policy to further spread an evaluation culture within the evaluation in Spain. In an attempt to move away different government structures. It is an effort that from more structured approaches, this new agency merits watching. is characterized by a very flexible view of policy and program evaluation. This constitutes an innova- tive approach to evaluation in Spain, in line with a tendency of other OECD countries to expand the scope of evaluations to include more complex policy matters. 7.33 This perspective is given all the more emphasis since AEVAL was created on the basis of the po- litical campaign statements of the current admin- 71 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 72 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA Bibliography Adolino, Jessica R. and Charles H. Blake. 2001. Comparing Public Policies: Issues and Choices in Six Industrialized Coun- tries, Washington, D.C.: CQ Press Agencia Estatal de Evaluación de las Políticas Públicas y Calidad de los Servicios (2007). Informe General de Actividad 2007. Ministerio de Administraciones Públicas, Gobierno de España. Allison, Graham, and Philip Zelikow (1999), Essence of Decision: Explaining the Cuban Missile Crisis, Second Edition, New York: Longman Althaus, Catherine; Bridgman, Peter, and Davis, Glyn (2007). The Australian Policy Handbook, 4th edition, Sydney: Allen & Unwin Anderson, Barry (2009), "The Changing Role of Parliament in the Budget Process", OECD Journal on Budgeting, Vo- lume 2009/1 Apfel, Kenneth (2010), Policy-Making Capacities in the USA: Relevant to Latin America?, Powerpoint presentation at the Seminar on "Strengthening Policy-Making Capacity in Latin America: What can be learned from the OECD?", June 1-2, 2010, World Bank, Washington, D.C. Arizti, Pedro, James Brumby, Nick Manning, Roby Senderowitsch and Theo Thomas (2010), Results, Performance Bud- geting and Trust in Government, Public Sector and Governance Unit for Latin America and the Caribbean, World Bank, Washington, DC. Beschel, Robert P. Jr., and Nicholas Manning (2001). "Central Mechanisms For Policy Formulation and Coordination", Chapter 2 of Salvatore Schiavo-Campo and Pachampet Sundaram, eds., To Serve and Preserve: Improving Public Administration in a Competitive World, Asian Development Bank, Manila Blind, P.K. (2006), Building Trust in Government in the Twenty-First Century: Review of Literature and Emerging Issues. Paper presented at the 7th Global Forum of Reinventing Government: Building Trust in Government, June 2007. Vienna, Austria. http://unpan1.un.org/intradoc/groups/public/documents/UN/UNPAN025062.pdf Bouckaert G., G. Peters and K. Verhoest (2009). Policy Coordination in OECD Countries 1980-2000, Basingstoke; Pal- grave. Bridgman, Peter & Davis, Glyn (1998). The Australian Policy Handbook, Sydney: Allen & Unwin Brudnick, I. (2008). The Congressional Research Service and the American Legislative Process, CRS Report for Congress. Buchs, M (2007). New Governance in European Social Policy. The Open Method of Coordination. Palgrave-Macmillan Cabinet Office, Strategic Policy Making Team (UK) (1999), Professional Policy Making for the Twenty First Century, London: Cabinet Office, September Centre of Excellence for Evaluation (2004). Policy Evaluation System for the Government of Canada. Treasury Board Secretariat of Canada. Comisión para el Estudio y Creación Agencia Estatal de Evaluación de la Calidad de los Servicios y de las Políticas THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 73 Publicas (2004), Informe, Ministerio de Administraciones Públicas. http://www.aeval.es/comun/pdf/Informe_co- mision_expertos_esp.pdf Dantzer, Ruth (2009), Management Accountability Framework (MAF) and the Way Forward, Powerpoint presentation at the seminar "Improving Public Service Performance in the OECS in Times of Crisis", Treasury Board of Canada, http://siteresources.worldbank.org/EXTLACREGTOPPUBSECGOV/Resources/Day2-Panel1-Ruth-Dantzer. ppt Davies, Philip (2004), Policy Evaluation in the United Kingdom, Government Chief Social Researcher's Office, Prime Minister's Strategy Unit, UK Cabinet Office, Paper to be presented at the KDI International Policy Evaluation Fo- rum, Seoul, Korea 19-21 May 2004. http://www.nationalschool.gov.uk/policyhub/docs/policy_evaluation_uk.pdf Evans, Gord, and Nick Manning (2003), Helping Governments Keep Their Promises: Making Ministers and Governments More Reliable Through Improved Policy Management, The World Bank, South Asia Region-Internal Discussion Pa- per, Report No. IDP-187, April Fischer, Frank, Gerald Miller, Mara S. Sidney (2007), Handbook of Public Policy Analysis: Theory, Politics, and Methods, CRC Press. Garde Roca, Juan Antonio (2007). La Agencia de Evaluación y Calidad de la Administración Española: Un Reto Institu- cional en el Espacio Público, XII Congreso Internacional del CLAD sobre Reforma del Estado y Administración Pública, Sto Domingo, República Dominicana, 30 otc. Geva-May, I. (2005). Thinking like a Policy Analyst. Policy Analysis as Clinical Profession, Palgrave Macmillan. Hoefer, R. (1994). "Corporatism, Pluralism and Swedish Interest Group Influence in Social Welfare Policymaking", Governance 7, 165-181. House of Commons,Public Administration Select Committee (2007),Governing the Future,Second Report of Session 2006­ 07,Volume I.Report,together with formal minutes,Ordered by The House of Commons to be printed 22 February 2007, HC 123-I,6 March,London:The Stationery Office Limited http://www.la-moncloa.es/Espana/EspaniaHoy/default.htm http://www.notre-europe.eu/uploads/tx_publication/Semi22-en.pdf Kraan, Dirk-Jan (2009). Programme Budgeting in OECD Countries. OECD. Manning, Nick et. al. (1999). Strategic Decisionmaking in Cabinet Government ­ Institutional Underpinnings and Obstacles. Sector Studies Series, The World Bank, October, No. 19935 Manning, Nick, and Geoffrey Shepherd (2009), "Reforma de la Gestión Pública: ¿Qué Debe Aprender América Latina de la OCDE?", Revista del CLAD Reforma y Democracia, No. 44, Junio McGann, James (2004), Scholars, Dollars and Policy Advice, www.fpri.org McGann, James, and Stephen Boucher (2004), Think Tanks in Europe and US: Converging or Diverging?, report of Seminar organized in Paris, December 13, 2004 Moran, Michael, Martin Rein, Robert E. Goodin, Eds.(2006), The Oxford Handbook of Public Policy, Oxford University Press 74 THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA National Audit Office (2001), Modern Policy Making: Ensuring Policies Deliver Value for Money, Report by the Comptro- ller and Auditor General, HC 209 Session 2001-2002, 1 November Newton, Michael T., and P. J. Donaghy (1997), Institutions of Modern Spain: A Political and Economic Guide, Cambridge University Press OECD (2003), Engaging Citizens Online for Better Policy-making, Policy Brief Ongaro, E. (2008), "Introduction: the Reform of Public Management in France, Greece, Italy, Portugal and Spain", International Journal of Public Sector Management, Volume 21 Issue 2, Pages 101-117. Page, E. C. and B. Jenkins (2006). Policy Bureaucracy, Oxford: Oxford University Press. Painter, Martin, and Jon Pierre, eds. (2005), Challenges to State Policy Capacity: Global Trends and Comparative Perspecti- ves, Palgrave Macmillan Peters, B. Guy (1996), The Policy Capacity of Government, Canadian Center for Management Development Research Paper 18 ( June), http://www.csps-efpc.gc.ca/Research/publications/pdfs/p62e.pdf Porter, R. B. (1980). Presidential Decision-Making: The Economic Policy Board, Cambridge: Cambridge University Press. Prabjakar, R. (2006). Rethinking Public Services. Government Beyond the Centre, Palgrave-Macmillan. Rokkan, S. (1967) "Norway: Votes Count but Resources Decide", in R. A. Dahl, ed., Political Oppositions in Western Democracies, Yale University Press. Rose, Richard (2006), "Giving Direction to Government in Comparative Perspective, Chapter 3, pp. 72-102, of Joel D. Aberbach, Mark A. Peterson, eds., The Executive Branch, Oxford University Press Schacter, Mark (1999), Cabinet Decision ­ Making in Canada: Lessons and Practices, Institute of Governance, Policy Brief Nº 1, April. Secretaría de Estado de Comunicación (2008). España Hoy, Ministerio de la Presidencia, Gobierno de España. Shaw, Richard, and Chris Eichbaum (2005), Public Policy in New Zealand: Institutions, Processes and Outcomes, Pearson Education New Zealand Sørenson, E. and J. Torfing (2007). Theories of Democratic Network Governing, Basingstoke; Palgrave. State Services Commission(1999), "State Services Commissioner's annual report on the State services", Government of New Zealand. Available at http://www.ssc.govt.nz/display/document.asp?NavID=82&DocID=5002 Stein, Ernesto, Mariano Tommasi, Koldo Echebarría, Eduardo Lora, Mark Payne, eds. (2005), The Politics of Policies: Economic and Social Progress in Latin America 2006 Report, Harvard University Press for the Interamerican Deve- lopment Bank. Stone, Diane (2001), "Guidance for Governance in Great Britain", Chapter 4 of Weaver and Stares (2001) THE PRACTICE OF POLICY-MAKING IN THE OECD: IDEAS FOR LATIN AMERICA 75 Stein, E. H., M. Tommasi, P. Spiller and C. Scartascini, eds. (2008). Policymaking in Latin America. How Politics Shapes Policies, Inter-American Development Bank, Chapter 5. Stilgoe, Jack, Alan Irwin, Kevin Jones (2006), The Received Wisdom: Opening up Expert Advice, London: Demos, http:// www.demos.co.uk/files/receivedwisdom.pdf Theodoulou, Stella S. (2001), Policy and Politics in Six Nations: A Comparative Perspective on Policy Making, Upper Saddle River, NJ: Prentice Hall Tiernan, Anne (2007), "Building Capacity for Policy Implementation", Chapter 10 of Wanna, John, ed. (2007), Im- proving Implementation: Organisational Change and Project Management, ANU E Press, http://epress.anu.edu.au/ imp_citation.html Treasury Board Secretariat (2004), "Canada's Performance Annual Report to Parliament", Governemnt of Canada, Ottawa. Verhoest, K. et. al (2010), Autonomy and Control of State Agencies ­ Comparing States and Agencies. Public Sector Organi- zation Series, Palgrave Macmillan. Weaver, R. Kent, and Paul B. Stares, eds. (2001), Guidance for Governance: Comparing Alternative Sources of Public Policy Advice, Tokyo: Japan Center for International Exchange World Bank (2001), Webpage "Center of Government", last updated 8 February 2001, http://www1.worldbank.org/ publicsector/civilservice/center.htm World Bank (2009). Case Study on the Management of Chile's Social Security Reform, Washington, DC.