Policy Research Working Paper 8873 From Theory to Practice Open Government Data, Accountability, and Service Delivery Michael Christopher Jelenic Governance Global Practice June 2019 Policy Research Working Paper 8873 Abstract Open data and open government data have recently countries? To answer this question, the paper constructs a attracted much attention as a means to innovate, add value, unique data set that operationalizes open government data, and improve outcomes in a variety of sectors, public and government accountability, service delivery, as well as other private. Although some of the benefits of open data ini- intervening and control variables. Relying on data from 25 tiatives have been assessed in the past, particularly their countries in Sub-Saharan Africa, the paper finds a number economic and financial returns, it is often more difficult of significant associations between open government data, to evaluate their social and political impacts. In the public accountability, and service delivery. However, the findings sector, a murky theory of change has emerged that links suggest differentiated effects of open government data the use of open government data with greater government across the health and education sectors, as well as with accountability as well as improved service delivery in key respect to service provision and service delivery outcomes. sectors, including health and education, among others. In Although this early research has limitations and does not the absence of cross-country empirical research on this attempt to establish a purely causal relationship between topic, this paper asks the following: Based on the evidence the variables, it provides initial empirical support for claims available, to what extent and for what reasons is the use about the efficacy of open government data for improving of open government data associated with higher levels of accountability and service delivery. accountability and improved service delivery in developing This paper is a product of the Governance Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The author may be contacted at mjelenic@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team From Theory to Practice: Open Government Data, Accountability, and Service Delivery Michael Christopher Jelenic 1 Keywords: Open Data, Open Government Data, Transparency, Accountability, Service Delivery JEL Classification: H11, H51, H52, H83, L38, O55 1Michael Christopher Jelenic is a Public Sector Specialist in the World Bank’s Governance Global Practice. This working paper was prepared for Open Data Research Symposium, which took in Buenos Aires, Argentina as part of the 2018 International Open Data Conference (IODC). This research has benefitted from the comments provided by the participants of the IODC conference as well as the detailed feedback provided by members of the World Bank’s Open and Transparent Government Community of Practice (CoP), including Stephen Davenport (Senior Governance Specialist, CoP Chair), Tiago Peixoto (Senior Public Sector Specialist), Jonathan Mellon, and Fredrik Sjoberg. Additional internal review and comments were provided by Timothy Herzog (Senior Data Scientist, DEC), Craig Hammer (Program Manager, DEC), as well as the World Bank Africa Region’s Chief Economist’s office (AFRCE). External peer review was provided by Stefaan Verhulst (Co-Founder and Chief Research and Development Officer, Governance Laboratory at NYU), by Cédric Dupont (Professor, International Relations/Political Science, Institut de Hautes Études Internationales et du Développement, IHEID), as well as through a double-blind review from the Centre for Open Data Research (School of Media and Communication of the Pan-Atlantic University in Lagos, Nigeria). Special thanks to Renaud Seligmann (Practice Manager, Governance Global Practice), Haishan Fu (Director, Development Data Group), and the World Bank Policy Research Working Paper publications team for supporting this work as well as to Augustin Velasquez (Research Assistant, IHEID) and Elizabeth Boomer (Research Assistant, IHEID) for their invaluable feedback. The findings, interpretations, and conclusions expressed in this paper are those of the author alone, and do not represent or otherwise reflect the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Any errors, omissions, and typos are unintentional and will be corrected in subsequent iterations of this working paper. I. INTRODUCTION Context Over the past decade, the production of data has grown exponentially around the world, and now more than ever, complex data are available at the touch of a button—from the grand, such as DNA sequencing and macroeconomic modeling, to the more mundane, such as weather maps, trash collection schedules, and public transport timetables. Expanding at a rapid pace, experts estimate that there will be a further 4,300 percent increase in annual data generation, reaching a level of 35 zettabytes (equivalent to 35 trillion gigabytes) by 2020. 2 As the sheer amount of available data in the world increases—driven by advances in technology and computing—terms such as “Big Data” and “Open Data” have become common parlance, reflecting the importance of data use in our everyday lives. In particular, Open Data—that is, “digital data that is made available with the technical and legal characteristics necessary for it to be freely used, reused, and redistributed by anyone, anytime, anywhere.” 3—has attracted much attention as a means to innovate, add value, and improve outcomes in a variety of sectors. Based on its attributes of “availability and access, reuse and redistribution, and universal participation”, 4 it has been argued that Open Data has the potential to deliver a number of benefit streams. In particular, Open Data’s potential to foster innovation, efficiency, jobs, profits, and economic growth has been touted by the private sector, which has widely emphasized the potential financial and economic returns of Open Data. For instance, McKinsey estimates that “Open Data can help unlock between US$3 trillion to US$5 trillion in economic value annually across seven sectors, including education, transportation, consumer products, electricity, oil and gas, health care, and consumer finance.” 5 Likewise, previous work by the World Bank has suggested that Open Data can yield economic benefits across five archetypical types of businesses, including suppliers, aggregators, developers, enrichers, and enablers of Open Data. 6 Other economic analyses of the topic include: a European Union study that estimated aggregate direct and indirect economic benefits of €200 billion 7; a UK study that “conservatively” estimated the direct economic benefits of public sector information at around £1.8 billion a year 8; and a study produced in Spain that found that the “infomediary” sector employs around 4,000 people in the country and generates €330 million to €550 million annually. 9 Compared with the attention given to these financial and economic benefits—however mind-bogglingly large they are purported to be—considerably less effort has been made in providing a sufficiently critical evaluation of the public-sector benefits, including the potential of Open Government Data (OGD). As a subset of Open Data, OGD is that which is “held by national, regional, local, and city governments, international governmental bodies, and other types of institutions in the wider public sector”. 10 As defined by the OECD, Open Government Data can be seen as comprised of two elements: “(i) Government Data, 2 See “Big Data Universe beginning to explode,” at http://www.csc.com/insights/flxwd/78931- big_data_universe_beginning_to_explode. Accessed September 15, 2015. 3 Open Data Charter, https://opendatacharter.net/wp-content/uploads/2015/10/opendatacharter-charter_F.pdf. Accessed January 21, 2019. 4 Open Data Handbook, http://opendefinition.org/od/index.html (May 12, 2012). Accessed September 15, 2015. 5 McKinsey Global Institute, “OGD: Unlocking Innovation and Performance with Liquid Information,” McKinsey Center for Government (October 2013). 6 A. Stott, “OGD for Economic Growth: Transport and ICT Global Practice,” World Bank (June 25, 2014). 7 Graham Vickery, “Review of Recent Studies on PSI Reuse and Related Market Developments,” Information Economics, Paris. 8 United Kingdom, Department for Business, Innovation & Skills, “Shakespeare Review: An Independent Review of Public Sector Information,” London (2013). 9 Dinand Tinholt, Characterization Study of the Infomediary Sector, Cap-Gemini, Spanish Open Data Portal Report 2013. Open Data Charter, https://opendatacharter.net/wp-content/uploads/2015/10/opendatacharter-charter_F.pdf. 10 Accessed January 21, 2019. 2 which are any data and information produced or commissioned by public bodies; and (ii) Open Data, which are data that can be freely used, re-used and distributed by anyone, only subject to (at the most) the requirement that users attribute the data and that they make their work available to be shared”. 11 Common types of OGD include, but are not limited to, data on maps, land ownership, census, government budget, government spending, company registration, legislation, public transport, international trade, health, education, crime statistics, environment statistics, election results, and contracts. While there is considerable conjecture about the precise nature and categorization of public sector benefits that can be derived from these types of data, OGD has been tied to a number of positive public externalities. For its part, the World Bank has cited a number of benefit streams, including: (i) fostering economic growth and job creation; (ii) improving the efficiency, effectiveness, and coverage of public services; (iii) increasing government transparency, accountability, and citizen participation; and (iv) facilitating better information sharing within government itself. 12 Likewise, the OECD describes similar benefits, which include: (i) improving government accountability, transparency, responsiveness and democratic control; (ii) promoting citizens’ self-empowerment, social participation and engagement; (iii) building the next generation of empowered civil servants; and (iv) creating value for the wider economy. 13 14 Recognizing this potential, dozens of governments have created OGD platforms to support the proactive release of high-value data sets that can be used to improve government transparency, accountability, and performance. Notable examples include the United States and the United Kingdom, who pioneered the release of government data sets in an effort to make government more transparent through platforms such as data.gov and data.gov.uk. Since then, numerous countries have created similar OGD portals to provide a plethora of government information. 15 16 At a global level, international institutions and nongovernmental organizations (NGOs) have begun providing technical assistance and capacity building for Open Data initiatives around the world. 17 In particular, the drive toward greater government transparency has been supported by the Open Government Partnership (OGP), which was formed in 2011 to function as “an international platform for domestic reformers committed to making their governments more open, accountable, and responsive to citizens.” 18 Likewise, in order to deepen commitments to government transparency, an Open Data Charter was signed in 2013 by the G-8 countries, setting out the principle of “open by default,” which has subsequently prompted calls for a “data revolution” to take center stage in the post-2015 development agenda, including the dialogue on the Sustainable Development Goals. 19 Other notable initiatives include the Open Data for Development Partnership (OD4D), comprising 65 bi-laterals, multilaterals, foundations, and NGOs; the Open Data Working Group of the OGP; the Open Contracting Data Partnership; and the Global Partnership for Sustainable Development Data; as well as tools such as the Open Data Barometer, which provides information on data availability in more than 90 countries. 11 Ubaldi, B. (2013), "Open Government Data: Towards Empirical Analysis of Open Government Data Initiatives", OECD Working Papers on Public Governance, No. 22, OECD Publishing, Paris. 12 World Bank, “OGD for Sustainable Development,” Transport and ICT Global Practice Policy Note 1 (2015). 13 Ubaldi, B. (2013), "Open Government Data: Towards Empirical Analysis of Open Government Data Initiatives". 14 Another useful organization of Open Data's Impacts is provided by Verhulst and Young (2016) who organize case studies into four main themes: (i) Improving Government; (ii) Empowering Citizens; (iii) Creating Opportunity; and (iv) Solving Public Problems (see: http://odimpact.org/). 15 Hania Farhan, Debra D’Agostino, and Henry Worthington, Web Index 2012, World Wide Web Foundation. 16 To date, 59 countries have included commitments to Open Data in their National Action plans. See Open Government Partnership Website at http://www.opengovpartnership.org/ . Accessed on May 3, 2015. 17 A growing number of organizations have become involved in the current dialogue on OGD, including international organizations such as the World Bank, the African Development Bank (AfDB), the European Union (EU), and United Nations, as well as a large number of think tanks, universities, and CSOs. 18 See https://www.opengovpartnership.org/ 19 Open Data Institute, “Supporting Sustainable Development with OGD” (2015). 3 Given the proliferation of international initiatives to support Open Government, and OGD in particular, it is increasingly evident that OGD is a key concern for political leaders in developing and developed countries alike. As such, the motivation for initiating and promulgating such initiatives can be viewed as a product of public policy priorities at the national level, which may differ significantly across countries. For developed countries, OGD is often guided by domestic legislation related to freedom of information (FoI) and access to information (A2I) as well as fiscal pressures, which make it imperative to demonstrate improved administrative efficiency and value for money in service delivery. For developing countries, these same motivations exist, but are amplified by a number of related issues, including, inter alia, an interest to satisfy donors and attract international aid; to meet international commitments such as the Sustainable Development Goals (SDGs); and to improve perceptions of transparency and accountability, which can help to improve the business climate and attract foreign investment. Problem Statement Based on its potential promise, a murky theory of change seems to have emerged among Open Data advocates, practitioners, and “evangelists,” which suggests that OGD has the potential to make governments more transparent in their operations and performance, thus providing an evidentiary basis for citizens to hold politicians to account—in a principle-agent manner—for spending public resources and providing public services. However, despite the appeal of this logic, there is very limited cross-country empirical data to support the purported relationships between these issues. In its place, it would appear that advocates have increasingly turned to a “faith-based” view on the efficacy of OGD, which has a number of limitations. One strand of literature on the efficacy of OGD comes in the form of “success stories” drawn from different settings around the world. Such qualitative case study analysis—if it can be labeled as such— tends to express the effects of Open Data in “aspirational and speculative” terms, often providing little evidence that Open Data “outputs” such as policies, platforms, dashboards, and the like are actually impacting public sector “outcomes” such as improved accountability, service provision, or service outcomes. 20 Moreover, the situation-specificity of these cases—often referring to a particular government, in a particular country, in a particular sector, at a particular moment in time—provides limited external validity for other contexts. Likewise, from a quantitative perspective, there is a significant body of impact evaluation literature utilizing randomized controlled trials, as well as quasi-experimental research designs, which has found statistical associations between government transparency initiatives and improved government accountability and service delivery. However, while this strand of literature has evaluated government transparency and accountability initiatives (TAIs) more broadly—including fiscal transparency programs, community scorecards, and access to information legislation, among others—few, if any, impact evaluations have been performed on the treatment effects of OGD initiatives in particular. Moreover, almost all of these impact evaluations are done at the country or subnational level, focusing on very specific populations and specific moments in time. As with the qualitative case study literature described above, the existing quantitative evidence therefore has a number of spatial and temporal limitations, which reduces its applicability to other country contexts. Research Question and Methods Given the limitations of the current research available, whether qualitative case studies or quantitative impact evaluations, it appears that a significant gap exists in the literature between theory and practice. In an attempt to fill this lacuna, the present research asks: 20 Stefaan G. Verhulst and Andrew Young (2017). "Open Data in Developing Countries: Toward Building an Evidence Base on What Works and How." The GovLab, New York University. July, 2017. 4 Based on the evidence currently available, to what extent and for what reasons is the use of OGD associated with higher levels of accountability and improved service delivery in developing-country contexts? To begin to answer this question, this research will first attempt to determine whether a statistical association actually exists to support the purported relationships between the variables. In doing so, the research will rely on a unique data set that operationalizes OGD, government accountability, and service delivery—including both provision and outcomes—while operationalizing other intervening variables that capture critical conditions for improved accountability and service delivery. Given the paucity of cross- country comparative studies on this topic, this research will utilize data for over 25 countries in Sub-Saharan Africa over a time horizon for which data currently exist. Significance and Contributions The present research aims to investigate the relationship between OGD, accountability, and service delivery in order to provide one of the first statistical assessments of the available evidence in a cross-country comparative context. Importantly, it is not the intent of the present research to provide a definitive or causal assessment of the relationship among OGD, accountability, and service delivery. Rather, given the limitations of the data available, the empirical contribution of this research is to flesh out potential correlations between variables in order to provide an initial assessment of OGD’s efficacy. In doing so, this research attempts to identify key indicators—which operationalize a conceptual framework gleaned from the literature—to test what associations may exist between variables. Moreover, the empirical contribution of this research seeks to tease out and test the effects of a number of intervening variables, as suggested by the literature, in order to have a fuller sense of the political economy dynamics at play. In addition to this empirical contribution, however small, this research also seeks to make a contribution to the theoretical literature by providing a conceptual framework, which captures current discussions and debates on OGD, transparency, accountability, and service delivery. In addition, this research seeks to contribute to the collective understanding of the necessary and sufficient conditions that support OGD, government accountability, and improved service delivery more broadly. In particular, this research hopes to demonstrate the enabling role that access to information and political participation can play in terms of leveraging sources of OGD to hold governments answerable and accountable to citizens. Likewise, in political environments where these factors are legally and institutionally constrained, this research seeks to test alternative hypotheses with respect to public service delivery, including the managerial role played by bureaucrats themselves as well as transparency and social accountability movements more broadly. A final practical contribution of this work is to provide a better understanding of how OGD is released and utilized in developing country contexts with respect to accountability and service delivery. As such, the countries included in this assessment were chosen given that OGD has the potential to more significantly “move the needle” with respect to accountability and service delivery in these environments than in high- income country contexts, where sophisticated accountability institutions and service delivery structures are already in place. 21 Finally, and perhaps most significantly, the focus of this research on developing countries is especially salient as these often represent the weakest governance environments, which are in most need of improvements to public accountability and service delivery institutions. 21 The logic behind this stems from the notion of marginally decreasing returns to investment, whereby investment in low-income countries yields higher returns than in more advanced economies due to the scarcity of capital; however, these initial returns quickly decline as levels of investment increase. As such, OGD performance in the sample countries is currently very small as are indicator scores measuring accountability and service delivery. Accordingly, this research argues that in these developing country contexts, the potential growth of OGD may have a significant bearing on broader governance outcomes than in countries on the frontier in terms of accountability and service delivery performance. 5 II. LITERATURE REVIEW The central research question proposed above sits at the intersection of a wide range of theoretical and empirical literature relevant to modern public-sector management and international development. In particular, this research touches on previous work related to the delineation of “Big Data,” of which “Open Data” and “Open Government Data” are extensions. Taken further, OGD as a concept can be viewed as a more specific element of a larger issue area related to government transparency and accountability initiatives (TAIs), which may take a myriad of manifestations including, among others, open budgeting, asset disclosure, and declassification initiatives, as well as freedom of information (FOI) and access to information (A2I) legislation. For its part, the literature on transparency is often tightly linked with literature on government accountability, including key theoretical concepts such as democratic accountability, social accountability, and social capital as well as those related to citizen voice, participation, and engagement. Underpinning these conceptions of accountability is a large body of literature on public management and administration, including that related to new public management, public choice theory, and welfare economics, which provides a theoretical foundation for analyzing the public provision of social services. Given the diversity of themes surveyed, the literature review that follows is presented in two parts. The first element is a theoretical section, which provides an overview of the key concepts, including Open Government Data, transparency, accountability, and service delivery. Given that there is significant conjecture among authors on the exact meaning of these terms, this first section seeks to provide a plurality of perspectives, as well as to specify the definitions that will be used throughout this analysis, including the conceptual framework put forth in the following section. The second part of the literature review presents the available qualitative and quantitative empirical literature, which highlights the current understanding of the relationships between OGD, transparency, accountability, and service delivery. Theoretical Perspectives, Definitions, and Differentiations The primary topic of this research—Open Government Data (OGD)—touches on a very active and ongoing contemporary debate on the boundaries of a number of concepts which have emerged over the past decade. While not entirely scholarly in its nature, numerous governments, international institutions, and private sector entities have proffered differing definitional propositions with respect to terms such as “Big Data,” “Open Data,” and “Open Government Data,” albeit with significant overlap and interconnection. Accordingly, as a first step, it is necessary to disentangle the various concepts to best specify a clear, working definition of OGD, which is the main topic of this research. Distinctions in terminology have been provided by various sources, with differing levels of conceptual specificity. For instance, McKinsey and Co., refer to “Big Data” as data sets that are “voluminous, diverse, and timely,” with modifier “big” referring to the size and complexity of these sets. “Open,” in McKinsey’s definition, describes how liquid the data are—that is, how “transferable” these data are across platforms and users. 22 Relatedly, the World Bank’s research on the topic posits that the term “Big Data” describes very large and complex data sets that must be processed using advanced analytic techniques, whereas the key defining characteristic of “Open Data” is that it is made available as a public good. 23 These distinctions are related to those of other authors, such as those of the Open Data Barometer, who describe knowledge “as ‘open’ if anyone is free to access, use, modify, and share it—subject, at most, to measures that preserve provenance and openness.” 24 Drawing on these notions, the Open Data Institute likewise highlights centrality of shareability and access in its definition of Open Data as “Data that is made available by 22 McKinsey Global Institute, OGD: Unlocking Innovation and Performance with Liquid Information (October 2013). 23 World Bank, Open Data for Sustainable Development (March 2016). 24 World Wide Web Foundation, OGD Barometer Global Report, 2nd Edition (January 2015). 6 governments, businesses, and individuals for anyone to access, use, and share.” 25 However, a more nuanced definition comes from the Open Data Handbook, which delineates a number of functions that data must fulfill to be deemed as “open”. As such, Open Data must possess three key attributes: 26 • Availability and Access: “Data must be available as a whole and at no more than a reasonable reproduction cost, preferably by downloading over the Internet.” • Reuse and Redistribution: “Data must be provided under terms that permit reuse and redistribution including the intermixing with other data sets.” • Universal Participation: “Everyone must be able to use, reuse, and redistribute; there should be no discrimination against fields of endeavor or against persons or groups.” Other authors have taken more parsimonious perspecives on the relationship among Big Data, Open Data, and OGD, which speaks to the nauce of the concepts and further elucidates the subject of this research. Gurin (2014) notes that Big Data are usually passivley generated, and often kept private, including “data that retailers keep on customers’ buying habits, that cell phone companies keep on their mobile users, or that hospitals collect about their patients.” 27 Alternately, Gurin characterizes Open Data as “public” and “purposful”—and as such, “information that has a particular goal in mind, such as fueling new businesses, improving public health, or identifing wasteful spending.” 28 Further illustrating the overlap and interaction of these terms, the World Bank likewise notes the following: “Open data are those that are freely and easily accessible, machine-readable, and explicitly unrestricted in use. Open data aren’t necessarily big, and big data aren’t necessarily open.” 29 Drawing on these conceptions, the Open Data Charter, which was first signed in July 2013, further delineates the difference between Open Data, and what constitutes Open Government Data, which is the central concept of the present research. In particular, the Charter “recognizes that the term “government data” includes, but is not limited to, data held by national, regional, local, and city governments, international governmental bodies, and other types of institutions in the wider public sector.” 30 Given the plurality of perspectives on these closely interrelated topics, the present research will rely on—for the sake of simplicity and clarity—a two-part definition of OGD as put forth by the OECD. 31 Such a conception marries two constituent elements: (i) “government data,” which includes “any data and information produced or commissioned by public bodies;” and “open data,” which, similar to other definitions, “can be freely used, re-used and distributed by anyone, only subject to (at the most) the requirement that users attribute the data and that they make their work available to be shared as well.” 32 The current research question also relates to a growing literature on transparency and conceptions of “openness” in government. Similar to what constitutes OGD, transparency has no single definition nor is there a single indicator to measure it. Instead, transparency takes on a variety of different meanings and measures across organizations and within the literature. Khemani (2016) provides one of the simplest and most straightforward definitions of transparency as “citizen access to publicly available information about 25 Open Data Institute, “Supporting Sustainable Development with Open Data” Division for Public Administration and Development Management of the United Nations Department of Economic and Social Affairs (2017). 26 See Open Data Handbook at http://opendatahandbook.org/guide/en/what-is-open-data/ 27 J. Gurin, Open Data Now: The Secret to Hot Startups, Smart Investing, Savvy Marketing, and Fast Innovation (2014) at p. 13. 28 Ibid. 29 World Bank Group World Development Report: Digital Dividends (2016) at p.244. 30 See https://opendatacharter.net/history/. Accessed January 5, 2018. 31 B. Ubaldi, "Open Government Data: Towards Empirical Analysis of Open Government Data Initiatives", OECD Working Papers on Public Governance, No. 22, OECD Publishing, Paris (2013). 32 Ubaldi at p. 6. 7 the actions of those in government and the consequences of those actions.” 33 Similarly, Trapnell (2014) operationalizes transparency as “greater availability of information between government departments as well as greater clarity about government processes, rules, and definitions”. 34 Somewhat more colorfully, Transparency International, one of the world’s leading authorities on government transparency, describes it as such: “Transparency is about shedding light on rules, plans, processes and actions. It is knowing why, how, what, and how much. Transparency ensures that public officials, civil servants, managers, board members and businesspeople act visibly and understandably, and report on their activities. And it means that the general public can hold them to account. It is the surest way of guarding against corruption and helps increase trust in the people and institutions on which our futures depend.” 35 For the purposes of the current research, Khemani’s definition of transparency is perhaps most useful in that it delineates two distinct elements—one in providing information about performance, and another about the outcomes of that performance. As such, Khemani (2016) notes that “information provided through transparency must be specific about both policy actions and the resulting outcomes, so that citizens can use this information to select and sanction leaders.” 36 In the context of OGD and service delivery, this would amount to having information not only on what was budgeted for public services, such as health and education, but also how and to what extent those budgets were executed, what was the quality and quantity of the services provided, and what were the eventual outcomes in terms of human development. The idea of having information about both actions and outcomes is similarly captured by Fox (2007), who argues that transparency can be either “clear” or “opaque”. 37 Fox differentiates between these two conceptions of transparency, with “opaque” transparency referring to “the dissemination of information that does not reveal how institutions actually behave in practice, whether in terms of how they make decisions, or the results of their actions.” 38 In the context of OGD, Access to Information (A2I) and Freedom of Information (FoI) legislation may enable the public to access large data sets and a government may even proactively disclose this information; however, not all of this information may provide insights into government performance or policy outcomes, thus making it “opaque”. A case in point would be a “data-dump” where an emormous quantity of data may be disclosed, but the quality, assessablity, and usefulness of the data are limited. Fox contrasts such “fuzzy” forms of transparency with “clear” transparency, which is “reliable information about institutional performance, specifying officials' responsibilities as well as where public funds go.” 39 Linking this to Khemani’s example above, “clear” transparency would likewise include information about budgeted resources, actual spending, provision, and related outcomes. A critical element of this research is to determine how and to what extent OGD—as a trasparency initiative in and of itself—makes government more accountable. This question touches upon the historical aspects of Open Government and OGD, including its linkage to transparency and accountability. As such, Yu and Robinson (2012) argue that a level of “ambiguity” has emerged with respect to OGD, noting that the concept no longer refers only to policies that support improved government accountability. Rather, “the term ‘Open Government Data’ might refer to data that make the government as a whole more open (that is, more publicly accountable), or instead might refer to politically neutral public sector disclosures that are easy to 33 S. Khemani, “Making Politics Work for Development: Harnessing Transparency and Citizen Engagement.” World Bank Policy Research Report (2016). 34 S. Trapnell (ed.) Right to Information: Case Studies on Implementation. World Bank (2014). 35 http://www.transparency.org/ what-is-corruption/#what-is-transparency. 36 Khemani at p. 220. 37 J. Fox, “The Uncertain Relationship between Transparency and Accountability,” Development in Practice (2007) at p. 405. 38 Ibid. 39 Ibid. 8 reuse, even if they have nothing to do with public accountability.” 40As a result of this “new ambigutiy,” Yu and Robinson argue that the release of Open Data, or the creation of special websites and OGD portals, may be enough to deem a government “open”—but may not actually make it more transparent or accountable. This distinction between the terms—as well as the distinctions related to “clear” and “opaque” transparency noted above—suggest that researchers cannot assume a merely linear relationship between these concepts. Rather, as noted by Kosack and Fung (2019), these relationships are often intermediated by a number of contextual factors that determine the extent to which information can effectively inform individual choices, assist with collaborative problem solving, increase pressure on service providers, enable top-down reforms, as well as support countervailing power structures. 41 The central research question engages with the debate on what constitutes accountaiblity, including public sector accountability more specifically. However, just as transparency and OGD have a wide variety of interpretations, the concept of accountability is equally challenging to define. A very broad and useful definition is provided by Tisné (2010), who notes that: “Accountability refers to the process of holding actors responsible for their actions…it is the concept that individuals, agencies, and organizations (public, private, and civil society) are held responsible for executing their powers according to a certain standard (whether set mutually or not).” 42 Building on the conception of accountability as a process, there is considerable debate in the literature regarding how these mechanisms function over time and with respect to different actors. Using a principle- agent framework, the World Bank has conceptualized accountability as a “set of relationships among actors (e.g. individuals, agencies, and organizations)—both public and private—responsible for their actions and executing their powers according to a certain standard”. 43 Such a rationale assumes that principles— namely, citizens—delegate their sovereignty to elected representatives to undertake certain public governance responsibilities as well as to allocate the resources—in the form of taxes and transfers—to pay for these services. As agents, these actors—namely, politicians, policy makers, and providers—are responsible to perform the delegated tasks and to inform the principle about their performance. Based on this information, the principles can then enforce sanctions and rewards on the agents, both through electoral and non-electoral channels. Challenging the functioning of this principle-agent relationship, the World Bank’s 2017 World Development Report 44 identifies a number of “power asymmetries,” which may emerge. These include: (i) “exclusion,” whereby certain individuals and groups may not enjoy the same access to certain services; (ii) “capture,” whereby elite individuals or groups influence politicians to adopt policies that serve a narrow special interest, often at odds with the public good; and (iii) “clientelism,” whereby political support is exchanged for preferential access to certain goods and services. 45 Similar aberrations are noted by Devarajan and Khemani (2016), who describe how such forms of “unhealthy political engagement” can similarly “invert” the principle-agent relationships between citizens and government officials. 46 Other authors note that government transparency and government accountability are often conflated at the conceptual level, and there is a general predisposition to assume that transparency automatically leads to 40 H. Yu and D. Robinson, “The New Ambiguity of Open Government,” UCLA. L. Rev. Disc (2012) at p. 1. 41 Stephen Kosack and Archon Fung, Does Transparency Improve Governance? (May 2014). Annual Review of Political Science, Vol. 17, pp. 65-87, 2014. 42 M. Tisné, “Transparency, Participation and Accountability: Definitions,” unpublished Background Note for Transparency and Accountability Initiative, Institute of Development Studies (2010). 43 World Bank. Making Services Work for Poor People. World Development Report 2004. 44 World Bank. Governance and the Law. World Development Report 2017. 45 WDR 2017 (p.11). 46 Shantayanan Devarajan, and Stuti Khemani. 2016. “If politics is the problem, how can external actors be part of the solution?” Policy Research working paper; no. WPS 7761. Washington, D.C. World Bank Group. 9 improved accountability. To take on this misconception, numerous authors have argued that while governments can become more “transparent,” they do not automatically become more “accountable,” unless there are appropriate mechanisms in place for that data to be acted upon by either the government or citizens more largely. To be sure, Fox (2007) argues that accountability can be either “soft” or “hard,” which reflects two critical dimensions: “answerability,” that is, the right/capacity to demand answers, as well as “enforceability,” with the capacity to sanction, compensate, or remedy. 47 The underlying relationships alluded to in this description are also touched upon by Peixoto (2013), who further argues that “publicity” and “political agency” conditions are necessary to translate transparency initiatives, such as OGD, into greater public accountability. 48 49 Other useful conceptions of accountability related to the present research come from the larger political science scholarship on democratic accountability. Conceptions of the citizens ceding their sovereignty to elected officials and holding them accountable through electoral and administrative channels can be traced back as far as the work of Alexis de Tocqueville, who saw these democratic accountability mechanisms as the greatest checks against despotism and tyranny. 50 However, while foundational, this early work only began to lay the foundations of more complex notions of how accountability functions in democratic societies. More recently, Dahl and Lindblom (1976) emphasize the convergence of a number of fundamental processes of control, including market systems, democratic institutions, bargaining among leaders, as well as effective control of non-leaders through hierarchical relationships. 51 Likewise, Borowiak (2011) further defines the attributes of democratic accountability with reference to the principle that “the governed should have the opportunities to sanction and demand answers from the powers that govern them”. 52 A common theme uniting all of this related literature is an implicit recognition of the principle- agent relationship within accountability processes. Taking an international perspective, other authors have written more broadly on the role of democratic accountability in the organizations—such as international development institutions, multilateral transparency initiatives, and other global charters and regimes—which support global agendas such as OGD and transparency. Dahl (1999) examines the extent to which that these organizations can likewise ensure the appropriate levels of democratic accountability demanded from their members and clients. In particular, the “democratic deficits” that can emerge when delegated responsibilities make it difficult for citizens at the domestic level to exercise effective control over larger “bureaucratic bargaining systems.” 53 Likewise, Grant and Keohane (2005) provide a typology of accountability mechanisms, including hierarchical, supervisory, fiscal, legal, market, peer, and reputational—which are more relaxed and nuanced than the strict democratic accountability relationships that function at the domestic level. 54 Similarly, Keohane (2011) differentiates between accountability mechanisms, which may influence the way that international agendas (e.g. OGD) are managed in terms of authorization, support, and impact. 55 While these arguments are very salient in terms of the debates on democratic accountability at the international level, they only 47 J. Fox, “The Uncertain Relationship between Transparency and Accountability,” Development in Practice 17 (2007) at p. 405. 48 T. Peixoto, “The Uncertain Relationship between Open Data and Accountability: A Response to Yu and Robinson’s The New Ambiguity of ‘Open Government,’” UCLA. L. Rev. Disc. 60 (2013) p. 200–248. 49 These concepts will be further described in the subsequent conceptual framework. 50 Alexis de Tocqueville, Democracy In America, edited and abridged edition by Richard Heffner (1956). 51 Charles E Lindblom and Robert A. Dahl, Politics, Economics, and Welfare, Chicago: University of Chicago Press (1976). 52 Craig T. Borowiak: Accountability and Democracy: The Pitfalls and Promise of Popular Control. Oxford: Oxford University Press (2011) p.9. 53 R. Dahl, “Can international organizations be democratic? A skeptic's view.” in I. Shapiro & C. Hacker-Cordón (Eds.), Democracy's Edges, Cambridge: Cambridge University Press (1999) p. 19-36. 54 R. Grant and R. Keohane, “Accountability and Abuses of Power in World Politics,” American Political Science Review, 99(1) (2005) p. 29-43. 55 Robert Keohane, Global Governance and Democratic Accountability (2011). 10 tangentially touch upon the type of “domestic” democratic accountability referred to in this present research, which essentially is located at the national and sub-national levels where services are delivered. Moving from accountability relationships to the service delivery mechanisms themselves, one of the most significant analyses of a principle-agent conception of service delivery is provided by the World Bank’s 2004 World Development Report (WDR 2004), 56 which developed a “Service Delivery Framework” 57 comprised of four constituent elements. Recognizing that public intervention is necessary in the provision of social services, a “long route” to accountability and service delivery would involve citizens—acting as principles—using their “voice” 58 to demand improvements in services from policy makers, who serve as agents. For their part, policy makers rely on “compacts” 59 with service provider organizations, such as ministries of health and education or regional hospitals or school districts. In this “long route”, provider organizations then “manage” 60 the frontline service providers—the doctors, nurses, teachers, and engineers—to more effectively deliver services. A final step in this accountability chain is the role of “client power” 61—a condition in which end users express their preferences and levels of satisfaction to frontline providers. 62 While such a model of service delivery is rooted in conceptions of democratic accountability, it can likewise be contextualized with respect to the growth of the welfare state since the end of the second World War and the emergence of new public management theory in the early-1990s. Both of these phenomena have created an environment where the role of the government has changed to become, respectively, both a provider and manager of public goods such as social services. From a demand side, the rise of the welfare state largely explains the growth of social service provision. To be sure, Myles and Quagagno (2009) argue that since the 1960s, expenditures on social transfers have grown considerably as a result of social insurance schemes as well as national social service programs in health and education. 63 From a supply side, new changes in government management techniques have put a premium on delivering services in an accountable, efficient, and evidence-based manner. 56 World Bank, World Development Report 2004: Making Services Work for Poor People (WDR 2004), Washington, DC (2003). 57 “The four service-related actors—citizens/clients, politicians/policy makers, organizational providers, frontline professionals— and the four relationships of accountability that connect them, [including]: (i) voice and politics, which connect citizens and politicians; (ii) compacts, which connect politicians/policy makers and providers; (iii) management, which connects provider organizations with frontline professionals; and (iv) client power, which connects clients with providers.” World Bank, WDR (2004, p. 48). 58 As noted by the World Bank, Voice "connects citizens and politicians and comprises many formal and informal processes, including voting and electoral politics, lobbying and propaganda, patronage and clientelism, media activities, and access to information. Citizens delegate to politicians the functions of serving their interests and financing governments through their taxes. Politicians perform by providing services, such as law and order or communities relatively free of pathogens. Citizens enforce accountability through elections and other less definitive means, such as advocacy, legal actions, and naming and shaming campaigns." World Bank, WDR (2004, p. 48). 59 Compacts, as defined by the WDR 2004, are “The broad, long-term relationship of accountability connecting policy makers to organizational providers. This is usually not as specific or legally enforceable as a contract. But an explicit, verifiable contract can be one form of a compact” (World Bank WDR 2004) p. 54. 60 Management, as defined by the 2004 WDR, is “The relationship of accountability connecting organizational providers and frontline professionals, comprising internal processes for public and private organizations to select, train, motivate, administer, and evaluate frontline professionals. These processes may be rule-bound in large bureaucracies, or idiosyncratic and ad hoc in small, private providers” (World Bank WDR 2004) p. 54. 61 Client Power, as defined by the 2004 WDR, is “The relationship of accountability connecting clients to the frontline service providers, usually at the point of service delivery, based on transactions through which clients express their demand for services and can monitor supply and providers” (World Bank WDR 2004) p. 54. 62 This “short route” to service delivery as well as other concepts will be further explored and operationalized in the subsequent conceptual framework. 63 J. Myles and J. Quagagno, “Political Theories of the Welfare State,” Social Service Review 76 (1), 75th Anniversary Issue (2009) p. 34–57. 11 These debates on the production and allocation of public services, particularly in the context of a growing welfare state, are captured in the vast literature on public choice theory, including the work of, inter alia, Black (1948), 64 Arrow (1951), 65 Downs (1957), 66 Buchanan and Tullock (1962) 67 and Olson (1965). 68 As such, the underlying rationale is that the production of services will be dictated by a confluence of interests and actors within the political sphere, whereby self-interest ultimately determines the allocation of resources. As such, certain goods such as health and education, may be undersupplied by market forces, necessitating the government to step in where the private market cannot—or will not—provide them. The idea of differentiating types of goods, including public goods that may be undersupplied due to market failures, can be linked to the early work of Samuelson (1954), who defined “collective consumption goods” as those “which all enjoy in common in the sense that each individual's consumption of such a good leads to no subtractions from any other individual's consumption of that good.” 69 Conceptions of “non-rivalry” and “non-excludability” in describing public goods and services are further emphasized by the work of Ostrom (2005), who treats the production of goods and services as a matter of public choice and bargaining based on diverse individual incentives. 70 Marrying the work of Samuelson and Ostrom, OGD in this context can be seen as a collective consumption good, which is both non-excludable and non-rivalrous. It is precisely at this junction where the present research on OGD links to public administration theory, including “new public management” (NPM), which recognizes government’s special role in service provision, financing, and regulation. 71 As first defined by Hood (1991), 72 NPM has been postulated to provide for better provision of public services, particularly in contexts of heightened public accountability, declining fiscal space, and focus on performance results. In particular, Gruening (2001) highlights the diverse antecedents of NPM, as a mixture of public choice theory, managerialism, policy analysis, principle-agent theory, property rights theory, transaction-cost economics, and methodological individualism, 73 which allows for a more flexible, outcome-driven approach to public sector management. As such, key characteristics of NPM include, among others: accountability for performance, performance auditing, privatization, decentralization, competition, performance measurement, freedom to manage, and increased use of information technology. 74 In the context of OGD, NPM theory is especially relevant because it provides the basis for performance measures—in terms of outcomes and not just program inputs—that can help determine the efficacy of service delivery as well as allow for evidence-based decision making and subsequent action based on managerial discretion. Importantly, it should be noted that while not incompatible, NPM departs from the classical public administration theory, which emphasizes the role of bureaucratic inputs managed by a top-down power structure. As such this classical public administrative theory—as first put forth by Max Weber’s seminal Bureaucracy (1922) 75—makes a case for a top-down control mechanism in the form of a “monocratic hierarchy” whereby policy is formulated at a high level and then executed by a series of descending 64 D. Black, The Theory of Committees and Elections. Cambridge: Cambridge University Press (1958). 65 K.J. Arrow, Social Choice and Individual Values, (1951, 2nd ed., 1963). 66 A. Downs, An Economic Theory of Democracy. Cambridge: York: Cambridge University Press (1957). 67 J.M. Buchanan and Gordon Tullock, The Calculus of Consent: Logical Foundations of Constitutional Democracy (1962). 68 M. Olson Jr., The Logic of Collective Action. Cambridge: Harvard University Press (1965, 2nd ed., 1971). 69 P.A. Samuelson, "The Pure Theory of Public Expenditure". Review of Economics and Statistics 36 (4) (1954) p. 387–89. 70 E. Ostrom, Understanding Institutional Diversity. Princeton, NJ: Princeton University Press (2005). 71 World Bank, WDR (2004). 72 Christopher Hood. "A Public Management for All Seasons?" Public Administration Vol. 69 (1991) p. 3-19. 73 G. Gruening, “Origin and Theoretical Basis of New Public Management,” International Public Management Journal 4 (2001). 74 Ibid, p. 2. 75 Max Weber. Bureaucracy (1922) as discussed in Weber’s Rationalism and Modern Society: New Translations on Politics, Bureaucracy and Social Stratification, ed. Tony Waters and Dagmar Waters, Palgrave MacMillan (2015) p. 73-128. 12 offices. 76 Within this context, Weber described the role of the civil servant as “vested in his ability to execute conscientiously the order of the superior authorities.” 77 This classical model is also a hallmark of “Taylorism”—based on Taylor’s Principles of Scientific Management 78—which emphasized the importance of control and planning mechanisms to best ensure efficiency and accountability. Although NPM does move away from these classical public administration perspectives, it provides an important antecedent for the conceptual model described in the next section, particularly, the ability of government to manage public servants to deliver services. Empirical Evidence and Linkages between Concepts Current empirical evidence of the effects of OGD remains somewhat sparse; however, efforts have been made to study the relationship between OGD and accountablity, albeit with mixed results. As such, Davies (2014) challenges an implicitly linear theory of change and argues that policy makers need a more nuanced understanding of how Open Data leads to outputs, outcomes, and impact. 79 Using qualitative case studies, Davies finds that while there is evidence of Open Data outputs such as applications, dashboards, and new analysis, information on outcomes is limited. Nevertheless, his case study review highlights the importance of legal frameworks, domain knowledge, and technical skills as important background resources for reaching outputs such as applications, reports, analysis, and new derived data sets. 80 Similarly, other case study research on Open Data’s impacts in Kenya’s urban slums and rural settlements suggests that data are being used by these communities and that they are demanding even greater quantities of data; however, the study finds no clear association between Open Data and improved accountability or service delivery. 81 A similar stream of research investigates the role of transparency initiatives more broadly, particularly the role played by Freedom of Information (FoI) and Access to Information (A2I) legislation—which have the ostensible goal of improving government accountability. As such, empirical studies have suggested a connection between improved accountability and government responsiveness, although the linkage between FoI/A2I and accountability is less clear. For instance, it has been postulated that FoI initiatives, particularly those that are community based, have been linked to improved citizen rights related to housing and water in South Africa. 82 Similarly, Hazell, Worthy, and Glover (2010) provide a systematic evaluation of FoI laws in Britain, finding that such legislation can help improve government decision making and public awareness. 83 Likewise, Peisakhin and Pinto (2010) analyze FOI legislation in India and find that it has a significant impact in motivating government officials to process applications for certain entitlement programs, including reducing the time needed to collect benefits. 84 Pandey et al. (2009) find that FOI has reduced absentee rates of public school teachers in rural India, albeit with limited effect on eventual learning 76 J. Pfiffner, “Traditional Public Administration versus the New Public Management: Accountability versus Efficiency,” in Institutionenbildung in Regierung und Verwaltung: Festschrift fur Klaus Konig, ed. A. Benz, H. Siedentopf, and K.P. Sommermann (2004) p. 443–54. 77 M. Weber, “Politics as a Vocation,” in From Max Weber, ed. H. H. Gerth and C. W. Mills, Oxford University Press (1946) p. 95. 78 Frederick Winslow Taylor, The Principles of Scientific Management, New York and London: Harper & Brothers (1911). 79 T. Davies, “Open Data in Developing Countries: Emerging Insights from Phase 1,” World Wide Web Foundation (2014). 80 Ibid. 81 Jesuit Hakimani Centre, “Open Government Data for Effective Public Participation: Findings of a Case Study Research Investigating Kenya’s Open Data Initiative in Urban Slums and Rural Settlements,” Nairobi Kenya (2014). 82 ODAC, “The Right to Know, the Right to Live: Turning the Right to Information into a Living Reality,” Cape Town (2010). 83 R. Hazell, B.Worthy, and M. Glover, "The Impact of the Freedom of Information Act on Central Government in the UK: Does FOI Work?" Palgrave Macmillan (2010). 84 L. Peisakhin and P. Pinto, "Is transparency an effective anti-corruption strategy? Evidence from a field experiment in India," Regulation & Governance 4(3) (2010) p. 261–280. 13 outcomes in the schools. 85 Finally, Awortwi and Nuvunga (2019) investigate the role that information disclosure regimes in the extractives sector—such as the Extractives Industry Transparency Initiative (EITI)—can play in improving accountability. Surveying 17 different empirical factors, the authors find that unless there is a risk of a ruling party losing power, information disclosures are unlikely to make governments more accountable on their own. 86 Relatedly, there is a relatively large literature that supports the proposition that the users of OGD—for instance the press, academia, and civil society—can use this information to hold governments to account. For instance, Mungiu-Pippidi (2014) investigates—among other variables—press freedom and corruption. Using a large-n cross-country analysis, as well as many of the same corruption indexes used by this present paper, the author finds a robust correlation between the variables. 87 Similarly, Ferraz and Finan (2008) find that in Brazil, the proactive disclosure of audit outcomes significantly impacted electoral performance of incumbents, particularly when corrupt practices were uncovered. Interestingly, these effects seem to be amplified in places where public radio was active in transmitting this information, which suggests the importance of the media in perpetuating the accountability-enhancing effects of OGD. 88 Similar results were obtained in Mexico by Larreguy, Marshall, and Snyder (2015), who studied the performance of infrastructure grants that targeted the poor. The authors found that when provided information on the use and outcomes of these funds, voters tend to punish poorly performing politicians, but only in areas covered by local media—again, suggesting the key role that the media can play in disseminating information to hold government to account. 89 In addition to investigating the role of OGD, the present research also seeks to build on previous efforts (Joshi 2013) investigating the efficacy of transparency and accountability initiatives, the findings of which are mixed in terms of service delivery. 90 To be sure, Gaventa and McGee (2014) have evaluated the efficacy of a number of transparency and accountability mechanisms, 91 and note that where there is “positive evidence in one setting, this is often not corroborated—and sometimes even contradicted—by findings in another setting where different, or even similar, methods have been used.” 92 Likewise, Carothers and Brechenmacher (2014, 2018) note that evidence on the impact of accountability, transparency, participation, and inclusion is “often long term, indirect, and difficult to isolate from other factors, and the evidence base to date is still too thin to arrive at firm conclusions.” 93 85 Priyanka Pandey, Sangeeta Goyal, and Venkatesh Sundararaman, “Public Participation, Teacher Accountability, and School Outcomes: Findings from Baseline Surveys in Three Indian States,” World Bank Policy Research Working Paper Series (November 1, 2008). 86 N. Awortwi and A. Nuvunga (2019) Sound of One Hand Clapping: Information Disclosure for Social and Political Action for Accountability in Extractive Governance in Mozambique, IDS Working Paper 523, Brighton: IDS 87 Alina Mungiu-Pippidi, “Quantitative Report on Causes of Performance and Stagnation in the Global Fight against Corruption.” available at http://anticorrp.eu/publications/quantitative-report-on-causes/ (2014). 88 Claudio Ferraz and Frederico Finan. "Exposing Corrupt Politicians: The Effects of Brazil’s Publicly Released Audits On Electoral Outcomes." The Quarterly Journal of Economics (2008). 89 Horacio A. Larreguy, John Marshall, James M. Snyder, Jr. "Revealing Malfeasance: How Local Media Facilitates Electoral Sanctioning of Mayors in Mexico." NBER Working Paper No. 20697 (2014). 90 A. Joshi, “Background Paper on Service Delivery: Review of Impact and Effectiveness of Transparency and Accountability Initiatives,” Institute of Development Studies (2013). 91 These include, among others, citizen report cards, organizational score cards, and asset discloures of public officials, as well as more “participatory” mechanisms such as community monitiring, social audits, public expenditure tracking surveys, and participatory budgeting processes. 92 J. Gaventa and R. McGee, “The Impact of Transparency and Accountability Initiatives,” Development Policy Review 31: s3–s28 (2013). 93 T. Carothers and S. Brechenmacher. “Accountability, Transparency, Participation, and Inclusion: A New Development Consensus?” Carnegie Endowment for International Peace (2014). p. 26. 14 Nevertheless, there is a growing body of large-n impact evaluation literature which provides some empirical evidence of transparency and accountability initiatives impacting service delivery, particularly in the education sector. For instance, Reinikka and Svensson (2004; 2011) have estimated that the dissemination of budgetary information has led to less leakage of public funds in Uganda under certain conditions, 94 and Barr et al. (2012) have found that participatory monitoring has been linked to improved education outcomes. 95 Similarly, Pandey et al. (2008) provide empirical evidence from India which suggests that dissemination of budget information has improved teacher effort including reduced absenteeism. 96 In a seminal study in Kenya, Duflo, Dupas, and Kremer (2012) have similarly demonstrated a positive association between community-based hiring of teachers with improved attendance and test scores. 97 Just as in the education sector, there is a growing body of empirical evidence that similarly links transparency and accountability initiatives with improved provision and outcomes in the health sector. Bjorkman, de Walque, and Svensson (2009; 2014) have provided evidence that access to service delivery information and related outcomes can have an effect on how services are delivered. 98 Using a randomized controlled trial in the Uganda health sector, the study found that better service delivery and outcomes were associated with the treatment group that received a report card on staff and health center performance. Similarly, Mishra (2007) has demonstrated that community scorecards have improved satisfaction in health service delivery in India, 99 which suggests the importance of community members having objective and quantitative information regarding staff behavior in monitoring accountability. Additional work on these issues has likewise been completed by Ravindra (2004), who found that citizen report cards actually improved local service delivery in certain settings in India, 100 and Goldfrank (2006) who demonstrates that participatory budgeting initiatives have been shown to improve public service delivery in Brazil. 101 With respect to electoral accountability mechanisms, the present research seeks to confirm earlier evidence, which supports the proposition that elections can have a significant effect on citizens rewarding and sanctioning politicians for their performance in delivering services. To be sure, there is a significant body of literature that suggests that performance in service delivery is an important metric that voters use to choose politicians, and to vote out of office those who they believe are performing poorly. Using the Italian municipal level to test this hypothesis, Kendall et al. (2015) find that information on “valence” issues, such as past service delivery performance, increased support for candidates much more than information about their particular policy positions. 102 Similar work by Carruthers and Wanamaker (2015) finds that the enfranchisement of women in elections during the early part of the 20th century in the United States 94 Ritva Reinikka and Jakob Svensson, “The Power of Information: Evidence from a Newspaper Campaign to Reduce Capture,” Quarterly Journal of Economics (2004: 119 [2]); Ritva Reinikka and Jakob Svensson, “The Power of Information in Public Services: Evidence from Education in Uganda,” Journal of Public Economics (2011: 95). 95 Abigail Barr, Frederick Mugisha, Pieter Serneels, and Andrew Zeitlin, “Information and Collective Action in Community-Based Monitoring of Schools: Field and Lab Experimental Evidence from Uganda” Working Paper (2012). 96 Priyanka Pandey, Sangeeta Goyal, and Venkatesh Sundararaman, “Public Participation, Teacher Accountability, and School Outcomes: Findings from Baseline Surveys in Three Indian States,” World Bank Policy Research Working Paper Series (November 1, 2008). 97 Esther Duflo, Pascaline Dupas, and Michael Kremer, “School Governance, Teacher Incentives, and Pupil -Teacher Ratios: Experimental Evidence from Kenyan Primary Schools,” National Bureau of Economic Research Working Paper 17939, Cambridge, MA (2012). 98 M. Bjorkman, D. de Walque, and J. Svensson, “Information Is Power: Experimental Evidence on the Long-Run Impact of Community-Based Monitoring,” Policy Research Working Paper 7015, World Bank (2014). 99 V. Misra, “Pilot Study 1, Andhra Pradesh, India: Improving Health Services through Community Score Cards: Learning Notes,” Social Accountability Series, Washington, DC, World Bank (2007). 100 A. Ravindra, “An Assessment of the Impact of Bangalore Citizen Report Cards on the Performance of Public Agencies,” Evaluation Capacity Development Working Paper 12, Washington, DC, World Bank, Operations Evaluation Department (2004). 101 B. Goldfrank, “Lessons from Latin American Experience in Participatory Budgeting,” Presentation at the Latin American Studies Association Meeting, San Juan, Puerto Rico (March 2006). 102 Chad Kendall, Tommaso Nannicini, and Francesco Trebbi, “How Do Voters Respond to Information? Evidence from a Randomized Campaign.” American Economic Review 105 (1) (2015) p. 322–53. 15 correlated with an increase in public school expenditure, suggesting that formal electoral mechanisms can have an effect on service provision. 103 Such findings have likewise been found in the health sector, where enfranchisement of the poor has been shown to have a significant effect on health provision and outcomes. Fujiwara (2015) finds that the rise of electronic voting in Brazil resulted in the de-facto enfranchisement of poor and less educated voters, which had the secondary effect of increasing health care provision, including services for prenatal and newborn health. 104 Similarly, in terms of health outcomes, Kudamatsu (2012) utilizes surveys conducted in 28 African countries to compare infant survival rates before and after the emergence of democracy in these countries and finds that infant mortality declined after democratization, 105 which suggests the effect that formal democratic accountability channels may have on improving health outcomes, particularly in a developing country context. Gaps in the Literature While the theoretical literature reviewed above provides an understanding of how the concepts of OGD, transparency, accountability and service delivery are defined—and how they relate to larger theoretical concepts and debates—the empirical literature that supports a relationship between these ideas is slightly less complete. As noted above, existing case studies are often aspirational or speculative in nature, and do not provide sufficiently rigorous evidence that OGD inputs translate into accountability or service delivery outcomes. Moreover, the situation-specificity of these cases provides limited external validity for other contexts. A final drawback to this literature is a paucity of analysis of the proximate conditions in place— that is, defining what is necessary and sufficient from a political economy perspective—to ensure that OGD has its intended impacts. Likewise, with respect to the empirical literature, numerous studies have been completed—some using sophisticated large-n experimental and quasi-experimental designs—however, few have been able to link Open Data, and OGD in particular, to changes in levels of government accountability or the provision and outcomes of health and education services. Rather, the literature available focuses mainly on the effects of other types of transparency and accountability initiatives—such as community score cards, participatory budgeting, and social audits—and not OGD itself. As is the case with such granular quantitative analysis, many studies focus on particular countries and regions at particular moments in time, which again, reduces their comparability to other contexts. III. CONCEPTUAL FRAMEWORK In an attempt to answer the question posed above, this research will evaluate four different propositions— or premises—that can be extrapolated from the literature, including previous theoretical and empirical contributions on transparency, accountability, and service delivery. As described below, this research will also test the political economy “conditions” that are necessary and/or sufficient for the purported relationships to hold—including the role played by the larger information-availability environment (i.e. “publicity” condition) and formal citizen voice (i.e. “political agency” condition) as well as government capacity (i.e. “public sector management” condition) and informal social accountability channels (i.e. “client power” condition). 103 C. K. Carruthers & M. H. Wanamaker, "Municipal Housekeeping: The Impact of Women's Suffrage on Public Education," Journal of Human Resources, vol 50(4) (2015) p. 837-872. 104 T. Fujiwara, “Voting Technology, Political Responsiveness, and Infant Health: Evidence from Brazil.” Econometrica 83 (2) (2015) p. 423–64. 105 M. Kudamatsu, “Has Democratization Reduced Infant Mortality in Sub-Saharan Africa? Evidence from Micro Data.” Journal of the European Economic Association 10 (6) (2012) p. 1294–317. 16  Premise 1: Higher levels of OGD make government more accountable. Under this premise, OGD provides an evidentiary basis through which citizens can be better informed on how government is performing. In particular, for this information to be effective—for it to be a type of “clear” transparency as noted by Fox (2007)—two elements would need to be present. In line with Khemani’s earlier findings, 106 this would first include information on sectoral performance, both in terms of provision of services (e.g. number of teachers, number of vaccinations, etc.) as well as service delivery outcomes (e.g. completion rates, infection rates, etc.). Second, in order for the efficiency of such service provision and outcomes to be determined, transparent information is needed both on how much was budgeted for these services as well as how much was actually spent on these services (e.g. execution rate). Only by having both types of data can “demand-side” consumers—that is, citizens, civil society organizations, researchers, journalists, and other “infomediaries”—make informed judgements as to whether the government is adequately executing its mandate and meeting its responsibilities. However, it should be noted that transparent access to sectoral and budgetary data is just one element; in order to hold government officials accountable, citizens need mechanisms to exercise their “voice”, either formally or informally. In line with the earlier work of Fox (2007), such voice channels could provide a degree of “answerability” and “enforceability”, which is crucial to ensuring a principle-agent element in the citizen-state interface. Formal citizen voice channels would include electoral mechanisms in the form of free and fair elections, which are held at regular intervals in line with legal provisions, and with no de jure or de facto impediments to participation. Other non-electoral, or social accountability mechanisms, could include, inter alia: ICT-enabled platforms 107 for collecting citizen feedback, satisfaction surveys, complaint and redress mechanisms, petitions, protests, and other participatory institutions for citizens. Given the need for both information to be publicly available as well as for mechanisms to be in place for citizens to express their voice and hold service providers to account, the research will investigate a number of enabling “conditions,” which have been postulated by the earlier work of Peixoto (2013), as noted above. These including the following: • Publicity Condition: A vital element necessary for supporting citizen voice is the “publicity condition,” 108 which provides for an enabling environment whereby “disclosed information actually reaches and resonates with its intended audiences”. 109 Conceptualized as such, the publicity condition would presuppose the existence of a necessary degree of political freedoms and civil rights so that the disclosed data can reach citizens—if not directly, then through third-party intermediators such as CSOs, social movements, academics, and the media. At minimum, this condition would imply, inter alia: constitutional protections for freedom of expression, speech, press (and internet); relevant freedom of information or access to information legislation; media freedom, impartiality, and protections against censorship; as well as the ability for civil society organizations, academia, unions, and other civic organizations to operate free from harassment, discrimination, or persecution. • Political Agency Condition: A second critical element is the presence of a “political agency condition,” 110 which provides the legal and institutional “mechanisms through which citizens can 106 World Bank. 2016. Making Politics Work for Development: Harnessing Transparency and Citizen Engagement. Policy Research Report. Washington, DC. p 220. 107 Examples of such ICT platforms include “Maji Voice” in Kenya; “I Change My City” in India: “Pressure Pan” in Brazil; and “U-Report” in Uganda. 108 This condition relates to other concepts in the literature such as “answerability”, “soft” accountability, and “yelp.” 109 Peixoto, “The Uncertain Relationship.” 110 This condition relates to other concepts in the literature such as “enforceability,” “hard” accountability, and “teeth.” 17 sanction or reward public officials”. 111 Conceptualized as such, the most obvious mechanism for citizens to express their voice would be through free and fair elections; however, this condition would imply the existence of other non-electoral mechanisms for public participation, including civil participation in political parties, CSOs, NGOs, as well as other participatory institutions. Evidence that this condition is in place would include, inter alia: evidence of political participation through free and fair elections; necessary oversight and electoral monitoring mechanisms; as well as political pluralism, including freedom of political parties, CSOs, and NGOs from discrimination, or persecution. In order for this premise to be true, there would be a number of observable implications. First, we would expect to see higher levels of OGD produced in terms of sector performance data (e.g. in health and education) as well as higher levels of data on budget allocation and execution. Second, we would expect government accountability indicators to likewise increase over time, with a significant correlation to improvements in OGD. Finally, in line with the literature, we would expect publicity and political agency conditions to be positively and significantly correlated with increases in accountability. This premise would be falsified if either there was an inverse relationship between OGD and government accountability, or, indicators measuring publicity and political agency were not positively correlated with OGD and government accountability.  Premise 2: More accountable governments enable improved service delivery. According to this second premise, policy makers and elected representatives would be held accountable to respond to citizens “voice”—whether formally or informally expressed—or risk the consequences of inaction. As such, these policy makers must be able to discipline government agencies that execute the public policy directives, priorities, and initiatives that they are advocating. Moreover, each service delivery agency must be able to ensure that civil servants—including the frontline providers responsible for service delivery—are adequately rewarded/sanctioned for their performance in carrying out their delegated tasks. While publicity and political agency are needed to translate OGD into an accountability tool, policy makers must likewise have the capacity to actually enact the policies they are seeking to implement. 112 However, in many developing country contexts, it is often the case that a significant “implementation gap” emerges between de jure legislation, policies, or regulations and de facto policy implementation. While impossible to generalize across all developing countries, such implementation gaps tend to emerge in two particular instances. The first is when politicians are not able to effectively delegate to line ministries—as their agents—to carry out policy implementation. In many developing country contexts, this is the result of competing mandates, poor coordination, unclear reporting relationships, poor prioritization, and limited institutional capacity. The second instance where implementation gaps may emerge, is at the level of the frontline service providers—that is, the doctors, nurses, teachers, engineers, sanitation workers, who physically interact with citizens in providing services. Policy implementation at this level is also prone to failure if the necessary resource allocation, human resource capacities, and performance incentives are not in place. For instance, in a context where high absenteeism is prevalent and disciplinary measures are not enforced, it is doubtful that service delivery provision and outcome targets will be reached. Returning to the literature, both of these phenomena have been well documented empirically and theoretically. With respect to the relationship between the politicians and the line ministries they oversee, previous research highlights the importance of “compacts” with service provider organizations, such as ministries of health and education or regional hospitals or school districts. As such, policy makers are 111Peixoto, “The Uncertain Relationship.” 112 As noted by the World Bank’s 2017 World Development Report: “Capacity, often considered a prerequisite for policy effectiveness, is certainly important, and in many cases it is even an overriding constraint…however, policies may still be ineffective if groups with enough bargaining power have no incentives to pursue implementation.” WDR 2017 (p. 11). 18 capacitated as principles, who provide resources and delegate power to these provider organizations and have the capacity to reward or sanction their agents if the need arises. With respect to the relationship between line ministries and frontline service providers, it is necessary that provider organizations effectively “manage” the frontline service providers—the doctors, nurses, teachers, engineers, and so forth—using the appropriate sanctions, rewards, and incentives. Therefore, according to this premise—where higher levels of government accountability lead to higher levels of service delivery—we would expect to see a well-functioning system of institutional “compacts” as well as strong “management” capacity. In order to capture these two somewhat fuzzy concepts, the research will investigate the effects of an additional “condition” as follows: • Public Management Condition: In this step of the accountability chain, policy makers function as the principles to demand that their agents—namely, service delivery organizations, such as ministries of health and education—are taking the necessary actions to improve services. In addition, service provider organizations must be able to effectively manage their frontline service providers, who in-turn serve as their agents. As noted above, in order to operationalize this chain of principle-agent relationships, governments rely on “compacts” and “management”, which would imply a number of processes, procedures, and institutional capacities being in place, including inter alia: revenue mobilization and taxation capacity; capacity for adequate public financial management, including fiscal policy formulation, budget management, and multiyear planning; statistical capacity; as well as general public administration capacity, including HR and performance management mechanisms. In order for this premise to be true, there would be a number of observable implications. First, we would expect to see higher levels of government accountability corresponding with improved provision of services as well as improved outcomes in key social sectors, such as health and education. Second, we would expect that improved levels of service delivery are associated with increasing levels of public management capacity, similarly demonstrating a positive and significant association. Accordingly, this premise would be falsified if either there was an inverse relationship between levels of government accountability and service delivery, or, indicators measuring the strength of public management capacity were not positively correlated with improved service delivery.  Premise 3: Higher levels of OGD make governments more accountable, which subsequently enables improved service delivery. Often referred to as the “long-route” to service delivery, this third premise unites the earlier premises into an accountability chain that links citizens, to government, to providers in a series of principle-agent relationships. As such, OGD would provide an evidentiary basis for citizens to hold their politicians to account for their mandates, and these politicians would then exercise “compacts” over line ministries to develop and implement policies for service delivery, and these ministries would then be able to discipline front line service providers such as doctors, nurses, and teachers, through good “management.” Crucial to this hypothesis is the role that accountability plays as an intermediating variable, namely that OGD improves accountability, and because of this improved accountability, service delivery is improved. In order for this third premise to be true, we would expect OGD, accountability, and service provision and outcomes to have a significant, positive relationship. At the same time, we would expect the joint effects of OGD and accountability (i.e. the interaction effect) to have a higher and more significant coefficient than either that of OGD or accountability on their own. Such a statistical association would demonstrate that accountability intermediates the relationship between OGD and service delivery, thus supporting this premise. Accordingly, this premise would be falsified if either there was an inverse relationship between levels of OGD, government accountability, and service delivery—and no intermediating effects is found for accountability at the sectoral level. 19  Premise 4: Higher levels of OGD directly enable improved service delivery, excluding the effects of accountability. Under this fourth premise, formal accountability channels are by-passed, and a direct link is assumed between ODG and service delivery. As such, this direct route can be “bottom-up” (upward) or “top-down” (downward) in nature. With respect to the former, clients— namely citizens, with the help of CSOs and infomediaries—would have access to OGD that provide vital information on public performance. In such a case, access to OGD can provide the evidentiary basis through which citizens can make determinations about the quality of services in local hospitals and schools. Based on this information, citizens then communicate their preferences directly with frontline service providers, including the teachers, doctors, and other direct service delivery actors. For instance, in Madagascar and other developing country contexts, the use of parent– teacher committees provide a valuable monitoring mechanism for parents as well as a mechanism for parents to communicate with education service providers about the quality and content of the services they provide. As demonstrated in the literature, citizens may rely on other social mechanisms as well, such as community scorecards, participatory budgeting exercises, and social audits, in order to directly influence the quantity and quality of services provided. “Bottom-Up” (upward) Channel: In order for such a channel to be evident, citizens would not express their voice through traditional accountability mechanisms such as elections or political processes, but do so in a more direct manner, interfacing directly with local service providers. In line with the conceptual model advocated by the World Bank’s 2004 World Development Report, the research will investigate the effects of a final condition, as follows: • Client Power Condition: In contexts where there are weak electoral mechanisms or weak government management capacity, client power can be especially helpful in collectively organizing local communities. In this scenario, characterized by limited, formal political accountability mechanisms, it would be necessary for there to be an eco-system in place that allows clients to supervise the provision of services as well as to communicate their preferences, satisfaction, and complaints directly with service providers themselves. Such a condition would imply mechanisms similar to the civil transparency and social accountability mechanisms mentioned above, including participatory budgeting, citizen score cards, social audits, as well as parent-teacher councils and local health center supervisory councils. For these to be successful in expressing citizen voice to front line providers, a number of crucial elements would need to be in place. First, this would entail a strong presence and capacity of civil society organizations, which can organize around key issue areas, monitor performance, collectively represent constituent voices, and communicate communal level concerns directly with front line providers, for instance at schools and hospitals. This condition would also presume a level of IT, internet connectivity, mobile phone penetration, and data literacy, if not on the part of individual citizens, then by CSOs, journalists, researchers, or other “infomediaries,” who play a role in digesting and disseminating information to the general population. Finally, on a larger level, the functioning of strong client power relationships would rely on a baseline of political and civil rights, which would permit freedom of association, media, and speech as well as a legal framework that protects civic groups and the media from harassment and censorship. In order for this premise/channel to be true, there would be a number of observable implications. First, we would expect to see higher levels of OGD corresponding to improved service delivery. In this relationship, citizens, CSOs, and other “infomediaries” would be the direct consumers of this data and exercise their voice outside of formal accountability mechanisms. As such, we would expect to find a well-developed civil society ecosystem, the capacity for the sharing of data at the grass roots level, and social accountability mechanisms in place to allow citizens to engage providers directly. Accordingly, this premise would be 20 falsified if either there was an inverse relationship between levels of OGD and service delivery, or the control variable operationalizing the “Client Power” condition was not significant. “Top-Down” (downward) Channel: Under such a scenario, changes to government services can be made unilaterally based on the strength of public management capacity (e.g. “public management condition” as described above) in order to obtain whatever exigencies and incentives managers deem most important and feasible. While this hypothesis has not been widely advocated in the literature, numerous studies suggest that OGD is able to facilitate better information sharing within government. To be sure, in places where OGD portals have been introduced, a large proportion of the views or downloads are from civil servants themselves. 113 Examples of such behavior may be supported by non-democratic, “managerialist,” or “developmentalist” regimes, which have shown a proclivity for implementing service delivery reforms based on solid evidence, yet without electoral incentives to respond to citizen accountability pressures. In this premise, citizen voice—or direct “client power”—is removed, and service delivery improvements are predicated solely upon internal government power relationships and government capacity to command and control public servants. In place of “client power” and “citizen voice” are strong government “compacts” (described above) as well as a high degree of “management” in the form of sanctions and reward mechanisms to discipline or reward frontline service providers. In this step of the accountability chain, policy makers function as the principles to demand that their agents—namely service delivery organizations, such as ministries of health and education—are actually taking action to improve services. In order for this premise/channel to be true—we would expect to see higher levels of OGD lead to improved service delivery, irrespective of “client power” and formal political accountability channels. In their place, we would expect to see a significant and positive association with the “Public Management” condition described above. This premise would be falsified if either there was an inverse relationship between levels of OGD and service delivery, or the control variable operationalizing the “Public Management” condition was not significant. Figure 1: Theoretical Framework of Key Premises: 113 OGD Institute, “Supporting Sustainable Development with OGD” (2015). 21 IV. RESEARCH DESIGN AND METHODS With respect to the statistical model, this research utilizes ordinary least squares (OLS) multivariate regression equations to capture the associations between the variables 114 (see Annex 2 for a listing of all regression specifications). As such, in premise 1, the key independent variable is OGD and the regression specification aims to test its relationship on accountability, controlling for the Publicity condition, Political Agency condition, and log GDP. In premise 2, the key independent variable is accountability and the regression specification aims to test its relationship on service delivery—including both provision and outcomes—controlling for the Public Management condition and log GDP. Building on these first two regressions, in premise 3, the key independent variable is OGD and the regression specification aims to test its relationship on service delivery—including both provision and outcomes—controlling for the intermediating effects of accountability as well as log GDP. Finally, in premise 4, the key independent variable is OGD and the regression specification aims to test its relationship on service delivery—including both provision and outcomes—controlling for the Public Management condition, the Client Power condition, and log GDP. Sample Selection Based on the most recently available data, the sample will include 25 countries in Sub-Saharan Africa 115 over a four-year period, 2013-2016. As such, the unit of analysis is country by year (e.g. Ghana 2014, Tanzania 2016, etc.) for a total of 86 data points. 116 These countries were chosen because they are at reasonably similar levels of human development and per capita income, which allows for good comparability between cases. 117 Likewise, the country sample presents relatively good variation with respect to the key variables over the period, allowing the research to better isolate key associations. Finally, the country selection will allow comparison at the sub-regional level, as key differences may emerge between the southern/eastern cluster versus the western cluster of African countries. Primary Variable/Indicator Construction To perform these regressions, this research will rely on a customized data set that operationalizes the key variables, namely OGD, accountability, and service delivery provision and outcomes. All variables have indices that can be used to compare countries across the Sub-Saharan Africa region with respect to the levels of OGD and the resulting levels of accountability and service delivery. To address potential endogeneity issues, the direction of the relationship will be strengthened by the sequencing of data gathered, which would imply that OGD changes occur prior to changes in accountability and service delivery. 118 • Open Government Data: This index will draw upon the Open Data Barometer (ODB), produced on an annual basis by the World Wide Web Foundation, which scores countries on the openness of their data. In particular, the ODB analyzes global trends, and provides comparative data on countries and 114 In the future, more robust statistical techniques may be employed to the extent that data availability provides the necessary degrees of freedom to perform such analyses. The current limitations to doing so are discussed subsequently in the paper. 115 Twenty-five countries are included in the study, including: Benin, Botswana, Burkina Faso, Cameroon, Côte d’Ivoire, Democratic Republic of Congo, Ethiopia, Ghana, Kenya, Malawi, Mali, Mauritius, Mozambique, Namibia, Nigeria, Rwanda, Senegal, Sierra Leone, South Africa, Swaziland, Tanzania, Togo, Uganda, Zambia, and Zimbabwe. 116 While this number of data points allows for sufficiently significant and robust results, the n-size will be subsequently increased as more data become available in mid-/late-2019. 117 For instance, comparing countries in Sub-Saharan Africa with OECD countries, Latin America, or East Asia would not be as instructive as the latter are much closer to the frontier in key service delivery areas and would not have the same variation. 118 Fortunately, the data available support this method, since annual OGD evaluations are released in January of each year, and accountability and service delivery indicators are released in October, thus allowing a nine-month lag. 22 regions using an in-depth methodology that combines contextual data, technical assessments, and secondary indicators. 119 For each year, a country is given an overall data set score (1–100) for the completeness of OGD data sets based on a standardized rubric 120 across a number of areas, including: maps, land ownership, census, government budget, government spending, company registration, legislation, public transport, international trade, health, education, crime statistics, environment statistics, election results, and contracts. For the purposes of the present analysis, raw scores provided by the ODB were slightly remixed to obtain a certain operationalization of the variables based on the theoretical literature and conceptual framework. 121 • Accountability: Since the research will focus on Sub-Saharan Africa, accountability data were obtained from the Mo Ibrahim Index of African Governance (IIAG), which has rated accountability in virtually every African country since 2001. The data set relies on an accountability index based on an inclusive list of sub-indicators, including those listed Annex 3. As noted in the theoretical literature, there are many definitions of accountability and there are likewise many different measures of what constitutes accountability across different international institutions’ indices. Accordingly, such an indexed variable—which draws on many of the principle accountability indexes including those produced by the World Bank, African Development Bank, Economist Intelligence Unit, Bertelsmann, the World Economic Forum, and others—ensures that the concept is broadly operationalized. 122 • Service Delivery: Measuring service delivery can be difficult, as it has many different dimensions including access, quality, timeliness, price, efficacy, and satisfaction. For the sake of simplicity, service delivery will be operationalized in two dimensions: (i) provision of services and (ii) outcomes of services. The rationale for this is that provision (e.g. number of teachers in a district, number of books purchased) may be more easily impacted by improvements in OGD as opposed to outcomes (e.g. primary school completion, literacy, etc.). This index will likewise draw upon the IIAG, which rates education and health at the aggregate level. Based on a review of the sub-indicators, the IIAG aggregate indicators were remixed in order to better parse out provision and outcomes in both the health and education sectors. Annex 2 provides a listing of all the sub-indicators used. 119 See https://opendatabarometer.org/barometer/ 120 For instance, these indicators measure the existence and extent of OGD based on the following questions, for which countries receive a score between 1–100: Do the data exist? Are data available online from government in any form? Is the data set provided in machine-readable formats? Are the machine-readable data available in bulk? Is the data set available free of charge? Are the data openly licensed? Is the data set up to date? 121 In the regression specification related to the relationship between OGD on government accountability, the Open Data Barometer Raw Scores were used for the Overall OGD Index, which measure the extent to which data are open on a de facto basis across more than a dozen categories. This figure averages the performance across all data categories on a yearly basis in order to provide an aggregate snapshot of the openness of the data environment. These broad scores were used since they best reflect the actual situation on the ground, and have the most direct effect on accountability, irrespective of the different dimensions of OGD that they may cover. These include the aggregate score for data on maps, land ownership, census, government budget, government spending, company registration, legislation, public transport, international trade, health, education, crime statistics, environment statistics, election results, and contracts. Conversely, in the regression specification related to the relationship between OGD and service delivery, the Open Data Barometer Raw Scores were used only in selected data dimensions. Based on the literature, for governments to make informed policy, budgetary, and service delivery decisions based on data, data on sector performance are needed as well as data on budget allocation and execution. To be sure, the results would be skewed if for instance a country had a high degree of public transport data, but a low level of health sector data—and the aggregated levels of data openness were used. Accordingly, this analysis re-constructs the OGD variables as follows: Health OGD: quality of health sector data (50% weight), budget data (25% weight), spending data (25% weight). OGD related to the health sector typically includes data on Mortality and survival rates; Levels of vaccination; Levels of access to health care; Health care outcomes for particular groups; Patient satisfaction with health services; Waiting times for medical treatment; and Spend per admission. Education OGD: quality of education sector data (50% weight), budget data (25% weight), spending data (25% weight). OGD related to the education sector typically includes data on Test scores for pupils in national examinations (not only rates of approvals); School attendance rates; and Teacher attendance rates. 122 Importantly, in order to reduce the chance for a distorted correlation between OGD and accountability, three sub-indicators that are particularly endogenous were removed and the index was recalculated (see annex 2 for details).. 23 • Control and Intervening Variables: Based on the literature, and as highlighted in the conceptual framework above, it is evident that a number of control/intermediating variables should be included, which can affect the relationships between the key independent and dependent variables. While many additional factors can and should be considered, the statistical analysis will only operationalize the most obvious enabling factors, including the following: o Publicity Condition: This index relies on the IIAG sub-index for “Rights” which averages scores for freedom of expression, media freedom, media impartiality, censorship, freedom of association and assembly, trade unions, civil liberties, human rights, and the like. o Political Agency Condition: This index relies on the IIAG sub-index for “Participation” which averages scores for political participation, freedom of political parties, CSO/NGO participation, free and fair elections, election monitoring, legitimacy of the political process, and the like. o Public Management Condition: This index relies on the IIAG sub-index for “Public Management” which averages scores for public administration capacity, statistical capacity, budget management, fiscal policy, revenue collection, HR management, and the like. o Client Power Condition: This research proposes a customized index that relies on a variety of sub-indicators related to CSO/NGO freedom, IT literacy, mobile phone and internet penetration, media censorship, freedom of association and assembly, and access to information. o Log Per Capita GDP: In order to remove the income effects between countries, as well as to control for inflationary effects, this research uses log per capita GDP harmonized to 2011. V. STATISTICAL FINDINGS Using the multi-year data set created, each premise in the conceptual framework was tested in order to estimate associations between the principle independent variable (OGD) and the respective dependent variables (accountability and service delivery). In addition to these aggregates, key relationships were tested with respect to variables contained in the sub-indices of accountability, service provision, and service delivery outcomes, which may be of interest on their own—outside the hypothesized relationships noted in the conceptual framework. Finally, the four enabling “conditions,”—publicity, political agency, public management, and client power—were tested in order to provide important clues regarding the necessary and sufficient conditions in each relationship.  Premise 1: Higher levels of OGD make government more accountable. To test the first premise, the aggregate accountability index is regressed on the overall average score of OGD implementation (across all dimensions of OGD). First, a positive and statistically significant correlation is found between the indexes: for a 1-point increase in OGD, accountability is expected to increase by 0.375 points in the sample of countries surveyed. Second, when controlling for the Political Agency condition index and the Publicity condition index, the association between OGD and accountability remain significant in the case of the Political Agency condition and marginally significant 123 in the case of the Publicity condition, suggesting that all three variables are meaningful both independently and when taken together. As expected, higher levels of per capita GDP are associated with higher levels of accountability. 123Although the OGD coefficient in column 3 is not significant at the 90% confidence interval, the coefficient value was not drastically affected and remains approximately twice the standard error, suggesting that this relationship remains at least marginally significant. Moreover, it is likely that some of the significance was lost given that the broader OGD enabling environment is captured by the sub-indices used to operationalize the Publicity condition (see annex 2 for details). 24 (1) (2) (3) Accountability Accountability Accountability OGD Average 0.375** 0.336* 0.289 (0.156) (0.171) (0.180) Political Agency 0.203** (0.0770) Publicity 0.294*** (0.110) Log GDP 9.778*** 7.213*** 7.809*** (1.437) (1.420) (1.626) _cons -35.19*** -26.13*** -33.41*** (10.66) (9.792) (11.40) N 86 86 86 R2 0.421 0.469 0.486 In addition to the overall accountability index, positive and significant correlations are found with respect to other accountability sub-indices, including those produced by the World Bank and the African Development Bank—thus adding to the validity of the purported relationship. In addition, greater levels of OGD are positively associated with with less abuse of office (Bertlesmann) as well as less diversion of public funds (WEF). However, OGD does not seem to have a significant effect on popular corruption indexes, including Executive Corruption (VDem), Corruption in Public Officials (EIU), and Corruption in Bureaucracy (WB), which is a counterintuitive finding in the context of the present research.  Premise 2: More accountable governments enable improved service delivery. To test the second premise, education and health service delivery—in terms of both provision and outcomes—are regressed on the accountability index. First, in the education sector, no significant effect of accountability is found on education service delivery in terms of provision or outcomes. 124 Second, when controlling for the public management condition—which again, is the index used to measure administrative capacity—no significant association is found with respect to either education provision or outcomes. Finally, it should be noted that the coefficient for accountability becomes marginally negative and significant when controlling for public management, which is an unexpected result given that higher levels of public administration capacity would be expected to correlate with improved education outcomes. (1) (2) (3) (4) Education Provision Education Outcomes Education Provision Education Outcomes Accountability 0.0658 -0.140 0.162 -0.205* (0.115) (0.0915) (0.144) (0.111) Public Mgmt. -0.232 0.157 (0.194) (0.150) Log GDP 8.975*** 17.12*** 9.429*** 16.82*** (1.952) (1.447) (1.872) (1.487) _cons -19.36 -82.99*** -15.54 -85.58*** (11.70) (8.614) (12.84) (8.452) N 86 86 86 86 R2 0.428 0.741 0.437 0.743 124As explained subsequently, this is not to suggest that accountability has no import in the education sector; rather, given the particular political economy of how education services are produced and consumed, formal accountability and social accountability (Client Power) mechanisms may not translate in the same ways as expected in other service delivery sectors. 25 However, when the same regression is performed in the health sector, different associations emerge. First, a significant correlation of accountability is found with respect to health provision and outcomes, whereby a 1-point increase in the accountability index, is correlated with an increase in the health provision index by 0.406 points and the health outcomes index by 0.167 points for the average country in the sample. In these specifications, the effect of accountability on health service provision is larger and more significant than on outcomes, since outcomes may take more time to materialize. Second, when controlling for the public management condition, accountability remains significant for health provision, suggesting that it is a driving factor in this relationship. Conversely, for health outcomes, the opposite is true: accountability loses its significance and public management shows a positive and significant effect (0.259), suggesting that public management might be a more relevant driving factor than accountability for health outcomes. (1) (2) (3) (4) Health Provision Health Outcomes Health Provision Health Outcomes Accountability 0.406*** 0.167** 0.423** 0.0599 (0.148) (0.0668) (0.174) (0.0851) Public Mgmt. -0.0415 0.259* (0.220) (0.132) Log GDP -0.271 3.196** -0.190 2.689** (2.310) (1.304) (2.311) (1.290) _cons 57.37*** 45.64*** 58.05*** 41.37*** (12.95) (8.727) (14.04) (9.044) N 86 86 86 86 R2 0.233 0.288 0.233 0.312  Premise 3: Higher levels of OGD make governments more accountable, which subsequently enables improved service delivery. To test the third premise, service provision and outcomes in both the health and education sectors are regressed on sector-specific OGD, while studying the interaction effects of the accountability index. First, in the regression specification for education service delivery, OGD is estimated to be positively and significantly correlated with education provision and outcomes indices. Second, when controlling for the levels of accountability, the estimated coefficients for OGD on both provision and outcomes remains almost unchanged, suggesting that the link between OGD and education service delivery is independent from levels of accountability at the country level. As mentioned above, the regressions likewise find that the effect of accountability on education provision is not significant, while the effect on outcomes is marginally negative (which is an unexpected result). Finally, no interaction effects of accountability are found with respect to the relationship between OGD and education service delivery (not shown). (1) (2) (3) (4) Education Provision Education Outcomes Education Provision Education Outcomes OGD Education 0.201*** 0.171*** 0.199*** 0.184*** (0.0645) (0.0426) (0.0646) (0.0420) Accountability 0.0376 -0.167* (0.114) (0.0913) Log GDP 9.141*** 15.20*** 8.754*** 16.92*** (1.198) (0.874) (1.842) (1.370) _cons -21.62** -78.07*** -20.31* -83.87*** (9.568) (7.391) (11.02) (8.091) N 86 86 86 86 R2 0.487 0.759 0.488 0.775 26 In the regression specification for health service delivery and OGD, slightly differentiated relationships between variables emerge. First, when disaggregating between provision and outcomes, OGD is positively and significantly correlated with health outcomes but not provision. Second, when controlling for accountability, the relationship between OGD and health provision remains insignificant and the relationship between OGD and health outcomes remains significant. This would imply that in terms of health outcomes, levels of accountability and OGD can independently have an important effect, whereas accountability—and not OGD—is an important factor for provision. Finally, the interaction effects of accountability are tested with respect to OGD and health, and no significant effect is found (not shown). (1) (2) (3) (4) Health Provision Health Outcomes Health Provision Health Outcomes OGD Health 0.127 0.109** 0.0627 0.0859* (0.0861) (0.0448) (0.0667) (0.0451) Accountability 0.389** 0.143** (0.149) (0.0699) Log GDP 4.130*** 5.068*** -0.0216 3.539*** (1.540) (0.993) (2.328) (1.326) _cons 39.61*** 36.70*** 54.97*** 42.36*** (11.77) (8.046) (13.29) (8.910) N 86 86 86 86 R2 0.105 0.274 0.239 0.309  Premise 4: Higher levels of OGD directly enable improved service delivery, excluding the effects of accountability. To test the final premise, service provision and outcomes are regressed against OGD, omitting the role of accountability as suggested above with respect to the sectoral differentiation, but testing alternative control variables such as the Public Management condition and the Client Power condition. First, in the education sector, OGD is estimated to have a stronger correlation with education provision than outcomes, which may reflect the time needed for policy to translate into results; however, it is worth highlighting that both relationships are significant. Second, no evidence supports the claim that either Public Management or Client Power has relevance with education provision or outcomes. Finally, even controlling for these conditions, the relationship between OGD and service delivery in the education sector does not change in a meaningful way with respect to the coefficients. (1) (2) (3) (4) (5) (6) Education Education Education Education Education Education Provision Outcomes Provision Outcomes Provision Outcomes ODG Education 0.201*** 0.171*** 0.217*** 0.181*** 0.213*** 0.183*** (0.0645) (0.0426) (0.0661) (0.0436) (0.0671) (0.0442) Client Power -0.130** -0.083 (0.0476) (0.0679) Public Mgmt. -0.136 -0.148 (0.169) (0.134) Log GDP 9.141*** 15.20*** 10.37*** 15.99*** 9.973*** 16.10*** (1.198) (0.874) (1.383) (1.063) (1.586) (1.232) _cons -21.62** -78.07*** -24.66*** -80.03*** -21.34** -77.76*** (9.568) (7.391) (10.003) (7.491) (9.396) (7.632) N 86 86 86 86 86 86 R2 0.487 0.759 0.505 0.764 0.493 0.764 27 Drilling down deeper, strong associations are found between OGD and certain aspects of education service delivery. As mentioned, when education service delivery is regressed on the OGD education index, a positive and significant association is found in terms of provision. Importantly, the positive and significant correlation between education provision and OGD is strongly driven by indicators linked to quality and management of education provision. In terms of outcomes, when education service delivery outcomes are regressed on the OGD education index, a positive and significant association is found in terms of certain outcomes as well. For instance, the association between OGD and education completion is only significant for primary levels of education, whereas it is insignificant for secondary and tertiary completion. In the health sector, OGD displays associations which are heterogeneous. First, OGD is found to be positively associated with health outcomes, but there is not such statistical evidence of an association with health provision. Second, with respect to health provision, Public Management capacity appears to be a more important determining factor than OGD (see column 5). Likewise, when controlling for the public management condition, the OGD coefficient linked to health outcomes remains significant, but slightly declines, suggesting that the impact of OGD on health outcomes (see column 6) is partly driven by changes in the index of Public Management, although both are independently significant. Finally, when controlling for the client power condition, this relationship remains unaltered, suggesting that this condition has little impact on the relationship between OGD and health service delivery in the countries included. In summary, it can be shown that overall, public management plays a role in health provision and outcomes, while OGD seems to further contribute to the outcome channel but not provision. (1) (2) (3) (4) (5) (6) Health Health Health Health Health Health Outcomes Provision Outcomes Provision Outcomes Provision OGD Health 0.127 0.109** 0.136 0.105** 0.0877 0.0809* (0.0861) (0.0448) (0.0852) (0.0465) (0.0762) (0.0447) Client Power -0.0960 0.0412 (0.0660) (0.0049) Public Mgmt. 0.394* 0.289*** (0.229) (0.109) Log GDP 4.130*** 5.068*** 5.079*** 4.661*** 1.603 3.212** (1.540) (0.993) (1.785) (1.059) (2.242) (1.220) _cons 39.61*** 36.70*** 37.08*** 37.79*** 39.91*** 36.92*** (11.77) (8.046) (12.408) (8.075) (11.96) (7.782) N 86 86 86 86 86 86 R2 0.105 0.274 0.116 0.278 0.155 0.327 At a disaggregated level, a strong association is found between OGD and certain aspects of health service delivery. As seen in the previous table, OGD and health provision are not significantly correlated; however, there is a significant association between OGD and sub-indexes related to immunizations, including DPT and Hepatitis B. No correlations exist for other health provision indicators including measles immunization and provision of anti-retroviral treatments. With respect to specific outcome indictors, it should be noted that the OGD index is significantly correlated with the index of health outcomes, as well as with child mortality outcomes and WHO disease prevention statistics. 28 VI. CONCLUSIONS Principles and Generalizations Inferred from the Results With respect to the four original premises put forth above, and the various regression analyses carried out, a number of generalizations can be drawn with respect the effects of OGD. Importantly, while certain relationships may hold in each of the premises analyzed, when taken together, larger conclusions may be drawn with respect to relationships between variables. Accordingly, these initial conclusions can help to bolster the empirical evidence base for the various public sector theories advocated in the literature as well as to highlight areas for future research on the topic.  Premise 1: Higher levels of OGD make government more accountable. With respect to the first premise, this analysis confirms a strong positive association between OGD and accountability in the countries surveyed over the period. When controlling for other contextual factors such as the levels of political freedoms, civil rights, and access to information (e.g. publicity condition) as well as political accountability, including the presence of formal electoral mechanisms (e.g. political agency condition), this analysis likewise finds these factors significant, independent of OGD. This suggests that all three factors can be considered important elements in improving government accountability, in line with the premised theory. As such, while OGD is necessary for improved accountability, it is not sufficient, as publicity and political agency must also be included—thus confirming the earlier theoretical and empirical contributions. Even when using other accountability indexes—for instance, related accountability indices prepared by the World Bank and African Development Bank—these relationships hold, thus reinforcing the validity of this claim. Figure 2: Effects of OGD on Accountability 29  Premise 2: More accountable governments enable improved service delivery. With respect to the second premise, this analysis provides evidence that accountability—in particular, those governments exhibiting higher levels of accountability—correlates with improved service delivery; however, there are differentiated effects across education provision and outcomes and health provision and outcomes. This is an important finding in and of itself, as it highlights the fact that sectoral specificities between the health and education sectors—in terms of how decisions are made, how services are rendered, and how clients consume these services—have a strong determining effect. Another important specification highlighted by the research is the importance of differentiating between service provision, which may be more readily governed by public policy decisions and implementation, versus service outcomes, which may be influenced by a myriad of other contextual factors and take years to materialize. To be sure, in the education sector, this analysis finds no correlation between levels of accountability and education provision and outcomes. Even when controlling for levels of public sector management capacity, which is an important tool for governments to translate their accountability structures and practices into improvements in service delivery, this initial analysis finds no association. Returning to the literature, there can be a number of important explanations for this seemingly counterintuitive finding. For instance, in the education sector in particular, there are a number of sectoral-level political economy factors, which may confound the impacts of accountability and public management capacity. These include, among others: (i) the potential for excludability, which affects the power relationships between those who demand access to education (e.g. students, parents, etc.) and those who control access to those services (e.g. school administrators); (ii) principle-agent asymmetries, whereby parents, as agents, make decisions for their children, who are the “rightful” principals of the service; (iii) the fact that education is a “merit” good, whereby its consumption is compulsory for most clients (e.g. students themselves); (iv) education outcomes often require “co-production,” which means that frontline providers and recipients (e.g. students and parents) both need to invest their time in achieving outcomes; and (v) issues related to information asymmetries and visibility, whereby it is difficult to measure the quality of education inputs (e.g. curriculum and instructional quality), which can have knock-on effects with respect to the political salience of policy decisions. 125 All of these—and other—sectoral level dynamics may explain why a positive association with accountability or public management in the education sector is not apparent in the sample selection of countries and years studied. However, this finding should not be interpreted to suggest that government accountability is not an important determining factor in the delivery of education more broadly, as accountable institutions remain sine qua non for ensuring efficacy and efficiency of the public sector. Conversely, in the health sector, public accountability appears to have a significant association with service provision and outcomes. However, when controlling for levels of public sector management capacity, which is an equally important determinant in how services are delivered, the results become more nuanced. In particular, it would appear that accountability is a more important condition for health provision, whereas public management capacity seems to absorb the effect that accountability has with respect to health outcomes. These differentiated effects between health provision and outcomes may likewise have some grounding in the literature related to sector-level political economy. For instance, a number of characteristics make health service delivery significantly more “murky” than other sectors, including, among others: (i) the difference between preventative and curative services, the latter of which may dominate, as many health services are consumed on an “emergency” basis and do not involve continual monitoring; (ii) the presence of information asymmetries whereby patients do not have the same knowledge 125 While outside the scope of the present research, the determinants of service delivery in the education sector are explored in detail by Dan Harris, Richard Batley, Claire Mcloughline, and Joseph Wales, in “The Technical is Political: Understanding the Political Implications of sector Characteristics for Education Service Delivery” (ODI, 2013). 30 of the quality of service inputs as health care providers; (iii) the high level of individual discretion in providing health services, whereby providers are required to make complex decisions, which are difficult to monitor; and (iv) the professional dominance of health providers, who are able to better politically organize in ways that allow them to resist strict government control and regulation. 126 These—and other— sectoral characteristics may go a long way in providing explanations as to why health provision is more associated with public management capacity and accountability, whereas health outcomes have a strong linkage with public management but not accountability. While the intricacies of these relationships are outside the scope of this paper, these findings indicate a need for future qualitative political economy analysis at the sector level.  Premise 3: Higher levels of OGD make governments more accountable, which subsequently enables improved service delivery. With respect to the third premise, this analysis provides evidence that accountability can play an important role in translating OGD into improved service delivery, but again, there are important distinctions between education provision and outcomes and health provision and outcomes. As mentioned above, the principle- agent relationships of accountability in the education sector may differ significantly from those in the health sector, especially where responsibilities are delegated to head of household as opposed to the eventual consumers of these services. Likewise, the nature of service delivery can have important determinants regarding how accountability affects provision and outcomes, given for instance differences between how access, quality, and oversight may translate into achievements in the education sector and how preventative and curative services may alter monitoring, oversight, and compliance in the health sector. In the education sector, the findings suggest that OGD is positively and significantly correlated with both provision and outcomes. Importantly, these relationships maintain their significance when controlling for accountability, suggesting that the link between OGD and education service delivery is independent from levels of accountability at the country level. Returning to the finding noted above, accountability does not appear to be a significant condition in intermediating the relationship between OGD and education service delivery, which suggests a direct effect of OGD on provision and outcomes. Concretely, this means that the present analysis finds no evidence of the “long-route” to service delivery in the education sector, whereby OGD fosters increased accountability, which then leads to improved service delivery. 127 Rather, this may mean that policy makers can use OGD simply as a management tool—outside a democratic accountability process—to make top-down policy decisions that affect service delivery in education in the countries concerned. Alternatively, this may mean that citizens are able to directly influence service delivery from front line providers using social (non-formal) accountability mechanisms, including collective engagement through parent-teacher committees, participatory budgeting and oversight exercises, or direct citizen pressure on service providers such as petitions and protests. Both of these alternate “short- route” hypotheses are tested subsequently under the fourth premise. 126Likewise, the determinants of service delivery in the health sector are explored in detail by Dan Harris, Richard Batley, and Joseph Wales, in “The Technical is Political: What does this Mean in the Health Sector” (ODI, 2014). 127 Again, this result is not to suggest that accountability has no role to play in education service delivery; rather, that in the countries and years analyzed, it did not bear a statistically significant relationship. 31 Figure 3: Effects of OGD on Education Provision and Outcomes Conversely, in the health sector, OGD is not associated with improved service provision, but it is significantly correlated with improved outcomes. When controlling for the effects of accountability on this relationship, the analysis finds that it has an independent relationship with respect to service provision but works in tandem with OGD with respect to health outcomes. This would therefore suggest that there may be evidence of the “long-route” to service delivery in the health sector, whereby OGD fosters increased accountability, which then leads to improved service delivery—with the caveat that “service delivery” is conceptualized on the basis of outcomes, not provision. Again, the reasons for these differentiated effects are likely vast and varied and would likewise require a level of qualitative analysis outside of this initial statistical estimation. Future research on this topic should thus work to parse out the sector-level details to provide more adequate explanations supporting these associations. Figure 4: Effects of OGD on Health Provision 128 and Outcomes 32  Premise 4: Higher levels of OGD directly enable improved service delivery, excluding the effects of accountability. With respect to the fourth premise, this analysis finds differentiated effects of public sector management capacity and client power (e.g. social accountability) mechanisms. Again, these dynamics may be more largely related to the differences in how services are planned, executed, and evaluated in the sectors concerned. These differences are also linked to particular aspects with respect to provision, which may be more easily controlled by public policy and execution than outcomes, which represent the long-term effects of service delivery policies and implementation. In the education sector, additional testing of other enabling conditions indicates that neither public management capacity nor client power (social accountability) affects the relationship between OGD and education service delivery. OGD is found to have a more positive correlation with education provision than outcomes (although both are significant), which may reflect the time needed for policy to translate into results. For their part, client power and public management capacity appear to have little influence on these results, suggesting that some key explanatory gaps remain in the education sector, which may have several interpretations. First, in the absence of any additional control or intervening variables, it may suggest a direct relationship between OGD and education service delivery, meaning that OGD is an important determinant in the education sector. Conversely, it may suggest other lurking variables, since education provision and outcomes are likely to be intimately intertwined with other contextual factors such as family income, access disparities between urban and rural areas, as well as other cross sectoral issues such as infrastructure, nutrition, as well as conflict and gender dynamics. Finally, it may suggest that certain political economy dynamics, including rent seeking, clientelism, patronage, and other forms of policy capture, which as noted above, can distort principle-agent relationships between citizens and public officials. 129Again, while the present analysis does not attempt to answer these questions, this suggests the need for additional future quantitative and qualitative research that delves deeper into sectoral level service delivery dynamics. As noted above, in the health sector, the findings suggest a positive association between OGD and health outcomes, but not provision. Importantly, the analysis finds that part of the impact of OGD on health indicators is driven by changes in the index of public management, although not client power, which appears to have a direct and important impact on both health provision and outcomes. Accordingly, the findings point to the role that public management can play as a predictor of better health provision and outcomes, while such a relationship is not apparent in education (as was also demonstrated in Premise 2). This finding must also be taken in context with previous findings on the importance of accountability, which likewise has differentiated associations between health provision and outcomes. On the aggregate, these findings suggest that accountability and public management capacity—but not OGD—impact health provision, whereas OGD and public sector management capacity—but not accountability—impact health outcomes. Finally, as in education, client power does not seem to play a role in health provision and outcomes, which is a counterintuitive finding. Thus, future research may need to better conceptualize this variable or employ methods to better specify the effects of client power. 129To be sure, Devarajan et. al. (2011), note that "in political economy environments characterized by high degrees of clientelism and rent-seeking, such as are widespread in the Africa region, an unqualified faith in civil society as a force for good is more likely to be misplaced. [Rather] the evidence based on the organization of civil society suggests that historic institutions of poverty and inequality, or of ethnic identity, can inhibit collective action in the broader public interest, promote more narrow sectarian interests, and nourish clientelist political competition." (p. 3). Devarajan, Shantayanan, Stuti Khemani, and Michael Walton. 2011. Civil Society, Public Action and Accountability in Africa. HKS Faculty Research Working Paper Series RWP11-036, John F. Kennedy School of Government, Harvard University. 33 Limitations Although this research has considerable promise, there are a number of limitations which should be acknowledged, including most importantly that this analysis only claims to be an initial assessment of the relationship between the variables. While it would be ideal to have more years of data—and more observations included in the data set—the analysis uses data that are currently available, as OGD only started to be measured in 2012/2013. In this regard, OLS regressions are instructive in teasing out initial associations between variables; however, as more data become available, increasingly more sophisticated estimations can be performed in the future. For instance, it would be ideal to include a country fixed-effects estimation as it would demonstrate that the results are not driven by omitted variable bias, which are country-specific and do not change over time. Unfortunately, including country fixed-effects estimation at this point in time would lead to a significant loss of degrees of freedom to estimate the parameters of interest. 130 As such, there is an inherent trade-off between unbiasedness/consistency that would be provided by the fixed effects estimation versus being able to detect parameters using this initial OLS estimation. Second, given the fact that OGD is a rather recent phenomenon, it is difficult to suggest a purely causal relationship between variables. While a lag of nine months has been introduced between the availability of OGD statistics and changes in accountability and service provision, it is likely that these changes may take more time to materialize. Accordingly, the present analysis only purports to provide evidence of associations between variables, to help support or refute existing theoretical propositions. Relatedly, challenges exist with certain indicators and it is necessary to ensure that there are no significant issues of endogeneity and simultaneous causality. For instance, more accountable governments may be more inclined to support an OGD initiative in the first place, which would reverse the causal relationship. As such, it is difficult to argue at this early stage whether OGD proceeds accountability or accountability proceeds OGD. Further quantitative and qualitative methods will need to be employed (e.g. process tracing, case study analysis, and qualitative comparative analysis) to establish a higher degree of causal certainty in the future. Finally, there may be difficulties extrapolating the relationships demonstrated in the Sub-Saharan context to other countries, particularly upper-middle-income and high-income countries. To be sure, the analysis finds the coefficient relating OGD and accountability to be of a significant magnitude, which is due primarily to the low levels of OGD prevalent in the Sub-Saharan African region. For this reason, one cannot expect the same gains from OGD in countries with already high levels of accountably and service delivery as would be possible in country contexts with relatively low starting points. 131 This is precisely the reason why the present study omits other regions and focuses on a sample of countries in Sub-Saharan Africa, a place where OGD has the greatest potential for improving accountability and service delivery. 132 Recommendations and Future Research Given that this paper seeks to present an initial assessment of OGD, accountability, service delivery, as well as some of the other relevant “conditions” as gleaned from the theoretical literature, it is perhaps too early to put forth any substantive recommendations on what needs to be done to better enable OGD to reach its full potential. In its place, the initial findings presented here suggest a number of streams of future research that need to be pursued, including, inter alia: (i) deeper sectoral level analysis in education and health to better understand key provision and outcome dynamics as well as their linkages to accountability 130 N.B. Random effects would yield similar coefficients to those obtained with OLS, but with more precise estimation of standard errors. However, they may similarly be subject to omitted variable bias as the OLS. 131 The logic behind this stems from the notion of marginally decreasing returns to investment, whereby investment in low-income countries yields higher returns than in more advanced economies due to the scarcity of capital; however, these initial returns quickly decline with higher levels of investment. 132 Overall OGD for these countries is very small (the average level of OGD is approximately 16 points on a scale of 100), so it would not be advisable to extrapolate the findings of this study to countries with higher levels of development such as Norway. 34 relationships; (ii) more robust understanding of relationships that do not manifest themselves statistically— such as the client power condition and related social accountability mechanisms—which ostensibly have a key role in the impact of OGD; (iii) deeper country-level analysis using qualitative methods to establish typologies of country behavior, which can help to provide key explanations for the performance of OGD, accountability, and service delivery across different contexts; (iv) more research and recommendations for sub-national OGD, which is where most front-line service delivery takes places (e.g. hospitals, clinics, schools, etc.); (v) more granular analysis of the end users of OGD as well as the outputs of OGD use (e.g. dashboards, visualizations, journalism, etc.); and finally, as more years of data become available, (vi) the introduction of more robust statistical methods (e.g. fixed-effects models) as well as more sophisticated qualitative methods (e.g. process tracing and qualitative comparative analysis) in order to strengthen the causal links between OGD, accountability, and service delivery. 35 Annex 1: Regression Specifications  Premise 1: Higher Levels of Open Data Availability Make Government More Accountable = + 1 + 2 log + 3 + 4 +  Premise 2: More Accountable Governments Enable Improved Service Delivery . = + 1 + 2 log + 3 + . = + 1 + 2 log + 3 + ℎ = + 1 + 2 log + 3 + ℎ = + 1 + 2 log + 3 +  Premise 3: Higher Levels of Open Data Availability Enables Improved Service Delivery, Taking into Account the Effects of Political Accountability . = + 1 + 2 log + 3 + 4 ∗ + . = + 1 + 2 log + 3 + 4 ∗ + ℎ = + 1 ℎℎ + 2 log + 3 + 4 ℎℎ ∗ + ℎ = + 1 ℎℎ + 2 log + 3 + 4 ℎℎ ∗ +  Premise 4: Higher Levels of Open Data Availability Enable Improved Service Delivery, Excluding the Effects of Political Accountability . = + 1 + 2 log + 3 + 4 + . = + 1 2 + log + 3 + 4 + ℎ = + 1 ℎℎ + 2 log + 3 + 4 + ℎ = + 1 ℎℎ + 2 log + 3 + 4 + 36 Annex 2: Indexed Indicators Accountability Index (including indicator sources) Adjusted Accountability Index • Accountability, Transparency & Corruption in the Public Sector (AfDB/WB) • Accountability, Transparency & Corruption in the Public Sector (AfDB) • Accountability, Transparency & Corruption in the Public Sector (WB) • Corruption & Bureaucracy (WB) • Corruption in Government & Public Officials (EIU) • Diversion of Public Funds (WEF) • Accountability of Public Officials (EIU) • Public Sector Corruption Investigation (GI) • Public Sector Corruption Bodies (GI) • Corruption Investigation (GI) • Prosecution of Abuse of Office (BS) • Accountability, Transparency & Corruption in Rural Areas (IFAD) Sub-Indicators removed for endogeneity • Access to Information (GI) • Public Information (GI) • Access to Legislative Processes & Documents (GI) Source: Mo Ibrahim Index of African Governance Indicators, 2017 Service Delivery Indices (including indicator sources) Education Service Delivery Health Service Delivery Overall Education Index Overall Health Index (provision + outcomes) (provision + outcomes) Education Provision Sub-Index Health Provision Sub-Index • Education Provision & Quality (BS) • Basic Health Services (AFR) • Education System Quality (WEF) • Public Health Campaigns (GI) • Ratio of Pupils to Teachers in Primary School • Immunization (WB/WHO) (UNESCO) • Immunization against Measles (WB) • Immunization against DPT (WB) Education Outcomes Sub-Index • Immunization against Hepatitis B (WHO) • Primary School Completion (WB) • Antiretroviral Treatment (ART) • Secondary School Enrolment (UNESCO) • Provision (UNAIDS) • Tertiary Education Enrolment (UNESCO) • ART Provision (UNAIDS) • Literacy (UNESCO) • ART Provision for Pregnant Women (UNAIDS) • Access to Sanitation (WHO/UNICEF) • Access to Improved Sanitation (WHO/UNICEF) Health Outcomes Sub-Index • Child Mortality (IGME) • Maternal Mortality (MMEIG) • Undernourishment (WB) • Disease (WHO) Malaria (WHO) • Tuberculosis (WHO) • Open Defecation Sanitation (WHO/UNICEF) Source: Mo Ibrahim Index of African Governance Indicators, 2017 37 Conditional Indicators Publicity Condition Political Agency Condition • Freedom of Expression (BS/RSF/V-Dem/GI) • Political Participation (EIU/FH/V-Dem) • Freedom of Expression (BS) • Political Participation (EIU) • Media Freedom (RSF) • Political Pluralism (FH) • Media Impartiality (V-Dem) • Freedom of Political Parties (V-Dem) • Freedom of Expression (V-Dem) • Civil Society Participation (BS/V-Dem/GI) • Censorship (GI) • Civil Society Participation (BS) • Media Censorship (GI) • Civil Society Inclusion (V-Dem) • Online Censorship (GI) • Freedom of NGOs (GI) • Freedom of Association & Assembly (BS/GI) • Barriers to NGO Operations (GI) • Freedom of Association & Assembly (BS) • Persecution of NGOs (GI) • Freedom of Association (GI) • Harassment of NGOs (GI) • Freedom of Association (GI) • Free & Fair Elections (BS/CDD/V-Dem) • Trade Unions (GI) • Free & Fair Elections (BS) • Civil Liberties (BS/FH) • Free & Fair Executive Elections (CDD) • Protection of Civil Liberties (BS) • Free & Fair Elections (V-Dem) • Civil Liberties (FH) • Election Monitoring Agencies (V-Dem/GI) • Human Rights Conventions (UNOLA/OHCHR) • Election Management Bodies (V-Dem) • Human Rights Violations (EIU) • Election Monitoring Agencies (GI) • Protection against Discrimination (GI) • Election Monitoring Agency Independence (GI) • Protection against Ethnic Discrimination (GI) • Election Monitoring Agency Reporting (GI) • Protection against Religious Discrimination (GI) • Legitimacy of Political Process (BS) Public Management Condition Client Power Condition • Governmental Statistical Capacity (WB) • Access to Information (GI) • Civil registration (GI) • Access to Public Information (GI) • Birth Registration (GI) • Access to Legislative Information (GI) • Death Registration (GI) • Online Public Services (UNDESA) • Public Administration (AfDB/WB) • Social Unrest (EIU/ACLED) • Public Administration (AfDB) • Social Unrest (EIU) • Public Administration (WB) • Riots & Protests (ACLED) • Diversification (AfDB/OECD/UNDP) • Civil Society Participation (BS/V-Dem/GI) • Budget Management (AfDB/WB) • Civil Society Participation (BS) • Budget Management (AfDB) • Civil Society Inclusion (V-Dem) • Budget Management (WB) • Freedom of NGOs (GI) • Budget Balance (AfDB/AUC/UNECA) • Barriers to NGO Operations (GI) • Fiscal Policy (AfDB/WB) • Persecution of NGOs (GI) • Fiscal Policy (AfDB) • Harassment of NGOs (GI) • Fiscal Policy (WB) • Freedom of Expression (BS) • Revenue Mobilisation (ICTD/AfDB/WB) • Media Freedom (RSF) • Taxation Capacity (ICTD) • Media Impartiality (V-Dem) • Revenue Collection (AfDB/WB) • Freedom of Expression (V-Dem) • Revenue Collection (AfDB) • Censorship (GI) • Revenue Collection (WB) • Media Censorship (GI) • Transparency of State-owned Companies (GI) • Online Censorship (GI) • Freedom of Association & Assembly (BS/GI) • Freedom of Association & Assembly (BS) • Freedom of Association (GI) • Freedom of Association (GI) • Trade Unions (GI) • Digital & IT Infrastructure (EIU/ITU) • IT Infrastructure (EIU) • Mobile Phone Subscribers (ITU) • Household Computers (ITU) Source: Mo Ibrahim Index of African Governance Indicators, 2017 38 Annex 3: Summary Statistics Mean Median Standard Minimum Maximum Deviation OGD Overall 16.6 13.6 8.8 3.3 45.7 Score OGD 20.8 12.5 15.8 3.8 77.5 Education OGD Health 20.0 12.5 15.1 3.8 77.5 Accountability 49.4 47.9 14.6 12.3 83.4 Index Score Education 55.8 56.3 12.6 36.7 88.4 Provision Index Score Education 47.3 43.3 15.5 24.9 82.9 Outcomes Index Score Health 75.2 74.2 12.0 35.6 98.5 Provision Index Score Health 79.5 79.7 8.6 58.9 99.4 Outcomes Index Score Political 59.9 61.3 19.3 15.1 89.9 Agency Index Score Publicity 52.5 52.8 14.3 23.5 80.6 Index Score Client Power 54.7 55.4 15.6 19.7 82.7 Index Score Public 52.7 50.3 8.8 34.2 74.7 Management Index Score N =86 39 Country Observations by Year (n=86) 2013 2014 2015 2016 Benin X X X X Botswana X X X X Burkina Faso X X X X Cameroon X X X X Côte d’Ivoire X Democratic Republic of Congo X Ethiopia X X X X Ghana X X X X Kenya X X X X Malawi X X X X Mali X X X X Mauritius X X X X Mozambique X X X Namibia X X X X Nigeria X X X X Rwanda X X X X Senegal X X X X Sierra Leone X X X South Africa X X X X Swaziland X Tanzania X X X X Togo X Uganda X X X X Zambia X X X X Zimbabwe X X X X 40 Annex 4: Bibliography Articles Asunka, Joseph. “What People Want from Government: Basic Services Performance Ratings, 34 Countries,” AfroBarometer Network (2013). Awortwi, N. and Nuvunga, A. (2019) Sound of One Hand Clapping: Information Disclosure for Social and Political Action for Accountability in Extractive Governance in Mozambique, IDS Working Paper 523, Brighton: IDS Ballivian, Amparo, and William Hoffman. “Public-Private Partnerships for Data: Issues Paper for Data Revolution Consultation,” World Bank & World Economic Forum (January 15, 2015). Barr, Abigail, and Lawrence Bategeka (eds.). “Management and Motivation in Ugandan Primary Schools: An Impact Evaluation Report,” Working Paper 2012–14 (December 2012). Barr, Abigail, Frederick Mugisha, Pieter Serneels, and Andrew Zeitlin. “Information and Collective Action in Community-Based Monitoring of Schools: Field and Lab Experimental Evidence from Uganda” Working Paper (2012). Bjorkman, M., and J. Svensson. “Power to the People: Evidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda,” The Quarterly Journal of Economics (2009): 735–69. Bjorkman, M., D. de Walque, and J. Svensson. “Information Is Power: Experimental Evidence on the Long- Run Impact of Community-Based Monitoring,” Policy Research Working Paper 7015, World Bank (2014). Calland, R. “Background Paper on Freedom of Information: Review of Impact and Effectiveness of Transparency and Accountability Initiatives,” Institute of Development Studies, Brighton, UK (2010). Carothers, T., and S. Brechenmacher. “Accountability, Transparency, Participation, and Inclusion: A New Development Consensus?” Carnegie Endowment for International Peace (2014). Carruthers, Celeste K. and Marianne H. Wanamaker. 2015. "Municipal Housekeeping: The Impact of Women's Suffrage on Public Education," Journal of Human Resources 50 (4) (2015): 837-72. Collier, David. "Understanding Process Tracing," PS: Political Science and Politics 44: 4 (2011): 823–30. Dahl, Robert. “Can international organizations be democratic? A skeptic's view.” in I. Shapiro & C. Hacker- Cordón (Eds.), Democracy's Edges. Cambridge: Cambridge University Press (1999). Davies, T. “Open Data in Developing Countries: Emerging Insights from Phase 1,” World Wide Web Foundation (2014). Devarajan, Shantayanan, Stuti Khemani, and Michael Walton. 2011. Civil Society, Public Action and Accountability in Africa. HKS Faculty Research Working Paper Series RWP11-036, John F. Kennedy School of Government, Harvard University. Devarajan, Shantayanan, and Khemani, Stuti. 2016. If politics is the problem, how can external actors be part of the solution ? (English). Policy Research working paper; no. WPS 7761. Washington, D.C. : World Bank Group. 41 Diaz-Cayeros, Alberto, Beatriz Magaloni, and Alexander Ruiz-Euler. “Traditional Governance, Citizen Engagement, and Local Public Goods: Evidence from Mexico,” World Development (January 2014): 53. Duflo, Esther, Pascaline Dupas, and Michael Kremer. “School Governance, Teacher Incentives, and Pupil-Teacher Ratios: Experimental Evidence from Kenyan Primary Schools,” National Bureau of Economic Research Working Paper 17939, Cambridge, MA (2012). Escaleras, Monica, Shu Lin and Charles Register. "Freedom of information acts and public sector corruption," Public Choice, Springer, vol. 145(3) (2010). Ferraz, Claudio and Frederico Finan. "Exposing Corrupt Politicians: The Effects Of Brazil’s Publicly Released Audits On Electoral Outcomes." The Quarterly Journal of Economics 123 (2) (2008): 703-745. Fox, J. “Social Accountability: What Does the Evidence Really Say?” GPSA Working Paper 1 (September 2014). ———. “The Uncertain Relationship between Transparency and Accountability,” Development in Practice ([August 2007] 17: 405). Fujiwara, Thomas. “Voting Technology, Political Responsiveness, and Infant Health: Evidence from Brazil.” Econometrica 83 (2) (2015): 423–64. Gaventa, J., and R. McGee. “The Impact of Transparency and Accountability Initiatives.” Development Policy Review 31 (2013): s3–s28. Goldfrank, B. “Lessons from Latin American Experience in Participatory Budgeting,” Presentation at the Latin American Studies Association Meeting, San Juan, Puerto Rico (March 2006). Gonçalves, Sonia. “The Effects of Participatory Budgeting on Municipal Expenditures and Infant Mortality in Brazil,” World Development (2013). Grandvoinnet, Helene, Ghazia Aslam, and Shomikho Raha. Opening the Black Box: The Contextual Drivers of Social Accountability: New Frontiers of Social Policy, World Bank Group: Washington, DC (2015). Grant, Ruth & Robert Keohane. “Accountability and Abuses of Power in World Politics.” American Political Science Review 99(1) (2005). Gruening, G. “Origin and Theoretical Basis of New Public Management,” International Public Management Journal 4 (2001). Gurin, J. “Open Data Now: The Secret to Hot Startups,” Smart Investing, Savvy Marketing, and Fast Innovation (2014). Herzog, Timothy, and Jeanne Holm. “Financing Investments in Data Literacy and Use: Issues Paper for Data Revolution Consultation,” World Bank and NASA (January 15, 2015). Hogge, Becky. “OGD Study: Transparency and Accountability Initiative,” Open Society Foundation (2010). 42 Hood, Christopher. "A Public Management for All Seasons?" Public Administration Vol. 69 (1991): 3-19. Joshi, A. “Background Paper on Service Delivery: Review of Impact and Effectiveness of Transparency and Accountability Initiatives,” Institute of Development Studies, Brighton, UK (2013). Kariuki, Maureen. “Open Governance Partnership—Africa in the OGP: Why 2015 Is a Big Year” (March 20, 2015). Kendall, Chad, Tommaso Nannicini, and Francesco Trebbi. “How Do Voters Respond to Information? Evidence from a Randomized Campaign.” American Economic Review 105 (1) (2015): 322–53. Khokhar, Tariq. “Financing Investments in Data Innovations: Issues Paper for Data Revolution Consultation,” World Bank (January 15, 2015). Kosack, Stephen and Fung, Archon, Does Transparency Improve Governance? (May 2014). Annual Review of Political Science, Vol. 17, pp. 65-87, 2014. Kudamatsu, Masayuki. “Has Democratization Reduced Infant Mortality in Sub-Saharan Africa? Evidence from Micro Data.” Journal of the European Economic Association 10 (6) (2012): 1294–317. Larreguy, Haracio A., John Marshall and James M. Snyder, Jr. "Revealing Malfeasance: How Local Media Facilitates Electoral Sanctioning of Mayors in Mexico." NBER Working Paper No. 20697 (2014). Lathrop, Daniel, and Laurel Ruma (eds.). “Open Government: Transparency, Collaboration and Participation in Practice,” O'Reilly Media (February 2010). Little, Eric, and Carolyn Logan. “The Quality of Democracy and Governance in Africa: New Results for AfroBarometer Round 4,” Working Paper 108, AfroBarometer Network (2008). McKinney, James, Stephane Guidoin, and Paulina Marczak. “OGD Standards Inventory: Open Government Partnership OGD Working Group Standards Stream,” Open North (2015). Myles, J., and J. Quagagno. “Political Theories of the Welfare State,” Social Service Review 76 (1), 75th Anniversary Issue (2009: 34–57). O’Meally, Simon. Mapping Context for Social Accountability, World Bank Group: Washington, DC (2013). Pampel, F., and J. Weiss, “Economic Development, Pension Policies, and the Labor Force Participation of Aged Males: A Cross-National Longitudinal Study,” American Journal of Sociology 89 (September 1983): 350-72. Pandey, Priyanka, and Sangeeta Goyal (eds.). “Community Participation in Public Schools: The Impact of Information Campaigns in Three Indian States,” World Bank, Washington, DC (2008). Pandey, Priyanka, Sangeeta Goyal, and Venkatesh Sundararaman. “Public Participation, Teacher Accountability, and School Outcomes: Findings from Baseline Surveys in Three Indian States,” World Bank Policy Research Working Paper Series (November 1, 2008). Peisakhin, Leonid and Paul Pinto. "Is transparency an effective anti-corruption strategy? Evidence from a field experiment in India," Regulation & Governance 4(3) (2010): 261–280. 43 Peixoto, T. “The Uncertain Relationship between Open Data and Accountability: A Response to Yu and Robinson’s The New Ambiguity of ‘Open Government,’” UCLA. L. Rev. Disc (2013): 60, 200–48. Peixoto, T., and J. Fox. “When Does ICT-Enabled Citizen Voice Lead to Government Responsiveness?” Background Paper, 2016 World Development Report, World Bank (2015). Pfiffner, J. “Traditional Public Administration versus the New Public Management: Accountability versus Efficiency,” in Institutionenbildung in Regierung und Verwaltung: Festschrift fur Klaus Konig, ed. A. Benz, H. Siedentopf, and K.P. Sommermann (Berlin: Duncker & Humbolt) (2004) p. 443–54. Pradhan, Menno, and Daniel Suryadarma (eds.). "Improving Educational Quality through Enhancing Community Participation: Results from a Randomized Field Experiment in Indonesia," Policy Research Working Paper Series 5795, World Bank (2011). Ravindra, A. “An Assessment of the Impact of Bangalore Citizen Report Cards on the Performance of Public Agencies,” Evaluation Capacity Development Working Paper 12, Washington, DC, World Bank, Operations Evaluation Department (2004). Reinikka, Ritva, and Jakob Svensson. “The Power of Information: Evidence from a Newspaper Campaign to Reduce Capture,” Quarterly Journal of Economics 119(2) (2004). ———. “The Power of Information in Public Services: Evidence from Education in Uganda,” Journal of Public Economics (2011: 95). Samuelson, Paul A. "The Pure Theory of Public Expenditure", Review of Economics and Statistics 36 (4) (1954): 387–89. Stott, Andrew. “OGD for Economic Growth: Transport and ICT Global Practice,” World Bank (June 25, 2014). Suk Kim, P., J. Halligan, N. Cho, C. H. Oh, and A. M. Eikenberry. “Toward Participatory and Transparent Governance: Report on the Sixth Global Forum on Reinventing Government,” Public Administration Review 65(6): 646–54 (2005). Tisné, M. “Transparency, Participation and Accountability: Definitions,” unpublished Background Note for Transparency and Accountability Initiative, Institute of Development Studies (2010). United Kingdom, Department for Business, Innovation & Skills. “Shakespeare Review: An Independent Review of Public Sector Information,” London (2013). Weber, M. “Politics as a Vocation,” in From Max Weber, ed. H. H. Gerth and C. W. Mills (Oxford University Press (1946): 95. Weber, Max. Bureaucracy. As translated in Weber’s Rationalism and Modern Society: New Translations on Politics, Bureaucracy and Social Stratification (Tony Waters and Dagmar Waters, ed.) Palgrave MacMillan (1922, 2015) p. 73-128. World Bank. “Tanzania—Open Government and Public Financial Management (OGPFM) Development Policy Operation Project,” Project Information Document, Washington, DC (2015). 44 Yu, H., and D. Robinson. “The New Ambiguity of Open Government,” UCLA. L. Rev. Disc (2012): 59, 178–208. Books Arrow, Kenneth J. 1951. 2nd ed., 1963. Social Choice and Individual Values. Yale University Press. Black, Duncan. 1958. The Theory of Committees and Elections. Cambridge: Cambridge University Press. Borowiak, Craig T. 2011. Accountability and Democracy: The Pitfalls and Promise of Popular Control. Oxford: Oxford University Press. Buchanan, James M. and Gordon Tullock. 1962. The Calculus of Consent: Logical Foundations of Constitutional Democracy. University of Michigan Press. Downs, Anthony. 1957. An Economic Theory of Democracy. Cambridge: York: Cambridge University Press. Keohane, Robert. 2011. Global Governance and Democratic Accountability. Lindblom, Charles E. and Robert A. Dahl. 1976. Politics, economics, and welfare. Chicago: University of Chicago Press. Olson, Mancur, Jr. 1965. 2nd ed., 1971. The Logic of Collective Action. Cambridge: Harvard University Press. Ostrom, Elinor. 2005. Understanding Institutional Diversity. Princeton, NJ: Princeton University Press. de Tocqueville, Alexis. 1956. Democracy In America. Edited and Abridged Edition by Richard Heffner. Mentor Books. Trapnell, Stephanie (ed.) 2014. Right to Information: Case Studies on Implementation. World Bank. Reports Global OGD Initiative. “Global OGD Initiative Survey and Interview Report.” Accessed March 23, 2015. www.opendatainitiative.org/survey-and-interview-report/. Hania Farhan, Debra D’Agostino, and Henry Worthington. Web Index 2012, World Wide Web Foundation. Hazell, R., and B. Worthy. “Impact of Freedom of Information on Central Government,” unpublished End of Award Report to the Economic and Social Research Council, University College London, Constitution Unit (2009). Jesuit Hakimani Centre. “Open Government Data for Effective Public Participation: Findings of a Case Study Research Investigating Kenya’s Open Data Initiative in Urban Slums and Rural Settlements,” Nairobi Kenya (2014). 45 Khemani, Stuti. 2016. Making Politics Work for Development: Harnessing Transparency and Citizen Engagement. World Bank Policy Research Report. McKinsey Global Institute. “OGD: Unlocking Innovation and Performance with Liquid Information.” McKinsey Center for Government (October 2013). Misra, V. “Pilot Study 1, Andhra Pradesh, India: Improving Health Services through Community Score Cards: Learning Notes,” Social Accountability Series, World Bank, Washington, DC (2007). Mungiu-Pippidi, Alina. 2014. “Quantitative Report on Causes of Performance and Stagnation in the Global Fight against Corruption.” Anti-Corruption Policies Revisited. ANTICORRP. Pillar 4, WP3. ODAC (Open Democracy Advice Centre). “The Right to Know, the Right to Live: Turning the Right to Information into a Living Reality,” Cape Town (2010). OGD Institute. “Benchmarking OGD Automatically,” Technical Report 000 (2015). ———. “OGD in Government: How to Bring About Change,” White Paper 001 (2015). ———. “Supporting Sustainable Development with OGD,” White Paper 000 (2015). OSJI (Open Society Justice Initiative). Transparency and Silence: A Survey of Access to Information Laws and Practices in 14 Countries, Open Society Institute, New York (2006). Peixoto, Tiago and Jonathan Fox. 2016. “When does ICT-Enabled Citizen Voice Lead to Government Responsiveness,” Institute of Development Studies, Vol. 47, Issue 1. Taylor, Frederick Winslow. The Principles of Scientific Management, New York and London: Harper & Brothers (1911). Tinholt, Dinand. Characterization Study of the Infomediary Sector, Capgemini, Spanish Open Data Portal Annual Report (2013). United Nations. “A World that Counts: Mobilizing the Data Revolution for Sustainable Development,” Independent Expert Advisory Group on Data Revolution for Sustainable Development (November 2014). United Nations, Department of Economic and Social Affairs. “United Nations E-Government Survey E- Government for the Future We Want,” New York (2014). Vickery, Graham. “Review of Recent Studies on PSI Reuse and Related Market Developments,” Information Economics, Paris. World Bank. “Country Policy and Institutional Assessment, Africa: Assessing Africa’s Policies and Institutions,” Office of the Chief Economist, Report 88991 (2014). ———. “Open Data for Sustainable Development,” Transport and ICT Global Practice (2015). ———. World Development Report 2004: Making Services Work for Poor People (WDR 2004), Washington, DC (2003). 46 ———. World Development Report 2017: Governance and the Law (WDR 2017), Washington, DC (2016). World Wide Web Foundation. OGD Barometer Global Report, Second Edition (January 2015). World Wide Web Foundation & Governance Lab. “Towards Common Methods for Assessing OGD: Workshop Report and Draft Framework,” New York University (June 10, 2014). 47