The Open Budget Survey is widely recognized by experts as one of the best sources of data on accountability in government budgets. The survey has been conducted for the past 16 years and its data is used by numerous governments, donor agencies, investment professionals, researchers, and civil society organizations. The data from the survey not only provides a snapshot of current practices in national budget transparency and accountability but also provides a timeseries on changes in practices in countries over time. The survey results are generally consistent with reports on fiscal accountability published by other reputable public finance bodies, including the World Bank, the International Monetary Fund, and the Organization for Economic Cooperation and Development.

Although the Open Budget Survey is a fairly comprehensive assessment of budget accountability, it has limitations and its results need to be contextualized against other trends and practices in countries. For example, despite the emergence of high-profile corruption scandals, some countries like Brazil, Mexico, Philippines, and South Africa perform highly on transparency measures in the Open Budget Survey. While this may seemingly appear to be a contradiction, it simply means (as discussed here) that fiscal transparency is a necessary but insufficient condition for successfully addressing the problem of corruption.

Further, the survey does not cover at all or inadequately covers some areas of public finance – such as extrabudgetary funds, quasi-fiscal activities, tax expenditures, public procurement, and classified expenditures (on national security and intelligence activities) – that can reduce fiscal accountability. Unfortunately, these ‘hidden corners’ of public finance cannot be easily assessed by civil society organizations through instruments like the Open Budget Survey. Instead, they are best undertaken by specialists from international financial institutions or government insiders with access to information that may not be in the public domain.

The lack of adequate coverage of these areas of public finance in the survey can sometimes yield results for individual countries – like Russia that is a top-15 performer on transparency measures in the Open Budget Survey – that might be at odds with current perceptions of opaque government practices. For example, there are at least three fiscal practices in Russia that are not adequately assessed in the Open Budget Survey and that likely impact the high transparency score received by the country.

One, a fairly large and growing proportion of the Russian national budget is classified as secret. As per an IMF evaluation, such classified expenditures increased from 10 percent in 2009 to 17 percent in 2018. Classified expenditures are only shown at the aggregate level in the budget and do not provide the details that are typically provided for other parts of the budget that are not classified.

Two, the same IMF evaluation estimates that more than one-third of public procurement contracts were classified as secret and therefore not subject to open and competitive tendering procedures. Public procurement transactions are not assessed in the Open Budget Survey.

And, three, the Open Budget Survey inadequately assesses the governance of extrabudgetary funds, including Russia’s large sovereign wealth fund (that it created through its oil revenues) that is used to directly finance certain activities, such as infrastructure projects and lending to foreign countries. Extrabudgetary funds often do not receive the same level of scrutiny as expenditures that are routed through the national budget. A recent report by the Peterson Institute for International Economics placed Russia’s sovereign wealth fund in the bottom category of countries on its scorecard that is based on international standards on governance and accountability of sovereign wealth funds.

These limitations in the transparency measures assessed by the Open Budget Survey do not make the data and results collected from the survey redundant. It does however imply that users of the data should look at all aspects of country performance on the survey. In addition to transparency measures, the survey also assesses the role of budget oversight institutions and opportunities for public participation in budgeting. Russia performs poorly on the survey’s public participation measures. Further, users should supplement the data from the Open Budget Survey with other governance and public finance data if they are interested in securing a more nuanced and comprehensive view of practices in countries.