Challenges in Budget Research

Comparing budgetary outcomes across time and space

The main challenge of comparing budgetary outcomes is consistency. Two forms of consistency need to be considered over time within a political system and across countries at a given time. An outline of specific challenges in data collection, coding, and analysis are listed below. This list draws from and extends on a discussion provided by Baumgartner, Jones, and Wilkerson's (2002, 29-46) chapter on studying policy dynamics in the United States.

• Comparison over Time

  
  Backward Comparability
  Since topic categories are revised and budget items are reshuffled by policy makers or bureaucrats, budget categories may not be consistent over time. In order to ensure reliability of categorization over time, a time-consistent categorization scheme should be used.
  

  Overcategorization
  In order to avoid the utilization of multiple keywords for indexing budget items, mutually exclusive and exhaustive budget categories need to be employed through the entire time period. Since budgeting is usually based on a detailed accounting system that provides mutually exclusive and exhaustive categories, overcategorization is occurring minimally. Overcategorization is more likely to be problematic for keyword searches of news coverage etc. than for budgeting though.

  Correspondence
  Some categories in a budget may not respond to substantively meaningful government activities (e.g. Function 950 "undistributed offsetting receipts" in the U.S. budget) and thus may not be considered for subsequent analysis.

• Comparison Across Space

  
  Uniqueness
  Budgetary data collected by a different entities, e.g. national or sub-national governments, often employ unique indexes. As a result, even similar main budget categories might included vastly different sets of sub-categories across entities. Similarly, budgetary data collected by international organizations, such as the World Bank, might employ a different categorization scheme and different levels of aggregation than the original national budgets (however, the European Union is in the process of moving toward a common budget scheme for all member states). Consequently, it is difficult to examine causal relationships across entities.

  Matching
  Given that data sets from different entities are often unique, the problem of matching categories across these entities emerges. The dilemma of matching is that one has to engage in the trade off between selecting a sub-set of comparable budget items while ignoring the budget as a whole and comparing the entities' budgets as a whole while disregarding nuances across budget entities.

  Availability and decisions rules
  Since the budget process varies vastly across countries (see for example the works of Hallerberg on the power of the ministry of finance) and some countries do not pass a budget laws, availability of budgetary data (especially budget authority data and outlays) might be limited.

• Comparison of Different Budget Classifcation Systems

  
  Public money can go to operating expenses , which fund on-going government activities, or capital expenses , which fund long-lasting projects such as a fire station or an aircraft carrier.   Some governments, such as the American states, separate capital from operating budgets, but many national governments, such as the US national government, do not.   For national governments, Outlays are actual expenditures, capital or otherwise, and budget authority is legislative authority for an agency to spend, more relevant to actual studies of decision-making. They should not be confused.
  

  US Office of Management and Budget tabulates outlays historically, but budget authority is consistent only after 1976. Consistent historical budget authority data is found at the Policy Agendas Project.

• Additional Issues

  
  Adjusting for Inflation
  When budgetary outcomes are compared across time and space, researchers need to determine whether the budgetary figures should be adjusted for inflation. The specific theoretical and empirical question of interest may help to determine the appropriateness of adjusting for inflation. For example, a comparison of public spending across countries over time may require adjusting for inflation.