Econometric evaluation of public policies for science and innovation: a brief guide to practice

It is widely documented that the private rate of return to innovation activities may be lower than the social rate. This divergence leads to a socially suboptimal rate of investment in science and innovation (Guellec and van Pottelsberghe, 2003). Indeed, most OECD countries have tried to delegate public resources directly to Research & Development (R&D) projects that are believed to boost the dynamic efficiency of an economic system and thus lead to a higher level of social welfare. More specifically, governments fund programs supporting basic scientific research and large-scale R&D projects aimed at reaching fundamental objectives (e.g., defense, transports, aerospace, health, or environmental concerns). Public support programs often involve also pre-competitive R&D with the goal of encouraging large spillover effects (e.g., through firms’ alliances and research consortia) and commercial R&D in order to solve possible market failures and increase innovation rates at the industrial level. However, there is little consensus among economists about the desirability of these public support measures, in particular those directed towards firms. There are at least two main reasons why these latter public programs may be ineffective from a social point of view. First, public money may crowd out private financing. In other words, there is no net increase in research investments, even if public resources are used productively and not wasted by the supported firms. More specifically, governments attempt to increase total R&D expenditures through granting public R&D funding to those projects which would normally be either underfinanced (in this case the policy intervention is said to aim at increasing the “intensive margin” in R&D expenditure) or not undertaken (i.e. the focus in this case is on the “extensive margin”). However, it may happen that prospective recipients do not promptly and fully respond to the public stimulus, or that subsidized firms replace their own R&D budget with the money they receive from the government (i.e. “crowding out effect). Second, it is quite reasonable to wonder how government agencies are able to pick “winners and losers”, namely to select research projects with high social benefits but insufficient private returns without interfering in market processes. For these reasons, assessing the impact of public research programs on social welfare is mainly an empirical question. As to econometrics is concerned, their quantitative evaluation in terms of general equilibrium effects must address several methodological issues. In fact, to properly evaluate the effects of such programs, an econometrician should answer the following question: “what is the effect of a program in place on participants and nonparticipants compared to no program at all or some alternative program?” (Heckman, 2001: p. 677). There is a strong debate in the literature on whether public research programs can be effectively evaluated without a randomized experiment. In fact, it is important to note that evaluating such programs is an exercise in counterfactual analysis; neither supported firms, nor firms not applying for funding, can be treated as a random sample. The econometric challenge in this setting is to find a valid control group of firms to be matched with supported firms. In order to do so, we must draw on quite recent advances in econometric methodologies used in evaluation studies of non-experimental data. The aim of this work is twofold. First, we review the most important econometric methods available to measure the effects of public research programs. Second, we provide some hints to deal with different methodological problems across various contexts of analysis. For the sake of simplicity, we focus on R&D subsidies at the firm level, but we develop an argument that can be easily extended to all types of public policies for science and innovation as well as to different levels of analysis (e.g., industry, country, private or social benefits).