(This article was originally published at Econometrics Beat: Dave Giles' Blog, and syndicated at StatsBlogs.)

What do we mean by a "significant result" in econometrics?

The distinction between "statistical significance" and "economic significance" has received a good deal of attention in the literature. And rightly so.

Think about the estimated coefficients in a regression model, for example. Putting aside the important issue of the choice of a

Marc Bellemare provided a great discussion of this in his blog a while back.

Here, I want to draw attention to a somewhat related issue - distinguishing between the statistical and economic overall

Almost 30 years ago, Hal Varian published a really nice paper on this topic. I'm surprised at how little attention it's received in recent times.

To set the scene, here's the abstract from Hal's paper (with my emphasis):

Back in 1999, Lindsay Tedds and I played around a bit with Hal's approach to economic goodness-of-fit, using demand systems based on the (constrained) maximization of various utility functions, and several data-sets. We never brought that work fully to fruition, but we came up with some interesting empirical results, and compared them with those for conventional (statistical) goodness-of-fit. Some slides for a talk that I gave on that work can be found here

The papers listed in the references below provide some additional insights into the debate over statistical

The distinction between "statistical significance" and "economic significance" has received a good deal of attention in the literature. And rightly so.

Think about the estimated coefficients in a regression model, for example. Putting aside the important issue of the choice of a

*significance level*when considering statistical significance, we all know that results that are significant in the latter sense may or may not be 'significant' when their*economic impact*is considered.Marc Bellemare provided a great discussion of this in his blog a while back.

Here, I want to draw attention to a somewhat related issue - distinguishing between the statistical and economic overall

*goodness-of-fit*of an economic model.Almost 30 years ago, Hal Varian published a really nice paper on this topic. I'm surprised at how little attention it's received in recent times.

To set the scene, here's the abstract from Hal's paper (with my emphasis):

"Conventional econometric tests of optimizing models typically involve embedding the optimizing model in a parametric specification and then examining the parametric restrictions imposed by the optimization hypothesis. The optimization hypothesis is rejected if the estimated parameters are significantly different, in the statistical sense, from the values implied by optimization. I argue thatContrary to a lot of what I see in the literature these days, microeconometrics should be all about the empirical analysis of optimization models. This makes Hal's work absolutely relevant!a more fruitful approach to testing optimizing behavior is to measure the departurefrom optimization using the estimated objective function, and see whether this departure issignificant in an economic sense.I discuss procedures for doing this that can be used in several sorts of optimizing models, and give a detailed illustration in the case of aggregate demand estimation."

Back in 1999, Lindsay Tedds and I played around a bit with Hal's approach to economic goodness-of-fit, using demand systems based on the (constrained) maximization of various utility functions, and several data-sets. We never brought that work fully to fruition, but we came up with some interesting empirical results, and compared them with those for conventional (statistical) goodness-of-fit. Some slides for a talk that I gave on that work can be found here

The papers listed in the references below provide some additional insights into the debate over statistical

*vs*. economic significance.Akerlof, G. and J. Yellen, 1985. Can small deviations from rationality make significant differences to economic equilibria?

Cochrane, J., 1989. The sensitivity of tests of intertemporal allocation of consumption to near-rational alternatives.

Cowell, F. A., E. Flachaire, & S. Bandopadhyay, 2009. Goodness-of-fit: An economic approach. Discussion Paper No. 444, Department of Economics, University of Oxford.

McCloskey, D. N., 1985. The loss function has been mislaid: The rhetoric of significance tests.

McCloskey, D. N. & S. T. Ziliak, 1996. The standard error of regressions.

*American Economic Review*, 75, 708-720.Cochrane, J., 1989. The sensitivity of tests of intertemporal allocation of consumption to near-rational alternatives.

*American Economic Review*, 79, 319-337.Cowell, F. A., E. Flachaire, & S. Bandopadhyay, 2009. Goodness-of-fit: An economic approach. Discussion Paper No. 444, Department of Economics, University of Oxford.

McCloskey, D. N., 1985. The loss function has been mislaid: The rhetoric of significance tests.

*American Economic Review*, 75, 201-205.McCloskey, D. N. & S. T. Ziliak, 1996. The standard error of regressions.

*Journal of Economic Literature*, 34, 97-114.Varian, H. R.

**, 1990. Goodness of fit in optimizing models.**

*Journal of Econometrics,*46, 125-140.

Ziliak, S. T. & D. N. McCloskey, 2004. Size matters: The standard error of regressions in the American Economic Review.

*Journal of Socio-Economics*, 33, 527-546© 2018, David E. Giles

**Please comment on the article here:** **Econometrics Beat: Dave Giles' Blog**