Home > RWER > RWER issue 57: Hollanders

RWER issue 57: Hollanders

Five methodological fallacies in applied econometrics

D.A. Hollanders   


This paper discusses five methodological problems in applied econometrics. These are the problem of measurement, data mining, publication bias, the Duhem-Quine critique, and the non-repeatable nature of historical events. These problems form a third source of error next to two other more common sources of error in econometrics, sampling error and specification error. The paper argues that these problems aggravate the already difficult task of testing, but can often be dealt with. In some circumstances however testing itself is inappropriate, and econometrics is better understood as a means for description than for testing. 

JEL classification: B40, C18, C50. Key words: applied econometrics, methodology, philosophy of science.

1. Introduction


            Econometrics is a collection of probability statements. That is, estimated regression-coefficients come with a disclaimer that they may be wrong. The disclaimer takes the form of the probability that the estimated coefficient would have popped up if the true value of the coefficient equals zero. If this probability is below 5%, the coefficient is said to be significant. In a world of finite data the probability a true hypothesis is rejected – the type I error-never equals zero and it has to be accepted as a fact of life, even if the model is correctly specified.

            A second concern, besides this Type I error, relates to the if-statement that the model is correctly specified. In its basic from, the ordinary least squares-model assumes regressors exogenous, error terms homoskedastic and uncorrelated, and sometimes normally distributed. Violations of these kind of model-assumptions form a second source of error, called specification error. Testing model-assumptions and thereby enabling correct inference is the core business of econometrics. A large part of econometrics consists then, in the words of Hendry [1980] of ‘test, test, test’.

            To deal with misspecification error and being aware of sampling error distinguishes good econometrics from bad econometrics. Nonetheless exclusively focusing on these two sources of error suggests there are no other possible sources of error. There are. The paper concerns itself with five such sources of error in econometrics; these fall outside the two categories mentioned, and together constitute what may be called methodological errors. These are (1) measurement error, (2) data mining, (3) Duhem-Quine critique, (4) publication bias, (5) historical events being sui generis.

            These methodological concerns are not new and no claim to novelty is made. All the same, they are treated, if mentioned at all, non-systematically in many econometric textbooks, and are dealt with, if treated at all, ad hoc in applied work. It may therefore be useful to group and categorize them. The five concerns all circle around the question

if and how it is possible to test, or if one likes, they form some epistemological disclaimers that comes with testing.

 2.1. Measurement problem and the problem of conceptualizing

 You may download the whole paper at:/ http://www.paecon.net/PAEReview/issue57/Hollanders57.pdf

  1. Claude Hillinger
    August 28, 2012 at 1:02 pm

    The natural sciences have done just fine with virtually no statistical significance testing. They try to find relationships that are as stable as possible over as wide a sample as possible and that are as simple as possible. They use the relationships if they seem to be usefull even though there is no claim that they are ‘true’ in any absolute sense. For example, it is convenient to assume that planetary motions are elliptical, though it is known that this is not exactly true.
    The basis for statistical inference is the assmption that disturbances are drawn from a random distribution. This is almost never plausible in the social sciences.
    Why are econometricians so fixated on testing? Because their methodlogy was defined at the Cowles Commission by statisticians; it did not evolve natuarally out of economics.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.