Home > Uncategorized > The main reason why almost all econometric models are wrong

The main reason why almost all econometric models are wrong

from Lars Syll

Since econometrics doesn’t content itself with only making optimal predictions, but also aspires to explain things in terms of causes and effects, econometricians need loads of assumptions — most important of these are additivity and linearity. Important, simply because if they are not true, your model is invalid and descriptively incorrect.  And when the model is wrong — well, then it’s wrong.

The assumption of additivity and linearity means that the outcome variable is, in reality, linearly related to any predictors … and that if you have several predictors then their combined effect is best described by adding their effects together …

catdogThis assumption is the most important because if it is not true then even if all other assumptions are met, your model is invalid because you have described it incorrectly. It’s a bit like calling your pet cat a dog: you can try to get it to go in a kennel, or to fetch sticks, or to sit when you tell it to, but don’t be surprised when its behaviour isn’t what you expect because even though you’ve called it a dog, it is in fact a cat. Similarly, if you have described your statistical model inaccurately it won’t behave itself and there’s no point in interpreting its parameter estimates or worrying about significance tests of confidence intervals: the model is wrong.

Andy Field

Let me take the opportunity to elaborate a little on why I find these assumptions of such paramount importance and ought to be much more argued for — on both epistemological and ontological grounds — if at all being used. 

Limiting model assumptions in economic science always have to be closely examined since if we are going to be able to show that the mechanisms or causes that we isolate and handle in our models are stable in the sense that they do not change when we “export” them to our “target systems”, we have to be able  to show that they do not only hold under ceteris paribus conditions and a fortiori only are of limited value to our understanding, explanations or predictions of real economic systems.

Our admiration for technical virtuosity should not blind us to the fact that we have to have a cautious attitude towards probabilistic inferences in economic contexts. We should look out for causal relations, but econometrics can never be more than a starting point in that endeavour since econometric (statistical) explanations are not explanations in terms of mechanisms, powers, capacities or causes. Firmly stuck in an empiricist tradition, econometrics is only concerned with the measurable aspects of reality. But there is always the possibility that there are other variables – of vital importance and although perhaps unobservable and non-additive, not necessarily epistemologically inaccessible – that were not considered for the model. Those who were can hence never be guaranteed to be more than potential causes, and not real causes. A rigorous application of econometric methods in economics really presupposes that the phenomena of our real world economies are ruled by stable causal relations between variables. A perusal of the leading econom(etr)ic journals shows that most econometricians still concentrate on fixed parameter models and that parameter-values estimated in specific spatio-temporal contexts are presupposed to be exportable to totally different contexts. To warrant this assumption one, however, has to convincingly establish that the targeted acting causes are stable and invariant so that they maintain their parametric status after the bridging. The endemic lack of predictive success of the econometric project indicates that this hope of finding fixed parameters is a hope for which there really is no other ground than hope itself.

Real-world social systems are not governed by stable causal mechanisms or capacities. The kinds of “laws” and relations that econometrics has established, are laws and relations about entities in models that presuppose causal mechanisms being atomistic and additive. When causal mechanisms operate in real-world systems they only do it in ever-changing and unstable combinations where the whole is more than a mechanical sum of parts. If economic regularities obtain they do it (as a rule) only because we engineered them for that purpose. Outside man-made ‘nomological machines’ they are rare, or even non-existent. Unfortunately, that also makes most of the achievements of econometrics – as most of the contemporary endeavours of mainstream economic theoretical modelling – rather useless. No matter how often you call your pet cat a dog, it is still nothing but a cat …

  1. Helen Sakho
    June 29, 2018 at 12:55 am

    Thank you Lars.
    In plain English “what spading the soil, I think of the flowers”, or “let’s call a spade a spade”.
    The very basics of the alphabet in this particular language, which in my humble view comparatively speaking is not such a difficult one to master.
    Do the readers believe that mainstream Economists can export a simple spade to where the land has been soiled for generations to come?

  2. June 29, 2018 at 6:11 am

    See Zaman, Asad: Lessons in Econometric Methodology: Axiom of Correct Specification, International Econometric Review, Vol 9, Iss 2, 2017. Regression models can be interpreted as correct descriptions of GENUINE patterns of correlation (and NEVER as correct descriptions of causal patterns) ONLY if all the axioms are satisfied. This is UNVERIFIABLE, since axioms involve assumptions about unobservable error terms — so actually the truth or falsity of these can NEVER be ascertained. This is exactly what allows econometricians to get away with murder (ie a massively ridiculous methodology). Just as no one can prove these assumptions are right, so no one can prove that these assumptions are wrong — which allows them to make ANY convenient assumption they like, and assume it is correct. Exactly along the lines of the joke that the Economist just assumed that the tin can was open and ate the food inside.
    Even though we cannot PROVE directly that assumptions made are wrong, we can show the the METHODOLOGY of making arbitrary assumptions is flawed by showing that we can use it to prove anything at all. I have made this argument in many different ways in many different papers and talks. See for example, Choosing the Right Regressors:
    https://weapedagogy.wordpress.com/2017/12/11/choosing-the-right-regressors/
    This post summarized the paper, and provides a link to a video lecture on the Axiom of Correct Specification and its implications for econometric methodology. This paper attempts to be constructive in the sense of trying to do the best you can with a fundamentally flawed methodology.
    A second talk on A Realist Approach to Econometrics explains why all of econometrics is a fraud because it is based on a nominalist/postivist approach which refuses to look beneath the surface of the observed numbers. Progress can only be made by trying to understand the real world process which generates these numbers — exactly the same observed numbers can be generated by two different real world processes, and would require DIFFERENT analyses despite being IDENTICAL in terms of data. This talk (needs to be written up for a paper – would anyone like to help in writeup?) and a summary is linked below:
    https://weapedagogy.wordpress.com/2018/05/04/a-realist-approach-to-econometrics/
    This paper is not constructive — rather, it proposed abandoning current methodology, and proposes a radically different approach to data analysis based on realist, as opposed to nominalist principles.

  3. David Harold Chester
    June 29, 2018 at 7:47 am

    How can an econometric model be wrong if it is based on the statistical results of past behavior? Surely it fits exactly with the system, as it was?

    What it is, is not wrong, but utterly unrealistic for the present-day conditions which are in a constant state of flux and whose behavior is not based on statistics. These conditions are based on the way our social system is phycially arranged…topologically, structurally and with the ability to quickly respond to every kind of change. There are better ways and logical/scientific methods for our understanding of the workings of our society.

  4. Frank Salter
    June 29, 2018 at 7:51 am

    A classic example of this invalid thinking is found in Solow (1957). He fitted five different forms of equation to his empirical data. He rejected, for statistical reasons, the only one which could have theoretical validity.

    In Salter (2017; pp, 154−159), I present a wider view of the fitted equations, with their peculiar excursions, and demonstrate why only the linear REJECTED equation might have a claim to theoretical validity. Ultimately, they are all wrong! My equation (31) on page 159 contains a typo. The dot, which should be directly over the “q” is over the first “1 –” on the right hand side. However this is the theoretically valid representation of the data. The coefficient values are only relevant to the Solow data, though the equation form is always valid.

    Salter, Frank M. (2017). “Transient Development”. In: Real World Economic Review (81), pp. 135–167
    Solow, R. M. (1957). ‘Technical change and the aggregate production function’. In: The Review of Economics and Statistics 39(3), pp. 312–320.

  5. July 16, 2018 at 5:38 am

    Thanks for all the discussion. But not necessary. Econometric models are wrong because they are models. Otherwise it would be unnecessary when back casting models to compare to historical events to ask and answer the question, “how close is close enough?”

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.