Home > Uncategorized > The main reason why almost all econometric models are wrong

The main reason why almost all econometric models are wrong

from Lars Syll

How come that econometrics and statistical regression analyses still have not taken us very far in discovering, understanding, or explaining causation in socio-economic contexts? That is the question yours truly has tried to answer in an article published in the latest issue of World Economic Association Commentaries:

The processes that generate socio-economic data in the real world cannot just be assumed to always be adequately captured by a probability measure. And, so, it cannot be maintained that it even should be mandatory to treat observations and data — whether cross-section, time series or panel data — as events generated by some probability model. The important activities of most economic agents do not usually include throwing dice or spinning roulette-wheels. Data generating processes — at least outside of nomological machines like dice and roulette-wheels — are not self-evidently best modelled with probability measures.

EGOBILD2017When economists and econometricians — often uncritically and without arguments — simply assume that one can apply probability distributions from statistical theory on their own area of research, they are really skating on thin ice. If you cannot show that data satisfies all the conditions of the probabilistic nomological machine, then the statistical inferences made in mainstream economics lack sound foundations.

Statistical — and econometric — patterns should never be seen as anything other than possible clues to follow. Behind observable data, there are real structures and mechanisms operating, things that are — if we really want to understand, explain and (possibly) predict things in the real world — more important to get hold of than to simply correlate and regress observable variables.

Statistics cannot establish the truth value of a fact. Never has. Never will.

  1. Buck
    July 18, 2018 at 2:59 am

    As crudely but accurately stated by skeptics for years; “[f]figures don’t lie, but liars figure”. Such are models, like big lies of economic success, predicated on free money to banks and almost so to corporate bosses, who create simple-minded catch phrases (Orwellian double -speak) akin to fictitious entities who talk without mouths, only corrupted mouthpieces on captured courts.

  2. July 18, 2018 at 3:50 am

    If “statistics cannot establish the truth value of a fact,” economics should be pursued by a research strategy (or strategies) different from before. I propose two new lines:

    (1) Make a long detour to arrive at the truth. Abandon macroeconomic predictions and concentrate your efforts to shorter causal links that may help you to understand how the economy works (even if it is partial understanding). Do not try to find a universal law but more accurate laws with clear domain of applicability.

    (2) Economy is an open system (T. Lawson). This fact excludes the possibility to construct a total closed system but it does not exclude the possibility to find firm relations (or patterns) between smaller number of variables, or phenomena in a narrow specific field. Mathematics may be helpful in understanding patterns in these phenomena. Those patterns may remain “algebraic” in the sense F. A. Hayek used this word (The Theory of Complex Phenomena). In many cases, we are interested in the recurrence of patterns. In these patterns, we are often unable to know the values that we should substitute for variables. These studies do not contribute to a prediction, but help us understand the economy. Rehabilitate theoretical analyses. This seems to be the thing that is lacking in many heterodox arguments.

    P.S. Hayek’s paper above cited well deserves to be read now. This is also a good guide for “Hard and soft science” (or difficult and easy science) debate.

    • July 18, 2018 at 2:00 pm

      The introduction at https://bazaarmodel.net/ftp/Project…/Theory%20of%20Complex%20Phenomena.pdf is also worth reading.

      My own view is that statistics can at least provide strong support to certain ‘negative facts’. For example, it seems to me that almost any attempt to analyse any reasonably long time-series of socially influenced data will amply illustrate the folly of ‘curve-fitting’ to the first half and expecting an extrapolation to fit the second half, thus tending to support the theory that social phenomena do not behave like (insert favourite deterministic or stochastic theory here), but are more complex than that, e.g. of infinite dimensionality, or at least of much greater dimensionality than has been assumed.

      By simple naïve induction, much mainstream theory seems to rest on assumptions that are palpably false, so one is surely obliged to abandon naïve induction or mainstream theory, or both.

      I also think that even though we would be foolish absolutely to rely on extrapolated statistics, they can be essential in informing effective action, as long as we recognize them for what they are.

      • July 18, 2018 at 5:23 pm

        Dave Marsay > statistics can at least provide strong support
        to certain ‘negative facts’

        Then, what do you propose as a possible strategy for economic science?

  3. Prof Dr James Beckman, Germany
    July 18, 2018 at 7:14 am

    Make the recent international soccer competition a microcosm for a national or all national economies. The assignment: model France’s victory, please. Only time for me to comment very incompletely, but the level of professional experience of players was an issue: you want “enough” but not so much as to have egotistical “star” players. Similarly, the four finalists were packed with athletes from immigrant families, who frankly had energy & aggressiveness mostly absent otherwise. Remind you of Uber, Tesla, & an equal number of Chinese competitors? Many older firms globally have pockets of the same “stuff”, not easy to model it seems to me.

    • July 18, 2018 at 2:04 pm

      Seconded.

      If you are in a competitive environment you need to practice your plays enough to be competent at them, but not so much that your perception of the game becomes anchored on them: you need to retain an open mind, just in case the other side innovates.

      I am more impressed by sports psychologists than mainstream economists.

  4. David Harold Chester
    July 18, 2018 at 9:22 am

    Which models are not wrong, and why?

    • Frank Salter
      July 18, 2018 at 10:48 am

      All economic models, which do not solve differential equations over time, are wrong.

      • July 18, 2018 at 2:10 pm

        In my view we can never be justified in claiming that any specific model or class of models actually corresponds to reality. All we can say is that we are doing our best to test it, and it has not – so far – been falsified. The first problem with mainstream economics, unlike biology, say, is that it has opted out any attempt to understand economies and resorted to scholasticism and dogma. The impact of this has been to throw modelling into disrepute. I think we need more modelling, better modelling and better understood modelling. Some argue that this is unachievable, and we should simply muddle through without modelling. But maybe we could try?

      • Prof Dr James Beckman, Germany
        July 18, 2018 at 7:45 pm

        Right on, Dave. In the May/Trump current trade fragmentation model, I tell my German clients (I have a few) that they might be able to pick up the pieces. Macrcon is trying to bring London’s bankers to Paris. A lot of German suppliers want to pick up from the UK’s suppliers who will be forced to drop out of the “just-in-time” model of today’s global supply chain. Etc. So for some firms/regions this can be the best of times, even if not for many others. Thus, all our macro models are apt to rather be regional for larger nations.
        The local German press today reported a giant shortage of apprentices in industry, but also in construction, hospitality & elder care. With small families & aging parents, there is an enormous need for young workers–aside from whatever Brexit will do to increase that. A part of my job at my Engineering faculty is to help our foreign students find jobs. It is like falling off a log if you have industry knowledge as I do, to find work for our mechanical, electrical & computer engineers. Thus, Germany’s attention to slow but steady economic growth will indeed make a place for most of those young refugees. We had one in our house today installing a new clothes washer. He is employed in hotel maintenance even as his family in the Middle East is being terrorized by new shelling. So in one family, ups/downs.

      • Frank Salter
        July 19, 2018 at 6:28 am

        Also all quantitative first principles (i.e. genuine axioms of reality — not some assumptions) analysis will provide appropriate and correct descriptions of reality.

    • July 18, 2018 at 2:18 pm

      Models which logically fit the facts so far, are contextualised within the appropriate theories, and with the appropriate caveats are not manifestly wrong. As long as they make no claim outside of the applicable logic, they may even be correct. Unfortunately there is always a gap between such models and what people want, such as ‘reliable predictions’.

      In a competitive or evolutionary setting, we expect even ‘not wrong’ models to be falsified in the long run. As well as being not wrong, a good theory will also have been severely tested and subject to appropriate review. I think that’s as good as it gets.

      • Frank Salter
        July 19, 2018 at 8:38 am

        ” we expect even ‘not wrong’ models to be falsified in the long run”:

        This may be true for specific models but it is not a universal truth. First principles analysis is totally different.

  5. Craig
    July 19, 2018 at 12:26 am

    The main reason why ALL present econometric models are wrong is they’re not looking directly at the economy, economists are largely ignorant of the identical natures of double entry bookkeeping and the pricing and money systems and hence they do not realize that policies that will resolve the major problematic situations in modern economies are possible by coming out of their habitual and near abstract fugues and LOOKING at the above tools and their significances to the economy.

    On an epistemological scale looking is right below the highest level of knowledge possible, that is, knowing.

  6. July 19, 2018 at 3:42 am

    I would say that the main reason econometric models are wrong is because of the “Ergodic” assumption built into every one other — the future will be like the past. There is no basis for this assumption. This is topped by the failure of most models to accurately describe the past. As long as econometrics remains a game of pattern recognition and extrapolation, there is no hope of progress. This pattern recognition and the prediction test of accuracy are both aspects of the nominalist/positivist methodology which currently underlies econometrics. We can only progress by making a radical shift to a “Realist” methodology, as I have explained in my post on: A Realist Approach to Econometrics:
    https://weapedagogy.wordpress.com/2018/05/04/a-realist-approach-to-econometrics/

    • July 19, 2018 at 3:55 am

      assumption built into every on OF THEM —

      • Frank Salter
        July 19, 2018 at 6:30 am

        So it is necessary to move to first principles analysis and ditch inappropriate assumptions.

    • Prof Dr James Beckman, Germany
      July 19, 2018 at 7:19 am

      Right on, Asad, as on a micro (group or regional groups’ level) things are always changing. Think something as simple as pay-day loans or supply chains which while very efficient in cost reduction terms are very expensive to modify. I laugh when experts tell me job loss in the US is due to technology, when I for 20 years was close enough to the Los Angeles/Long Beach Harbors to see the buildup of cargo containers & imported vehicles from the hundreds of thousands to the millions annually. Just try commuting behind thousands of trucks bearing Chinese-lettered cargo containers for a reality check, I suggest to the disbelievers.
      It is similar when I am teaching in China, watching the unbelievable Chinese construction figures–buildings, roads, airports, etc–so I see how the West has benefited with lower-priced & adequate quality goods, while we have done little to support our out-of-work, out-of-good health care workers. The politicians don’t want to get it any more than the economists, it seems to me.

      • July 19, 2018 at 10:56 am

        Mainstream trade theory cannot treat unemployment caused by international trade. This is simply so, because they assume that each nation economy is in equilibrium. It means full employment is already achieve. Mainstream trade economists do not understand that they are excluding unemployment by assumption and they are arguing that unemployment due to international trade is impossible.

        The trouble does not stop here. Many of heterodox economists accuse traditional trade theory and policy but do not try to build a new theory. Are they defeatists in theory?

        Finally, if they (heterodox economists) cannot produce their own theory, they should know that there exists already a new theory that do not assume full employment. See my paper:

        The New Theory of International Values: An Overview
        https://www.researchgate.net/publication/304717720_The_New_Theory_of_International_Values_An_Overview

      • Prof Dr James Beckman, Germany
        July 19, 2018 at 12:32 pm

        Yoshinori, thanks for your contribution. Perhaps all economists should intern with a firm which s subject to their theory–before they write their dissertations or journal articles.

      • July 19, 2018 at 1:32 pm

        Good idea! I once passed one-week training course in a small town factory where we learned how to work with lathe.

        Another possibility is to recruit graduate students who are graduated from departments of commerce, production management, accounting and others. Now we can find few number of heterodox economists in economics department, but there are relatively many number of them in business schools.

  7. July 26, 2018 at 1:02 pm

    Thanks for all the discussion. But not necessary. Econometric models are wrong because they are models. Otherwise it would be unnecessary when back casting models to compare to historical events to ask and answer the question, “how close is close enough?”

  8. Craig
    July 28, 2018 at 7:36 pm

    Where modern macro went wrong is in not realizing that private finance’s monopolistic money creating power….is post retail sale and hence illegitimate….because the actual economic/productive process ends precisely and definitively AT retail sale. And as a public national bank can more economically do everything good that private banks can do (and a hell of a lot less harm than they have continually done for the last 5000 years) such a banking system guided by the abundant, benevolent and gifting concept of grace….seems to be the way for macroeconomics to right and reverse its seeming inabilities to bring stability and continuing prosperity to the economy and the discipline.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.