Home > Uncategorized > Econometrics and the problem of unjustified assumptions

Econometrics and the problem of unjustified assumptions

from Lars Syll

There seems to be a pervasive human aversion to uncertainty, and one way to reduce feelings of uncertainty is to invest faith in deduction as a sufficient guide to truth. Unfortunately, such faith is as logically unjustified as any religious creed, since a deduction produces certainty about the real world only when its assumptions about the real world are certain …

economUnfortunately, assumption uncertainty reduces the status of deductions and statistical computations to exercises in hypothetical reasoning – they provide best-case scenarios of what we could infer from specific data (which are assumed to have only specific, known problems). Even more unfortunate, however, is that this exercise is deceptive to the extent it ignores or misrepresents available information, and makes hidden assumptions that are unsupported by data …

Econometrics supplies dramatic cautionary examples in which complex modellin​g has failed miserably in important applications …

Sander Greenland

Yes, indeed, econometrics fails miserably over and over again.

One reason why it does, is that the error term in the regression models used is thought of as representing the effect of the variables that were omitted from the models. The error term is somehow thought to be a ‘cover-all’ term representing omitted content in the model and necessary to include to ‘save’ the assumed deterministic relation between the other random variables included in the model. Error terms are usually assumed to be orthogonal (uncorrelated) to the explanatory variables. But since they are unobservable, they are also impossible to empirically test. And without justification of the orthogonality assumption, there is, as a rule, nothing to ensure identifiability:

Distributional assumptions about error terms are a good place to bury things because hardly anyone pays attention to them. Moreover, if a critic does see that this is the identifying assumption, how can she win an argument about the true expected value the level of aether? If the author can make up an imaginary variable, “because I say so” seems like a pretty convincing answer to any question about its properties.

Paul Romer

Nowadays it has almost become a self-evident truism among economists that you cannot expect people to take your arguments seriously unless they are based on or backed up by advanced econometric modelling​. So legions of mathematical-statistical theorems are proved — and heaps of fiction are being produced, masquerading as science. The rigour​ of the econometric modelling and the far-reaching assumptions they are built on is frequently not supported by data.

Econometrics is basically a deductive method. Given the assumptions, it delivers deductive inferences. The problem, of course, is that we almost never know when the assumptions are right. Conclusions can only be as certain as their premises — and that also applies to econometrics.

Econometrics doesn’t establish the truth value of facts. Never has. Never will.

  1. Helen Sakho
    August 25, 2019 at 2:02 am

    1+1= 0 occurred some decades ago, the first 2 figures relate to unjustifiable assumptions, and theories, and the 0 his the ultimate outcome. This (at the cost of harmful repetition) did not start overnight. And, as such, cannot be resolved overnight. We need to get real in order to deal with reality.

  2. Yoshinori Shiozawa
    August 25, 2019 at 6:23 am

    I am not a specialist in econometrics, but as an economist who has worked in economics more than 40 years I may have a useful point of view for econometrics people. My simple proposals are two:

    (1) Stop working on macroeconomic models. Macroeconomy is too complex a system and is adaptable to no macroeconomic models. In that case, the best adjustment for econometrics is to retreat from fitting macroeconomics models to an economy.

    (2) As economy is a structured and structuring complex, the first things to do are to investigate more narrow and restricted relations. For example, one may work by an econometric method how the investment decisions are made in firms. There are many instances that we may find a useful knowledge by working in this piecemeal science.

    Simply stated, the modern science started from Galileo Galilei’s law of falling bodies and Kepler’s three laws of planetary motion. Combining these two results, Newton has succeeded in constructing a universal theory. Even at that time, no electromagnetic forces are not included. Science proceed step by step. Meteorology became a practically useful science long after Newton, in the 20th century. Actual macroeconomics with the aid of econometrics seems like ruinous efforts that seek to predict the weather without having no principal theory like Navier-Stokes equations.

    I read many claims that economic should stop imitating natural sciences, but in my impression, the contrary effort is necessary. We should learn more from the history of modern natural sciences.

    • Robert Locke
      August 25, 2019 at 1:36 pm

      When dealing with uncertainty, deal with people involved in a “science” in which uncertainty Ungewisseheit is integral to what they study, e.g.., the science of war. Nobody can predict outcomes, as Clausewitz knew, because of the unknown in war, so try to develop a science, as he did, that deals with the unknown. See rwer, 2012, The basis of economics from Smith to Carl von Clausewitz.

      • Frank Salter
        August 26, 2019 at 10:20 am

        Before atomic weapons the ability to predict in very general terms was possible. Using the second world as example, once Germany and Japan’s initial advantage in preparedness and the advantage bestowed by the quantity of weaponry and size of their existing armies was withstood then the ability of the allies to outproduce the axis lead to their inevitable defeat. The only true unknown was how long and how much damage would be inflicted.

    • Rob
      August 26, 2019 at 3:26 am

      I read many claims that economic should stop imitating natural sciences, but in my impression, the contrary effort is necessary. We should learn more from the history of modern natural sciences. (Shiozawa Yoshinori, RWER: Econometrics and the problem of unjustified assumptions, 8/25/2019)

      It seems the only kind of “history” Shiozwa knows is a whig interpretation of history.

    • Frank Salter
      August 26, 2019 at 10:31 am

      May I suggest different wording. Rather than “Newton [has] succeeded in constructing a universal theory” it is “Newton succeeded in constructing an abstract theory of gravitation”. This separated the analysis from specific models and revealed the underlying mechanism in an abstract formulation.

  3. Yoshinori Shiozawa
    August 26, 2019 at 4:35 am

    In the beginning of this year (2019), Lars Syll posted an article with the title How to re-establish trust in economics as a science

    I missed this article when it appeared. I believe the basic spirit / orientation is sound and productive. I came to read this article suggested by the recent comments by Ken Zimmerman and Robert Locke, whom I must thank.

    Although I agree with Lars Syll’s basic proposals, the next recommendation seems to contain some misunderstandings about laws in physics and economics.

    (3) Stop pretending that there are laws in economics. There are no universal laws in economics. Economies are not like planetary systems or physics labs. The most we can aspire to in real economies is establishing possible tendencies with varying degrees of generalizability. (Lars Syll 2019/01/03)

    Planetary systems must refer to Kepler’s law of planetary motions. But they are not universal laws. They are precise empirical laws found under the rare circumstance that is called solar system. It is a system where almost all mass is concentrated in the Sun and interactions between planets were negligible. This is also the reason why Kepler could find his laws for Mars but not for Moon (In the case of Moon, one has to solve the three-body problem whose solution cannot be expressed by elementary functions like the case of elliptic curves).

    If Lars Syll wants to find a universal law which describes the (macro)economy, it is hopeless to find a universal law. He is right in this point. But, as the economy is structured and articulated complex, there is a possibility that we can find more solid laws that we find in many stylized facts. For example, circuitists have found that, whatever happens, the sum of money (bank note and credit money in bank accounts) does not change if there is no granting of credit nor return of bank debt. Or more generally, we have the formula:

    ΔM = Gc – Rd,

    where ΔM is the increase/decrease of money for a period, Gc the sum of granted credit (including the purchase of bonds and notes by banks), and Rd the return of bank credit (including the selling of bonds and notes by banks) for the same period.

    Kepler’s three laws of planetary motion and Galileo’s law of falling body and other laws (like isochronism of pendulum) are not universal laws

    Modern physics did not start from the discovery of universal laws (like Newton’s laws of motion and law of universal gravitation). Newton established those laws on the basis of various empirical formulae and this was extremely deductive and abductive efforts (in short theoretical efforts).

    To seek universal laws for macroeconomic system must be unproductive. It is true that we should not seek to explain everything as Lars’s point (1). We should restart our economics where we can build a sounder science: to find empirical formulae at very particular relations between small number of variables at a particular aspect of an economy and theories based on them.

  4. Yoshinori Shiozawa
    August 26, 2019 at 9:32 am

    Robert Rocke,

    Lars Syll wrote in the article I cited above:

    (5) Stop building models and making forecasts of the future based on totally unreal micro-founded macromodels with intertemporally optimizing robot-like representative actors equipped with rational expectations.

    In view of existence of true uncertainty in the economy, this recommendation should be stated more simply

    (5) Stop building (macroeconomic) models and making forecasts of the future.

    Economists since 1950’s are too much influenced (probably unconsciously) by an idea that a good economics should be predictive science. They are haunted by Milton Friedman and his “methodology of positive economics”.

    An economics that has no predictive power can be a useful and truthful science.

  5. Robert Locke
    August 26, 2019 at 9:55 am

    But one that can deal with the unexpected, “scientifically” can be equally if not more useful. That is why I wrote the article rwer, 2012, on Clausewitz, as an economist. In the late 19th century everybody was developing general staffs and playing war games, in the Prussian mode? The science of war is economics, too, in its methods.

    • Robert Locke
      August 26, 2019 at 10:07 am

      Also see, Charles de Gaulle, in an article published in 1934, he tallked about economic mobilization in the U.S. “Mobilisation economique a l’etranger.” cited in my book. The Collapse of the American Management Mystique, 1996, p.256

  6. Yoshinori Shiozawa
    August 26, 2019 at 2:32 pm

    Almost all Post Keynesians admit that uncertainty is important, even crucial factor to be taken in consideration. However, unconditional emphasis of uncertainty has no sense, because there are many economic relations that are not normally disrupted by uncertainty. Economy is a complex which has many articulated parts and some of those parts have solid structure. Because macroeconomists want to grasp overall laws of an economy, they could not arrive to grasp such a structure which lies beneath/inside of the macroeconomy.

    • August 27, 2019 at 12:10 pm

      “Economy is a complex which has many articulated parts and some of those parts have solid structure.” Though the trouble is that those solid parts are nothing more than definitions and identities.
      Human decision making is by its very nature uncertain and if it wasn’t we wouldn’t even need any of the social sciences (https://www.s-e-i.ch/archive/AI.htm).
      You may get headaches from this fact or love doing economics exactly because of the uncertain nature of humans. I prefer the latter while most mainstreamers, I kind of sense, hail from the former herd.

  7. Yoshinori Shiozawa
    August 27, 2019 at 2:40 pm

    Are you really worrying about you may not eat bread tomorrow? You are free to worry about it, but most people do not worry about it with reason, because economy is provided by a mechanism that assures you can buy bread whenever you want if you have enough money. To make clear how this mechanism works is a part of economics.

    Economic events have a large range of spectrum from very certain ones to very uncertain ones. To know structure of such spectrum is also a part of economics.

  8. Ken Zimmerman
    August 28, 2019 at 12:42 pm

    A few comments for Lars and others. First, as Lars says, econometrics has an illness. That illness is too much derivation from unverifiable assumptions and too little following empirical data. Human life and all the things humans create are characterized above all by uncertainty. It is irreducible. Physics is full of theoretical constructs like gravity, motion, etc. that cannot be explained, except by equations that assume the constructs exist as described by the equations. That’s circular in anyone’s book. A physicist friend joked with the question, why do metal rods stick to one another? The theoretical construct magnetism is the answer most often given. His answer, love. One’s just as likely as the other.

    Taking physics as its model, some in economics have constructed theoretical constructs like utility, price, market, etc. Some of these are used in ordinary life, also, but with very different meanings. And like physics the constructs are represented by equations based on the assumption that the constructs are as the equations describe them. Assumptions based on nothing, no or little empirical data.

    I agree with Yoshinori, stop building models and using them to make forecasts of the future. But I suggest we do a lot more of what Robert suggests. While no economics can predict the future, it can, if solidly based in detailed empirical studies help us deal with the unexpected effectively and prepare us to be ready for the future, no matter what it brings. This requires much more than theoretical constructs. And it requires empirically-directed and roll-up-your-sleeves economists who can work on-their-feet. Most economists have no such experience and as the field is organized today never will.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.