Home > Uncategorized > Keynes’ critique of econometrics — still valid after all these years

Keynes’ critique of econometrics — still valid after all these years

from Lars Syll

To apply statistical and mathematical methods to the real-world economy, the econometrician has to make some quite strong assumptions. In a review of Tinbergen’s econometric work — published in The Economic Journal in 1939 — John Maynard Keynes gave a comprehensive critique of Tinbergen’s work, focusing on the limiting and unreal character of the assumptions that econometric analyses build on:

(1) Completeness: Where Tinbergen attempts to specify and quantify which different factors influence the business cycle, Keynes maintains there has to be a complete list of all the relevant factors to avoid misspecification and spurious causal claims. Usually, this problem is ‘solved’ by econometricians assuming that they somehow have a ‘correct’ model specification. Keynes is, to put it mildly, unconvinced:

istheseptuagintaIt will be remembered that the seventy translators of the Septuagint were shut up in seventy separate rooms with the Hebrew text and brought out with them, when they emerged, seventy identical translations. Would the same miracle be vouchsafed if seventy multiple correlators were shut up with the same statistical material? And anyhow, I suppose, if each had a different economist perched on his a priori, that would make a difference to the outcome.

J M Keynes

(2) Homogeneity: To make inductive inferences possible — and being able to apply econometrics — the system we try to analyse has to have a large degree of ‘homogeneity.’ According to Keynes most social and economic systems — especially from the perspective of real historical time — lack that ‘homogeneity.’  It is not always possible to take repeated samples from a fixed population when we were analysing real-world economies. In many cases, there simply are no reasons at all to assume the samples to be homogenous.

(3) Stability: Tinbergen assumes there is a stable spatio-temporal relationship between the variables his econometric models analyze. But Keynes argued that it was not really possible to make inductive generalisations based on correlations in one sample. As later studies of ‘regime shifts’ and ‘structural breaks’ have shown us, it is exceedingly difficult to find and establish the existence of stable econometric parameters for anything but rather short time series.

(4) Measurability: Tinbergen’s model assumes that all relevant factors are measurable. Keynes questions if it is possible to adequately quantify and measure things like expectations and political and psychological factors. And more than anything, he questioned — both on epistemological and ontological grounds — that it was always and everywhere possible to measure real-world uncertainty with the help of probabilistic risk measures. Thinking otherwise can, as Keynes wrote, “only lead to error and delusion.”

(5) Independence: Tinbergen assumes that the variables he treats are independent (still a standard assumption in econometrics). Keynes argues that in such a complex, organic and evolutionary system as an economy, independence is a deeply unrealistic assumption to make. Building econometric models from that kind of simplistic and unrealistic assumptions risk producing nothing but spurious correlations and causalities. Real-world economies are organic systems for which the statistical methods used in econometrics are ill-suited, or even, strictly seen, inapplicable. Mechanical probabilistic models have little leverage when applied to non-atomic evolving organic systems — such as economies.

Building econometric models can’t be a goal in itself. Good econometric models are means that make it possible for us to infer things about the real-world systems they ‘represent.’ If we can’t show that the mechanisms or causes that we isolate and handle in our econometric models are ‘exportable’ to the real-world, they are of limited value to our understanding, explanations or predictions of real-world economic systems.

(6) Linearity: To make his models tractable, Tinbergen assumes the relationships between the variables he study to be linear. This is still standard procedure today, but as Keynes writes:

It is a very drastic and usually improbable postulate to suppose that all economic forces are of this character, producing independent changes in the phenomenon under investigation which are directly proportional to the changes in themselves; indeed, it is ridiculous.

To Keynes, it was a ‘fallacy of reification’ to assume that all quantities are additive (an assumption closely linked to independence and linearity).

2014+22keynes%20illo2The unpopularity of the principle of organic unities shows very clearly how great is the danger of the assumption of unproved additive formulas. The fallacy, of which ignorance of organic unity is a particular instance, may perhaps be mathematically represented thus: suppose f(x) is the goodness of x and f(y) is the goodness of y. It is then assumed that the goodness of x and y together is f(x) + f(y) when it is clearly f(x + y) and only in special cases will it be true that f(x + y) = f(x) + f(y). It is plain that it is never legitimate to assume this property in the case of any given function without proof.

J. M. Keynes “Ethics in Relation to Conduct” (1903)

Real-world social systems are usually not governed by stable causal mechanisms or capacities. The kinds of ‘laws’ and relations that econometrics has established, are laws and relations about entities in models that presuppose causal mechanisms and variables — and the relationship between them — being linear, additive, homogenous, stable, invariant and atomistic. But — when causal mechanisms operate in the real world they only do it in ever-changing and unstable combinations where the whole is more than a mechanical sum of parts. Since statisticians and econometricians have not been able to convincingly warrant their assumptions of homogeneity, stability, invariance, independence, additivity as being ontologically isomorphic to real-world economic systems, Keynes’ critique is still valid.

In his critique of Tinbergen, Keynes points us to the fundamental logical, epistemological and ontological problems of applying statistical methods to a basically unpredictable, uncertain, complex, unstable, interdependent, and ever-changing social reality. Methods designed to analyse repeated sampling in controlled experiments under fixed conditions are not easily extended to an organic and non-atomistic world where time and history play decisive roles.

Econometric modelling should never be a substitute for thinking. From that perspective, it is really depressing to see how much of Keynes’ critique of the pioneering econometrics in the 1930s-1940s is still relevant today.

The general line you take is interesting and useful. It is, of course, not exactly comparable with mine. I was raising the logical difficulties. You say in effect that, if one was to take these seriously, one would give up the ghost in the first lap, but that the method, used judiciously as an aid to more theoretical enquiries and as a means of suggesting possibilities and probabilities rather than anything else, taken with enough grains of salt and applied with superlative common sense, won’t do much harm. I should quite agree with that. That is how the method ought to be used.

Keynes, letter to E.J. Broster, December 19, 1939

  1. Craig
    March 31, 2018 at 4:15 am

    “Unresolvable” problems do not resolve when a new paradigm and its new insights are not perceived.

  2. edward ross
    March 31, 2018 at 4:29 am

    Thank you Lars Syll for your academic explanation of why many of Keynes ideas are still valid today. My support for Keynes begins the fact that I was born in 1936 an may have only begun to understand the world around me. As a result I grew up listening to my elders accounts of how the Keynesian ideas lifted them out of the abject poverty they had experienced during the great depression. Furthermore as I have often mentioned before Keynes was very much against free trade, the international mobility of capital and foreign ownership of national assets. Here while I acknowledge that to some extent the world has changed since then essence of Keynes principles are just as valuable today as they were then.Ted

  3. Frank Salter
    March 31, 2018 at 7:28 am

    Keynes was right to condemn the manner in which econometrics are used. This still continues.

    That there is there so little condemnation of elementary errors of analysis implies that there are major failings in economics education. Elementary errors include fitting fractional powers to or using logarithms of parameters and then to claim the fitted relationship describes some underlying theoretical truth. Such fits can only be methods of interpolating data. The mathematical manipulations preclude the very possibility of their being a valid theoretical theory. But still papers claiming validity are accepted by eminent peer reviewed journals. From which it must be concluded that the majority of academic economists do not understand the constraints which a correct use of mathematical physics requires.

    Without this failure being addressed, the present situation will continue indefinitely. This is illustrated admirably by:

    “Moreover, the production function has been a powerful instrument of miseducation. The student of economic theory is taught to write 𝑂 = 𝘧 (𝐿, 𝐶) where 𝐿 is a quantity of labour, 𝐶 a quantity of capital and 𝑂 a rate of output of commodities. He is instructed to assume all workers alike, and to measure 𝐿 in man-hours of labour; he is told something about the index-number problem involved in choosing a unit of output ; and then he is hurried on to the next question, in the hope that he will forget to ask in what units 𝐶 is measured. Before ever he does ask, he has become a professor, and so sloppy habits of thought are handed on from one generation to the next.”
    Joan Robinson, The Production Function and the Theory of Capital, Review of Economic Studies, 21(2), 1953−1954, p. 81.

    • edward ross
      April 1, 2018 at 5:02 am

      Yes Frank Slater I agree with you, there is a problem with economic education .Tony Lawson has also mentioned this some time ago, as some other contributors to RWEA have done. Some of them have suggested the earlier this starts in their education the better. For some time I have been trying to write paper that relates to real people and students concerns with the present mainstream economic system, not a lot of confusing irrelevant academic theory and models. I may be wrong but I think this sort of approach could encourage more students to take up economic study, Ted

      • Frank Salter
        April 1, 2018 at 9:51 am

        You refer to “a lot of confusing irrelevant academic theory and models”.

        This is exactly the point I have been making. So much of what is being taught can be eliminated by a better understanding of what are essentially the ‘rules’ of physical reality. All analysis claiming production functions to be theoretically valid, not just curve fits to data, can be rejected out of hand. A google-scholar search for “production function” registers “about 871,000”. A quick look at titles which include “production function” suggests that most are presenting them as of theoretical validity.
        It is the tools to understand physical reality correctly which are necessary for critical analysis by economists. It appears that these are taught to scientists and engineers but not to economists, why not?

  4. April 1, 2018 at 2:02 am

    It is not so much that his criticisms of econometrics are “still” valid so much as the validity of his criticisms is nearing a cyclical peak as more than half a century of the “don’t tax business” crowd have had a massive influence on the industry to the point that we are once again in dangerous territory.

  5. Jan Milch
    April 1, 2018 at 8:08 am

    Lay to that J.M Keynes, as a ardent student of Alfred Marshall´s econometrics simple efforts in this area,say in the General Theory that:
    “It is a great fault of symbolic pseudo-mathematical methods of formalizing a system of economic analysis…that they expressly assume strict independence between the factors involved and lose all their cogency and authority if this hypothesis is disallowed; whereas, in ordinary discourse, where we are not blindly manipulating but know all the time what we are doing and what the words mean, we can keep “at the back of our heads” the necessary reserves and qualifications and the adjustments which we shall have to make later on, in a way in which we cannot keep partial differentials “at the back” of several pages of algebra which assume that they all vanish. Too large a proportion of recent “mathematical” economics are mere concoctions, as imprecise as the initial assumptions they rest on, which allow the author to lose sight of the complexities and interdependencies of the real world in a maze of pretentious and unhelpful symbols.” (pp. 297-298)

  6. Norman L. Roth
    April 1, 2018 at 2:58 pm

    When it comes to your commentary about the {Cobb-Douglas only ?}production function, most of you fellas sound like “the last cowboys to discover horses” .I don’t even know where to begin. But, instead of proclaiming the “everything has to start from scratch again” amateurishness, I would strongly suggest that you carefully introduce yourselves to TELOS & TECHNOS, especially Chapter 3. AND Chapter 5, Capital. There you will find ,in considerably less indignant language {and all the fancy geometry you can digest} the roots of the fallacies ,in the otherwise well intentioned construct of the “Production Function”. AND, a remedial paradigm that cannot be derived from the kind of thinking that begat the “Production Function. Especially as formulated by it’s most famous advocate, the, in many ways admirable, hard-working socialist-minded, & later U.S. Senator, Paul Douglas. He was certainly NOT a “sloppy-minded” professor who intended to “miseducate” ! But a man of his own time. Indeed,All the K & L’s of the Production Function in all their aggregative,”homogeneous” & spuriously quantified glory, were rooted in a quasi-Marxist intellectual fuzziness: i.e In order to make the case for rewarding the more “virtuous” productivity of L{“homogeneous” } labor} vs. the {assumed} shady claims to reward for productive virtue claimed by {undefined & ‘homogeneous} K {das KAPITAL}. Douglas and his contemporaries were well aware of the deficiencies of their mathematical symbolism. But they made an honest try at making these concepts more “mathematically tractable”. It should also be emphasized that the construct of the PRODUCTION FUNCTION was more related to manufacturing industries than any other sector of the economy.
    Thank you for your patience. Please GOOGLE: {1} Norman L. Roth {2} Norman L. Roth, RWER

    • Frank Salter
      April 2, 2018 at 7:01 am

      Norman L. Roth presents a clear and accurate commentary of what Cobb and Douglas intended. They ended their analysis with the possibility of other functional forms being more appropriate for other data.

      Since Cobb and Douglas’ paper a multitude of better fitting relationships to specific data have been introduced and each treated as if were a better theory rather than a better fit to different data. There are some 4000 (google) papers with production function in their titles. not one able to be valid theory theory.

      I have experience of discussing, with its authors, a paper dealing with a subject with which I have a great deal of expertise and trying to explain to them that what they described had a very different explanation than the one they offered. Their curve fitting had excluded very important factors and they had conflated some data inappropriately. They airily dismissed my concerns with “the error terms” will deal with all problems. The fact that different parameters in their fitted equations would have produced different conclusions appeared not to enter their thinking in any way.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: