Home > Uncategorized > The econometric dream-world

The econometric dream-world

from Lars Syll

Trygve Haavelmo — with the completion (in 1958) of the twenty-fifth volume of Econometrica — assessed the role of econometrics in the advancement of economics, and although mainly positive of the “repair work” and “clearing-up work” done, he also found some grounds for despair:

We have found certain general principles which would seem to make good sense. Essentially, these principles are based on the reasonable idea that, if an economic model is in fact “correct” or “true,” we can say something a priori about the way in which the data emerging from it must behave. We can say something, a priori, about whether it is theoretically possible to estimate the parameters involved. And we can decide, a priori, what the proper estimation procedure should be … But the concrete results of these efforts have often been a seemingly lower degree of accuracy of the would-be economic laws (i.e., larger residuals), or coefficients that seem a priori less reasonable than those obtained by using cruder or clearly inconsistent methods.

Haavelmo-intro-2-125397_630x210There is the possibility that the more stringent methods we have been striving to develop have actually opened our eyes to recognize a plain fact: viz., that the “laws” of economics are not very accurate in the sense of a close fit, and that we have been living in a dream-world of large but somewhat superficial or spurious correlations.

Another of the founding fathers of modern probabilistic econometrics, Ragnar Frisch, shared Haavelmo’s doubts on the applicability of econometrics:

sp9997db.hovedspalteI have personally always been skeptical of the possibility of making macroeconomic predictions about the development that will follow on the basis of given initial conditions … I have believed that the analytical work will give higher yields – now and in the near future – if they become applied in macroeconomic decision models where the line of thought is the following: “If this or that policy is made, and these conditions are met in the period under consideration, probably a tendency to go in this or that direction is created”.

Ragnar Frisch

Econometrics may be an informative tool for research. But if its practitioners do not investigate and make an effort to provide a justification for the credibility of the assumptions on which they erect their building, it will not fulfil its tasks. There is a gap between its aspirations and its accomplishments. Without more supportive evidence to substantiate its claims, critics will continue to consider its ultimate argument as a mixture of rather unhelpful metaphors and metaphysics. Maintaining economics should be a science in the ‘true knowledge’ business, yours truly remains a sceptic of the pretences and aspirations of econometrics. So far, I cannot really see that it has yielded very much in terms of relevant, interesting economic knowledge.

The marginal return on its ever-higher technical sophistication in no way makes up for the lack of serious under-labouring of its deeper philosophical and methodological foundations that already Keynes complained about. The rather one-sided emphasis of usefulness and its concomitant instrumentalist justification cannot hide that the legions of probabilistic econometricians who give supportive evidence for their considering it “fruitful to believe” in the possibility of treating unique economic data as the observable results of random drawings from an imaginary sampling of an imaginary population, are skating on thin ice. After years having analyzed ts ontological and epistemological foundations, I cannot but conclude that econometrics on the whole has not delivered ‘truth,’ nor robust forecasts.

  1. Econoclast
    May 2, 2023 at 9:46 pm

    I vaguely (as an old man with memory problems) recall learning in 60s grad school that Milton Friedman stated that the accuracy of assumptions doesn’t matter, as long as the results work. Or something to that effect. Poisoning the well of United States economics, as well as that in Chile, Russia, and elsewhere.

  2. Steven Klees
    May 3, 2023 at 6:43 pm

    I really believe that econometrics is the emperor’s new clothes. I think that future generations will look back in astonishment that so many otherwise intelligent people could believe that useful results can come from torturing some data with what are necessarily wildly inaccurate models to get answers that are ALWAYS contested by someone else’s regression. I believe I have convincingly shown this in my article: “Inferences from Regression Analysis: Are They Valid?” Real World Economics Review, April 2016, 74, 85-97 that can be found at: http://www.paecon.net/PAEReview/issue74/Klees74.pdf
    I have never heard a sensible refutation.

  3. lars syll
    May 3, 2023 at 8:17 pm

    Great article in a great journal:)

  4. gerald holtham
    May 3, 2023 at 10:49 pm

    A careful qualitative analysis of a particular historical event can, of course, produce convincing stories about what caused what. But Steven Kees’ assertion that such analysis is potentially “generalisable” surely is subject to all the objections he directs at quantitative analysis. Given the myriad factors potentially involved and the complex web of causation how can we possibly know whether what we have learned about situation X will be informative about situation Y? Theories don’t have to be expressed mathematically to be mis-specified. What about all the potential factors that, contingently, were not at work in X so did not enter the story? How can we know that some of them will not dominate the outcome in Y? Problems in epistemology are not solved by refusing to use numbers.
    Leamer’s point about no-one taking other people’s data seriously is well made. All too often empirical research in social studies is all about making a case and establishing personal priority and no serious effort is made to explain and encompass the research of others where it differs from one’s own. This vice is common in econometricians but is not confined to them. And while common it is not inevitable. The bottom line is that many questions do not yield to regression analysis but some can.
    Oddly enough, we have learned a few broad things about the way an economy behaves. Historicall or case studies, cross-tabulation and statistical analysis have all played a part in that. The results are more meagre than they should be because of the massive misallocation of intellectual resources in the economics trade. We can probably agree on that.
    I shall not attempt a reply to Lars. He repeats himself in misunderstanding the basis of econometrics in sampling theory. I would be repeating myself if I said more.

  5. Steven Klees
    May 7, 2023 at 10:07 pm

    I have no problem with quantification. Crosstabs are real, face valid data. Regression coefficients are not. The latter comes from torturing those crosstabs to get the effect of one variable holding others constant. We know crosstabs can’t do that – well, neither can regressions. They take useful quantitative data and turn it into nonsense numbers, whose causal inference is easily disputed by someone else’s model. The issue is not generalizability. You can’t generalize from nonsense numbers. You can reason your way to causal inferences from crosstabs and generalize those inferences if you have a good sample — but it is clear that crosstabulations can support different inferences. But at least they are based on real data. And, of course, you can make sensible causal inferences from small sample qualitative data and it is clear too, that those may be contested. Good theories and clear reasoning combined with valid quantitative (crosstabs) and/or qualitative data are all we have to support our research and policy arguments.

  6. gerald holtham
    May 7, 2023 at 11:22 pm

    i don’t think it is appropriate to rule out any method of empirical investigation. Regression analysis can be misused but it can identify leading indicators for forecasting purposes and it can test a specified theory. Any research should use the most appropriate method given the particular data available and the question being addressed

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.