Home > Uncategorized > Models and forecasts

Models and forecasts

from Lars Syll

Yesterday John Kay had an interesting article about models and forecasting in Financial Times:

A bane of this economist’s life is the belief that economics is clairvoyance. I should, according to this view, be offering prognostications of what gross domestic product growth will be this year and when the central bank will raise interest rates …

What was the right answer on January 1 1989 to the question “will the Berlin Wall be pulled down in 1989?” A shrewd commentator would have said (though few did) something like “almost certainly the Wall will stand but you should understand the potentially destructive forces undermining the Soviet engine and the East German state”. That type of response combines probabilistic and narrative thinking.

But people long for certainties, though they know they cannot have them. I have learnt that few really want answers when they ask me to predict GDP growth or advise whether interest rates will rise in the third quarter. It is usually easy to move the subject on to something more interesting than macroeconomic forecasting.

Kay’s remarks — and Tony Yates comments on them — made me think about an article that Oxford macroeconomist Simon Wren-Lewis wrote on models and forecasts a couple of years ago, saying that “macroeconomic forecasts are always bad,” but, on the other hand, since they are “probably no worse than intelligent guesses” and anyway are “not obviously harmful,” we have no reason to complain.

The thing is that Wren-Lewis is wrong. These forecasting models and the organizations and persons around them do cost society billions of pounds, euros and dollars every year. And if they do not produce anything better than “intelligent guesswork,” I’m afraid most taxpayers would say that they are certainly not harmless at all!

Mainstream neoclassical economists often maintain – usually referring to the methodological individualism of Milton Friedman – that it doesn’t matter if the assumptions of the models they use are realistic or not. What matters is if the predictions are right or not. But, if so, then the only conclusion we can make is – throw away the garbage! Because, oh dear, oh dear, how wrong they have been!

When Simon Potter a couple of years ago analyzed the predictions that the Federal Reserve Bank of New York did on the development of real GDP and unemployment for the years 2007-2010, it turned out that the predictions were wrong with respectively 5.9% and 4.4% – which is equivalent to 6 millions of unemployed. In other words — the “rigorous” and “precise” macroeconomic mathematical-statistical forecasting models were wrong. And the rest of us have to pay.

Potter is not the only one who lately has criticized the forecasting business. John Mingers comes to essentially the same conclusion when scrutinizing it from a somewhat more theoretical angle.

The empirical and theoretical evidence is clear. Predictions and forecasts are inherently difficult to make in a socio-economic domain where genuine uncertainty and unknown unknowns often rule the roost. The real processes that underly the time series that economists use to make their predictions and forecasts do not confirm with the assumptions made in the applied statistical and econometric models. Much less is a fortiori predictable than standardly — and uncritically — assumed. The forecasting models fail to a large extent because the kind of uncertainty that faces humans and societies actually makes the models strictly seen inapplicable. The future is inherently unknowable — and using statistics, econometrics, decision theory or game theory, does not in the least overcome this ontological fact. The economic future is not something that we normally can predict in advance. Better then to accept that as a rule “we simply do not know.”

So, to say that this counterproductive forecasting activity is harmless, simply isn’t true. Spending billions after billions of hard-earned money on an activity that is no better than “intelligent guesswork,” is doing harm to our economies.

In New York State, Section 899 of the Code of Criminal Procedure provides that persons “Pretending to Forecast the Future” shall be considered disorderly under subdivision 3, Section 901 of the Code and liable to a fine of $250 and/or six months in prison. Although the law does not apply to “ecclesiastical bodies acting in good faith and without fees,” I’m not sure where that leaves macroeconomic model-builders and other forecasters …

In an interesting discussion on the hopelessness of accurately modeling what will happen in the real world, Nobel laureate Kenneth Arrow – in Eminent Economists: Their Life Philosophies (CUP 1992) – pretty well sums up what the forecasting business is all about:

It is my view that most individuals underestimate the uncertainty of the world. This is almost as true of economists and other specialists as it is of the lay public. To me our knowledge of the way things work, in society or in nature, comes trailing clouds of vagueness … Experience during World War II as a weather forecaster added the news that the natural world as also unpredictable. cloudsAn incident illustrates both uncertainty and the unwillingness to entertain it. Some of my colleagues had the responsi-bility of preparing long-range weather forecasts, i.e., for the following month. The statisticians among us subjected these forecasts to verification and found they differed in no way from chance. The forecasters themselves were convinced and requested that the forecasts be discontinued. The reply read approximately like this: ‘The Commanding General is well aware that the forecasts are no good. However, he needs them for planning purposes.’

  1. January 11, 2016 at 3:16 pm

    My review of John Kay’s excellent new book OTHER PEOPLES MONEY , is at http://www.seekingalpha.com . His honest appraisal of the state of economic models and that of finance itself is damning ! I gave it a ” must read “.

  2. blocke
    January 11, 2016 at 3:26 pm

    OK, if the predication business is not good, then what do we do? I remember reading an analysis of the failure of German economists to predict accurately labor market needs, which led to very poor educational planning, how many electronics engineers, etc. will be needed in ten years, completely missing the forecasts. But the commentators noted that it did not matter because the labor force was sufficiently educated for it to adapt rapidly to actual needs when the demand did not mirror forecasts. Have a labor force that is adaptable to demands whatever they are is the answer. That is the sort of education that is necessary: highly numerate and literate; they’ll serve the needs well regardless of the predictions.

  3. January 11, 2016 at 7:35 pm

    The belief, and actions based on that belief by not just economists, but sociologists, statisticians, and psychologists that either an individual’s or a society’s actions can be predicted is an interesting study in social history. I know. I do such work for many of my clients. But social scientists are just the latest in a long line of humans and human institutions to believe in and take on the task of creating forecasts of all aspects of human actions and natural world events. The Delphi Oracle of Ancient Greece in fact so much captures the imagination of current “scientific” forecasters that a method of forecasting is named after it. Described in the textbooks as “…a structured communication technique or method, originally developed as a systematic, interactive forecasting method which relies on a panel of experts.” But forecasting or predicting is not about knowing the future. It is rather about “believing” that we can know the future and thus control our own destinies. It’s a crutch to help humans feel better about their place in the world, today and tomorrow. The Ancient Greeks knew this. This brings us to Lars’ issue of the cost for this crutch. Certainly we cannot eliminate the crutch totally. But on the other hand economists’ predictions take the crutch too far. They say in effect, our predictions are, unlike some others important and greatly useful, so depend on them wholly. This is not just expensive in terms of money, but also in terms of dysfunctional actions. Plus in most instances preparing for and adapting to the future works better if we don’t “put all our eggs in one forecasted basket.” And that is the difference between predicting and planning.

  4. January 11, 2016 at 8:08 pm

    Error! Everything can predict with great accuracy if there is a good model and initial conditions are known and possible impacts, but the options may be many. Uncertainty in quantum mechanics is not due to random variables, but the many options. Currently, however, the prognosis is very easy. The Great Depression of the century begins and it will last very long.

  5. January 11, 2016 at 10:17 pm

    In Germany we do have the “Ifo Geschäftsklima Index” (roughly translates into “Ifo Business climate index”) which is estimated on a quarterly basis. In which we ask a selected number of business individuals about their expectations how the market will develop in the upcoming quarter. And we have others who estimate a prediction of this precise index. So we have people estimating what the gut feeling of the business insiders is about the market climate in the future. And this is one of the most important business cycle indexes in Germany.

  6. Oleg
    January 11, 2016 at 10:26 pm

    I was always puzzled why economists think their job is to explain what happened in the past and forecast what will happen in future. Both activities if they are not a part of something bigger are pretty useless.

    We need to develop decision recommendations – you do this and it will lead to that result. Isn’t that obvious?.. I guess not for the economists.

  7. January 12, 2016 at 2:02 am

    One can go to far with this unpredictability stuff. The other half – perhaps the more pernicious half – of bad economists’ pretense of knowing that hard things like completely predicting the future is easy, is the pretense that easy things are hard. Thinking that one cannot predict or control the future at all is a worse “crutch” than a pretense at infallibility.

    For instance, it is very easy to accurately predict that a sane society could always accurately predict its unemployment rate. Zero. For it would perform the very easy task of completely eliminating unemployment. Simply by deciding to.

    And the prediction that this would deliver great benefits to the society as a whole and the individuals composing – that there would be no drawbacks – is completely logical, accurate and well-founded.

    Unpredictabilityism gone amok is very like the unforgettable coin toss scenes in No Country For Old Men, where Anton Chigurh insists that the best he can do is flip a coin to decide to kill or not kill people who he doesn’t particularly need or want to kill. Our “advanced” modern societies decide the fate of the jobless, the poor, the disadvantaged, the unlucky, the homeless, who of course the upper classes don’t particularly need or want to kill – in much the same way.

    To insistence that one cannot do better, that one cannot predict or control, that this is just fate or the way things are, that it is unavoidable, that this is not a simple decision – the answer is ” Do you have any idea how crazy you are?”

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s