Home > Uncategorized > On the non-applicability of statistical theory

## On the non-applicability of statistical theory

from Lars Syll

Eminent statistician David Salsburg is rightfully very critical of the way social scientists — including economists and econometricians — uncritically and without arguments have come to simply assume that one can apply probability distributions from statistical theory on their own area of research:

We assume there is an abstract space of elementary things called ‘events’ … If a measure on the abstract space of events fulfills certain axioms, then it is a probability. To use probability in real life, we have to identify this space of events and do so with sufficient specificity to allow us to actually calculate probability measurements on that space … Unless we can identify [this] abstract space, the probability statements that emerge from statistical analyses will have many different and sometimes contrary meanings …

Kolmogorov established the mathematical meaning of probability: Probability is a measure of sets in an abstract space of events. All the mathematical properties of probability can be derived from this definition. When we wish to apply probability to real life, we need to identify that abstract space of events for the particular problem at hand … It is not well established when statistical methods are used for observational studies … If we cannot identify the space of events that generate the probabilities being calculated, then one model is no more valid than another … As statistical models are used more and more for observational studies to assist in social decisions by government and advocacy groups, this fundamental failure to be able to derive probabilities without ambiguity will cast doubt on the usefulness of these methods.

Wise words well worth pondering on.

As long as economists and statisticians cannot really identify their statistical theories with real-world phenomena there is no real warrant for taking their statistical inferences seriously.

Just as there is no such thing as a ‘free lunch,’ there is no such thing as a ‘free probability.’ To be able at all to talk about probabilities, you have to specify a model. If there is no chance set-up or model that generates the probabilistic outcomes or events – in statistics one refers to any process where you observe or measure as an experiment (rolling a die) and the results obtained as the outcomes or events (number of points rolled with the die, being e. g. 3 or 5) of the experiment – there strictly seen is no event at all.

Probability is — as strongly argued by Keynes — a relational element. It always must come with a specification of the model from which it is calculated. And then to be of any empirical scientific value it has to be shown to coincide with (or at least converge to) real data generating processes or structures – something seldom or never done!

And this is the basic problem with economic data. If you have a fair roulette-wheel, you can arguably specify probabilities and probability density distributions. But how do you conceive of the analogous ‘nomological machines’ for prices, gross domestic product, income distribution etc? Only by a leap of faith. And that does not suffice. You have to come up with some really good arguments if you want to persuade people into believing in the existence of socio-economic structures that generate data with characteristics conceivable as stochastic events portrayed by probabilistic density distributions!

1. February 3, 2017 at 4:18 pm

My view is yeah, no model in general can be assumed to be more true than another. Also, often one find a statistical model which is equilvent to a deterministic one (or any other kind, though to me these basicaly are the 2 main classes—unless one sees computer simulations as another class–and these have both aspects).

General relativity—determinisitc–has recently been recast and derived from statistics. Chaos theory is based on this concept as well.

So in a sense there is justification ‘a priori’ for any model, and also none for making a model—-eg writing something down about the world using words, equations….Newton didnt need to model the universe. People do this for fun, interest, out of habit or maybe due to genetics. ‘I wonder why i wonder why i…”—R Feynman. A model is like a piece of art or maybe fashionable or functional clothing—useful for some things and not others. (Some suggest alot of published models are mostly useful to owners of journals—-they fill pages which can be sold or used as part of CV to jusify jobs abd grants.)

Long ago the catholic church had no use for Galileo’s model. Though flawed that model whther correct or flawed was useful for making planes, rockets….I talked to an econoist recently (basically about a job) who sort of does ‘bayesian econophysics’ . He didnt have much use for the kinds of models i was interested in developing. (Actually i wanted to first spend a bit of time studying all the existing related models, and maybe not even get past that, since improved models may be difficult to find. So that might have turned into a history of ideas and modeling, along the lines of North’s book ‘measuring the universe’ which summarized the history of models of the universe which gradually ended up with Einstein’s.)

There is so much data (sometimes contradictory) in economics its hard to ‘unfiy it’. People i know who have gotten jobs in economic usually use one small set of data and model one phenomena (consumer buying, allocation of reources i firms). That way you get a publisashable paper, but to me often on what are trivial topics , and also have ‘rigor mortis’—–filled with equations, statistical analyses, etc.

(nowadays eople are using stats to study logical systems—eg proofs–which are basically deterministic, though have some statistical features. Traveling salesman type problems ar similar.) .