Home > Uncategorized > Why statistics does not give us causality

Why statistics does not give us causality

from Lars Syll

If contributions made by statisticians to the understanding of causation are to be taken over with advantage in any specific field of inquiry, then what is crucial is that the right relationship should exist between statistical and subject-matter concerns …

introduction-to-statistical-inferenceWhere the ultimate aim of research is not prediction per se but rather causal explanation, an idea of causation that is expressed in terms of predictive power — as, for example, ‘Granger’ causation — is likely to be found wanting. Causal explanations cannot be arrived at through statistical methodology alone: a subject-matter input is also required in the form of background knowledge and, crucially, theory …

Likewise, the idea of causation as consequential manipulation is apt to research that can be undertaken primarily through experimental methods and, especially to ‘practical science’ where the central concern is indeed with ‘the consequences of performing particular acts’. The development of this idea in the context of medical and agricultural research is as understandable as the development of that of causation as robust dependence within applied econometrics. However, the extension of the manipulative approach into sociology would not appear promising, other than in rather special circumstances … The more fundamental difficulty is that, under the — highly anthropocentric — principle of ‘no causation without manipulation’, the recognition that can be given to the action of individuals as having causal force is in fact peculiarly limited.

John H. Goldthorpe

Causality in social sciences — and economics — can never solely be a question of statistical inference.  Causality entails more than predictability, and to really in depth explain social phenomena require theory. Analysis of variation — the foundation of all econometrics — can never in itself reveal how these variations are brought about. First, when we are able to tie actions, processes or structures to the statistical relations detected, can we say that we are getting at relevant explanations of causation.

5cd674ec7348d0620e102a79a71f0063Most facts have many different, possible, alternative explanations, but we want to find the best of all contrastive (since all real explanation takes place relative to a set of alternatives) explanations. So which is the best explanation? Many scientists, influenced by statistical reasoning, think that the likeliest explanation is the best explanation. But the likelihood of x is not in itself a strong argument for thinking it explains y. I would rather argue that what makes one explanation better than another are things like aiming for and finding powerful, deep, causal, features and mechanisms that we have warranted and justified reasons to believe in. Statistical — especially the variety based on a Bayesian epistemology — reasoning generally has no room for these kinds of explanatory considerations. The only thing that matters is the probabilistic relation between evidence and hypothesis. That is also one of the main reasons I find abduction — inference to the best explanation — a better description and account of what constitute actual scientific reasoning and inferences.

For more on these issues — see the chapter “Capturing causality in economics and the limits of statistical inference” in my On the use and misuse of theories and models in economics.

In the social sciences … regression is used to discover relationships or to disentangle cause and effect. However, investigators have only vague ideas as to the relevant variables and their causal order; functional forms are chosen on the basis of convenience or familiarity; serious problems of measurement are often encountered.

Regression may offer useful ways of summarizing the data and making predictions. Investigators may be able to use summaries and predictions to draw substantive conclusions. However, I see no cases in which regression equations, let alone the more complex methods, have succeeded as engines for discovering causal relationships.

David Freedman

Some statisticians and data scientists think that algorithmic formalisms somehow give them access to causality. That is, however, simply not true. Assuming ‘convenient’ things like faithfulness or stability is not to give proofs. It’s to assume what has to be proven. Deductive-axiomatic methods used in statistics do no produce evidence for causal inferences. The real causality we are searching for is the one existing in the real world around us. If there is no warranted connection between axiomatically derived theorems and the real-world, well, then we haven’t really obtained the causation we are looking for.

  1. Ken Zimmerman
    June 25, 2019 at 11:53 am

    Consider this regarding causation. Most facts have many different, possible, alternative explanations, but we want to find the best of all contrastive (since all real explanation takes place relative to a set of alternatives) explanations. But here is no means, and never has been for humans to pick the “best” explanation. Our approach to causation needs to change. Causation is not a thing. It is a story. A story that explains within the framework of a specific culture how events or actors relate to one another, and how both relate to human culture and society. For example, every society have a creation story that explains the origins of that society and its place in the “universe.” That’s causation, a story. A believable story we can share. But this “believable” story is still part of complex events. Complexity makes identifying causality in a predictive way impossible.

    The central question is why did humans create, invent causality? It was invented as a way for humans to do what they do best, make up a story. In this case a story that explains to them why some event or action occurred as it did, when it did. An example from science. What’s the cause of a nuclear explosion? Nuclear explosions occur when atoms from atomically unstable elements such as Uranium 238 are forced to collide with one another in a confined space. In colliding the atoms are broken apart or fused into larger atoms. The strength of the explosion is the result how many and how quickly these atoms collide, and which element is chosen. The mass of the element necessary for the explosion is called “critical mass.” The explosion ends either when the atoms are no longer confined or when the element no longer produces enough atoms to maintain the collisions. This is the story of nuclear explosions. Perhaps at another time humans will create another story to explain nuclear explosions. This one satisfies for the present.

  2. Helen Sakho
    June 26, 2019 at 2:32 am

    The same old story of “9 out of 10” economists agree that X is caused by Y, but to prove otherwise buy 11 text books to prove them wrong!

    That is if you can afford the cost, as most students are so burdened by the debt they have to repay their masters that in all probability they will just regurgitate whatever they are taught to get a degree in the hope of securing a permanent job in the world of finance management. Even the latter is increasingly difficult.

    • Ken Zimmerman
      June 26, 2019 at 9:07 am

      Helen, that’s only part of the problem. The stories about causation told by economics only gratify economists. And not even all of them. It also seems causation stories told by economists intentionally explain nothing. Designed more to dazzle and confuse than provide insight.

  3. Mark Brooks
    July 23, 2019 at 1:22 am

    So basically, the deeper your understanding of the context, the deeper your understanding of the chances of correlation. I ran a link to your post on BehavioralEconomist.com, thanks.

  4. gerald holtham
    July 25, 2019 at 2:41 am

    Of course statistics cannot prove causation. As Popper observed no sequence of observations can prove a general proposition. As Lar Sylls says, you have to have a theory or a model which asserts causation. Statistics can then test the model. It can never prove it but it can show if it is tenable or highly unlikely to be true. Unfortunately economic and social systems are so complex that the defenders of a theory can always qualify it and assert that the statistical test was biased because it omitted relevant factors. This makes it difficult to kill off useless theories, like the neutrality of money or rational expectations, both of which still have adherent despite repeatedly failing statistical tests.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.