Consistency and validity is not enough!
from Lars Syll
Neoclassical economic theory today is in the story-telling business whereby economic theorists create make-believe analogue models of the target system – usually conceived as the real economic system. This modeling activity is considered useful and essential. Since fully-fledged experiments on a societal scale as a rule are prohibitively expensive, ethically indefensible or unmanageable, economic theorists have to substitute experimenting with something else. To understand and explain relations between different entities in the real economy the predominant strategy is to build models and make things happen in these “analogue-economy models” rather than engineering things happening in real economies.
Formalistic deductive “Glasperlenspiel” can be very impressive and seductive. But in the realm of science it ought to be considered of little or no value to simply make claims about the model and lose sight of reality. As Julian Reiss writes:
There is a difference between having evidence for some hypothesis and having evidence for the hypothesis relevant for a given purpose. The difference is important because scientific methods tend to be good at addressing hypotheses of a certain kind and not others: scientific methods come with particular applications built into them … The advantage of mathematical modelling is that its method of deriving a result is that of mathemtical prof: the conclusion is guaranteed to hold given the assumptions. However, the evidence generated in this way is valid only in abstract model worlds while we would like to evaluate hypotheses about what happens in economies in the real world … The upshot is that valid evidence does not seem to be enough. What we also need is to evaluate the relevance of the evidence in the context of a given purpose.
Neoclassical economics has since long given up on the real world and contents itself with proving things about thought up worlds. Empirical evidence only plays a minor role in economic theory, where models largely function as a substitute for empirical evidence. Hopefully humbled by the manifest failure of its theoretical pretences, the one-sided, almost religious, insistence on axiomatic-deductivist modeling as the only scientific activity worthy of pursuing in economics will give way to methodological pluralism based on ontological considerations rather than formalistic tractability. To have valid evidence is not enough. What economics needs is sound evidence.
Discussing Paul Romer’s “mathiness” concept, Peter Dorman yesterday criticized economists’ belief that theories and models being “consistent with” data somehow make the theories and models a success story. And Chris Dillow elaborates on the weakness of this “consistent with” error in a post today:
If a man has no money, this is “consistent with” the theory that he has given it away. But if in fact he has been robbed, that theory is grievously wrong. Mere consistency with the facts is not sufficient.
This is a point which some defenders of inequality miss. Of course, you can devise theories which are “consistent with” inequality arising from reasonable differences in choices and marginal products. Such theories, though, beg the question: is that how inequality really emerged?** And the answer, to put it mildly, is: only partially. It also arose from luck, inefficient selection, rigged markets, rent-seeking and outright theft …
The Duhem-Quine thesis warns us that facts under-determine theory: they are “consistent with” multiple theories. This is perhaps especially true when those facts are snapshots. For example, a Gini coefficient – being a mere snapshot of inequality – tells us nothing about how the inequality emerged.
So, how can we guard against the “consistent with” error? One thing we need is history: this helps tell us how things actually happened. And – horrific as it might seem to some economists – we also need sociology: we need to know how people actually behave and not merely that their behaviour is “consistent with” some theory. Economics, then, cannot be a stand-alone discipline but part of the social sciences and humanities – a point which is lost in the discipline’s mathiness.
Yes indeed, history helps. And if we’re not to ‘busy’ doing the things we do, but once in a while take a brake and do some methodological reflection on why we do what we do — well, that takes us a long way too.