Home > Uncategorized > What so many critiques of economics gets right

What so many critiques of economics gets right

from Lars Syll

Yours truly had a post up the other day on John Rapley’s Twilight of the Money Gods.

In the main I think Rapley is right in his attack on contemporary economics and its ‘priesthood,’ although he often seems to forget that there are — yes, it’s true — more than one approach in economics, and that his critique mainly pertains to mainstream neoclassical economics.

Noah Smith, however, is not too happy about the book:

strw manThere are certainly some grains of truth in this standard appraisal. I’ve certainly lobbed my fair share of criticism at the econ profession over the years. But the problem with critiques like Rapley’s is that they offer no real way forward for the discipline … Simply calling for humility and methodological diversity accomplishes little.

Instead, pundits should focus on what is going right in the economics discipline — because there are some very good things happening.

First, economists have developed some theories that really work. A good scientific theory makes testable predictions that apply to situations other than those that motivated the creation of the theory. Slowly, econ is building up a repertoire of these gems. One of them is auction theory … Another example is matching theory, which has made it a lot easier to get an organ transplant …

Second, economics is becoming a lot more empirical, focusing more on examining the data than on constructing yet more theories.

Noah Smith maintains that new imaginative empirical methods — such as natural experiments, field experiments, lab experiments, RCTs — help us to answer questions concerning the validity of economic theories and models.  

Yours truly begs to differ. Although one, of course, has to agree with Noah’s view that discounting empirical evidence is not the right way to solve economic issues, when looked at carefully, there  are in fact few real reasons to share his optimism on this so called ’empirical revolution’ in economics.

Field studies and experiments face the same basic problem as theoretical models — they are built on rather artificial conditions and have difficulties with the ‘trade-off’ between internal and external validity. The more artificial conditions, the more internal validity, but also less external validity. The more we rig experiments/field studies/models to avoid the ‘confounding factors’, the less the conditions are reminicent of the real ‘target system.’ You could of course discuss the field vs. experiments vs. theoretical models in terms of realism — but the nodal issue is not about that, but basically about how economists using different isolation strategies in different ‘nomological machines’ attempt to learn about causal relationships. I have strong doubts on the generalizability of all three research strategies, because the probability is high that causal mechanisms are different in different contexts and that lack of homogeneity and invariance doesn’t give us warranted export licenses to the ‘real’ societies or economies.

If we see experiments or field studies as theory tests or models that ultimately aspire to say something about the real ‘target system,’ then the problem of external validity is central (and was for a long time also a key reason why behavioural economists had trouble getting their research results published).

Assume that you have examined how the work performance of Chinese workers A is affected by B (‘treatment’). How can we extrapolate/generalize to new samples outside the original population (e.g. to the US)? How do we know that any replication attempt ‘succeeds’? How do we know when these replicated experimental results can be said to justify inferences made in samples from the original population? If, for example, P(A|B) is the conditional density function for the original sample, and we are interested in doing a extrapolative prediction of E [P(A|B)], how can we know that the new sample’s density function is identical with the original? Unless we can give some really good argument for this being the case, inferences built on P(A|B) is not really saying anything on that of the target system’s P'(A|B).

As I see it, this is the heart of the matter. External validity and generalization is founded on the assumption that we could make inferences based on P(A|B) that is exportable to other populations for which P'(A|B) applies. Sure, if one can convincingly show that P and P’ are similar enough, the problems are perhaps surmountable. But arbitrarily just introducing functional specification restrictions of the type invariance and homogeneity, is, at least for an epistemological realist far from satisfactory. And often it is — unfortunately — exactly this that I see when I take part of mainstream neoclassical economists’ models/experiments/field studies.

By this I do not mean to say that empirical methods per se are so problematic that they can never be used. On the contrary, I am basically — though not without reservations — in favour of the increased use of experiments and field studies within economics. Not least as an alternative to completely barren ‘bridge-less’ axiomatic-deductive theory models. My criticism is more about aspiration levels and what we believe that we can achieve with our mediational epistemological tools and methods in the social sciences.

Many ‘experimentalists’ claim that it is easy to replicate experiments under different conditions and therefore a fortiori easy to test the robustness of experimental results. But is it really that easy? If in the example given above, we run a test and find that our predictions were not correct – what can we conclude? The B ‘works’ in China but not in the US? Or that B ‘works’ in a backward agrarian society, but not in a post-modern service society? That B ‘worked’ in the field study conducted in year 2008 but not in year 2016? Population selection is almost never simple. Had the problem of external validity only been about inference from sample to population, this would be no critical problem. But the really interesting inferences are those we try to make from specific labs/experiments/fields to specific real world situations/institutions/ structures that we are interested in understanding or (causally) to explain. And then the population problem is more difficult to tackle.

The increasing use of natural and quasi-natural experiments in economics during the last couple of decades has led, not only Noah Smith, but several prominent economists to triumphantly declare it as a major step on a recent path toward empirics, where instead of being a deductive philosophy, economics is now increasingly becoming an inductive science.

In randomized trials the researchers try to find out the causal effects that different variables of interest may have by changing circumstances randomly — a procedure somewhat (‘on average’) equivalent to the usual ceteris paribus assumption).

Besides the fact that ‘on average’ is not always ‘good enough,’ it amounts to nothing but hand waving to simpliciter assume, without argumentation, that it is tenable to treat social agents and relations as homogeneous and interchangeable entities.

Randomization is used to basically allow the econometrician to treat the population as consisting of interchangeable and homogeneous groups (‘treatment’ and ‘control’). The regression models one arrives at by using randomized trials tell us the average effect that variations in variable X has on the outcome variable Y, without having to explicitly control for effects of other explanatory variables R, S, T, etc., etc. Everything is assumed to be essentially equal except the values taken by variable X.

Limiting model assumptions in economic science always have to be closely examined since if we are going to be able to show that the mechanisms or causes that we isolate and handle in our models are stable in the sense that they do not change when we ‘export’ them to our ‘target systems,’ we have to be able to show that they do not only hold under ceteris paribusconditions and a fortiori only are of limited value to our understanding, explanations or predictions of real economic systems.

Real world social systems are not governed by stable causal mechanisms or capacities. The kinds of ‘laws’ and relations that econometrics has established, are laws and relations about entities in models that presuppose causal mechanisms being atomistic and additive. When causal mechanisms operate in real world social target systems they only do it in ever-changing and unstable combinations where the whole is more than a mechanical sum of parts. If economic regularities obtain they do it (as a rule) only because we engineered them for that purpose. Outside man-made ‘nomological machines’ they are rare, or even non-existant.

I also think that most ‘randomistas’ really underestimate the heterogeneity problem. It does not just turn up as an external validity problem when trying to ‘export’ regression results to different times or different target populations. It is also often an internal problem to the millions of regression estimates that economists produce every year.

Just as econometrics, randomization promises more than it can deliver, basically because it requires assumptions that in practice are not possible to maintain.

Like econometrics, randomization is basically a deductive method. Given the assumptions (such as manipulability, transitivity, separability, additivity, linearity, etc.) these methods deliver deductive inferences. The problem, of course, is that we will never completely know when the assumptions are right. And although randomization may contribute to controlling for confounding, it does not guarantee it, since genuine ramdomness presupposes infinite experimentation and we know all real experimentation is finite. And even if randomization may help to establish average causal effects, it says nothing of individual effects unless homogeneity is added to the list of assumptions. Real target systems are seldom epistemically isomorphic to our axiomatic-deductive models/systems, and even if they were, we still have to argue for the external validity of the conclusions reached from within these epistemically convenient models/systems. Causal evidence generated by randomization procedures may be valid in ‘closed’ models, but what we usually are interested in, is causal evidence in the real target system we happen to live in.

When does a conclusion established in population X hold for target population Y? Only under very restrictive conditions!

‘Ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right ‘closures.’ Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. ‘It works there ‘s no evidence for ‘it will work here.’ Causes deduced in an experimental setting still have to show that they come with an export-warrant to the target population/system. The causal background assumptions made have to be justified, and without licenses to export, the value of ‘rigorous’ and ‘precise’ methods — and ‘on-average-knowledge’ — is despairingly small.

So, no, I find it hard to share Noah Smith’s and others enthusiasm and optimism on the value of (quasi)natural experiments and all the statistical-econometric machinery that comes with it. Guess I’m still waiting for the export-warrant …

I would, contrary to Noah Smith’s optimism, argue that although different ’empirical’ approaches have been — more or less — integrated into mainstream economics, there is still a long way to go before economics has become a truly empirical science.

Taking assumptions like utility maximization or market equilibrium as a matter of course leads to the ‘standing presumption in economics that, if an empirical statement is deduced from standard assumptions then that statement is reliable’ …

maxresdefaultThe ongoing importance of these assumptions is especially evident in those areas of economic research, where empirical results are challenging standard views on economic behaviour like experimental economics or behavioural finance … From the perspective of Model-Platonism, these research-areas are still framed by the ‘superior insights’ associated with early 20th century concepts, essentially because almost all of their results are framed in terms of rational individuals, who engage in optimizing behaviour and, thereby, attain equilibrium. For instance, the attitude to explain cooperation or fair behaviour in experiments by assuming an ‘inequality aversion’ integrated in (a fraction of) the subjects’ preferences is strictly in accordance with the assumption of rational individuals, a feature which the authors are keen to report …

So, while the mere emergence of research areas like experimental economics is sometimes deemed a clear sign for the advent of a new era … a closer look at these fields allows us to illustrate the enduring relevance of the Model-Platonism-topos and, thereby, shows the pervasion of these fields with a traditional neoclassical style of thought.

Jakob Kap

  1. Alan.
    July 28, 2017 at 4:26 pm

    There’s a much better picture of the discipline’s very own straw man here: https://ronaldthomaswest.com/2015/04/04/jeremy-bentham/

    “Menacing his posterity” indeed.

  2. July 29, 2017 at 1:49 am

    We need a grand strategy of our research program.

    Lars Syll is right in emphasizing the limit of experimental (or field) research, because experimentation is limited by its nature to narrow range of situation. Syll critiques that experimental research is based on the randomization and that it is deductive rather than empirical. In my opinion, this criticism is missing the right target.

    Economy is a closely related large system. It is quite hard to find any single closed small subsystems, which are the proper target of experiments. To deal with this complex object, research strategy must be adapted to its characteristic. We have to know that economics is facing a much more difficult situation than that modern physics faced when it started its research .

    Economics should be a systemic science. (This does not mean it must be a systematic science.) As a consequence, economics as a whole must contain at least two parts: empirical research and theoretical research. Without theoretical research, empirical research only accumulates diffuse knowledge. It may make economics a natural history but cannot produce a coherent science. We have to ask our strategy by which to attack complex system called economy.

  3. Enquiring Mind
    July 30, 2017 at 5:35 pm

    Another critique of economics looks at what it leaves out. There is no discussion of the morality or ethics of decisions. Those omissions facilitate anything that is not proscribed. If it isn’t illegal, then it must be legal. Define what you will, take what you can.

    That approach leads to all manner of bad actions that harm people, in the name of efficiency. Why stop at squeezing out 90% of consumer surplus when that extra 10% can be had.
    Why not lie, cheat and steal to get what you want and who cares about others?
    At some point, people will react and the results will not be pretty. They have several times in the past.

    A Republic, if you can keep it.

  4. July 31, 2017 at 12:44 pm

    This article leans heavily on a “theoretical construct” called causation. I’d like to consider that construct for bit. Causality has a spotty history in physics and the other physical sciences. The founder of Newtonian physics looked at causation as the operation of the basic forces of the universe. Many physicists after Newton did not agree. After Einstein and Relativity Newton’s notions of causality were replaced by relational causation, in which causation is relationships between the basic building blocks of the universe (mass, energy, light, time, speed, etc.).

    Causation in the social sciences and history is more problematic. Before Chaos notions and complexity, causation in social science is a belief that events occur in predictable ways and that one event leads to another. That effect(s) follows cause(s). For some social scientists, causation relates to moral and legal rules. Rules are the basis of society. Enforcing them controls actions and keeps society functioning. No rule can be ignored. Humans must be held accountable to all the rules. Some psychologists take an empirical approach to causality, investigating how people and non-human animals detect or infer causation from sensory information, prior experience, and innate knowledge. Other social scientists contend that people filter observations of events through a basic belief that causes have the power to generate (or prevent) their effects, thereby inferring specific cause-effect relations. Others believe our view of causation depends on what we consider to be the relevant events in any situation. Still others look at causation in terms of theories or models, assumed to provide the connections to establish causation. Statistics and economics usually employ pre-existing data or experimental data to infer causality by regression methods. Omitting the regression many “interpretive” sociologists describe causation in terms of probabilities. And so on. Hume and Bertrand Russell say causation does not exist. But human history could not exist without causation, inside and outside the physical and social sciences.

    So, how should social scientists and economists use causation, or should they use it at all? Many social scientists, most philosophers, and physical scientists generally point out that causation is a tool invented by humans to make sense of observations. To place them into categories of thought that make humans comfortable. Historical studies certain bear out this hypothesis. Not necessarily a bad thing. Two issues I see with using causation in the social sciences and economics. One, how far should social scientists and economics use causation as an explanatory device? Two, don’t we need to examine the cultural processes through which our notions of causation are constructed?

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s