Home > Uncategorized > Econometrics versus reality

Econometrics versus reality

from Asad Zaman

Underlying Philosophy of Science

Many important structures of the real world are hidden from view. However, as briefly sketched in previous lecture on Ibnul Haytham: First Scientist, current views say that science is only based on observables. Causation is central to statistics and econometrics, but it is not observable. As a result, there is no notation available to describe the relationship of causation between two variables. We will use X => Y as a notation for X causes Y. Roughly speaking, this means that if values of X were to change, then Y would have a tendency to change as a result. This is not observable for two separate reasons. ONE because it is based on a counterfactual. In another world, where the value of X was different from what was actually observed in our current world, this change would exert pressure on Y to change. TWO X exerts an influence on Y, but there are other causal factors which are also involved. Thus Y might not actually change in the expected direction because the effect of X  might be offset by other causal factors which we have not accounted for. For both of these reasons, causality is not directly observable.

Achieving Conceptual Clarity  read more

  1. March 31, 2021 at 8:41 pm

    “current views say that science is only based on observables” – not true, or at least not true of most scientists or most philosophers
    “there is no notation available to describe the relationship of causation between two variables” – not true – Judea Pearl developed a rigorous language for describing this relationship, as well as mapping out multiple causal forces; when I say “rigorous”, this is literally true: the diagrams are as rigorous as equations

  2. March 31, 2021 at 10:15 pm

    I should have added that although Pearl is not an economist, his methods have been used in economics, e.g. by Kevin Hoover in “Causality in macroeconomics”.

  3. April 1, 2021 at 4:42 am

    I am well aware of Pearl, and my complaints are directed against ECONOMETRICS. They are taken from his chapter on “How Statisticians Inflicted Causal Blindness on Itself”. Even though Pearls methods are used and accepted widely in the social sciences, there are currently NO textbooks in econometrics and statistics which make use of them, and there is no notation in use in journals of these disciplines for the concept.

  4. April 1, 2021 at 4:43 am

    Similarly, in philosophy of science, “realism” is an embattled minority position. Anti-Realists, as well as nominalists, rule the field.

  5. April 1, 2021 at 12:37 pm

    Thanks for your reply Asad. I thought you must know about Pearl, but if so, why not make that part of the article about his method, and use his notation? There are a lot of informative things you could say about his work, that may be new to some readers. His approach to causal inference has become quite influential in some domains, e.g. epidemiology. If anyone wants to follow this up, I recommend his “Causal inference in statistics: An overview” from 2009 in Statistics Surveys – it condenses his main analysis into about 50 pages. My view would be to add, there are some elements in econometrics that epidemiologists could usefully learn from, notably the systematic use of natural experiments, and instrumental variable methods, to infer causality. I know that IVs are often misused, but the basic idea is good.

    I agree with you that many if not most philosophers have a problem about realism. My take on this is that a baked-in attitude in philosophy is, the more sceptical you sound, the more credit you get. But my impression is that few of them go as far as to say that “science is only based on observables” (which they would label as positivism). That dominated for much of the 20th century, but what I see in philosophy now is an ambiguous take on science, along the lines of “on the one hand this, on the other hand that” – e.g. empiricists would argue one line and realists would argue something else. I “attended” a webinar recently by Alexander Bird, who is well known – there were more than 100 attendees – on the topic. The title was “Against empiricism”, with an abstract beginning “Most philosophers of science are realists. Most philosophers of science are, at least implicitly, empiricists. But, I argue, it is not reasonable to be both an empiricist and a realist, because …”. I criticised it for having a static view of science, and thus misunderstanding the cumulative nature of science. So that is why I say that “many if not most philosophers have a problem about realism”. They mostly accept that somehow scientists do find out how things work, at a deeper level than surface phenomena and observations – they just don’t understand how this happens. In fact, it’s an iteration between observations and hypotheses/conjectures/theories, but they try and press it into a static syllogism-like logical structure. Not unlike how economists look at the market mechanism in terms of a static equilibrium, balancing a supply schedule and a demand schedule conceived of as mathematical functions, like simultaneous equations. Rather than as a sequence of causal processes operating across time and in the context of multiple causation, which would be the realist way of portraying it.

    If you want a brief description of how science works in e.g. biology, and how this could be applied in economics, see my “Causal theories, models and evidence in economics—some reflections from the natural sciences”, at https://www.tandfonline.com/doi/full/10.1080/23322039.2017.1280983.

  6. April 1, 2021 at 1:04 pm

    This is actually just a lecture in an introductory course on statistics, to students with minimal background – none in philosophy. So it treats these topics at a superficial level. My experience is that such students have little tolerance for, and little understanding of, these topics. So I try to get to the practical implications as soon as possible – in terms of the practice of regression and how this fails to give reasonable results, and alternatives.

  7. Gerald Holtham
    April 1, 2021 at 3:31 pm

    i fear for those students.

  8. Gerald Holtham
    April 1, 2021 at 4:32 pm

    Evidencebas, I am in substantial agreement with the points you make in the article for which you provide a link. Of course economics is not homogenous and some people have worked more in the way you prescribe but the biases to which you point are unquestionably dominant in economic theorising. Much of the problem stems from a failure to distinguish procedural rationality in a situation that one does not fully understand from an ability to always find an optimal solution. Moreover that implausible form of “rationality” is not regarded as an empirical issue for testing but as a necessary axiom in economic theorising. Indeed some influential schools regard that axiom as defining the boundary between economics and other social studies. Theories that introduce “non-rational” elements are ipso facto “sociological” rather than economic. This concern for the purity of the economist’s domain has effectively cut it off from the real world and means that conventional economic theorising can only provide illumination by accident since it departs from axioms not observations.
    These are indeed errors based on a defective methodology but they are a good deal more superficial than debates about the justification for empiricism. I can agree with you and see the problems in economics without moving beyond empiricism. I still believe we can only learn from observation and that you can’t get an “is” from an “ought”.

    • April 1, 2021 at 11:35 pm

      Thanks Gerald. I agree that treating rationality as an axiom and a defining feature of economic analysis to be considered legitimate is a serious problem. The same could be said about other concepts, e.g. equilibrium and optimization. I think optimization is perhaps the most serious, because it requires that there is an automatic, determined response by the economy, which in turn means that economic outcomes are responses to non-economic determinants, namely technology and preferences. You have to assume a given set of technology and preferences, and then the economic effects flow directly from them. It is a stimulus-response model, which allows no room for specifically economic causation. As I have recently outlined in this forum, my view is that many if not most economic phenomena are due to feedback or other aspects of complexity; in the case of feedback it is quite feasible to map out the actual causal processes (this is more difficult with other aspects of complexity – although Benoit Mandelbrot does do this for fractals in his book). Sometimes this feature is described as “nonlinearity”, but I find this an inadequate term, because (a) it only says what it is *not*, and (b) because there are many ways a system can be nonlinear, e.g. having a different functional form such as quadratic, but still a stimulus-response concept; or something like causal potentiation or conversely inhibition.

      But the issue you point to, i.e. taking something like rationality (or the others I’ve mentioned) to be “a necessary axiom in economic theorising” is the central methodological flaw in neoclassical theorizing. It means that when empirical studies turn out not to fit with the theory – which is the usual case – the response is to try and explain the *discrepancy* from theory, rather than the phenomenon itself. There are two “stages”: orthodox theory, and then something that explains how reality deviates from it (I use this phrase deliberately: it does often seem that for some economists, it is reality that deviates, rather than the theory!). Whereas a better strategy is to try and explain the phenomenon in one “stage”. This is seen clearly in behavioral economics, where a major tendency (not the only one) is to start from neoclassical theory, and then try to explain the *bias* – made explicit in Rabin’s concept of PEEMs. See my “Mechanism in behavioural economics” in the Journal of Economic Methodology, 2019, at https://www.tandfonline.com/doi/abs/10.1080/1350178X.2019.1625214?journalCode=rjec20

  9. Gerald Holtham
    April 2, 2021 at 4:29 pm

    We are violently agreeing. The sad fact is that none of these observations is new. Herbert Simon was pointing all this out in the 1970s and 80s, when he got the Nobel prize. Unfortunately he was right but not persuasive and things took a sharp turn for the worse with Lucas and Sargent et al and their representative agents and rational expectations. It is sad to see a subject regress for fifty years but it happened in macroeconomics.
    There have been some advances in microeconomics with the study of situations of asymmetrical information and principal-agent issues. They have flowed from the usual axiom-based theorising but in taking account of uncertainty they have developed results with some relevance to reality. As ever, the real work starts in identifying situations to which they apply and how they must be adapted for the situation under study. Economic models are seldom “plug and play”.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.