Home > ethics, The Economics Profession > Error Correction and Ethics

Error Correction and Ethics

from Peter Radford

You probably have missed it, but there is a major furor within the economics profession concerning the findings of an academic paper written by Carmen Reinhart and Kenneth Rogoff in 2010. The profession issues a torrent of papers annually, most of which remain scarcely read and massively under-appreciated. Probably deservedly so since the sole objective of most is to meet the check-box requirement of publication that dominates academia. This desperation to publish to build reputation and to demonstrate mastery of the subject to a determinedly self-referential peer group is one of the causes of the rapid decline within economics: it encourages ever more fragmentation of the subject into ever less relevant sub-disciplines, and has resulted in a near total elimination of a common understanding – or common memory – of its development.

It has also produced sloppiness.

The R-R paper has now become a cause-celebre of such sloppiness.

This would not normally be worthy of passing along to a broader audience – who cares what academic economists write after all? – were it not for the particular topic that R-R covered. 

Recall that R-R wrote a very popular and useful book, published in 2009, called “This Time Is Different”, which was a long history of financial crises and the reverberation that such crises tend to have. The book is loaded with good data and reinforced  the longstanding view that recoveries from recessions whose source is a financial crisis is always longer and more arduous than is recovery from a more ‘normal’ recession. It is a terrific book worth reading by anyone trying to get their arms around how we arrived where we are today.

So the R-R paper published a year later was naturally going to attract a lot of attention.

Which it did. Especially from the ranks of austerity advocates who seized on one of the paper’s major findings: notably that economies where the ratio of government debt to GDP rises above 90% appear to underperform radically. To be precise, R-R found that economies with debt ratios below that magic 90% mark tend to grow in the 3% – 4% range annually, whereas those with ratios above that mark see growth slump to a minuscule 0.1%. That’s a big difference. One that captured everyone’s attention.

Almost immediately R-R’s reputation as gurus and specialists in our crisis rocketed upward. They became near omnipresent in the financial press and other media, and their work was used to advance the cause of austerity everywhere.

There is one small problem.

R-R made a mistake. Not a small mistake. A very significant mistake.

Correct that mistake, as Robert Pollin, Michael Ash, and Thomas Herndon now have, we learn that the awful 0.1% GDP growth of countries above the magic 90% ratio is actually a more robust, if not stellar, 2.2%.

In other words the R-R conclusion, a bedrock of austerity policy in recent times, melts into thin air.


More than whoops: oh dear!

Ordinary people around the world have lost their jobs due to austerity policies being crammed down the throats of various economies in no small part due to the R-R paper. Regular folks have had their lives ruined. Those innocent people have the right to wonder about the skill and the ethics of any profession that wields such power and has such authority, and yet has so little control over its various representatives.

The economics profession has just arrived at a crucial ethical point.

Its theoretical underpinnings are a mess. There is no point in beating that particular dead horse any longer. The myriad obvious, deep, and yet totally accepted flaws in modern – and much past – theory are a shameful testament to the increasing separation of economics from the real world. But the subject’s influence still grows. Its better known advocates garner reputations and reward unrelated to the actual efficacy of the advice they give. They infest policy making agencies the world over.

Were the correction of R-R to have taken place in a more arcane backwater of academia it would be a subject for little comment. But this is economics. This is far from arcane. Livelihoods are at stake. It is reasonable, therefore, for society to expect more than simple correction.

It isn’t just the error that ought raise the hackles of those affected by policies relying on R-R, but it is the way in which the error stood. My speculation is that R-R allowed the error to stand because the result was to their liking. They saw the magic line appear, it confirmed their pre-existing commitment or predilection for austerity, and thus they saw no need to check their calculation. They rushed out their paper in the afterglow of the acclaim of their book, and were swept up in the glaring light of celebrity. They sought fame not rigor.

This is a clear example of the pervasive nature of political worldview muddying what appears on the surface as academic work. I have no complaint about that because I think it is unavoidable, but it ought to be explicit. Social science is a lot more social than it is science. And it is holistic in that economics is irretrievably intertwined with politics, psychology, and sociology etc. no matter what the much-splintered halls of academia would have us all believe to the contrary. To study one is to get involved in them all. So all pure economics is incomplete and is tainted with ideology whether we like it or not. But error is error. And error allowed to stand because it reinforced an ideological predisposition is a weed difficult to eradicate.

Yes, the workings of ordinary academic challenge and response have worked. And error has been revealed. A paper has been debunked.

But, and I have to keep saying this, economics is no ordinary discipline. It pontificates and influences policy. It preaches certain social structures. It gives great intellectual credence to political action. It has, therefore, outsized power. Every single year its theories are absorbed by countless students. They learn its tools. They learn its doctrines. They adopt its messaging.

It had better damn well be right if is taught without revealing its different traditions.

It is not at all proper, ethical, or appropriate that errors, even when corrected, are without further consequences. Does anyone reading this truly believe that R-R will suffer similar economic consequences to those suffered by the victims of austerity policies built upon R-R’s error? Of course not.

That, in short, is the ethical challenge economists, particularly academic economists. Refuse to grapple with.

And it is no good resorting to the old argument that the establishment and enforcement of some form of ethical standard would   cloud academic freedom. Of course it would. That’s the entire point of incentives: to constrain and limit options.

For a subject that preaches the efficacy and effectiveness of incentives, economics is singularly unwilling to accept the creation of incentives for itself.

We now have a live experiment: what consequences will R-R suffer? Demotion? Loss of income? Loss of employment? Loss of reputation? What exactly will happen to discipline them? I wish none of those things on anyone. But this is economics. This is a subject that daily spreads the gospel of incentives. Of the fear that limits and modifies behavior. Of the cleansing effect of market forces that correct errors and allow us to arrive at ‘optimal’ solutions. The market, economics teaches, is the ultimate corrective mechanism. It does not suffer fools.

So why does economics?

  1. paolo leon
    April 23, 2013 at 3:58 pm

    The point is not so much the error, but the fact that for some 20 years it was possible for such ideas to conquer the high ground. Was’nt it Rogoff who chocked for the Nobel to Stiglitz?

  2. Dave Raithel
    April 23, 2013 at 4:02 pm

    Am I mistaken to think I recall the earliest and first critique of R & R was from the MMT proponents, who pointed out (then supposing the arithmetic was correct) 1) the R&R analysis conflated countries borrowing in currencies of others with those that issue their own and 2) the correct analogy to the US is Great Britain of the 19th Century (the debts of which nearly always exceeded the “safe” level, but which kinda sorta ruled the World….)?

  3. Roger Erickson
    April 23, 2013 at 4:25 pm

    Peter’s prognosis is dead-on, since it’s an axiom for networked, i.e., organized, systems that group intelligence is held in the body of discourse and/or interactions – and especially in the specific permutations of feedback exchanged that best address a given context. That is WHY we have sexual, cultural and “option” recombination. This aint’ rocket-science, folks. Any ancient tribal member could tell you that full-group consultation always wins.

    A disconnected set of insular components is a mob of idiot savants, not an organized group.

    Walter Shewhart summed this well 80 years ago: “Data is meaningless without context.” R&R compiled a lot of data, but were clueless about context. The aftermath was inevitable.

    And yes, many people knew that the very premise of R&R’s work was so bad it wasn’t even wrong, it was irrelevant.

  4. Victoria Chick
    April 23, 2013 at 4:39 pm

    Although I share the author’s concerns with good practice and careful analysis, I am somewhat alarmed by his paragraph beginning ‘It isn’t just the error that ought…’, which imputes to R-R motives and even actions (allowing the error to stand) that he cannot possibly know. They may not have known of the error. The paragraph verges on libel, which is no more attractive than the ethical lapses attributed to R-R.

    Time, too, to heap praise on UMass economists for assigning replication as a project for their research students. Replication is tedious, which is why it is seldom done, but it is one of the few weapons we have against sloppiness and worse.

    Those interested in this article may like to know of a conference forthcoming in Cambridge, England, ‘Coding Economics? Professional Conduct and Scholarly Values after the Crisis’. See http://goo.gl/HKWnq for details.

    • April 24, 2013 at 4:00 am

      The error was so egregious that simply glancing at the data showed that it could not average out to what R-R said it averaged out as. Granted, I graduated from a real-world school of adding up large columns of numbers in my head called “pizza delivery driver” (in a pre-handheld-card-scanner era), but we’re not talking rocket science here. R-R at the very least were almost criminally negligent in not performing the most basic of sanity tests upon their results. Impugning their motivations for not performing even the most basic of sanity tests upon their results may be crossing a line, but it’s a line that’s so glaringly *there* that it’s hard not to cross it.

  5. JnKg
    April 23, 2013 at 7:16 pm

    Los médicos matan de a uno; los ingenieros de a cientos… Los economistas de a millones…

  6. Ken Zimmerman
    April 23, 2013 at 7:18 pm

    In “What is Mathematics Really?” ” mathematician Reuben Hersh said this about mathematics,…mathematics must be understood as a human activity, a social phenomenon, part of human culture, historically evolved, and intelligible only in a social context. I call this viewpoint ‘humanistic.” Most mathematicians know this, but for a lot of reasons (including the academic position of mathematics and the expectations for scientists) often fail to acknowledge it publically. Scientists other than mathematicians simply use what they’re given, as best they can regarding mathematics. This includes economists. In another book Hersh and his co-author Philip J Davis comment that most mathematicians are frightened when they see mathematics used by social scientists and economists. They worry about the damage such use could do. It appears their worries were not misplaced. Of the many things economists abuse mathematics is one of the worst.

  7. davetaylor1
    April 23, 2013 at 7:45 pm

    I don’t give a damn about disciplining R & R. When (and how?) is the economics profession going to put right the damage R & R have done to the lives of millions of innocent people by advocating totally unjustified austerity instead of demanding the sacking, delicencing and, where appropriate, prosecution of self-serving financial crooks, bullies and saboteurs?

  8. April 23, 2013 at 9:30 pm

    Far from an exception, R-R is typical of many examples of economists passing data they don’t
    understanding, through mathematical tools (Excel and statistical packages) they have no mastery of, mis-interpreting the results (as statistically significant), and then drawing non sequitur conclusions about a world they have no experience, all the while asserting that they have discovered “objective” knowledge from empirical data. Applied econometrics is in a sad state.

  9. Juan de la O
    April 23, 2013 at 11:03 pm

    Strikes me as simply another example of economics as ideology.

  10. Dave Raithel
    • merijnknibbe
      April 24, 2013 at 5:51 pm

      How’s the job search going?

  11. April 24, 2013 at 6:51 pm

    The “publish or perish” model of academia has produced so much noise that real progress is difficult. Mindless business management of universities has been substantially responsible. Quantity is the number of published papers. Quality is ranking of journals, which are dictated by money and cronyism, which are dominated by the Ivy League.

    Once quality is defined by where you come from, “correct” ideas become the province of a few elite centers and form the basis of the economic paradigm which nearly all academics subscribe, business accept and governments use to implement policy.

    Such an institutional structure has enabled the domination of a false neoclassical theory for the past few decades. The rapidity with which R-R’s sloppy work was accepted and influenced public
    policy shows how flawed the structure is.

    The flaw was exposed with the essential help of the internet, because without it, criticisms of elite
    publications would never be widely known and would not be published in journals. The intellectual corruption has been entrenched for many years and only now being more widely exposed.

  12. mcdruid
    April 25, 2013 at 4:28 am

    The problem was not that it was “a mistake,” it was a series of errors and sloppiness. One mistake is just a mistake, so many mistakes is an embarrasment.

    • Roger Erickson
      April 25, 2013 at 4:52 am

      “One mistake is just a mistake, so many mistakes is an embarrasment.”

      … that eventually illustrates the truth.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.