Home > Uncategorized > Randomised controlled trials — a retreat from the bigger questions

Randomised controlled trials — a retreat from the bigger questions

from Lars Syll

Unknown1Nobel prizes are usually given in recognition of ideas that are already more or less guaranteed a legacy. But occasionally they prompt as much debate as admiration. This year’s economics award, given to Abhijit Banerjee, Esther Duflo and Michael Kremer … recognised the laureates’ efforts to use randomised controlled trials (RCTs) to answer social-science questions … RCT evangelists sometimes argue that their technique is the “gold standard”, better able than other analytical approaches to establish what causes what. Not so, say some other economists … Results are contextually dependent in ways that are hard to discern; a finding from a study in Kenya might not reveal much about policy in Guatemala …

Advanced economies grew rich as a result of a broad transformation that affected everything from the aspirations of working people to the functioning of the state, not by making a series of small, technocratic changes, no matter how well-supported by evidence …

Indeed, some economists have a sneaking suspicion that the rise of RCTs represents a pivot not just to smaller questions but also to smaller ambitions … Researchers are still guided by theory, which shapes the empirical questions that get asked and whether results are interpreted as capturing some deeper aspect of an economy’s nature. But a world in which economists are mostly policy-tweakers—or “plumbers”, in Ms Duflo’s phrase—is very different from the one to which many economists once aspired.

The Economist

It is nowadays widely believed among mainstream economists that the scientific value of randomization — contrary to other methods — is totally uncontroversial and that randomized experiments are free from bias. When looked at carefully, however, there are in fact few real reasons to share this optimism on the alleged ’experimental turn’ in economics. Strictly seen, randomization does not guarantee anything.

‘Ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right ‘closures.’ Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. Causes deduced in an experimental setting still have to show that they come with an export-warrant to their target populations.

the-right-toolThe almost religious belief with which its propagators — like Duflo, Banerjee and Kremer — portray it, cannot hide the fact that RCTs cannot be taken for granted to give generalizable results. That something works somewhere is no warranty for us to believe it to work for us here or even that it works generally.

The present RCT idolatry is dangerous. Believing there is only one really good evidence-based method on the market — and that randomization is the only way to achieve scientific validity — blinds people to searching for and using other methods that in many contexts are better. RCTs are simply not the best method for all questions and in all circumstances. Insisting on using only one tool often means using the wrong tool.

This year’s ‘Nobel prize’ winners think that economics should be based on evidence from randomised experiments and field studies. Duflo et consortes want to give up on ‘big ideas’ like political economy and institutional reform and instead go for solving more manageable problems the way plumbers do. But that modern time ‘marginalist’ approach sure can’t be the right way to move economics forward and make it a relevant and realist science. A plumber can fix minor leaks in your system, but if the whole system is rotten, something more than good old fashion plumbing is needed. The big social and economic problems we face today is not going to be solved by plumbers performing RCTs.

  1. Craig
    November 30, 2019 at 12:13 am

    “The big social and economic problems we face today is not going to be solved by plumbers performing RCTs.”

    Couldn’t agree more. Marginal reforms are small and palliative. Paradigms changes are pattern changing and permanently progressive. History tells us that. And when all of the thorny problems in the current/old paradigm revolve around money, banks, debt and financing and they’re all resolved by the new paradigm of Direct and Reciprocal Monetary Gifting….what else is there to say and do???

  2. November 30, 2019 at 11:02 am

    This “plumbers” argument is reminiscent of Lakatos (in “Criticism and the Growth of Knowledge”) on science normally advancing by patching failings in theory – until the point where all the leaks have been patched and the theory still doesn’t work. That’s the time for Kuhn’s revolutionary science taught practically by change of “paradigm” (obvious example).

    It seems to me Craig is right about the way we do things now – i.e.the existing paradigm – exemplifying the common understanding of “money, banks, debt and financing”, i.e. banks creating money as debt to them. The “more that needs to be said” about his proposed alternative of “Direct and Reciprocal Monetary Gifting” is that it doesn’t address the point raised so eloquently just now by Ikonoclast: the issue being the use and renewal of real resources, not their increased use and equitable distribution. Craig’s formula is near enough right as a principle, but his paradigm is “50% discount at point of retail sale”, which seems merely to leave consumers with more money to buy more, which have to be made, using more resources. His “Copernican” revolution is genuine enough, exchanging debts (taking) for giving (both in money). But what is money? Leave money out (the formula can then include real things) and one is giving either a loan (at interest) or credit (for doing something). The “Copernican” reversal, merely of the way we are looking at something, can now become paradigmatic. The old paradigm is represented by a debit card (spending “credit” ultimately loaned someone by a bank) and the new one by a credit card (recognising our own credit worthiness given our doing anything needing resources). What you can do with the one you can do with the other. What comes out of your account goes into the vendors: the honest version of Craig’s double entry book keeping.

    As a literate adult with a known pension I can manage my credit card budgeting via the numbers in my account. Children and other innumerates can’t. I thus see a need to retain token money for children learning to earn pocket money and small informal purchases at events like farmer’s markets and collections for charities. Again, this changes nothing. The understanding of money has changed from portable valuables like gold, through paper money and cheques to electric credit transfers. If we accept that money refers only to paper money and coins, our accessing it through ATMs is already recorded in our accounts and will eventually be written off in the vendor’s.

    • Craig
      November 30, 2019 at 6:20 pm

      Doubling purchasing power does not equate to a doubling of consumption or commodity through put.

      We haven’t acted/created effective policy for climate change and renewable sources of energy for almost 50 years. The policies I’ve suggested will FINALLY enable and streamline what we should have been doing for that period.

      The new monetary paradigm ends the paradox of thrift and with its directness greatly homogenizes the so called “fallacy of composition” in economics and the money system.

      Nobody has suggested a program nearly as comprehensive and immediately effective as my new monetary and financial paradigm. MMT only talks about government debt. Public Banking is a mere reform. UBI/QE for the people is a mere bone toss to we serfs. Hudson’s diatribes against finance are spot on but there’s no policy program to do anything effective about financial illegitimacy. Keen exposes financial instability, but again has no insightful or comprehensive way to accomplish the new paradigm he says we need, and then squirrels off to dealing with energy. Indeed, a very important area of research, but an excellent example of focusing on problems to the point of the neglect of solutions. We don’t need intellect nearly as much as we need wisdom….which of course includes intellect.

      • December 1, 2019 at 9:22 am

        Craig, I sometimes wonder if you can read! I outline the argument for credit cards as a paradigm for giving people “money” when they need it, paid for by their earning it, and you come back saying “Nobody has suggested a program nearly as comprehensive and immediately effective as my new monetary and financial paradigm”!

        So you want us to give back money to those who already have too much, but not to penniless producers turned off their land so trees can be chopped down to grow stuff for beef-burgers and margerine?

        It seems you have never thought about Malthus’ problem, never mind our current need for self-imposed austerity in both consumption and reproduction (i.e. overall birthrates), hence the need for local control information and global mobilisation of tree-replanters. But are you SEEING what I am saying, or as usual just LISTENING to yourself?

      • Craig
        December 1, 2019 at 6:32 pm

        Dave, a credit card is not a monetary gift, it is a DEBT….and that places you unmistakably in the private banker’s mindset and paradigm of DEBT ONLY as in a monopoly paradigm. The monopolistic onlyness, especially in supposedly COMPETITIVE free enterprise economic systems, is precisely what is problematic and wrong. Comprende?

        The Adamic curse was lifted over 2000 years ago. It’s time we applied it to the economy and money system. Or does it not apply there?

        Now if you’re an entrepreneur or business person and you want or need to increase your production, or a home buyer the national publicly administered non-profit banking system will be happy to create and LEND you the required funds…at o%…unless you’re a hard core pornographer or drug king pin or something….then you’ll have to go to the private financiers with someone’s already created and saved profits or incomes and they’ll intermediate your project at a rate of interest that I’m sure would be a “deal”.

        “So you want us to give back money to those who already have too much,”

        Of course of those who have much, much is required, but the politics of envy are largely irrelevant….and old paradigm as well.

        “but not to penniless producers”

        Not relevant critique. As I showed Ken, my program of a dividend and a job guarantee enables his homeless Portlander to have over $52k worth of purchasing power per year. Solidly middle class for an individual.

        “turned off their land so trees can be chopped down to grow stuff for beef-burgers and margerine?”

        A sovereign, and under the new paradigm’s policies an obviously benevolent government, people will not be extortable and having knowledge of their prior oppression will undoubtedly be sensitive to the caprices of commercial interests who also being under the gracious but sovereign purview of the government, will not have free reign to exploit.

        “It seems you have never thought about Malthus’ problem, never mind our current need for self-imposed austerity in both consumption and reproduction (i.e. overall birthrates), hence the need for local control information and global mobilisation of tree-replanters. But are you SEEING what I am saying, or as usual just LISTENING to yourself?”

        Doubling purchasing power does not equate with a doubling of commodity consumption.

        Select additional subsidization of green consumer products and the enabling of funding for the mega projects necessary for ecological survival…are a primary part of my program and end the idiocy of such being “too costly” under the current financial paradigm of Debt Only.

        I’m sure there’s a gracious integration of population control within ecological survival…but it won’t have any possibility of enactment unless we awaken to the new monetary and financial paradigm FIRST. The new zeitgeist is grace and the current zeitgeist is power. So maybe you should figure out what that means.

  3. Ed Zimmer
    November 30, 2019 at 8:46 pm

    Craig: Where can I find a definitive description of your Direct and Reciprocal Monetary Gifting proposal?

    • Craig
      November 30, 2019 at 11:07 pm

      In my book on Amazon entitled: “The New Monetary Paradigm and Its Policies….Change Everything” which I’m re-writing presently for conciseness and policy emphasis. Give me a week to finish the changes.

      The actions of the 50% Discount/Rebate policy at retail sale perfectly describe and express the new paradigm. All merchants opting into the policy (read all of them because to opt out is to become completely uncompetitive) agree to reduce any and all product or service prices by 50% and open an account for all sales entitled discounts. The total amount of their discounts to consumers is rebated back to them by the monetary authority whether that is a new actually publicly administered federal level monetary authority or whatever authority is mandated to fulfill the rebate, and in so doing make every merchant whole on their overheads and margins of profit.

      As retail sale is the terminal ending point of the entire economic/productive process for every consumer item from a package of chewing gum to an automobile or home and is by definition also the terminal expression point for all economic and monetary factors, like inflation for instance, because it is the point where production becomes consumption. If possession is 90% of the law, then possession as in consumption at retail sale is 99.9% of economics. That means everyone’s purchasing power just got doubled at the very least (the dividend of course is additionally doubled purchasing power and enables transfer taxation elimination).

      Price inflation is typically a small single digit percentage so the net percentage discount insures beneficial price deflation, and even if after the taxation discouragements and further sanctions I recommend in the new paradigm program to inhibit greedy anti-social businesses from inflating still somehow results in say 1-4% overall inflation….you just make the discount 51-54%.

      Hyper-inflations never happen without several specific and disastrous circumstances happening like a prior war and the destruction of most of a nation’s productive capacity and then a compliant central bank that leverages up speculators who short the currency that initiates the actual final hyper-inflation. That plus high-tech capital intensive economic systems are costly and still competitive meaning they can’t just raise their prices arbitrarily high without losing market share. So never mind the straw man/rabbit hole of the hyperinflationistas.

  4. Ed Zimmer
    December 1, 2019 at 8:14 pm

    Craig: To make sure we have the same view of the economy, here’s my view:

    GDP is the measure of our productive economy. GDP is the sum of household, business and government spending (and likewise the income of those sectors equals that spending, because all spending is someone else’s income). Our economy depends on household spending (2/3 of GDP). That spending is limited by household income (which comes only from those three sectors). Business provides that income to the extent demand (business opportunity) exists, and government provides the rest (by way of bookkeeping entries to household bank accounts). All that’s important to the economy is maintaining this flow, and with a fiat currency (whose value, by definition, depends only on currency-users perception), there are no limits other than that perception.

    The household spending portion of GDP is the FINAL sale of products & services. I’m assuming that is specifically what you’re proposing to discount & that the reimbursement will occur by crediting the vendors’ bank account upon receipt of their submission of their sales report. (The report format & timing requirements would need to be revised & the reporting fully automated (ie, paper eliminated) for the benefit of both vendors & BEA. Full automation would be especially important to ease the burden on small retailers, landlords & service providers, especially those in the household construction and maintenance sectors.)

    GDP would dramatically increase, but not necessarily double (with a 50% discount), the PCE/GDP ratio would significantly increase (which I’d deem a significant improvement in manageability of the economy) & I’d expect to see a significant reduction in payday loans & some increase in personal savings. If staple goods/services constitute the major portion of GDP, inflation should not be a problem, due to competition.

    The advantage over a UBI would be that the government spending flows directly into household spending (rather than some of it going into savings & investment). I expect that vendor resista/nce would be the most difficult part of selling the program, which could be overcome with a well-designed reporting/reimbursement system (mobile reporting, instant reimbursement). Political resistance could be minimized by starting with a small discount, increasing as the program proves itself. All in all, seems like a reasonable proposal.

    • Craig
      December 1, 2019 at 8:49 pm

      That appears to be good detail to me. I would say however that integrating the discount/rebate monetary policy with both a universal dividend and a job guarantee is perfectly doable because of the finality of it being implemented at the terminal ending point at retail sale, and because of how those policies would enable us to completely step out of the current paradigm and implement fully direct personal and fiscal distributism. Taxation is a legitimate sovereign power and right to discourage economic vices and encourage economic virtues, but re-distributive taxation is old paradigm and unnecessary when direct fiscal distributism is made possible by the 50% Discount/Rebate policy. I see no reason to keep any unnecessary foot in the old paradigm.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.