The Efficiency Myth

from Peter Radford

Efficiency, it seems, is entirely contextual. What works well today and thus appears to be the height of elegant engineering, with efficiency fairly oozing from every corner, will collapse in an undignified heap tomorrow when the earth shifts, the environment or tastes change, or when new technologies simply make it all seem so quaint.

So I hate efficiency because it feels and looks like a fool’s game.

I say keep something in reserve. Because you never know.

The problem is that other people adore efficiency.

Like the bankers who thought nothing of gambling with their balance sheets by deploying a 40:1 leverage ratio. Why worry? They had thought through every possibility. Their thinking allowed them to stretch to the very edge. Efficiency demanded they squeeze every ounce of profit from their equity and not allow it to sit around under deployed. That word “under” implies inefficient deployment.

Efficient people always feel fine going to extremes. They know the future sufficiently to do so. They see no reason to build in some spare capacity. They just know the answer.

Besides it is cheaper being efficient. Cheaper because every ounce of resource is at work. Nothing is left idle. There is no need for a reserve. No need for a shock absorber.

At the heart of the efficiency error is a dichotomy to do with knowledge and the way we store and use it.

When I discuss knowledge in the context of business I like to refer to “primary” and “secondary” kinds of knowledge. Dinosaurs are a good example of relying exclusively on the primary sort. Primary knowledge is compressed into simple routines. It is the kind of knowledge that says “when this happens, respond by doing x”. Easy. Cheap to store. Easily encoded. Easily replicated. Very easy to manage. And produces the same result every time. Businesses love this kind of knowledge. It lies at the heart of the dumbing down in every large business. It makes the cost of management lower because you don’t need much management overhead to get consistent results.

Until, of course something changes. As in the environment shifting.

Then all that supremely efficient knowledge is rendered not just useless, but dangerous. Organizations who pride themselves on their efficiency are betting that their environment will justify their knowledge. They have, either explicitly or implicitly, planned that they know the future.

DNA works this way. Living things adapt to their environments through trial and error and then pass into future generations all those survival tricks nicely encoded into DNA. Most living things don’t add to this knowledge. They simply re-enact the actions of their ancestors whenever a similar set of problems pops up. This is why Popper called all life problem solving. DNA represents an accumulated set of responses to an historical set of problems. The bet being that this set of problems will be a good example of what will happen in the future. that’s a big bet. And it works only over limited periods. Change inevitably invalidates it.

But if that bet pays off repeatedly the living organism gradually concentrates itself on its niche and works its way up a fitness peak. Such a peak being a representation of repetitive success driving ever more specialization at the tasks that appear to work. The higher the fitness peak, the greater the past success of whatever knowledge the organism deploys in order to survive. In other words the more efficient the organism is with respect to its surroundings.

We all know where this goes.

The more specialized the knowledge, or the more leveraged against its knowledge something is, the more vulnerable it is to change.

Efficiency then becomes a very bad thing.

Imagine that you have just struggled to squeeze every ounce of cost out of your system. It costs a great deal to carry around excess knowledge. Brains use a great deal of energy. The less you need them the lower cost you can be. So you arrive at your low cost/high leverage peak and look out across the landscape. And. Darn. There’s a higher peak. Someone is more efficient. Worse. You are in competition with that other person. You are doomed.

Efficiency is not just relative to the environment. It is also relative to other solutions to the same problem. That means to other knowledge. You can never rest assured you have the right answer. Someone else may have a better idea.

Having climbed laboriously to the top of your hill you are faced with deconstructing your efficiency and starting over in order to compete. Most likely you will succumb before you succeed in re-engineering yourself. Too bad. You should have left space for the very wasteful, but entirely necessary secondary knowledge.

Such secondary knowledge is embodied in what I refer to as roles – as opposed to the routines of primary knowledge. The crucial difference being that a role allows ad hoc response based upon an evaluation of changes in context. It is still rule based, but now it incorporates feedback. It thus involves learning. Secondary knowledge, by its nature, is high cost to deploy. It involves lugging around all sorts of unused rules that may or may not ever be deployed in action. It involves time to implement the feedback loop. It involves approximation rather than exact response. It allows for making do, rather than locating and then replicating a perfect idea. It also is very difficult to control since it allows each instance of a problem to produce a unique response. “Whatever works” becomes the operating plan. Rather than “stick to this since we know it works”.

There is always a tension between primary and secondary knowledge. Business prefers primary at all times since it is cheaper. Adaptation requires secondary since it allows change. Evolution has used both, but the emphasis is on primary knowledge with the result that failed knowledge implies extinction. Dinosaurs being a good example. Perfect for a very long time. Constant evolution along a path that then became, suddenly, a poor one. Highly efficient. And then not at all efficient.

The preferable strategy seems to be a bit of both.

Lots of embedded primary knowledge, coupled with a goodly amount of adaptive learning ability. Just enough redundancy to pivot when necessary. And enough efficiency to motor through the smooth periods at relatively low cost. This balancing act puts a great emphasis on sufficiency. “Satisficing” as Simon called it. Just enough of everything to manage. With some latitude along the way.

Humans are a great example of this. Poorly designed eyes. Rotten hearing. Weak sense of smell. Not too big. Not too small. We get by. We remain the same tubular body plan of our distant ancestors – think about the topology for a moment – but with a ton of adaptive equipment added. We manage our environment as best we can so that we avoid being entirely context dependent. We can occupy all sorts of niches. We rely heavily on secondary knowledge to solve problems on the fly. In fact we are supremely ad hoc. We get by.

Which makes me wonder why we constantly throw up dreamlike ideas of perfection. So many great thinkers have been beguiled into thinking that they can develop the perfect system. That epitome of efficiency we all long for. Or so they presume. Our history is full of utopias that look silly in the light of consequent events. We are afflicted by the lingering seduction or many more. They all promise efficiency. They feed our apparent need to lower the emotional and physical cost of surviving. We all dream of a life unencumbered by the vagaries of uncertainty. Death in particular seems to be a problem. So we circumvent uncertainty by pretending to know and control the future. We aspire to be efficient.

The result of this arrogant folly is always the same. Whenever the utopia is translated into action or policy here in the uncertain world, it ends up being a flop. Sometimes momentously so.

But uncertainty is relentless. And the past no map for the future.

So learning and adaptation are necessary for survival or growth.

Whereas efficiency allows for neither.

Beware of the utopias that incorporate efficiency. They are dangerous and misleading. Avoid them as the plague they are. Stick with the wasteful and vague comfort of muddling through. Piecemeal progress that allows for adaptation and does not presume to know the future may feel intellectually messy, but the mess won’t be you being trampled underfoot by something you didn’t plan for. Better yet: you won’t be taken for one of those hubris ridden fools who thinks they know everything.

  1. March 16, 2011 at 11:47 am

    Very very very good comment!

  2. Aksa
    March 16, 2011 at 2:09 pm


  3. March 16, 2011 at 3:01 pm

    many thanks!at long last someone who despises the concept as much as i do

  4. s h a r o n
    March 16, 2011 at 3:44 pm

    I think there might be a confusion here between the word “efficient” and “effective.
    Efficient relates to waste–as in having little or no _waste_ in its operation.

    Effective is usually meant to imply a successful outcome, given the _intent_ of the effort/plan.

  5. March 16, 2011 at 3:48 pm

    Agree, great blog. Gillian Tett’s piece in the FT today: Japan’s supply chain risk reverberates around globe, makes a similar point.

  6. Ken S
    March 16, 2011 at 3:55 pm

    I don’t see this as an attack on efficiency but rather an attack on thinking one knows what efficiency is… what you describe as an alternative actually sounds like a very efficient system in the long run.

  7. March 16, 2011 at 5:03 pm

    don’t know. that might be a waste of words (‘innefficient’ use of electrons on a blog).
    drill baby drill. waste energy. don’t be a dinosaur. be too big to fail.

    while i just glanced at the post, there are arguments that for example DNA is efficient in the sense of shannon’s information theory (an optimal code). while the arguments are rigorous, they are at this point speculative.

  8. March 16, 2011 at 5:12 pm

    Inefficiency implies waste. If we reject efficiency, we accept waste. The optimal policy is sustainable efficiency within the constraint of equity. Sustainable efficienty implies evolution. Dinosaurs that evolved into birds were sustainably efficient.

  9. Janus
    March 16, 2011 at 9:18 pm

    Too much efficiency damages resilience.

  10. Ken Zimmerman
    March 17, 2011 at 12:20 am

    John Law opens his book, “After Method: Mess in Social Science Research” with picture that can be found on the site here (if you can’t find it reply with your email address and I’ll send along the picture)
    He then asks, “If this is an awful mess …. then would something less messy make a mess of describing it?” He then quotes from Alice in Wonderland, “There is no use in trying said Alice, ‘one cannot believe impossible things.’ ‘I dare say you haven’t had much practice,’ said the Queen. ‘When I was your age I always did it for half an hour a day. Why, sometimes I’ve believed as many as six impossible things before breakfast.” Law’s point, this is what goes on every day and always has with everything we think we know and love.

    Peter, your points are well stated. However your beginning assumptions are wrong. You call life and the future uncertain. They’re more than uncertain; they’re a bloody mess and always has been and will be. No explanation can capture the world or the actions of the actors in it. As William James noted a hundred years ago there’s always enough data and flexibility in the world to justify every explanation offered. In this kind of mess it’s not surprising that humans as one of the actors invovled sometimes cook-up schemes to bring certainty to events, a la James. That can be science, religion, economics, mathematics, etc. They all fall before James’ observation. Our worlds are not even as orderly and explicable as Peter’s miminalist explanations imply. The really amazing thing to observe is that durable order of some sort is observed to occur, sometimes lasting years, or even centuries or millenia. This order emerges out of what Law calls “relational materiality,” that all entities and actors we know emerge from interactions, from relationships. The world and its actors (being really one-and-the-same) are the results of ongoing relationships. So efficiency thus emerged, along with its justification and explanation. This may be a bit frightening when you first consider it. But you soon see it as just the ongoing stream of what is and could be.

    But all this doesn’t answer your basic question, Peter. What should we do about efficiency? Efficiency like all that emerges from relational materiality will fail, more than once and eventually will be surpassed by a different certainty. You can hurry that along by targeting its destruction. I’d say take your best shots, on the widest canvas possible, for as long as your strength holds out. But what replaces it will be no different ultimately from what it replaced. That’s all the certainty, to use your term there is.

  11. Alice
    March 17, 2011 at 10:17 am

    Efficiency belongs with words like functional, rational, competitive, productive. It does not belong with words like motivated, inclusive, imaginative and creative and thats whats gone missing in the race to become “more efficient”.
    I reject the notion of efficiency and the word itself really grates on my nerves. No one knows where it is or when to stop looking for more of it. Its a mmirage. It exists only in the desert and some organisations run their own human capital over hot sand trying to find it.

  12. Michael
    March 17, 2011 at 4:20 pm

    A couple of things:

    1) Dinosaurs are actually the opposite of what you think — recall that some dinosaurs evolved feathers, so that they were able to withstand the climate shock and evolve into the enormously successful birds that now occupy every ecological niche imaginable. That is, it was the spread of different types of dinosaurs which made their continued success possible. Further, it was the specialization of mammals into cold-weather and low-resource niches which made our freak success possible. And then it was human adaptability which made us capable of succeeding in every climate zone. Luck is luck.

    2) The problem is that specialization = risk = prosperity. They are all the same thing, and they are all tied together. Unemployment is a phenomenon of prosperity; by definition, if everyone is doing the same thing, all we do is reduce one another’s marginal productivity. It’s when we specialize and compete/cooperate in the same action that we build one another up and are able to leave meaningful capital behind for our children.

    So in a world where our prosperity is tied up in individual risk, does it make sense for our large institutions to create more risk in search of more prosperity, or does it make sense for them to mitigate risk and paradoxically avoid productiveness to create more long-term success? The answer lies in the following parable: during snowstorms, all of the city workers are pulled from their regular duties to snow removal.

  13. Alice
    March 18, 2011 at 8:17 am

    My greatest sense of whats truly grossly inefficient is the amount of money being thrown at the Japanese banks by Japan. Still going down the same path (helicopter Ben’s path)…imagine what else they could have done with that money right now?
    This bailout will cost the Japanese dearly….but just shows another country, another neoliberal remedy destined to fail miserably.
    See how effective it turns out to be and stack it on the evidence shelf.

  14. ben
    March 18, 2011 at 5:17 pm

    Bernard Lietaer has an interesting take on efficiency versus resiliency in economics (also based on a natural metaphor but a bit more grounded in natural science). He talks about the efficiency and resilience and how those are functions of diversity and interconnectivity.

    Also, Barry Lynn has an interesting historical bit on efficiency in his book Cornered. He notes that “efficiency” has been used a defense for the centralization of political and economic power for at least a hundred years and thus basically sets up a dichotomy between efficiency and freedom. His argument is that in the 1970s efficiency started being defined in relation to consumers instead of producers, which enabled Reagan’s Justice Dept to circumvent all sorts of antitrust legislation. This led to things like Wal-Mart and a variety of other firms that were effectively monopolies but were happily allowed to exist so long as they didn’t pass costs on to consumers.

    • Alice
      March 21, 2011 at 8:03 am

      Words like efficiency and competitiveness have no inherent meaning at all. Im with Peter. I hate its (their) use in economics with a passion also. Ive long known they mean nothing – so firms want to save a few costs here and there – how can they become more “competitive”?? Textbook answer – by becoming more “efficient”.

      Wow – thats instructive (not) – most wouldnt have a clue whether they had passsed the point of efficiency and were now screwing the creativity, morale and poductivity of their human capital but nonetheless – some useless piece of human garbage is paid a fortune to ruin the lives of employees, creditors and others with interest in the firm.

      Firms have stated they needed to pay unnecessary salaries to appoint known fly by night con men to manage their companies in order to be more “competitive” and also more “efficient”. These same two words are also used to exploit and mistreat labour – in order to be “more competitive” and “more efficient” we need to slash the workforces, bust unions, casualise labour, give them no rights, commandeer their salaries and force them to save for their own retirements in real money we then use to pay exorbitant financial sector bonuses.

      Efficiency for some, competitiveness for some, misery for millions (and the rising problem of unemployment taking us further away from so called efficiency) all in the name of competitiveness and efficiency.

      How can the word have any meaning if the excessive pursuit of it, creates its own anthithesis?

      Do these icon words of modern day economics deliver and for whom do they deliver? The very core of MSM economics twisted to something no decent economist would ever put their name to.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.