Home > Uncategorized > Discrimination and the use of ‘statistical controls’

Discrimination and the use of ‘statistical controls’

from Lars Syll

The gender pay gap is a fact that, sad to say, to a non-negligible extent is the result of discrimination. And even though many women are not deliberately discriminated against, but rather self-select into lower-wage jobs, this in no way magically explains away the discrimination gap. As decades of socialization research has shown, women may be ‘structural’ victims of impersonal social mechanisms that in different ways aggrieve them. Wage discrimination is unacceptable. Wage discrimination is a shame.

You see it all the time in studies. “We controlled for…” And then the list starts … The more things you can control for, the stronger your study is — or, at least, the stronger your study seems. Controls give the feeling of specificity, of precision. But sometimes, you can control for too much. Sometimes you end up controlling for the thing you’re trying to measure …

paperAn example is research around the gender wage gap, which tries to control for so many things that it ends up controlling for the thing it’s trying to measure …

Take hours worked, which is a standard control in some of the more sophisticated wage gap studies. Women tend to work fewer hours than men. If you control for hours worked, then some of the gender wage gap vanishes. As Yglesias wrote, it’s “silly to act like this is just some crazy coincidence. Women work shorter hours because as a society we hold women to a higher standard of housekeeping, and because they tend to be assigned the bulk of childcare responsibilities.”

Controlling for hours worked, in other words, is at least partly controlling for how gender works in our society. It’s controlling for the thing that you’re trying to isolate.

Ezra Klein

Trying to reduce the risk of having established only ‘spurious relations’ when dealing with observational data, statisticians and econometricians standardly add control variables. The hope is that one thereby will be able to make more reliable causal inferences. But — as Keynes showed already back in the 1930s when criticizing statistical-econometric applications of regression analysis — if you do not manage to get hold of all potential confounding factors, the model risks producing estimates of the variable of interest that are even worse than models without any control variables at all. Conclusion: think twice before you simply include ‘control variables’ in your models!

piled-up-dishes-in-kitchen-sinkWhen I present this argument … one or more scholars say, “But shouldn’t I control for everything I can in my regressions? If not, aren’t my coefficients biased due to excluded variables?” … The excluded variable argument only works if you are sure your specification is precisely correct with all variables included. But no one can know that with more than a handful of explanatory variables …

A preferable approach is to separate the observations into meaningful subsets—internally compatible statistical regimes … If this can’t be done, then statistical analysis can’t be done. A researcher claiming that nothing else but the big, messy regression is possible because, after all, some results have to be produced, is like a jury that says, “Well, the evidence was weak, but somebody had to be convicted.”

Christopher H. Achen

Kitchen sink econometric models are often the result of researchers trying to control for confounding. But what they usually haven’t understood is that the confounder problem requires a causal solution and not statistical ‘control.’ Controlling for everything opens up the risk that we control for ‘collider’ variables and thereby create ‘back-door paths’ which gives us confounding that wasn’t there to begin with.

  1. Ikonoclast
    June 14, 2021 at 11:05 pm

    “If you make a model, after a while you get suckered into it. You begin to forget that it’s a model and think of it as the real world.” – James Lovelock.

  2. Ken Zimmerman
    June 15, 2021 at 9:24 am

    Lovelock is correct in my view. But I am suggesting something that may look like a model but is not. It is an approach followed by physical scientists and historians. Simulation attempts to illustrate (not replicate) some specific set of past events’, not models, which show how a system has worked in the past, but also how it will work in the future.  John Gaddis pits both scientists and historians, who operate with complex and chaotic systems with many variables, against social scientists, who seek to understand reality by breaking it up into its various parts.  The latter seem to have a ‘reductionist’ and the former an ‘ecological’ view of reality. Among social scientists, economists are radically reductionist. Simulations can be reconstructed and reconfigured over time to make them more exact illustrations of the events of interest to the historian or scientist. And thus more useful.

  3. Ikonoclast
    June 15, 2021 at 11:50 pm

    For sure, there is nothing wrong with models if they are empirically validated and used correctly. “Used correctly” includes understanding where they might and might not apply.

    Take the “Orbit Equation”. (See the Wikipedia entry.) The equation we can see as a model or as the mathematical distillation of a model. The model comes with a set of defining conditions which are already formalized

    “Consider a two-body system consisting of a central body of mass M and a much smaller, orbiting body of mass m, and suppose the two bodies interact via a central, inverse-square law force (such as gravitation). In polar coordinates, the orbit equation can be written as…”

    I won’t try to type mathematical notation here. Notice that the problem is defined and limited. The equation is only for a two-body system. It assumes a central inverse-square law force (such as gravitation). There are other limiting conditions.

    One has only to take matters one step further (the three-body problem) and new difficulties arise.

    “The three-body problem is a special case of the n-body problem. Unlike two-body problems, no general closed-form solution exists, as the resulting dynamical system is chaotic for most initial conditions, and numerical methods are generally required.”

    “In mathematics, a closed-form expression is a mathematical expression expressed using a finite number of standard operations.”

    There is no longer a neat, finite equation with a perfect solution or perfect set of solutions. The problems of numerical method relate to the computational and algorithmic load and to issues of approximation and convergence. Even at the level of the three-body problem, the cosmos ceases to neatly mathematicizable and computable. At least, this is my interpretation of the issue. Accurate model become less and less feasible, expecially when we begin to consider multiple feedbacks and the impacts of human calculations and feelings as feedback into a system when those calculations and feelings are acted upon. This renders economics an n-body problem, a n-feed-back problem and real system / formal system / agent system interaction problem. At this point, we ought to realize the complexity and the ultimate “un-modelability” of economic problems or more properly political-economy and social power problems.

    This suggests that we need to attempt. implement or apply heuristic solutions, asymptotic analysis and quite frankly moral philosophy tenets. What does this mean? It means in a nutshell, rules of thumb, making sure we don’t push systems, natural and human, beyond their limits and the application of ethics. This is bare bones and would take a lot more fleshing out than is possible in a short blog post. It would mean abandoning money and finance calculations as the guiding algorithms of our civilization and instead employing ethics, democracy and science. At least, that is how I see it.

    Capitalism is about to suffer a huge defeat. This defeat will come from the natural earth systems which will demonstrate that capitalist precepts cannot manage a civilization on a planet without catastrophically disrupting vital earth system by crossing planetary boundaries. If humans survive this catastrophe one hopes that the lessons become part of our secular philosophical, scientific and political economy “wisdom literature”. The capitalist file must be closed and stamped “Never Again”.

    (All quotes from Wikipedia.)

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.