Nov 2, 2013

Correlation, causation, families and poverty

It's not really possible to support nuclear families and support public policies that support nuclear family stability without, it seems, being really stupid about the difference between correlation and causation. It should be possible. It really seems like it should be possible. It is not necessary that pro-family public policies be based on illiterate, illogical claims.

And yet.

A recent example: At the Atlantic, W. Bradford Wilcox reports on his research on connections between family structures and children's future success. He shows a connection, but then claims that connection means more than he shows.

He writes,
My own research using individual-level data from the Add Health dataset for the Home Economics Project, a new joint initiative between the American Enterprise Institute and the Institute for Family Studies, indicates that adolescents raised in intact, married homes are significantly more likely to succeed educationally and financially.
Wilcox calls this the "marriage bump," and cites some interesting numbers that seem to show this connection. In no case, though, does he show that that connection is causal. Instead, he shows that people whose parents were married are 44 percent more likely to go to college. People whose parents are married generally (though not always) earn more money. Yet are those outcomes produced by the parents' marriages, or is there another factor at work, which results in both the family stability and the child's success? Wilcox assumes correlation implies causation and doesn't seem to know why one would ask that question. But the question is still there: where's the proof that one thing causes the other?

It's entirely plausible that family stability has the effects that Wilcox and those like Wilcox say it does. The "bump" may well bump. It has to be shown, though. Wilcox is just wiggling his eyebrows and calling it proof.

That shouldn't be good enough.

It shouldn't be good enough especially if it's being used to support arguments for important public policies. Getting the correlation-causation relationship wrong has some real consequences.

Wilcox of course knows that there are some questions about this argument he's been making for a while now, though he frames the criticism as criticism of the idea. He says it "strikes some as a spurious correlation" that "marriages have such strong spillover effects." What's spurious isn't the idea, though. What's spurious is the correlation claiming to be something more without any evidence of more.

But there is evidence of a causal relationship, too. MIT economist John Gruber, studying the effect of divorce on later incomes, found that adults "exposed to unilateral divorce regulations as children are less well educated, have lower family incomes, marry earlier but separate more often, and have higher odds of suicide."
This, of course, has exactly the same problems as Wilcox's own research. That's not evidence. That's correlation. Lots and lots of correlations don't add up to causation. They're different things. At least as presented, the problem of what-caused-what is still a problem with this other study, despite the fact that Wilcox is offering this as evidence of causation.

Maybe he doesn't know what causation would look like, anymore.

There are scholars who are more cautious about this. They have attempted to figure out if the connection between family structure and child poverty are causal. They do this by not just charting the correlations, but by attempting to control for variables, come up with tests, and hypotheses that could be falsified.

They do this by being honest about the correlation-causation problem with their research, and trying to deal with it.

In three different studies, for example, in 1996, 2002 and 2003, researchers simulated hypothetical marriages. They attempted to run the data to see how things would be different if single men and women were paired up, how rates of child poverty and family income would change. They did find some change. In one study, the simulation showed that with marriage rates from 1970, childhood poverty in 1998 would have dropped by 3.4 percent. In another, the poverty rates of single mothers getting no financial support from their children's fathers was reduced by half during the simulated marriage calculation. Before, 86 percent of single mothers not being financially supported by fathers were in poverty. After, when calculating that number as if they were married, only 46 percent were in poverty.

It's not entirely clear those were more successful than Wilcox in showing a meaningful connection between marriage and children's success, but those studies were also more circumspect in their claims. As Adam Thomas and Isabel Sawhill wrote in 2005, interpreting the data is tricky:
Do these findings necessarily mean that differences in family structure have created these economic disparities? Might it not be hat the sorts of people who are most likely to divorce or have children out of wedlock are also the sorts of people who are most likely to have limited incomes, regardless of their living arrangements? Could it be that economic distress helps to bring about marital dissolution? If the answers to such questions are yes, then one would expect to see a correlation between family structure and family economic well-being, even if the former had no effect on the latter ...

In light of this problem, how can one be certain whether family structure is helping to drive the differences in the incomes of married-parent, cohabiting, and lone-parent families? The short answer is that we cannot be absolutely sure.
That's such a better place to start.

Thomas and Sawhill and people taking that more measured approach show how support for nuclear families and support for public policies that support those families don't have to be predicated on correlation-causation confusion. It's not necessary that this research all start and end with waving hands and unsupported claims.

After all, just because stupidity about logical relationships strongly correlates with support for certain political positions doesn't mean it actually causes them.