Vaniver comments on Understanding Simpson's Paradox - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (19)
The main line I'm thinking of is:
I don't think this story quite captures the data, because I can construct a model where both of these are true but we don't get this effect. If you have the same link between income and education for each group conditioned on knowing group membership (and a net positive relationship without knowing group membership), but you have the blue group mean only to the right (i.e. more educated) than the green group mean, then you don't have this effect because equal education lines don't have blues earning more than greens (they earn less; this is a straightforward 'discrimination against blues' story).
I would use the language of B to mean "in the three node model which has color, education, and income, the direct effect of education on income is positive," which does not appear to be true in the graphs, which look like they could be generated from a E<-C->I model. While it could also be used to mean "in the two node model which has education and income, the direct effect of education on income is positive," that seems unnatural in a case where you know that the link from E to I is spurious (that is, it flows backwards along the causal path from C to E; changing your level of education can't change your color). But this could just be me expecting an unreasonable level of precision from your wording, since the straightforward interpretation, though unnatural, does fit the data (although I think it reduces the strength of the "this doesn't show discrimination" claim, because it does show that what looked like a virtuous education-income link is now a questionable color-income link).
It's very possible I've imagined the difference / misunderstood what you've written. My appreciation of the filtering effect of the slices is also very recent, and I may think it's more important as I think about it more.
It seems that I'm quick to jump to a graphical model with nodes that captures the effects between these groups, and want to keep direct, indirect, and total effects separate. I think that's why I see the fundamental broad mechanism here as a reversal effect: if you learn about a node C that was previously hidden, the direct path from node A to node B can reverse sign if the indirect path from A to C and C to B takes over the strength of the old connection from A to B. (This has requirements on C, which I think matches up with your statement about inhomogenous data.)
In this view, noise is just a very convenient way to add new nodes which can cause these reversals, especially when the noise is multidimensional. So when I look at Simpson's paradox and RRE, the underlying mechanism I see is that there's a reversal effect going on in each, and so they look fundamentally similar to me. I got the impression from your post that you think there's a fundamental difference between them, and I don't see that difference- but I might have misread you, or overestimated what you think the size of the difference is.