Hi Vaniver! =D
On the commentary: your eyeballing seems good, but I don't think I ever said anything about relative comparisons between correlation coefficients (namely just overall correlation is positive). As you observed, I could easily make all 3 correlations (blue-only, green only, or blue+green) positive. I don't have any interesting things to say about their relative degrees.
I don't quite see the difference in interpretation from this writing. I agree with basically all the stuff you've written? The fact that the slicing "behaves as a filter", if I interpret it correctly, is exactly the problem here.
I don't know what "have a different origin than Simpson's paradox" means exactly, but here are a few ways they differ and why I say they are "different":
a fundamental assumption on Simpson's paradox is that there's some imbalance with the denominators; in your 2x2x2 matrix you can't arbitrarily scale the numbers arbitrarily; all the examples you can construct almost relies on (let's say we are using the familiar batting averages example) the fact that the denominators (row sums) are different.
the direct cause of the reversal effect is, as you said, the noise; I don't think Simpson's paradox has anything related to the noise.
Idea: my steel-man version of your argument is that reversal effects arise when you have inhomogenous data, and this is definitely the more general common problem in both situations. In that case I agree. (this is how I teach this class at SPARC, at least).
I don't think I ever said anything about relative comparisons between correlation coefficients (namely just overall correlation is positive).
The main line I'm thinking of is:
the data is telling a very simple story, which is that A) blue men are more educated and B) more educated people get paid more.
I don't think this story quite captures the data, because I can construct a model where both of these are true but we don't get this effect. If you have the same link between income and education for each group conditioned on knowing group membership (an...
An article by Judea Pearl, available here. It's quick at 8 pages, and worth reading if you enjoy statistics (though I think people who already are familiar with the math of causality1 will get more out of it than others2). I'll talk here about the part that I think is generally interesting:
I've never really liked the name "paradox," because what it seems to mean is "unintuitive phenomenon." (Wikipedia puts it as "something which seems false and yet might be true.") The trouble is that "unintuitive" is a two-place word, and it makes sense to think like reality, so that true things seem true to you, instead of still seeming false. (For example, when I first learned about Zeno's Paradox, I already knew calculus, and so Zeno's position was the one that seemed confusing and false.)
What I like most about Pearl's article is that it explicitly recognizes the importance of fully dissolving the paradox,3 and seems to do so. Simpson's Paradox isn't an unsolvable problem in statistics, it's a straightforward reversal effect--only if you use the language of causality.
1. My review of Causality gives a taste of what it would look like to be familiar with the math, but you'd need to actually read the book to pick it up. The Highly Advanced Epistemology 101 for Beginners sequence is relevant, and contains Eliezer's attempt to explain the basics of causality in Causal Diagrams and Causal Models.
2. Pearl discusses how you would go about using simulations to show that do calculus gives you the right result, but leaves it as an exercise for the reader.
3. How An Algorithm Feels From Inside is probably a better place to start than Dissolving the Question, and I can't help but echo a question from it: "So what kind of math design corresponds to [Simpson's Paradox]?"
See also: bentarm's explanation of Simpson's Paradox.