Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

StuartBuck comments on Why the tails come apart - Less Wrong

116 Post author: Thrasymachus 01 August 2014 10:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (90)

You are viewing a single comment's thread.

Comment author: StuartBuck 28 July 2014 03:58:16AM 31 points [-]

It's not just that the tails stop being correlated, it's that there can be a spurious negative correlation. In any of your scatterplots, you could slice off the top right corner (with a diagonal line running downwards to the right), and what was left above the line would look like a negative correlation. This is sometimes known as Berkson's paradox.

Comment author: Thrasymachus 02 August 2014 02:07:38AM 4 points [-]

There's also a related problem in that population substructures can give you multiple negatively correlated associations stacked beside each other in a positively correlated way (think of it like several diagonal lines going downwards to the right, parallel to each other), giving an 'ecological fallacy' when you switch between levels of analysis.

(A real-world case of this is religiosity and health. Internationally, countries which are less religious tend to be healthier, but often within first world countries, religion confers a survival benefit.)

Comment author: tjohnson314 09 June 2015 08:45:12PM 1 point [-]

Another example I've heard is SAT scores. At any given school, the math and verbal scores are negatively correlated, because schools tend to select people who have around the same total score. But overall, math and verbal scores are positively correlated.

Comment author: owencb 14 August 2014 01:10:17PM 1 point [-]

Looks like you can get this if you cut the corner off in a box shape too, which may be more surprising.