= 27d567bc2d48d9f40e9ccc20ea370b15)
taw comments on Sociosexual Orientation Inventory, or failing to perform basic sanity checks - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (41)
You never say what the problems are. So what if the subjects lie? Maybe "orientation" as measured by what people want to project is more useful than their actual behavior.
This reminds me of Shalizi's complaints about IQ (not his complaints about intergroup differences that Johnicholas linked). One difference is that for SOI there's a definite number (actual turnover of sex partners) that could be confused with it. But the name seems designed to discourage that confusion.
These are real problems, but it's not obvious that the "basic sanity checks" you suggest would lead to better measures.
Shalizi's complaints are semi-valid, that if you throw a huge amount of somewhat correlated data at PCA, you will most likely get a small number of components, with one explaining most of the variance. And when you start removing data that doesn't correlate highly enough (as obviously "testing something else"), the leading component will only seem statistically stronger.
I'm quite surprised but it mirrors very closely what I think about the Big Five personality traits - factors on their own don't really prove anywhere as much as is commonly stated, and can as easily be statistical artifacts.
This criticism doesn't mean that either IQ or big 5 are invalid, but it does mean that the case should be made for them independently of "they show up as big factors in PCA". It seems to be so for IQ, and I'm not that terribly convinced it's also true for the Big Five.