gwern comments on Beware of WEIRD psychological samples - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (28)
I'd be worried about trusting the students. It's like giving them a test and your answer key, and telling them 'hey, we did our best in getting the right answers, but please work through all the problems again and see whether we made any mistakes'. This sort of thing only works if you don't get too much garbage in your replications.
The students might be honest enough to actually do all the work professionally, but I'm not sure I'd trust American students (a summer/semester isn't that long, and if they're in India, there are things to do there that could fill a lifetime; the temptation to just fudge up some data and go do all those awesome things would be tremendous), much less Indian ones.
You have way too much trust in the professors. Just a few students naive enough to do what they're supposed to would be an improvement on the status quo.
The problem is, we already have replications being done by Indian and Chinese scientists and... they're not very good. Here's one: "Local Literature Bias in Genetic Epidemiology: An Empirical Evaluation of the Chinese Literature", 2005:
The huge amount of data that could be gathered should allow for checking; data that is both different from what westerners would expect, and consistent over several independent students, is likely to be accurate. Or at least, not inaccurate because of lazy students.