CronoDAS comments on She Blinded Me With Science - Less Wrong

13 Post author: Jonathan_Graehl 04 August 2009 07:10PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (38)

You are viewing a single comment's thread. Show more comments above.

Comment author: Douglas_Knight 04 August 2009 08:47:23PM 7 points [-]

I suspect that many authors are hesitant to subject themselves to the sort of scrutiny they ought to welcome.

Normative language ("ought") is not helpful here. Journals that nominally require publication of data or calculations don't enforce it, either.

One way to deal with selection bias and fraud that I have occasionally seen, and only in economics and parapsychology ("the control group for science"), is to compare the effect size to the study size. If it's a real effect, it will not depend on the study size. But if it's fake, it will always just barely be statistically significant and thus it will decline with study size.

This kind of meta-analysis come from not trusting one's peers. This is rude, hence rare. But it's a lot more useful than pooling the data, the usual meta-analysis.

Comment author: CronoDAS 04 August 2009 09:22:08PM 1 point [-]

Parapsychology: The control group for science.

Excellent quote. May I steal it?

Comment author: Tyrrell_McAllister 04 August 2009 09:26:28PM 0 points [-]

It's too good to ask permission for. I'll wait to get forgiveness ;).