Desrtopa comments on Follow-up on ESP study: "We don't publish replications" - Less Wrong

71 Post author: CarlShulman 12 July 2011 08:48PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (50)

You are viewing a single comment's thread. Show more comments above.

Comment author: Desrtopa 19 July 2011 09:00:08PM 3 points [-]

It's not clear to me that he was talking about studies being ignored because they're not interesting enough to cite, rather than studies being ignored because they're not trustworthy enough to cite.

In any case, I think both are dubious safety mechanisms. John Ioannidis found that even most the most commonly cited studies in medical research are highly likely to be false. If researchers are basing their trust in studies on the rate at which they're cited, they're likely to be subject to information cascades, double counting the information that led other researchers to cite the same study.

In the case of only citing papers that contain numerous citations, this is helpful if the papers contain many redundant citations, demonstrating that the factual claims have been replicated, but if a paper relies on many uncertain findings, then its own uncertainty will be multiplied. The conclusion is at most as strong as its weakest link.