Is "my grandfather is still alive, in no pain and walking eight miles a day, when six months ago he was given two months to live" stronger evidence for vitamin C effectivity than a reviewed paper saying "we have conducted a study on 1000 patients with terminal cancer; the survival rate in the group treated with large doses of vitamin C was not greater than the survival rate in the control group"? If so, why?
It would depend on the methodology used. I have seen enough examples of horribly bad - not to say utterly fraudulent - science in medical journals that I would actually take publication in a medical journal of a single, unreplicated, study as being slight evidence against the conclusion it comes to. (As an example, the Mayo clinic published a study with precisely those results, claiming to be 'unable to replicate' a previous experiment, in the early 80s. Except that where the experiment they were trying to 'replicate' had used intravenous doses, they used...
Today's post, Some Claims Are Just Too Extraordinary was originally published on 20 January 2007. A summary (taken from the LW wiki):
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was A Fable of Science and Politics, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.