Fair enough. I don't think the biases are symmetrical though: these people have a real and life-threatening disease, so they approach any intervention hoping strongly that it will work; hence we should expect them to yield more false positives than false negatives compared to whatever an equal medical trial would yield. On the other hand, when we're looking at the chatrooms of hypochondriacs & aspartame sufferers, I think we can expect the bias to be reversed: if even crazy people find nothing to take offense to in something, that something may well be harmless.
This yields the useful advice that when looking at any results, we should look at whether the participants have an objectively (or at least, third-party) validated problem. If they do, we should pay attention to their nulls but less attention to their claims about what helps. And vice versa. (Can we then apply this to self-experimentation? I think so, but there we already have selection bias telling us to pay little attention to exciting news like 'morning faces help my bipolar', and more attention to boring nulls like 'this did nothing for me'.)
Kind of a moot point I guess, because the fakes do not seem to be well-organized at all.
I think you're probably right in general, but I wouldn't discount the possibility that, for example, a rumour could get around the ALS community that lithium was bad, and be believed by enough people for the lack of blinding to have an effect. There was plenty of paranoia in the gay community about AZT, for example, despite the fact that they had a real and life-threatening disease, so it just doesn't always follow that people with real and life-threatening diseases are universally reliable as personal judges of effective interventions.
Similarly if the wi-...
When you're suffering from a life-changing illness, where do you find information about its likely progression? How do you decide among treatment options?
You don't want to rely on studies in medical journals because their conclusion-drawing methodologies are haphazard. You'll be better off getting your prognosis and treatment decisions from a social networking site: PatientsLikeMe.com.
PatientsLikeMe.com lets patients with similar illnesses compare symptoms, treatments and outcomes. As Jamie Heywood at TEDMED 2009 explains, this represents an enormous leap forward in the scope and methodology of clinical trials. I highly recommend his excellent talk, and I will paraphrase part of it below.