Eliezer_Yudkowsky comments on Med Patient Social Networks Are Better Scientific Institutions - Less Wrong

37 Post author: Liron 19 February 2010 08:11AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (49)

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 19 February 2010 06:22:37PM 18 points [-]

You don't want to rely on studies in medical journals because their conclusion-drawing methodologies are haphazard.

I dispute none of this, but so far as I can tell or guess, the main thing powering the superior statistical strength of PatientsLikeMe is the fact that medical researchers have learned to game the system and use complicated ad-hoc frequentist statistics to get whatever answer they want or think they ought to get, and PatientsLikeMe has some standard statistical techniques that they use every time.

Also, I presume, PatientsLikeMe is Bayesian or Bayes-like in that they take all available evidence into account and update incrementally, while every medical experiment is a whole new tiny little frequentist universe.

This is not really an article about PatientsLikeMe being strong, it is an article about the standard statistical methods of academic science being weak and stupid.

Comment author: PhilosophyTutor 13 May 2012 09:02:39PM 0 points [-]

What is your evidence for the claim that the main thing powering the superior statistical strength of PatientsLikeMe is the fact that medical researchers have learned to game the system and use complicated ad-hoc frequentist statistics to get whatever answer they want or think they ought to get? What observations have you made that are more likely to be true given that hypothesis?

Comment author: toto 20 February 2010 08:23:15PM *  5 points [-]

I dispute none of this, but so far as I can tell or guess, the main thing powering the superior statistical strength of PatientsLikeMe is the fact that medical researchers have learned to game the system and use complicated ad-hoc frequentist statistics to get whatever answer they want or think they ought to get, and PatientsLikeMe has some standard statistical techniques that they use every time.

1) I'd like to see independent evidence of their "superior statistical strength".

2) On the face of it, the main difference between these guys and a proper clinical trial is an assumption that you can trust self-reports. Placebo effect be damned.

In particular, I'd really, really like to see the results for some homeopathic "remedy" (a real one, not one of those that silently include real active compounds).

Comment author: conchis 23 February 2010 06:33:01PM 0 points [-]

Isn't the main difference just that they have a bigger sample. (e.g. "4x" in the hardcore group).