Nick_Tarleton comments on Practical Advice Backed By Deep Theories - Less Wrong

42 Post author: Eliezer_Yudkowsky 25 April 2009 06:52PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (112)

You are viewing a single comment's thread. Show more comments above.

Comment author: badger 26 April 2009 08:58:16AM *  4 points [-]

The graph of Roberts's weight compared to fructose water intake on p. 73 of "What makes food fattening?" is very persuasive in my mind. I don't think there is any evidence that it is effective in the population at large, but I think it is clear cut that it worked for Roberts.

I don't think the cynical explanation gets very far. The details of the diet are freely available. There is only a single, cheap, slim book that Roberts published so that someone could learn about the diet in a format other than his website. Roberts could easily be mistaken, but I think his tone has consistently been "here is a little-known, easy technique that was highly effective for me; I have a theory why it could work for you too". It's hard to make money by telling someone to take three tablespoons of extra-light olive oil a day in addition to whatever other diet they are following.

Comment author: timtyler 26 April 2009 11:32:12AM 2 points [-]

One rat is just not statistically significant evidence - especially not when the rat is also the salesman. I don't know whether Roberts is motivated by wealth, fame, or whatever - nor do I care very much.

Comment author: Luke_A_Somers 20 November 2011 10:42:35PM *  3 points [-]

Many tests on the same rat can be statistically significant! Do X, Y changes in the rat. Undo it, Y changes back. Repeat until it's statistically certain connection...

We just have no particular reason to expect that it'll generalize well to others.

This really stands out to me as a physicist because we do things like one rat tests all the time. Well, usually we get a few other 'rats', but we rely heavily on the notion that identically prepared matter is... identical. Biology, of course, doesn't allow that shortcut.

Clinicians sometimes have a cohort of 1 for rare diseases... but of course that's simply the best they can do under the circumstances.

Comment author: timtyler 21 November 2011 12:10:58AM 2 points [-]

Many tests on the same rat can be statistically significant! Do X, Y changes in the rat. Undo it, Y changes back. Repeat until it's statistically certain connection...

True - but it won't be too convincing if self-experimenting on yourself with your own diet. Science is based on confirmations of experiments by other scientists.

Comment author: Luke_A_Somers 21 November 2011 03:25:20PM 2 points [-]

The rat being the salesman is the more serious issue there, yes.