= 27d567bc2d48d9f40e9ccc20ea370b15)
Eugine_Nier comments on Alternative Places to Get Ideas (Also, "In Defense of Food") - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (36)
We have to be careful when thinking about "science" as a single entity; the science that physicists do, the science that biologists, and the science that nutritionists do are each very different.
My take on what happened to nutrition science is that the nutrition science research community settled on a paradigm (controlled dietary studies followed by measuring indirect proxies for health) that was inadequate. They then put out a bunch of studies, each of which was only very weak evidence and had an extraordinarily long list of caveats. This got amplified first by reporting p-values which failed to account for those caveats, and then again by the media; and the result was a bunch of dietary recommendations that were some combination of noise, echo chamber effects, and deliberate manipulation, with barely any signal.
But that isn't a failure of science, per se. That's a failure of the research and publication methodologies of one particular field. It is concerning that other fields are using similar publication methodologies (especially the use of p-values), and there are some other fields where there is reason to suspect that the signal to noise ratio is also bad. The lesson I take from nutrition science is that you can't trust a community's output just because they call their work "science" and have all the trappings thereof; you have to look closely, see if it makes sense, and see how far above the noise floor their models' predictions really are.
I mostly agree but am bothered by the fact that from an outside view this sounds like No True Scotsman.
This is a valid point. However, there is an objective fact that's different between physics/biology and nutrition: in the former, there is a lot of historical progress: stuff discovered and promoted at a high confidence tends to be supported and replicated. In the latter, stuff promoted at high confidence by the media is fairly likely to be contradicted again soon after. So it's significantly more reasonable to ignore the results of nutrition science when deciding what to eat than it is to ignore the predictions of, say, biology when deciding whether to vaccinate your children.
I suspect much of medicine especially the newer stuff is probably nearly as bad as nutrition.
Edit: See Robin Hanson's many posts on the subject.
Why especially the "newer stuff"?
The two most obvious reasons are:
1) Once the low-hanging fruit is exhausted, people are more likely to make stuff up.
2) Newer stuff has had less time for problems to get exposed.
Just curious, as I've heard the opposite asserted with confidence.
1) Very little of the Hansonian critique of medicine involves researchers making stuff up, and I doubt this is a major problem.
2) True, although hopefully research methodology is improving.
This analysis may interest you, I seem to recall it supports your suspicion.
Sorry about that, I didn't meant more generate results based on statistical noise, then outright faking research.
Ah. Then yeah that's a problem, but I'm not sure why this would be worse with recent research.
This article gives a pretty good overview of the shortcomings of medical statistics, and includes one of my favorite lines ever:
Because the earlier stuff, e.g., sanitation, vaccines, and antibiotics, had a stronger effect and thus was easier to notice above the noise.