Estimate Effect Sizes
I think the following recommendation might reduce a particular negative reaction to blog posts and claims by 10%-50%, and generally improve the precision of discussion by a slight amount.
There are many phenomena that we observe that we aren't certain how to explain, and discussing potential explanations is a good way to converge on better explanations. Many phenomena are complicated, and so simple explanations are unlikely to explain everything- but a combination of simple explanations might explain most (or even all) of the phenomena.
One common way to approach this issue is to throw out a list of possibilities that could explain the phenomenon. This can run into trouble several ways: a list that is meant to explain only a fraction of the total effect could be read as exclusive, the list might come across as mutually exclusive instead of cooperative, or the relative importance of the various proposed possibilities might not be clear. It can also be bothersome to have to list the entirety of the possible alternatives or co-factors whenever you want to focus on a particular explanation.
So consider estimating the effect size as a way to clarify and limit your claims. Numerical estimates are likely to be more communicative than verbal estimates (what does it mean to be a "minor" factor, or a "significant" factor?), but harder to generate. Leading with your estimate, rather than adding it as a caveat at the end, is likely to be more communicative. This often degrades the flow of an argument, but as the point is often to prevent people from disconnecting from the argument, flow afterwards is not going to be meaningful for those people.
Counterargument: Robin's post Against Disclaimers seems relevant. This post was inspired by the negative reaction to this article by Yvain.
[LINK] Will Eating Nuts Save Your Life?
TLDR: Study on death avoidance, which interests a lot of people here, and commentary on what sort of informative priors we should have about health hypotheses.
From Steve Sailer, who is responding to Andrew Gelman, who got sent this study. An observational study showed that people who consumed nuts were less likely to die; Gelman points out that the study's statistics aren't obviously wrong. Sailer brings up an actual RCT of Lipitor from the 90s:
The most striking Lipitor study was one from Scandinavia that showed that among middle-aged men over a 5-year-period, the test group who took Lipitor had a 30% lower overall death rate than the control group. Unlike the nuts study, this was an actual experiment.That seemed awfully convincing, but now it just seems too good to be true. A lot of those middle-aged deaths that didn't happen to the Lipitor takers didn't have much of anything to do with long-term blood chemistry, but were things like not driving your Saab into a fjord. How does Lipitor make you a safer driver?I sort of presumed at the time that if they had taken out the noisy random deaths, that would have made the Lipitor Effect even more noticeable. But, of course, that's naive. The good folks at Pfizer would have made sure that calculation was tried, so I'm guessing that it came out in the opposite direction of the one I had assumed. Guys who took Lipitor everyday for five years were also good about not driving into fjords and not playing golf during lighting storms and not getting shot by the rare jealous Nordic husband or whatever. Perhaps it was easier to stay in the control group than in the test group?Here’s how I would approach claims of massive reductions in overall deaths from consuming some food or medicine:Rank order the causes of death by how plausible it is that they are that they are linked to the food or medicine. For example:1. Diabetes2. Heart attacks3. Strokes4. Cancer5. Genetic diseases6. Car accidents7. Drug overdoses8. Homicides9. Lightning strikesIf this nuts-save-your-life finding is valid, then the greater effects should be found in causes of death near the top of the list (e.g., diabetes). But if it turns out that eating nuts only slightly reduces your chances of death from diabetes but makes you vastly less likely to be struck by lighting, then we’ve probably gotten a selection effect in which nut eaters are more careful people in general and thus don’t play golf during thunderstorms, or whatever.
Table 3 of the paper breaks out the hazard ratios by cause of death. The most impressive effects (as measured by the right tail of the 95% CI for pooled men and women for any nut)1 are Heart Disease, All Causes, Other Causes, Cancer, Respiratory Disease, Stroke, Infection, Diabetes, Neurodegenerative Disease, and Kidney Disease.
Steve's categories and the paper's categories don't overlap very well. But it looks to me like if you follow Steve's logic, it's reasonable to believe that nuts have a protective effect against heart disease, and then most of the other effects or non-effects have a common cause with nut consumption, like healthiness / conscientiousness / whatever, rather than being caused by nut consumption. Note the strong negative relationships between nut consumption and BMI or smoking, and the strong positive relationships between nut consumption and physical activity or intake of fruits, vegetables, or alcohol. The hazard ratios are calculated controlling for those variables, but it's still reasonable to see there being a hidden 'health-consciousness' node which noisily affects all of those nodes.
It's also interesting to look at the negative results- the hazard ratio for neurodegenerative disease and stroke was roughly 1, implying that nut-eaters and non-nut eaters had comparable risks, despite 'other causes' having a hazard ratio of 0.87. That weakly implies to me that either health consciousness has no impact on neurodegenerative disease and stroke, or that nuts are harmful for those two categories.
Since heart disease is a huge killer (24% of all deaths in the study group), this study seems like moderate evidence in favor of eating nuts, but it's likely that the total study's effect is overstated. (The study also suggests that tree nuts are probably superior to peanuts; I know various QS people have raised concerns that the kind of nut matters significantly.)
1. This is a heuristic for impressiveness, not the point estimate. It looks like nuts have the strongest effect for kidney disease, with a mean hazard ratio estimate of 0.69- but the upper bound of the 95% CI is 1.26, because only a handful of people died due to kidney disease. The heart disease hazard ratio estimate is 0.74 (0.68-0.81), which is much more believable, even though the point estimate is slightly higher. The point estimate for diabetes is 0.80 (0.54-1.18), which has a mean estimate that's only slightly worse, but diabetes again killed far fewer than heart disease. If you order them by point estimates, the paper is stronger evidence for nuts being useful for dietary reasons, and which method you prefer depends on your priors for how representative this sample is.
Understanding Simpson's Paradox
An article by Judea Pearl, available here. It's quick at 8 pages, and worth reading if you enjoy statistics (though I think people who already are familiar with the math of causality1 will get more out of it than others2). I'll talk here about the part that I think is generally interesting:
Rationality Quotes September 2013
Another month has passed and here is a new rationality quotes thread. The usual rules are:
- Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
- Do not quote yourself.
- Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.
- No more than 5 quotes per person per monthly thread, please.
Harry Potter and the Methods of Rationality discussion thread, part 27, chapter 98
This is a new thread to discuss Eliezer Yudkowsky’s Harry Potter and the Methods of Rationality and anything related to it. This thread is intended for discussing chapter 98. The previous thread is at nearly 500 comments.
There is now a site dedicated to the story at hpmor.com, which is now the place to go to find the authors notes and all sorts of other goodies. AdeleneDawner has kept an archive of Author’s Notes. (This goes up to the notes for chapter 76, and is now not updating. The authors notes from chapter 77 onwards are on hpmor.com.)
The first 5 discussion threads are on the main page under the harry_potter tag. Threads 6 and on (including this one) are in the discussion section using its separate tag system.
Also: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26.
Spoiler Warning: this thread is full of spoilers. With few exceptions, spoilers for MOR and canon are fair game to post, without warning or rot13. More specifically:
You do not need to rot13 anything about HP:MoR or the original Harry Potter series unless you are posting insider information from Eliezer Yudkowsky which is not supposed to be publicly available (which includes public statements by Eliezer that have been retracted).
If there is evidence for X in MOR and/or canon then it’s fine to post about X without rot13, even if you also have heard privately from Eliezer that X is true. But you should not post that “Eliezer said X is true” unless you use rot13.
Rationality Quotes August 2013
Another month has passed and here is a new rationality quotes thread. The usual rules are:
- Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
- Do not quote yourself.
- Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.
- No more than 5 quotes per person per monthly thread, please.
Rationality Quotes July 2013
Another month has passed and here is a new rationality quotes thread. The usual rules are:
- Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
- Do not quote yourself.
- Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson.
- No more than 5 quotes per person per monthly thread, please.
Open Thread, July 1-15, 2013
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Harry Potter and the Methods of Rationality discussion thread, part 19, chapter 88-89
This is a new thread to discuss Eliezer Yudkowsky’s Harry Potter and the Methods of Rationality and anything related to it. This thread is intended for discussing chapter 88-89. The previous thread has passed 500 comments.
There is now a site dedicated to the story at hpmor.com, which is now the place to go to find the authors notes and all sorts of other goodies. AdeleneDawner has kept an archive of Author’s Notes. (This goes up to the notes for chapter 76, and is now not updating. The authors notes from chapter 77 onwards are on hpmor.com.)
The first 5 discussion threads are on the main page under the harry_potter tag. Threads 6 and on (including this one) are in the discussion section using its separate tag system. Also: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18.
Spoiler Warning: this thread is full of spoilers. With few exceptions, spoilers for MOR and canon are fair game to post, without warning or rot13. More specifically:
You do not need to rot13 anything about HP:MoR or the original Harry Potter series unless you are posting insider information from Eliezer Yudkowsky which is not supposed to be publicly available (which includes public statements by Eliezer that have been retracted).
If there is evidence for X in MOR and/or canon then it’s fine to post about X without rot13, even if you also have heard privately from Eliezer that X is true. But you should not post that “Eliezer said X is true” unless you use rot13.
Produce / Consume Ratios
I've been thinking about this a bit recently, and thought I'd do a dump of evidence and conjecture, and see what Less Wrong had to say.
There are lots of areas of life where activities can be partitioned into either producing products or consuming products. For those areas, it may be worthwhile to calculate one's Produce / Consume Ratio (PCR) and also contemplate what the optimal PCR is for that area.
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)