What I want to know is: why is it necessary to come up with ad-hoc solutions for these types of problems instead of using bayesian networks, which were invented precisely for this purpose (and have been wildly successful at it)?
The usual way is to create a set of DAGs with various latent variables and figure out the maximum likelihood estimate, such as what Judea Pearl did for analyzing the relationship between smoking and lung cancer: http://www.michaelnielsen.org/ddi/if-correlation-doesnt-imply-causation-then-what-does/
why is it necessary to come up with ad-hoc solutions for these types of problems instead of using bayesian networks
It's unlikely you will find any good objections to this here even if they exist.
In most cases where it's possible, why not just use RCTs? They're simple to arrange.
In most cases where it's possible, why not just use RCTs? They're simple to arrange.
A RCT for nut consumption with a power to pick up an effect like this is probably prohibitively expensive to set up in the current funding enviroment.
If you look at the table of results, the nuts did have a slightly lower all-cause mortality result than the control group (which brings us back to the power issue).
I think financing is the most constraining part of "where it's possible". Drug trials are far more expensive, but easier to finance. Food trials should be cheaper, but more difficult to finance.
Have any researchers tried kickstarter? I think there would be a lot of people who would be willing to pay for this kind of stuff rather than drug research.
We can’t have X causing Y causing Z causing X! At least, not without a time machine. Because of this we constrain the graph to be a directed acyclic graph
The many components of life don't seem acyclic in their interdependencies. Anyone care to explain?
Sure. In a lot of cases where there seems to be a cyclic dependency, it's actually an acyclic dependency when unrolled over time. So X1 causes Y1 causes Z1 causes X2 (number denotes timestep).
The simplest model of such type is simply where X1 directly causes X2 which directly causes X3 and so on, which is called a Markov chain, and is a type of Bayesian network.
it's actually an acyclic dependency when unrolled over time
Yes; by introducing time steps and indexing by "at time tn" you can "always" do away with loops by kind of transforming them into spirals going down in time, from the loops they once were. But I wonder if people don't seem to mind that the endless spiral still resembles a loop, that "X at time t1" and "X at time t2" may technically be different nodes, but only in a trivial and uninteresting sense?
You can work DAG magic on such a "linearized" causal structure, I just wonder whether it's actually cutting reality at its joints, and whether there are (commonly used) alternatives.
You should definitely look at the book "Probabilistic Graphical Models" by Daphne Koller, these concepts and more are explained in depth and with numerous examples. I kind of consider it the spiritual successor to Jayne's work.
The reason why having a DAG is important is because it makes inference much easier. There are very efficient and exact algorithms for doing inference over DAGs, whereas inference over general graphs can be extremely hard (often taking exponential time). Once you roll out a loop in this fashion, eventually you reach a point where you can assume that X1 has very little influence over, say, X100, due to mixing. http://en.wikipedia.org/wiki/Markov_chain#Ergodicity Thus the network can be 'grounded' at X1, breaking the cycle.
If you can convert a cyclic dependency to a hundred acyclic dependencies, it makes sense to do so, and not just because of computational concerns.
Often real-world events really do unroll over time and we have to have some way of modelling time dependencies. Hidden Markov models and the Viterbi algorithm are a good example of this in action.
You should definitely look at the book "Probabilistic Graphical Models" by Daphne Koller
I will (... put it on my "do() at some indeterminable time in the near to far future"-list. Sigh.). Thanks.
What I want to know is: why is it necessary to come up with ad-hoc solutions for these types of problems instead of using bayesian networks, which were invented precisely for this purpose (and have been wildly successful at it)?
It's not necessary. The sort of reasoning Steve is doing here looks to me like putting informative prior distributions on the causal effect parameters. If we run pcalg on the dataset and it tells us "nut consumption causes age," we'll probably say "well, something went wrong." But with a large study size like this, it's not clear to me that informative priors are going to push the final estimate around by much. Hopefully, the DAG discovery methods will discover common causes, but the priors may be most useful in establishing correlations between the parameters- "if nuts don't help with X, we don't expect they help with Y, because of an underlying similarity between X and Y"- but I don't know if that's better accomplished by just adding another node to the system.
My addition was eyeballing what the DAG results would look like, which is nowhere near as good as getting the data and running pcalg on it.
I didn't write the linked article's title. But the body of the article is sort of like that- "these are the sorts of life-saving which that would just be silly for."
Guys who took Lipitor everyday for five years were also good about not driving into fjords and not playing golf during lighting storms and not getting shot by the rare jealous Nordic husband or whatever. Perhaps it was easier to stay in the control group than in the test group?
Probably not. They would have given the control group a placebo pill for 5 years and taken compliance data on that, and so the study drop-out rates would not have been a confound.
Guys who took Lipitor everyday for five years were also good about not driving into fjords and not playing golf during lighting storms and not getting shot by the rare jealous Nordic husband or whatever
That's an interesting way to look at confounders to say the least. How many people in the test group actually dropped out compared to the control group and was it accounted for? Should be easy to find out if one actually wanted to do that instead of just making motivated assumptions.
It almost seems the guy doesn't understand what randomization is for.
It almost seems the guy doesn't understand what randomization is for.
Just because an experiment is randomized doesn't mean that this effect doesn't show up, just that it's less likely to than in an observational study where there are potentially several incoming causal arrows to the treatment node.* When talking about the most impressive RCT from one of the more spectacular drugs of the past, it's actually not that silly to suspect that the experimental group got lucky, and all-cause mortality is a measure that is less susceptible to luck.
*That is to say, lack of causation does not imply no correlation in any given sample, just no expected correlation, and no expected correlation does not mean the expected r^2 is 0.
it's actually not that silly to suspect that the experimental group got lucky
True, but you shouldn't simply assume that. How silly it is depends on the actual numbers. He hardly provided any evidence, and he didn't seem to think it was about mere luckiness, but also that there was some selection effect because of dropouts.
Do you know the trial he's talking about? I can't find it and would like to know if he's attacking a strawman.
It appears that he is talking about this trial which has a follow-up performed 8 years later here. This was enrolled in the late 90s, concerned Lipitor, enrolled ~80% men, and included patients from the UK, Ireland, and Nordic countries.
If that is the study, it doesn't actually agree with what Steve Sailer says. In the first study, there is a 36% reduction in (non-fatal heart attacks + fatal heart disease), but the reduction in all-cause mortality is only 13% and not statistically significant. The trial was stopped early because the treatment arm had become highly statistically significant, which means that the lack of significance in all cause mortality isn't necessarily surprising.
As you have already pointed out, people who eat nuts also engage in other healthy activities. It sort of reminds me of the studies on moderate drinkers and death. Perhaps people who are able to control their drinking after having one or two beers, have more self-control in other areas of their life, compared to those who are heavy drinkers or teetotalers who avoid it like the plague.
Even after controlling for all of this, I wonder if their is an optimal nut intake.
TLDR: Study on death avoidance, which interests a lot of people here, and commentary on what sort of informative priors we should have about health hypotheses.
From Steve Sailer, who is responding to Andrew Gelman, who got sent this study. An observational study showed that people who consumed nuts were less likely to die; Gelman points out that the study's statistics aren't obviously wrong. Sailer brings up an actual RCT of Lipitor from the 90s:
The most striking Lipitor study was one from Scandinavia that showed that among middle-aged men over a 5-year-period, the test group who took Lipitor had a 30% lower overall death rate than the control group. Unlike the nuts study, this was an actual experiment.That seemed awfully convincing, but now it just seems too good to be true. A lot of those middle-aged deaths that didn't happen to the Lipitor takers didn't have much of anything to do with long-term blood chemistry, but were things like not driving your Saab into a fjord. How does Lipitor make you a safer driver?I sort of presumed at the time that if they had taken out the noisy random deaths, that would have made the Lipitor Effect even more noticeable. But, of course, that's naive. The good folks at Pfizer would have made sure that calculation was tried, so I'm guessing that it came out in the opposite direction of the one I had assumed. Guys who took Lipitor everyday for five years were also good about not driving into fjords and not playing golf during lighting storms and not getting shot by the rare jealous Nordic husband or whatever. Perhaps it was easier to stay in the control group than in the test group?Here’s how I would approach claims of massive reductions in overall deaths from consuming some food or medicine:Rank order the causes of death by how plausible it is that they are that they are linked to the food or medicine. For example:1. Diabetes2. Heart attacks3. Strokes4. Cancer5. Genetic diseases6. Car accidents7. Drug overdoses8. Homicides9. Lightning strikesIf this nuts-save-your-life finding is valid, then the greater effects should be found in causes of death near the top of the list (e.g., diabetes). But if it turns out that eating nuts only slightly reduces your chances of death from diabetes but makes you vastly less likely to be struck by lighting, then we’ve probably gotten a selection effect in which nut eaters are more careful people in general and thus don’t play golf during thunderstorms, or whatever.
Table 3 of the paper breaks out the hazard ratios by cause of death. The most impressive effects (as measured by the right tail of the 95% CI for pooled men and women for any nut)1 are Heart Disease, All Causes, Other Causes, Cancer, Respiratory Disease, Stroke, Infection, Diabetes, Neurodegenerative Disease, and Kidney Disease.
Steve's categories and the paper's categories don't overlap very well. But it looks to me like if you follow Steve's logic, it's reasonable to believe that nuts have a protective effect against heart disease, and then most of the other effects or non-effects have a common cause with nut consumption, like healthiness / conscientiousness / whatever, rather than being caused by nut consumption. Note the strong negative relationships between nut consumption and BMI or smoking, and the strong positive relationships between nut consumption and physical activity or intake of fruits, vegetables, or alcohol. The hazard ratios are calculated controlling for those variables, but it's still reasonable to see there being a hidden 'health-consciousness' node which noisily affects all of those nodes.
It's also interesting to look at the negative results- the hazard ratio for neurodegenerative disease and stroke was roughly 1, implying that nut-eaters and non-nut eaters had comparable risks, despite 'other causes' having a hazard ratio of 0.87. That weakly implies to me that either health consciousness has no impact on neurodegenerative disease and stroke, or that nuts are harmful for those two categories.
Since heart disease is a huge killer (24% of all deaths in the study group), this study seems like moderate evidence in favor of eating nuts, but it's likely that the total study's effect is overstated. (The study also suggests that tree nuts are probably superior to peanuts; I know various QS people have raised concerns that the kind of nut matters significantly.)
1. This is a heuristic for impressiveness, not the point estimate. It looks like nuts have the strongest effect for kidney disease, with a mean hazard ratio estimate of 0.69- but the upper bound of the 95% CI is 1.26, because only a handful of people died due to kidney disease. The heart disease hazard ratio estimate is 0.74 (0.68-0.81), which is much more believable, even though the point estimate is slightly higher. The point estimate for diabetes is 0.80 (0.54-1.18), which has a mean estimate that's only slightly worse, but diabetes again killed far fewer than heart disease. If you order them by point estimates, the paper is stronger evidence for nuts being useful for dietary reasons, and which method you prefer depends on your priors for how representative this sample is.