lukeprog comments on Causal Diagrams and Causal Models - Less Wrong

61 Post author: Eliezer_Yudkowsky 12 October 2012 09:49PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (274)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 26 March 2013 10:52:20PM 3 points [-]

Thanks again.

One more question, since this is your field. Do you happen to know of an instance where some new causal effect was discovered from observational data via causal modeling, and this cause was later confirmed by an RCT?

Comment author: IlyaShpitser 26 March 2013 11:26:33PM *  1 point [-]

Well, I think smoking/cancer was first established in case control studies. In general people move up the "hierarchy of evidence" Kawoomba mentioned. At the end of the day, people only trust RCTs (and they are right, other methods rely on more assumptions). There is another good example, but let me double check before posting.


With case control studies you have the additional problem of selection bias, on top of confounding.

Comment author: gwern 27 March 2013 12:51:44AM 1 point [-]

I thought there were still no actual RCTs of smoking in humans.

Comment author: IlyaShpitser 27 March 2013 12:55:27AM *  0 points [-]

Right, you can't always RCT in humans. But a causal mechanism + RCTs in animals biologically close to humans is convincing for something like lung cancer where minor differences among mammals shouldn't matter much (although e.g. bears have evolved some crazy stuff to deal with all that fat they eat before hibernating).

Comment author: gwern 27 March 2013 01:58:55AM *  0 points [-]

where minor differences among mammals shouldn't matter much

I think you are entirely optimistic. I recently pointed out that the research indicates that animal studies routinely (probably usually) do not transfer, and as it happens, animal smoking studies are an example of this, according to Hanson. So the differences are often far from minor, and even if there were cancer in the animal studies, we could infer very little from it.

Comment author: IlyaShpitser 27 March 2013 02:52:15AM 0 points [-]

Out of curiosity, do you smoke?

Comment author: gwern 27 March 2013 02:56:45AM 0 points [-]

No.

Comment author: IlyaShpitser 27 March 2013 06:42:22PM -1 points [-]

I find much to agree with in Hanson's writings, but in this case I just don't find him convincing. One issue is that cancer is a scourge of a long-living animal. One hypothesis is that smoking causes long term cumulative damage, and you might not see effects in mice or dogs because they die too soon regardless. There is also the issue that we have a fair idea of the carcinogenic mechanism now, so if you think smoking does not cause harm, there also needs to be a story how that mechanism is foiled in humans.

Comment author: Vaniver 27 March 2013 08:07:52PM 1 point [-]

I find much to agree with in Hanson's writings, but in this case I just don't find him convincing.

His interpretation, or his evidence? I point this out because it looks to me like your position has shifted from "the smoking / lung cancer link is established by RCTs in animals" to "even though RCTs don't establish the smoking / lung cancer link for animals, we have other reasons to believe in the smoking / lung cancer link for humans."

Comment author: gwern 27 March 2013 08:05:51PM *  -1 points [-]

I find much to agree with in Hanson's writings, but in this case I just don't find him convincing...One hypothesis is that smoking causes long term cumulative damage, and you might not see effects in mice or dogs because they die too soon regardless.

So: heads I win, tails you lose? If the studies had found smoking caused cancer in animals, well, that proves it! And if they don't, well, that just means they didn't run long enough so we can ignore them and say we "just don't find them convincing"...

There is also the issue that we have a fair idea of the carcinogenic mechanism now, so if you think smoking does not cause harm, there also needs to be a story how that mechanism is foiled in humans.

You don't think there were plenty of 'fair ideas' of mechanisms floating around in the thousands of animal studies and interventions covered in my animal studies link? Any researcher worth his degree can come up with a plausible ex post explanation.