= 27d567bc2d48d9f40e9ccc20ea370b15)
Kindly comments on Beautiful Probability - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (109)
I think you're misunderstanding probability theory a little. Probability theory is the chance that x result means that the universe operates on principle y. The fact that the second researcher had no reason to stop until he tested 100 people means that the % of patients cured did not exceed 60% until his last batch of test subjects. Which significantly alters the chance that the universe operates on principle y. The first researcher could have had a % over 60 at any time during the experiment. Which is a physical difference. The probability used is not based on the researcher's private thoughts, it is based on their experimental procedure, which is different, regardless of the fact they ended with the same results.
Your skepticism is a good sign, but your mathematical intuition is lacking.
Suppose for concreteness that the second researcher did two batches of 50 experiments, and that we only consider two hypotheses: H1 (the treatment is 80% effective) and H2 (the treatment is 50% effective). Then you are quite correct in saying that Pr[n=100, r=70|H1] and Pr[n=100, r=70|H2] are different for the two scientists -- the second scientist needs, in addition, that the first batch of 50 had fewer than 30 successes. Algebraically, then, there seems to be no reason that the odds Pr[n=100, r=70|H1]/Pr[n=100, r=70|H2] would miraculously be the same for both scientists, right?
It turns out that the odds (which are all we care about) are, in fact, the same. This what we'd expect based on the general principle "the same data has only one interpretation".
But in case you're still worried, here's a way to help your intuitions get to the right thing. Fix some arbitrary sequence of patient outcomes with n=100 and r=70 which the second researcher could conceivably get. It's clear that the probability of following this sequence is the same for both researchers: this screens off the experimental procedure. So any particular sequence of patient outcomes gives the same odds, which means overall the odds are the same.
(There's also the sequences of outcomes that only the first researcher can get. But we don't worry about those because all n=100, r=70 outcomes are equally likely for the first researcher.)