= 27d567bc2d48d9f40e9ccc20ea370b15)
Eliezer_Yudkowsky comments on Beautiful Probability - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (109)
If there is a difference, it is not because the experiments went differently, it is because the experiments could have gone differently, and so the likelihoods of them happening the way they did happen is different.
The Monty Hall problem was mentioned above. I pick a door, Monty opens a door to reveal a goat, I can stick or switch (but can't take the goat). Whether Monty is picking a random door or picking the door he knows doesn't have the goat, the evidence is the same - Monty opened a door and revealed a goat. But if Monty what matters is what might have happened otherwise. If Monty always picks a door with a goat, then I win if I switch 2/3 of the time. If Monty might have picked the door with the car (and just happened not to), I win if I switch only 50% of the time.
Same evidence, different conclusions based solely on what someone might have done otherwise not based on what actually happened; and I am confident of the difference in the Monty Hall problem, as I have not only read about it but also simulated it.
In the situation given, Researcher 1 did stop at 100 experiments, but might have stopped at 49, or 280. Researcher 2 was sure to stop at 100. I am not unwilling to accept that this doesn't change the meaning of the evidence, in this case, but I do not understand at all why it should be "obvious" that it can't, given that it does in the case of the Monty Hall problem.
The difference is that depending on Monty's algorithm, there is a different probability of getting the exact result we saw, namely seeing a goat. The exact event we actually saw happens with different probability depending on Monty's rule, so Monty's rule changes the meaning of that result.
The researchers don't get a given exact sequence of 100 results with different probability depending on their state of mind - their state of mind is not part of the state of the world that the result sequence tells us about, the way Monty's state of mind is part of the world that generates the exact goat.
To look at it another way, a spy watching Monty open doors and get goats would determine that Monty was deliberately avoiding the prize. Watching a researcher stop at 100 results doesn't tell you anything about whether the researcher planned to stop at 100 or after getting a certain number of successes. So, just like that result doesn't tell you anything about the researcher's state of mind, knowing about the researcher's state of mind doesn't tell you anything about the result.
That makes sense. Thank you.
Suppose that the frequency of cures actually converges at 60%, and each researcher performs his experiment 100 times. Researcher A should end up with about 6000 cures out of 10000, and Researcher B should end up with .7n cures out of n. We would expect in advance, after being told that Researcher B ended up testing 10000 people, that he encountered 7000 cures.
It seems that once we know that Researcher B was not going to stop until the incidence of cures was 70%, we do not learn anything further about the efficacy of the treatment by looking at his data.
What's wrong about this?
You are proposing to partition the data into two observations, namely "number of trials that were performed" and "the results of those trials".
The information you have after observing the first part and not the second, does depend on the researcher's stopping criterion. And the amount of information you gain from the second observation also so depends (since some of it was already known from the first observation in one case and not in the other). But your state of information after both observations does not depend on the stopping criterion.
Also, the most likely outcome of your proposed experiment is not .7n cures for some n. Rather, it's that Researcher B never stops.
That explanation sounds plausible. Can you say it again in math?