= 27d567bc2d48d9f40e9ccc20ea370b15)
thomblake comments on Beautiful Probability - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (109)
If there is a difference, it is not because the experiments went differently, it is because the experiments could have gone differently, and so the likelihoods of them happening the way they did happen is different.
The Monty Hall problem was mentioned above. I pick a door, Monty opens a door to reveal a goat, I can stick or switch (but can't take the goat). Whether Monty is picking a random door or picking the door he knows doesn't have the goat, the evidence is the same - Monty opened a door and revealed a goat. But if Monty what matters is what might have happened otherwise. If Monty always picks a door with a goat, then I win if I switch 2/3 of the time. If Monty might have picked the door with the car (and just happened not to), I win if I switch only 50% of the time.
Same evidence, different conclusions based solely on what someone might have done otherwise not based on what actually happened; and I am confident of the difference in the Monty Hall problem, as I have not only read about it but also simulated it.
In the situation given, Researcher 1 did stop at 100 experiments, but might have stopped at 49, or 280. Researcher 2 was sure to stop at 100. I am not unwilling to accept that this doesn't change the meaning of the evidence, in this case, but I do not understand at all why it should be "obvious" that it can't, given that it does in the case of the Monty Hall problem.
Did you read the chapter linked at the end of the post?
A hopefully intuitive explanation: A spy watching the experiments and using Bayesian methods to make his own conclusions about the results, will not see any different evidence in each case and so will end up with the same probability estimate regardless of which experimenter he watched.
While the second experimenter might be contributing to publication bias by using that method in general, he nonetheless should not have come up with a different result.
It seems worth noting the tension between this and bottom-line reasoning. Could the second experimenter have come up with the desired result no matter what, given infinite time? And if so, is there any further entanglement between his hypothesis and reality?
Why would a spy watching Monty Hall be different?
Amongst other reasons, Monty isn't the experimenter. I'm really not sure in precisely what way Monty Hall is analogous to these experiments.
Monty Hall is analogous in that we are looking at evidence and trying to make conclusions about likelihoods. It is relevant because the likelihoods are different depending on what was in Monty's head in the past, after observing the same physical evidence. Monty is not the experimenter; where does that make a difference? Could one reformulate it so that he was? He would be running two different experiments, surely - but then why isn't that the case for the two researchers?