Reflective consistency, randomized decisions, and the dangers of unrealistic thought experiments
A recent post by Ape in the coat on Anthropical Paradoxes are Paradoxes of Probability Theory re-examines an old problem presented by Eliezer Yudkowsky in Outlawing Anthropics: An Updateless Dilemma. Both posts purport to show that standard probability and decision theory give an incorrect result in a fairly simple situation (which it turns out need not involve any "anthropic" issues). In particular, they point to a "reflective inconsistency" when using standard methods, in which agents who agree on an optimal strategy beforehand change their minds and do something different later on. Ape in the coat resolves this by abandoning standard probability theory in favour of a scheme in which a single person can simultaneously hold two different probabilities for the same event, to be used when making different decisions. Eliezer's resolution is to abandon standard decision theory in favour of one in which agents act "as if controlling all similar decision processes, including all copies of themselves". Here, I will defend something close to standard Bayesian probability and decision theory, with the only extension being the use of randomized decisions, which are standard in game theory, but are traditionally seen as unnecessary in Bayesian decision theory. I would also like to point out the danger of devising thought experiments that are completely unrealistic, in ways that are crucial to the analysis, as well as the inadvisability of deciding that standard methods are flawed as soon as you happen to come across a situation that is tricky enough that you make a mistake when analysing it. Here is the problem, in the formulation without irrelevant anthropic aspects: > Twenty people take part in an experiment in which one of two urns is randomly chosen, with equal probabilities, and then each of the 20 people randomly takes a ball from the chosen urn, and keeps it, without showing it to the others. One of the urns contains 18 green balls and 2 red balls. The other ur
I agree that individual control increases policy variance, which was sort of my point. Whether that's good or not seems to me to depend on what the default course of events is. If you think things are headed in a good direction, then low variance is good. But if the default course is likely to be disastrous, high variance at least provides a chance.
I don't understand your point about asymmetry. Doesn't that tend to make the default course bad?