Harry Potter and the Methods of Rationality discussion thread, part 5

NihilCredo

Harry Potter and the Methods of Rationality discussion thread, part 5 — LessWrong

Comment Permalink

The second formulation is simpler, but then leads to absurdities such as counterfactual mugging. This is a failure of the theory.

If you don't think so, try a counterfactual mugging on everyday people, and then try it at a LessWrong meeting. Which group do you think will be more likely to come out ahead, in this practical example?

As it says on the wiki:

If some particular ritual of cognition—even one that you have long cherished as "rational"—systematically gives poorer results relative to some alternative, it is not rational to cling to it.

WrongBot15y00

The counterfactual mugging requires that the deal be offered by an entity that is known to be both perfectly honest and a perfect predictor. If Omega tries to counterfactually mug you, you should pay him. If I try to counterfactually mug you, paying up would be significantly less wise.

A sufficiently good decision theory should get both of those cases right.

12Scott Alexander15y

The Less Wrong meeting, of course. I'm no Omega, but I'm smart enough to predict none of the regular people will take the deal, and most of the Less Wrongers will. That means I won't give any money to any everyday people, but after the coin flip I'll be handing out a whole bunch of suitcases with $10000 to the Less Wrongers (while also collecting a few hundred dollar bills). The average person in the Less Wrong meeting will come out $4950 richer than the person on the street. If you mean I should do the second part, the part where I take the money, but not the first part, then it's no longer a counterfactual mugging. Then it's just me lying to people in a particularly weird way. The Less Wrongers might do worse on the completely unrelated problem of whether they believe weird lies, but I don't see much evidence for this.

5ata15y

You can't "try a counterfactual mugging" unless you are Omega (or some other entity with a lot of money to throw away and some unusually and systematically accurate way of predicting people's behaviour under counterfactual interventions). And if you are... then those who are inclined to pay in counterfactual mugging will win more from it on average. That's the whole point. If you accept the premises of the problem (Onega is honest and flipped a fair coin, etc.), paying really is the winningest thing to do.

See in context