Eliezer_Yudkowsky comments on Harry Potter and the Methods of Rationality discussion thread, part 24, chapter 95 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (304)
Quirrell seems to have been counterfactually mugged by hearing the prophecy of the end of the world...which would mean his decision theory, and psychological commitment to it, are very advanced.
Assume Quirrell believes that the only possible explanation of the prophecy he heard is that the apocalypse is nigh. This makes sense: prophecies don't occur for trivial events like a visitor to Hogwarts destroying books in the library named "Stars in Heaven" and "The World," and the idea of "the end of the world" being a eucatastrophe hasn't occurred to him. Assume Quirrell believes that prophecies are inevitable once spoken. Then why is Quirrell bothering to try to save the world?
Given that he hears the prophecy, Quirrell can either try T or not try ~T to avert it. Given that he tries, Quirrell is either capable C or incapable ~C of averting it. If T and C, by inevitability Quirrell will never hear the prophecy, which means that it is less likely the end of the world will occur (massive events always produce a prophecy that is heard by a wizard, so either Time finds some way to stop the end of the world or someone else hears it but fails to avert it). Say the end of the world causes -100 utility to Quirrell, and trying to stop it causes -1 utility. Then if C, a Quirrell that would try never hears the prophecy, so he never loses any utility, while a Quirrell that would not try hears the prophecy, goes out in a blaze of hedonism rather than fighting the inevitable, and loses 100 utility from the end of the world. Unfortunately, the actual world is the ~C world, where T brings -101 utility and ~T brings -100. So T looks like an irrational choice, but actually maximizes Quirrell's utility across counterfactuals.
This isn't the only explanation for Quirrell's actions; he could just prefer to go out fighting, or be betting on the slim chance that prophecies actually can be averted, or just trying to delay the end of the world as long as possible, or acting on other, weirder motives. But it's an interesting illustration of how alien a being that has truly internalized a really sophisticated decision theory might be.
Upvoted for the word 'eucatastrophe'.