You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Eliezer_Yudkowsky comments on Harry Potter and the Methods of Rationality discussion thread, part 24, chapter 95 - Less Wrong Discussion

6 Post author: palladias 18 July 2013 02:23AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (304)

You are viewing a single comment's thread. Show more comments above.

Comment author: HonoreDB 18 July 2013 12:57:06PM 12 points [-]

Quirrell seems to have been counterfactually mugged by hearing the prophecy of the end of the world...which would mean his decision theory, and psychological commitment to it, are very advanced.

Assume Quirrell believes that the only possible explanation of the prophecy he heard is that the apocalypse is nigh. This makes sense: prophecies don't occur for trivial events like a visitor to Hogwarts destroying books in the library named "Stars in Heaven" and "The World," and the idea of "the end of the world" being a eucatastrophe hasn't occurred to him. Assume Quirrell believes that prophecies are inevitable once spoken. Then why is Quirrell bothering to try to save the world?

Given that he hears the prophecy, Quirrell can either try T or not try ~T to avert it. Given that he tries, Quirrell is either capable C or incapable ~C of averting it. If T and C, by inevitability Quirrell will never hear the prophecy, which means that it is less likely the end of the world will occur (massive events always produce a prophecy that is heard by a wizard, so either Time finds some way to stop the end of the world or someone else hears it but fails to avert it). Say the end of the world causes -100 utility to Quirrell, and trying to stop it causes -1 utility. Then if C, a Quirrell that would try never hears the prophecy, so he never loses any utility, while a Quirrell that would not try hears the prophecy, goes out in a blaze of hedonism rather than fighting the inevitable, and loses 100 utility from the end of the world. Unfortunately, the actual world is the ~C world, where T brings -101 utility and ~T brings -100. So T looks like an irrational choice, but actually maximizes Quirrell's utility across counterfactuals.

This isn't the only explanation for Quirrell's actions; he could just prefer to go out fighting, or be betting on the slim chance that prophecies actually can be averted, or just trying to delay the end of the world as long as possible, or acting on other, weirder motives. But it's an interesting illustration of how alien a being that has truly internalized a really sophisticated decision theory might be.

Comment author: Eliezer_Yudkowsky 18 July 2013 07:19:18PM 7 points [-]

Upvoted for the word 'eucatastrophe'.