Warrigal comments on Decision theory: Why we need to reduce “could”, “would”, “should” - Less Wrong

19 Post author: AnnaSalamon 02 September 2009 09:23AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (46)
Tags: decision_theory

Comments (46)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 02 September 2009 07:02:21PM *  2 points [-]

To be exact, the agent must not know "which action maximizes utility" not "which action it will do". If it does whichever action it knows it'll do, it can have Lob-type short circuits where you say "I know you'll shoot yourself" and it says "OK thanks!" and shoots itself.

http://lesswrong.com/lw/t8/you_provably_cant_trust_yourself/

Comment author: [deleted] 03 September 2009 07:24:51AM 3 points [-]

Here, let me go back in time and become timtyler and retroactively write the following instead, thereby averting this whole discussion:

Calculating what would happen if an agent took some action is not counter-factual from the POV of the agent. The agent doesn't know what action it is going to take. If it did, it would just wait for itself to take the action - not spend time calculating the consequences of its various possible actions.

Powered by Reddit Powered by Reddit