Emile comments on Open Thread, July 1-15, 2012 - Less Wrong

2 Post author: OpenThreadGuy 01 July 2012 10:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (150)
Tags: open thread open_thread

Comments (150)

You are viewing a single comment's thread. Show more comments above.

Comment author: Emile 09 July 2012 03:25:24PM 0 points [-]

I don't understand why "Yes" is the right answer. It seems to me that an agent that self-modified to answer "Yes" to this sort of question in the future but said "No" this time would generate more utility than an agent that already implemented the policy of saying yes.

If that first agent (that answers no, then self-modifies to answer yes) had been in the situation where the coin had fell heads, then it would not have got the million dollars; whereas an agent that can "retroactively precommit" to answer yes would have got the million dollars. So having a "retroactively precommit" algorithm seems like a better choice than having a "answer what gets the biggest reward, and then self-modify for future cases" algorithm.

Comment author: Incorrect 09 July 2012 10:36:40PM 0 points [-]

If that first agent (that answers no, then self-modifies to answer yes) had been in the situation where the coin had fell heads, then it would not have got the million dollars; whereas an agent that can "retroactively precommit" to answer yes would have got the million dollars.

But we know that didn't happen. Why do we care about utility we know we can't obtain?

So having a "retroactively precommit" algorithm seems like a better choice than having a "answer what gets the biggest reward, and then self-modify for future cases" algorithm.

For what goal is this a better choice? Utility generation?

Powered by Reddit Powered by Reddit