timtyler comments on Satisficers want to become maximisers - Less Wrong

21 Post author: Stuart_Armstrong 21 October 2011 04:27PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (67)

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 22 October 2011 07:25:14PM 0 points [-]

It is motivated to take an action that is a satisficing action, and 1) and 2) are equally satisficing.

Surey option 1 has a 10% chance of failing to satisfy.

Comment author: Stuart_Armstrong 23 October 2011 12:01:57AM 2 points [-]

Option 1) already satisfies. Taking option 1) brings the expected utility up above the threshold, so the satisficer is done.

If you add the extra requirement that the AI must never let the expected utility fall below the threshold in future, then the AI will simply blind itself or turn itself off, once the satisficing level is reached; then its expected utility will never fall, as no extra information ever arrives.

Comment author: timtyler 23 October 2011 02:10:43AM 2 points [-]

Sorry - a failure to reread the question on my part :-(