DanielLC comments on Defining a limited satisficer - Less Wrong

6 Post author: Stuart_Armstrong 11 March 2015 02:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (11)

You are viewing a single comment's thread.

Comment author: DanielLC 12 March 2015 07:08:38AM 1 point [-]

Would not effectively resist M(-u), a u-minimizer.

I'm not sure how that's supposed to work. S(u) won't do much as long as the desirability threshold is obtained, but if M(-u) comes along and makes this difficult, S(u) would use everything it has to stop M(-u). Are you using something beyond desirability threshold? Something where S(u) stops not when the solution is good enough, but when it gets difficult to improve?

Comment author: Stuart_Armstrong 12 March 2015 11:08:03AM 1 point [-]

See my edit above. "would use everything it has to..." is the kind of behaviour we want to avoid. So I'm more following the sastisficing intuition than the formal definition. I can justify this by going meta: when people design/imagine satisficers, they generally look around at the problem, see what can be achieved, how hard it is, etc... and then set the threshold. I want to automate "set a reasonable threshold" as well as "be a reasonable satisficer" in order to achieve "don't have a huge impact on the world".