You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

artemium comments on Anti-Pascaline satisficer - Less Wrong Discussion

3 Post author: Stuart_Armstrong 14 April 2015 06:49PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (7)

You are viewing a single comment's thread.

Comment author: artemium 15 April 2015 07:06:32AM *  0 points [-]

One possibility is to implement the design which will makes agent strongly sensitive to the negative utility when he invests more time and resources on unnecessary actions after he ,with high-enough probability , achieved its original goal.

In the paperclip example : wasting time an resources in order to build more paperclips or building more sensors/cameras for analyzing the result should create enough negative utility to the agent compared to alternative actions.

Comment author: Stuart_Armstrong 17 April 2015 03:13:39PM 0 points [-]

This has problems with the creation of subagents: http://lesswrong.com/lw/lur/detecting_agents_and_subagents/

You can use a few resources to create subagents without that restriction.