You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Stuart_Armstrong comments on Tools want to become agents - Less Wrong Discussion

12 Post author: Stuart_Armstrong 04 July 2014 10:12AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (81)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 04 July 2014 03:13:39PM 1 point [-]

I'm pointing out that "suggest a plan and stop" does not prevent the tool from suggesting a plan that turns itself into an agent.

Comment author: TheAncientGeek 04 July 2014 03:24:48PM 0 points [-]

My intention was that the X is stipulated by a human.

If you instruct a tool AI to make a million paperclips and stop, it won't turn itself into an agent with a stable goal of paper Clipping, because the agent will not stop.

Comment author: Stuart_Armstrong 04 July 2014 03:35:45PM 1 point [-]

Yes, if the reduced impact problem is solved, then a reduced impact AI will have a reduced impact. That's not all that helpful, though.

Comment author: TheAncientGeek 04 July 2014 04:42:20PM *  -1 points [-]

I don't see what needs solving. I f you ask Google maps the way to Tunbridge Wells, it doesn't give you the route to Timbuctu.