You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Toggle comments on The germ of an idea - Less Wrong Discussion

6 Post author: Stuart_Armstrong 13 November 2014 06:58PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (11)

You are viewing a single comment's thread.

Comment author: Toggle 14 November 2014 02:46:30AM *  0 points [-]

It seems very difficult to prevent an AI from routing around this limitation by going meta. It can't actually account for its utility penalty, but can it predict the actions of a similar entity that could, and notice how helpful that would be? An AI that couldn't predict the behavior of such an entity might be crippled in some serious fundamental ways.