Toggle comments on The germ of an idea - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (11)
It seems very difficult to prevent an AI from routing around this limitation by going meta. It can't actually account for its utility penalty, but can it predict the actions of a similar entity that could, and notice how helpful that would be? An AI that couldn't predict the behavior of such an entity might be crippled in some serious fundamental ways.