atucker comments on The Friendly AI Game - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (170)
Assuming that the utility function is written in a way that makes loss of utility possible (utility = dollars in bank or something), this is a failure mode:
AI stops short of the limit, makes another AI that prevents loss of utility, hits the bound, and then shuts down.
Second AI takes over the universe as a precaution against any future disutility.