Strange7 comments on What if AI doesn't quite go FOOM? - Less Wrong

11 Post author: Mass_Driver 20 June 2010 12:03AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (186)

You are viewing a single comment's thread.

Comment author: Strange7 21 June 2010 05:23:44AM 0 points [-]

An AI that "valued" keeping the world looking roughly the way it does now, that was specifically instructed never to seize control of more than X number of each of several thousand different kinds of resources,

Define 'seize control.' Wouldn't such an AI be motivated to understate it's effective resources, or create other nominally-independent AIs with identical objectives and less restraint, or otherwise circumvent that factor?