You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

lessdazed comments on Guardian Angels: Discrete Extrapolated Volitions - Less Wrong Discussion

0 Post author: lessdazed 25 September 2011 10:51PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (9)

You are viewing a single comment's thread. Show more comments above.

Comment author: lessdazed 26 September 2011 01:16:09PM 2 points [-]

I don't think undetectability solves things. Offensive subsystems could survive their creator's demise like two people in a grenade lobbing fight.

Suppose all hid a copy, the merged AI would still be more powerful than any hidden copies, and if it was destroyed everyone would be a small copy again. If there were many AIs, an individual would be banking on its ability to defeat a much larger entity. Offense is more powerful on most scales and technological levels but not by incomprehensible orders of magnitude.