jimrandomh comments on The Friendly AI Game - Less Wrong

38 Post author: bentarm 15 March 2011 04:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (170)

You are viewing a single comment's thread. Show more comments above.

Comment author: jimrandomh 16 March 2011 04:27:09PM 3 points [-]

One such set of Program A and Program B is 'attached' to every individual in the world. Being of equal intelligence, in THEORY they should compromise in order to avoid wasting resources.

Ecosystems of many cooperating agents only work so long as either they all have similar goals, or there is a suitable balance between offense and defense. This particular example fails if there is any one person in the world who wants to destroy it, because their AI can achieve this goal without having to compromise or communicate with any of the others.