Unknowns comments on AI cooperation in practice - Less Wrong

26 Post author: cousin_it 30 July 2010 04:21PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

You are viewing a single comment's thread. Show more comments above.

Comment author: Unknowns 30 July 2010 07:39:24PM 0 points [-]

However, B will as well, and you do want to cooperate with everybody who cooperates with you, so my proposed change works very well.

Comment author: JGWeissman 30 July 2010 07:43:21PM 2 points [-]

you do want to cooperate with everybody who cooperates with you

Not if they will cooperate regardless of what you do. Then you can win more by defecting.

Comment author: Unknowns 30 July 2010 07:44:35PM 0 points [-]

A knows B's source code, and so it knows that B will only output a 1 if A and B output the same thing, i.e. only if A and B both output 1.

Comment author: cousin_it 30 July 2010 08:29:26PM * 3 points [-]

No, A doesn't know that because it doesn't know B's proof checker is correct.

You really have to be triple extra careful when talking about this stuff. I'm trying to.

Comment author: Unknowns 30 July 2010 08:32:57PM 0 points [-]

Yes, I see that.

Comments (157)