Wei_Dai comments on AI cooperation in practice - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (157)
Edit: The following is not obviously possible, see this comment.
This can be generalized to provide even better algorithms for more general games. Let the following be a simplified outline of the algorithm for cooperation problem, with bounds omitted (players are A and B, our player is A):
The proof that the outcome is "Cooperate" basically hinges on the provability of Löb statement
from which we conclude that A()==B() and so A()=="Cooperate". The statement can be constructed from
and so
Now, we can get basically the same proof if we use a simpler algorithm for the players:
This gets rid of the more general relation, but achieves basically the same result. We can improve on this by defecting against cooperating rocks:
This suggests a general scheme for construction of decision-making algorithms: for all possible actions a and b of players A and B, sort the pairs <a,b> by utility for A, starting from highest, and implement the Löb algorithm for each possibility in turn:
This plays against arbitrary opponents quite well.
Suppose we have two possible actions X,Y and A prefers <X,Y> to <Y,X> to <X,X> to <Y,Y> and B prefers <Y,X> to <X,Y> to <X,X> to <Y,Y>. What will be the outcome if both of them use main4() with their respective orderings and the same proves() function?
Probably <X,X>, which is not quite optimal. This doesn't solve the bargaining problem, which is exactly what your example is.