Wei_Dai comments on Fairness and Geometry - Less Wrong

9 Post author: cousin_it 22 July 2009 10:44AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (34)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 22 July 2009 09:58:04PM *  0 points [-]

Of course, if I invented a system that was resistant to manipulation, there'd be much rejoicing. But this goal seems still far away if at all possible.

It's not hard to fix a system so that it is resistant to manipulation: given a set of players, compute how the players would want to modify their utility functions under the original system, then use the original system on the modified utility functions to pick an outcome, then make that the solution under the new system. In the new system, players no longer has any incentive to modify their utility functions.

Of course if you do this, the new system might lose some desirable properties of the original system. But since you're assuming that the players are self-modifying AIs, that just means your system never really had those properties to begin with.

Comment author: cousin_it 23 July 2009 10:03:45AM *  0 points [-]

Interesting.

How a player would want to modify their utility function depends on how other players modify theirs. Schelling introduces "strategic moves" in games: selectively reducing your payoffs under some outcomes. (This is a formalization of unilateral commitments, e.g. burning your bridges.) A simultaneous-move game of strategic moves derived from any simple game quickly becomes a pretty tricky beast. I'll have to look at it closer; thanks for reminding.