You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Eliezer_Yudkowsky comments on Even with default points, systems remain exploitable - Less Wrong Discussion

9 Post author: Stuart_Armstrong 19 July 2013 04:24PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (9)

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 20 July 2013 03:27:42PM 6 points [-]

The proof is nearly derisively brief: the player simply needs to reduce their declared utility on every other admissible pure outcome, until these are no longer admissible. Then O would be the only outcome available to the bargaining solution.

This is a most excellent proof (it makes everything obvious) and I am convinced by this that agents should not insist that their outcomes above all lie on a Pareto frontier, because some of the points on a Pareto frontier represent unfair splits of gains from trade a la Ultimatum. An auction system between multiple agents might resolve this, but meanwhile one must be willing to refuse certain proposed 'Pareto improvements' in order to possess any negotiating power about which Pareto improvement to make, and it may also be that by the nature of utility functions it is not possible to distinguish aliens that have cleverly rejiggered their utility functions from aliens which were born with them.