Eliezer_Yudkowsky comments on Even with default points, systems remain exploitable - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (9)
This is a most excellent proof (it makes everything obvious) and I am convinced by this that agents should not insist that their outcomes above all lie on a Pareto frontier, because some of the points on a Pareto frontier represent unfair splits of gains from trade a la Ultimatum. An auction system between multiple agents might resolve this, but meanwhile one must be willing to refuse certain proposed 'Pareto improvements' in order to possess any negotiating power about which Pareto improvement to make, and it may also be that by the nature of utility functions it is not possible to distinguish aliens that have cleverly rejiggered their utility functions from aliens which were born with them.