= 27d567bc2d48d9f40e9ccc20ea370b15)
Christopher_Russo comments on A Difficulty in the Concept of CEV - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (23)
Supposing we can do that, there's good and bad news.
Good news: the resulting utility function is transitive so the AI will function without being susceptible to Dutch book arguments. (The AI won't accept a series of bets that it is assured to lose because of cyclical preferences.)
Bad news: the aggregation procedure (adding utility functions, ignoring scaling) fails at least one of Arrow's fairness criteria. [Edit: As Qiaochu_Yuan points out, I may have misunderstood the adding procedure defined in Kalai's paper vs the one you proposed. Under the adding procedure defined in the paper, the criteria failed is independence of irrelevant alternatives.]
You care about this because human values are fragile, and an imperfect aggregation procedure for CEV will shatter those values. Unless overcome, Arrow's impossibility theorem ensures that any such procedure will be imperfect.
The adding procedure Eliezer describes isn't even covered by the setup of the paper you linked to. Eliezer is assuming that people have actual utility functions, whereas Kalai and Schmeidler implicitly assume that only equivalence classes of utility functions up to translation and scaling are meaningful. (The adding procedure that is well-defined in Kalai and Schmeidler's setup doesn't fail dictatorship, it fails independence of irrelevant alternatives as I pointed out in my comment.)
This is another reason not to take theorems too seriously, which is that they often have implicit assumptions (in the setup of the problem, etc.) that are easy to miss if you only look at the statement of the theorem.
Ah, oops! Please excuse my mistake.