AlexMennen comments on Harsanyi's Social Aggregation Theorem and what it means for CEV - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (86)
The situation analogous to Simpson's paradox can only occur if for some reason we care about some people's opinion more than others in some situations (this is analogous to the situation in Simpson's paradox where we have more data points in some parts of the table than others. It is a necessary condition for the paradox to occur.)
For example: Suppose Alice (female) values a cure for prostate cancer at 10 utils, and a cure for breast cancer at 15 utils. Bob (male) values a cure for prostate cancer at 100 utils, and a cure for breast cancer at 150 utils. Suppose that because prostate cancer largely affects men and breast cancer largely affects women we value Alice's opinion twice as much about breast cancer and Bob's opinion twice as much about prostate cancer. Then in the aggregate curing prostate cancer is 210 utils and curing breast cancer 180 utils, a preference reversal compared to either of Alice or Bob.
This is essentially just an example of Harsanyi's Theorem in action. And I think it makes a compelling demonstration of why you should not program an AI in that fashion.