G0W51 comments on Bayesian Adjustment Does Not Defeat Existential Risk Charity - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (89)
Bounded utility functions still seem to cause problems when uncertainty is involved. For example, consider the aforementioned utility function U(n) = 1 - (1 / (n^2)), and let n equal the number of agents living good lives. Using this function, the utility of a 1 in 1 chance of there being 10 agents living good lives equals 1 - (1 / (10^2)) = 0.99, and the utility of a 9 in 10 chance of 3^^^3 agents living good lives and a 1 in 10 chance of no agents living good lives roughly equals 0.1 * 0 + 0.9 * 1 = 0.9. Thus, in this situation the agent would be willing to kill (3^^^3) - 10 agents in order to prevent a 0.1 chance of everyone dying, which doesn't seem right at all. You could modify the utility function, but I think this issue would still to exist to some extent.