Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

gjm comments on Value Uncertainty and the Singleton Scenario - Less Wrong

8 Post author: Wei_Dai 24 January 2010 05:03AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (28)

You are viewing a single comment's thread.

Comment author: gjm 24 January 2010 11:26:02AM 2 points [-]

The difference between the Hanson and Bostrom-Ord proposals is very like the difference between the mean and the median. The former lets you take more notice of probably-wrong data, at the cost of being more sensitive to disruption by distant/low-probability outliers.

I'm inclined to think that the presence of uncertainty about one's values (still more about the shape of one's overall ethical system) the more cautious/robust/insensitive approach is perferable. But I'm by no means certain. Well, let's say I'm 80% sure of the former and 20% of the latter, and do an expected-utility calculation or a vote ... :-)

Seriously, I think we're a long way from having a good handle on how to deal with the limitations, unreliability, and nontransparency of our own hardware, and the tendency for attempts to do so to end up in this sort of "but what if we're uncertain about the uncertainty in our own uncertainty" recursion is a symptom of this. I suspect that the specific open question Wei_Dai has identified here is best viewed as a special case of that higher-level problem.