Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Vladimir_Nesov comments on The Fun Theory Sequence - Less Wrong

28 Post author: Eliezer_Yudkowsky 25 January 2009 11:18AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (23)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Vladimir_Nesov 26 January 2009 06:57:22PM 0 points [-]

@Roko: As I understood, one of the points you made was about how preferences of both individual people and humanity as a whole are quite coarse-grained, and so strong optimization of environment is pointless. Beyond certain precision, the choices become arbitrary, and so continuing systematic optimization, forcing choices to be non-arbitrary from the updated perspective, basically consists in incorporating noise into preferences.

I reply that a formula for pi can be written down in much fewer bytes that it'd take to calculate 10000th digit in its decimal expansion. Human embodying a description of morality, just like note containing the formula for pi, can have no capacity for imagining (computing) some deeper property of that description, and still precisely determine that property. What we need in both cases is a way around the limitations of media presenting the description, without compromising its content.