Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

timtyler comments on Value is Fragile - Less Wrong

38 Post author: Eliezer_Yudkowsky 29 January 2009 08:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (82)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 21 February 2013 01:11:45AM *  0 points [-]

If that's the case I don't think we disagree about anything substantial. We probably just disagree about what percentage of resources should go to UIV and what should go to terminal values.

It's a tricky and controversial issue. The cost of preserving our values looks fairly small - but any such expense diverts resources away from the task of surviving - and increases the risk of eternal oblivion. Those who are wedded to the idea of preserving their values will need to do some careful accounting on this issue, if they want the world to run such risks.

While the phrase "universal instrumental values" has the word "instrumental" in it, that's just one way of thinking about them. You could also call them "nature's values" or "god's values". You can contrast them with human values - but it isn't really an "instrumental vs terminal" issue.