timtyler comments on Value is Fragile - Less Wrong

41 Post author: Eliezer_Yudkowsky 29 January 2009 08:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (88)
Tags: ai fai complexity_of_value fragility_of_value

Comments (88)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Ghatanathoah 21 February 2013 01:05:32AM -1 points [-]

It's likely to not be a binary decision. We may well be able to trade preserving values against a better chance of surviving at all......

....How to deal with this tradeoff is a controversial issue. However, it certainly isn't obvious that we should struggle to preserve our human values - and resist adopting universal instrumental values. That runs a fairly clear risk of screwing up the future for all our descendants.

If that's the case I don't think we disagree about anything substantial. We probably just disagree about what percentage of resources should go to UIV and what should go to terminal values.

Since I see our values heavily overlapping with universal instrumental values, adopting them doesn't seem too bad to me

You might be right to some extent. Human beings tend to place great terminal value on big, impressive achievements, and quickly colonizing the universe would certainly involve doing that.

Comment author: timtyler 21 February 2013 01:11:45AM *  0 points [-]

If that's the case I don't think we disagree about anything substantial. We probably just disagree about what percentage of resources should go to UIV and what should go to terminal values.

It's a tricky and controversial issue. The cost of preserving our values looks fairly small - but any such expense diverts resources away from the task of surviving - and increases the risk of eternal oblivion. Those who are wedded to the idea of preserving their values will need to do some careful accounting on this issue, if they want the world to run such risks.

While the phrase "universal instrumental values" has the word "instrumental" in it, that's just one way of thinking about them. You could also call them "nature's values" or "god's values". You can contrast them with human values - but it isn't really an "instrumental vs terminal" issue.

Powered by Reddit Powered by Reddit