shokwave comments on David Chalmers' "The Singularity: A Philosophical Analysis" - Less Wrong

33 Post author: lukeprog 29 January 2011 02:52AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (202)

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 29 January 2011 04:20:04PM *  4 points [-]

I've linkified the grandparent a bit - for those not familiar with the ideas.

The main idea is that many agents which are serious about attaining their long term goals will first take control of large quantities of spactime and resources - before they do very much else - to avoid low-utility fates like getting eaten by aliens.

Such goals represent something like an attractor in ethics-space. You could avoid the behaviour associated with the attractor by using discounting, or by adding constraints - at the expense of making the long-term goal less likely to be attained.

Comment author: shokwave 29 January 2011 06:15:50PM 2 points [-]

These sound instrumental; you take control of the universe in order to achieve your terminal goals. That seems slightly different from what Newsome was talking about, which was more a converging of terminal goals on one superterminal goal.

Comment author: timtyler 29 January 2011 06:20:54PM *  1 point [-]

Thus one the proposed titles: "Universal Instrumental Values".

Newsome didn't distinguish between instrumental and terminal values.