You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

steven0461 comments on BOOK DRAFT: 'Ethics and Superintelligence' (part 1) - Less Wrong Discussion

11 Post author: lukeprog 13 February 2011 10:09AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (107)
Tags: singularity book meta-ethics

Comments (107)

You are viewing a single comment's thread. Show more comments above.

Comment author: steven0461 14 February 2011 07:44:23PM 2 points [-]

Because many humans, to put it extremely politely, have preferences that are distinctly different to what I would call 'right'.

Before or after extrapolation? If the former then why does that matter, if the latter then how do you know?

Comment author: wedrifid 15 February 2011 02:22:09AM *  3 points [-]

Before or after extrapolation? If the former then why does that matter, if the latter then how do you know?

Former in as much as it allows inferences about the latter. I don't need to know with any particular confidence for the purposes of the point. The point was to illustrate possible (and overwhelmingly obvious) failure modes.

Hoping that CEV<humanity> is desirable rather than outright unfriendly isn't a particularly good reason to consider it. It is going to result in outcomes that are worse from the perspective of whoever is running the GAI than CEV<that person> and CEV<group more closely identified with>.

The purpose of doing CEV<humanity> at all is for signalling and cooperation (or, possibly, outright confusion).

Powered by Reddit Powered by Reddit