RomeoStevens comments on Superintelligence 24: Morality models and "do what I mean" - Less Wrong

7 Post author: KatjaGrace 24 February 2015 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (47)

You are viewing a single comment's thread. Show more comments above.

Comment author: RomeoStevens 24 February 2015 03:20:47AM 2 points [-]

MR seems correct if we have evidence that points to CEV containing unacceptable amounts of unavoidable suffering because we are just unlucky and our true values suck.

Limited CEV seems correct when we have evidence that some class of agents don't have moral worth and shouldn't have their preferences taken into account. Most conceptions of CEV have some sort of limitation that isn't obviously a correct carving of reality at the joints. Choosing between "all humans CEV" and "all humans + extrapolation of uplifted minds" and "CEV of various future minds that we would value" is non-obvious for example.

Comment author: skeptical_lurker 25 February 2015 11:32:41AM 0 points [-]

Choosing between "all humans CEV" and "all humans + extrapolation of uplifted minds" and "CEV of various future minds that we would value" is non-obvious for example.

Take an average.

Comment author: RomeoStevens 25 February 2015 04:51:22PM *  1 point [-]

Weighting the uncertainty and value of each CEV is non-obvious, so you've pushed the uncertainty up one level.

Comment author: skeptical_lurker 26 February 2015 08:54:38AM -1 points [-]

True, but it seems easier to compromise on.