You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

wedrifid comments on In favour of a selective CEV initial dynamic - Less Wrong Discussion

12 [deleted] 21 October 2011 05:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (110)

You are viewing a single comment's thread.

Comment author: wedrifid 21 October 2011 07:51:59PM *  6 points [-]

Furthermore, desirability of the CEV ouput to the average human in existence today should be weighed against the desires of (for example) sentient human uploads created in a post-singularity scenario.

No, it really shouldn't. CEV is inclusive. That is, if we care about post human sentient uploads then CEV<us> accounts for that better than we can. That's the whole point. (This holds unless 'desirability' is defined to mean some other arbitrary thing independent of volition of the type we are talking about.)

Comment author: [deleted] 21 October 2011 08:43:31PM *  4 points [-]

That depends who "we" is referring to. What I meant to say is that if the FAI programmers and other intellectuals believe that extrapolated humans in general might not care about harming uploads (future humans with no voice in that CEV) - whereas a selective CEV of intellectuals is expected to do so - then they should consider this when deciding whether to allow all of humanity to have a say in the CEV, rather than a subset of minds whom they consider to be safer in that regard.

So even if the implementation of a universal CEV as initial dynamic can be expected to reflect the desires of humanity en masse better than a selective initial CEV, this doesn't define the total moral space that should be of concern to those responsible for and having the power to influence that implementation.