Lightwave comments on Two questions about CEV that worry me - Less Wrong

29 Post author: cousin_it 23 December 2010 03:58PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (137)

Comments (137)

You are viewing a single comment's thread. Show more comments above.

Comment author: Lightwave 23 December 2010 10:49:31PM *  1 point [-]

They may include values or desires we would consciously prefer to eradicate entirely, such as a drive for fighting or for making war. If you thought that 1) most people in history enjoyed and desired war, and 2) this was due to a feature of their builtin cognitive architecture that said "when in situation X, conquer your neighbors" - would you want to include this in the CEV?

CEV is supposed to incorporate not only the things you want (or enjoy), but also the things you want to want (or don't want to enjoy, in this case).

Comment author: DanArmak 23 December 2010 10:59:50PM 7 points [-]

Supposed to based on what evidence?

As Vladimir Nesov said, there are builtin preferences (which CEV takes into account), and then there are our conscious desires or "state of knowledge about preference". The two may be in conflict in some cases.

How do you know that CEV won't include something that all the humans alive today, on the conscious level, would find hateful?

Powered by Reddit Powered by Reddit