You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Lumifer comments on CEV: coherence versus extrapolation - Less Wrong Discussion

14 Post author: Stuart_Armstrong 22 September 2014 11:24AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (17)

You are viewing a single comment's thread.

Comment author: Lumifer 22 September 2014 04:49:12PM 12 points [-]

To summarise: the human mindspace is much narrower is near mode than in far mode.

The conclusion I draw from your examples is that the human mindspace is much narrower when constrained by reality then when allowed to engage in flights of fantasy.

I agree that that there is tension between "coherent" and "extrapolated", however it looks to me secondary to more basic problems. The "extrapolated" part essentially assumes that all humans have the same value system at the core, and the "coherent" part implies that human preferences are coherent when, empirically speaking, they are not.

Comment author: Stuart_Armstrong 22 September 2014 05:02:55PM 4 points [-]

when constrained by reality

Yes. But it's mainly constrained by social reality, which gives you convergence for cheap.

Comment author: Lumifer 22 September 2014 05:22:16PM 3 points [-]

The social reality gives convergence of behavior, not necessarily of values. Or, rather, the convergence of behavior is immediate and easily enforced; the convergence of values is a much slower and less controlled process.