= 27d567bc2d48d9f40e9ccc20ea370b15)
CronoDAS comments on Holden's Objection 1: Friendliness is dangerous - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (428)
"A point I may not have made in these posts, but made in comments, is that the majority of humans today think that women should not have full rights, homosexuals should be killed or at least severely persecuted, and nerds should be given wedgies. These are not incompletely-extrapolated values that will change with more information; they are values. Opponents of gay marriage make it clear that they do not object to gay marriage based on a long-range utilitarian calculation; they directly value not allowing gays to marry. Many human values horrify most people on this list, so they shouldn't be trying to preserve them."
This has always been my principal objection to CEV. I strongly suspect that were it implemented, it would want the death of a lot of my friends, and quite possibly me, too.
Regarding CEV: My own worry is that lots of parts of human value get washed out as "incoherent" - whatever X is, if it isn't a basic human biological drive, there are enough people out there that have different opinions on it to make CEV throw up its hands, declare it an "incoherent" desire, and proceed to leave it unsatisfied. As a result, CEV ends up saying that the best we can do is just make everyone a wirehead because pleasure is one of our few universal coherent desires while things like "self-determination" and "actual achievement in the real world" are a real mess to provide and barely make sense in the first place. Or something like that.
(Universal wireheading - with robots taking care of human bodies - at least serves as a lower bound on any proposed utopia; people, in general, really do want pleasure, even if they also want other things. See also "Reedspace's Lower Bound".)
I would like to see more discussion on the question of how we should distinguish between 1) things we value even at the expense of pleasure, and 2) things we mistakenly alieve are more pleasurable than pleasure.
Surely if there is something I will give up pleasure for, which I do not experience as pleasurable, that's strong evidence that it is an example of 1 and not 2?
Yes, but there are other cases. If you prefer eating a cookie to having the pleasure centers in your brain maximally stimulated, are you sure that's not because eating a cookie sounds on some level like it would be more pleasurable?
I'm not sure how I could ever be sure of such a thing, but it certainly seems implausible to me.