= 27d567bc2d48d9f40e9ccc20ea370b15)
cousin_it comments on Two questions about CEV that worry me - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (137)
In questions like this, it's very important to keep in mind the difference between state of knowledge about preference (which corresponds to explicitly endorsed moral principles, such as "slavery bad!"; this clearly changed), and preference itself (which we mostly don't understand, even if our minds define what it is). Since FAI needs to operate according to preference, and not out state of knowledge about preference, any changes in our state of knowledge (moral principles) is irrelevant, except for where they have a chance of reflecting changes in the actual preference.
So the idea is that 21st century American and caveman Gork from 40000 BC probably have very similar preference, because they have very similar cognitive architecture, even though clearly they have different explicitly endorsed moral principles. This property is a "sanity check" on a method of defining preference, not an explicit requirement.
In other words, finding similar preferences in people from different eras is about consistency expected between different maps of the same territory, not about adding a rule that demands consistency from the maps of the territory, even if the changes thus introduced aren't based in fact.
If you're saying actual human preference is determined by human biology and brain architecture, but mostly independent from brain content, this is a very new claim that I don't remember hearing ever before. You'll need pretty strong arguments to defend it. I'd bet at about 80% odds that Eliezer would disagree with it.
Hmm, I think I've said this many times already. Of course beliefs are bound to change preference to some extent, but shouldn't be allowed to do this too much. On reflection, you wouldn't want the decisions (to obtain certain beliefs) of your stupid human brain with all its biases that you already know not to endorse, to determine what should be done with the universe.
Only where such decisions manage to overcome this principle, will there be change, and I can't even think of a specific example of when that should happen. Generally, you can't trust yourself. The fact that you believe that X is better than Y is not in itself a reason to believe that X is better than Y, although you might believe that X is better than Y because it is (because of a valid reason for X being better than Y, which your belief in X being better than Y isn't).
So when beliefs do change your preference, it probably won't be in accordance with beliefs about preference.
As opposed our biology and brain architecture, which were designed by the blind idiot god.
But don't our biological preferences imply pressing pleasure buttons? Isn't it just for our cultural/learnt preferences (brain content) that we assign low utility to drug induced happiness and push-button pleasure?