= 27d567bc2d48d9f40e9ccc20ea370b15)
saturn comments on Post Your Utility Function - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (273)
Of course, because the immediate pain of the thought of choosing B would outweigh the longer-term lesser pain of the thought of losing contact with your sister.
This has nothing to do with whether the events actually occur, and everything to do with your mapping of the experience of the conditions, as you imagine them for purposes of making a decision.
That is, the model you make of the future may refer to a hypothetical reality, but the thing you actually evaluate is not that reality, but your own reaction to that reality -- your present-tense experience in response to a constructed fiction made of previous experiences
It so happens that there is some correspondence between this (real) process and the way we would prefer to think we establish and evaluate our preferences. Specifically, both models will generate similar results, most of the time. It's just that the reasons we end up with for the responses are quite different.
But calling that latter concept "territory" is still a category error, because what you are using to evaluate it is still your perception of how you would experience the change.
We do not have preferences that are not about experience or our emotional labeling thereof; to the extent that we have "rational" preferences it is because they will ultimately lead to some desired emotion or sensation.
However, our brains are constructed in such a way so as to allow us to plausibly overlook and deny this fact, so that we can be honestly "sincere" in our altruism... specifically by claiming that our responses are "really" about things outside ourselves.
For example, your choice of "A" allows you to self-signal altruism, even if your sister would actually prefer death to being imprisoned on Mars for the rest of her life! Your choice isn't about making her life better, it's about you feeling better for the brief moment that you're aware you did something.
(That is, if you cared about something closer to the reality of what happens to your sister, rather than your experience of it, you'd have hesitated in that choice long enough to ask Omega whether she would prefer death to being imprisoned on Mars.)
And what if he did ask?
Then, as I said, he cares about something closer to the reality.
The major point I've been trying to make in this thread is that because human preferences are not just in the map but of the map, is that it allows people to persist in delusions about their motivations. And not asking the question is a perfect example of the sort of decision error this can produce!
However, asking the question doesn't magically make the preference about the territory either; in order to prefer the future include his sister's best interests, he must first have an experience of the sister and a reason to wish well of her. But it's still better than not asking, which is basically wireheading.
The irony I find in this discussion is that people seem to think I'm in favor of wireheading because I point out that we're all doing it, all the time. When in fact, the usefulness of being aware that it's all wireheading, is that it makes you better at noticing when you're doing it less-usefully.
The fact that he hadn't asked his sister, or about his sister's actual well-being instantly jumped off the screen at me, because it was (to me) obvious wireheading.
So, you could say that I'm biased by my belief to notice wireheading more, but that's an advantage for a rationalist, not a disadvantage.
Is human knowledge also not just in the map, but exclusively of the map? If not, what's the difference?
Any knowledge about the actual territory can in principle be reduced to mechanical form without the presence of a human being in the system.
To put it another way, a preference is not a procedure, process, or product. The very use of the word "preference" is a mind projection - mechanical systems do not have "preferences" - they just have behavior.
The only reason we even think we have preferences in the first place (let alone that they're about the territory!) is because we have inbuilt mind projection. The very idea of having preferences is hardwired into the model we use for thinking about other animals and people.
You never answered my question.
You said, "if not, what's the difference", and I gave you the difference. i..e, we can have "knowledge" of the territory.
So, knowledge exists in the structure of map and is about the territory, while preference can't be implemented in natural artifacts. Preference is a magical property of subjective experience, and it is over maps, or about subjective experience, but not, for example, about the brain. Saying that preference exists in the structure of map or that it is about the territory is a confusion, that you call "mind projection" Does that summarize your position? What are the specific errors in this account?
No, "preference" is an illusory magical property projected by brains onto reality, which contains only behaviors.
Our brains infer "preferences" as a way of modeling expected behaviors of other agents: humans, animals, and anything else we perceive as having agency (e.g. gods, spirits, monsters). When a thing has a behavior, our brains conclude that the thing "prefers" to have either the behavior or the outcome of the behavior, in a particular circumstance. In other words, "preference" is a label attached to a clump of behavior-tendency observations and predictions in the brain -- not a statement about the nature of the thing being observed.
Thus, presuming that these "preferences" actually exist in the territory is supernaturalism, i.e., acting as though basic mental entities exist.
My original point had more to do with the types of delusion that occur when we reason on the basis of preferences actually existing, rather than the idea simply being a projection of our own minds. However, the above will do for a start, as I believe my other conclusions can be easily reached from this point.
Do you think someone is advocating the position that goodness of properties of the territory is an inherent property of territory (that sounds like a kind of moral realism)? This looks like the lack of distinction between 1-place and 2-place words. You could analogize preference (and knowledge) as a relation between the mind and the (possible states of the) territory, that is neither a property of the mind alone, nor of the territory alone, but a property of them being involved in a certain interaction.
I feel your frustration, but throwing the word "magical" in there is just picking a fight, IMO. Anyway, I too would like to see P.J. Eby summarize his position in this format.
I have a certain technical notion of magic in mind. This particular comment wasn't about frustration (some of the others were), I'm trying out something different of which I might write a post later.