That preference is not universal, which to me makes it absolutely part of the map.

I don't see how it supposed to follow from the fact that not everyone prefers not-being-decieved, that those who claim to prefer not-being-deceived must be wrong about their own preferences. Could you explain why you seem to think it does?

The claim others are defending here (as I understand it) is not that everyone's preferences are really over the territory; merely that some people's are. Pointing out that some people's preferences aren't about the territory isn't a counterargument to that claim.

You've just ignored Cyan's counterexample, and presented a few of your own that support your point of view.

If you are operating under an assumption that nobody agrees with, you are wasting anyone's time (assumption: map is about the map, territory be damned), as the argument never goes anywhere. As a compromise, compose your best argument as a top-level post (but only if you expect to convince at least someone).

IOW, our statements about reality are about the intersection of some portion of "observable" reality and our particular mapping (division and labeling) of it. And it cannot be otherwise, since to even talk about it, we have to carve up and label the "reality" we are discussing.

It's funny that you talk of wordplay a few comments back, as it seems that you're the one making a technically-correct-but-not-practically-meaningful argument here.

If I may attempt to explore your position: Suppose someone claims a preference for "blue skies". The wirehead version of this that you endorse is "I prefer experiences that include the perception I label 'blue sky'". The "anti-wirehead" version you seem to be arguing against is "I prefer actual states of the world where the sky is actually blue".

You seem to be saying that since the preference is really about the experience of blue skies, it makes no sense to talk about the sky actually being blue. Chasing after external definitions involving photons and atmospheric scattering is beside the point, because the actual preference wasn't formed in terms of them.

This becomes another example of the general rule that it's impossible to form preferences directly about reality, because "reality" is just another label on our subjective map.

As far as specifics go, I think the point you make is sound: Most (all?) of our preferences can't just be about the territory, because they're phrased in terms of things that themselves don't exist in the territory, but at best simply point at the slice of experience labeled "the territory".

That said, I think this perspective grossly downplays the practical importance of that label. It has very distinct subjective features connecting in special ways to other important concepts. For the non-solipsists among us, perhaps the most important role it plays is establishing a connection between our subjective reality and someone else's. We have reason to believe that it mediates experiences we label as "physical interactions" in a manner causally unaffected by our state of mind alone.

When I say "I prefer the galaxy not to be tiled by paperclips", I understand that, technically, the only building blocks I have for that preference are labeled experiences and concepts that aren't themselves the "stuff" of their referents. In fact, I freely admit that I'm not exactly sure what constitutes "the galaxy", but the preference I just expressed actually contains a massive number of implicit references to other concepts that I consider causally connected to it via my "external reality" label. What's more, most people I communicate with can easily access a seemingly similar set of connections to their "external reality" label, assuming they don't talk themselves out of it.

The territory concept plays a similar role to that of an opaque reference in a programming language. Its state may not be invariant, but its identity is. I don't have to know any true facts concerning its actual structure for it to be meaningful and useful. Just as photons aren't explicitly required to subjectively perceive a blue sky, the ontological status of my territory concept doesn't really change its meaning or importance, which is acquired through its intimate connection to massive amounts of raw experience.

Claiming my preferences about the territory are really just about my map is true in the narrow technical sense that it's impossible for me to refer directly to "reality", but doing so completely glosses over the deep, implicit connections expressed by such preferences, most primarily the connection between myself and the things I label "other consciousnesses". In contrast, the perception of these connections seems to come for free by "confusing" the invariant identity of my territory concept with the invariant "existence" of a real external world. The two notions are basically isomorphic, so where's the value in the distinction?

Whether your preferences refer to your state, or to the rest of the world is indeed a wirehead-related issue. The problem with the idea that they refer to your state is that that idea tends to cause wirehead behaviour - surgery on your own brain to produce the desired state. So - it seems desirable to construct agents that believe that there is a real world, and that their preferences relate to it.

if by "good for you", you mean health, persistence, or success in general

The argument is precisely about what "good for you" means, so this would be assuming the conclusion that needs to be established.