Assuming "wanting" is basically the dopamine version of "liking" seems more plausible and strictly simpler
Why assume? It's there in the brain. It's okay to model reality with simpler stuff sometimes, but to look at reality and say "not simple enough" is bad. The model that says "it would be rewarding, therefore I must want it" is too simple.
than assuming there's a really complex hypothetical calculation based on states of the world being performed.
Except the brain is a computer that processes data from sensory organs and outputs commands - it's not like we're assuming this from nothing, it's an experimental result. I'm including all sorts of thing in "the world" here (maybe more than you intended), but that's as it should be. And ever since mastering the art of peek-a-boo, I've had this concept of a real world, and I (i.e. me, my brain) use it in computation all the time.
Also, I suspect you are understanding wireheading as too narrow here. It's not just the pleasure center [...] The intuition "I get wireheaded and still feel like I want something else" is false, which is why I used "rewards" instead of "pleasure".
This is part of why I referenced expected utility maximizers. Expected utility maximizers don't choose what just makes them feel like they've done something. They evaluate the possibilities with their current utility function. The goal (for an agent who does this) truly isn't to make the utility meter read a big number, but to do things that would make their current utility function read a big number. An expected utility maximizer leading a worthwhile life will always turn down the offer to be overwritten with orgasmium (as long as one of their goals isn't something internal like "get overwritten with ogasmium").
I'd much rather side with revealed preferences, which show that plenty of people are interested in crude wireheading
And plenty of people aren't, or will play tetris but won't do heroin. And of course there are people who will lay down their lives for another - to call wireheading a revealed preference of humans is flat wrong.
This is part of why I referenced expected utility maximizers. Expected utility maximizers don't choose what just makes them feel like they've done something.
Correct and I don't disagree with this. An actual expected utility maximizer (or an approximation of one) would have no interest in wireheading. Why do you think humans are best understood as such utility maximizers? If we were, shouldn't everyone have an aversion, or rather, indifference to wireheading? After all, if you offered an expected paperclip maximizer the option of wireheading, it would si...
I've been thinking about wireheading and the nature of my values. Many people here have defended the importance of external referents or complex desires. My problem is, I can't understand these claims at all.
To clarify, I mean wireheading in the strict "collapsing into orgasmium" sense. A successful implementation would identify all the reward circuitry and directly stimulate it, or do something equivalent. It would essentially be a vastly improved heroin. A good argument for either keeping complex values (e.g. by requiring at least a personal matrix) or external referents (e.g. by showing that a simulation can never suffice) would work for me.
Also, I use "reward" as short-hand for any enjoyable feeling, as "pleasure" tends to be used for a specific one of them, among bliss, excitement and so on, and "it's not about feeling X, but X and Y" is still wireheading after all.
I tried collecting all related arguments I could find. (Roughly sorted from weak to very weak, as I understand them, plus link to example instances. I also searched any literature/other sites I could think of, but didn't find other (not blatantly incoherent) arguments.)
(There have also been technical arguments against specific implementations of wireheading. I'm not concerned with those, as long as they don't show impossibility.)
Overall, none of this sounds remotely plausible to me. Most of it is outright question-begging or relies on intuition pumps that don't even work for me.
It confuses me that others might be convinced by arguments of this sort, so it seems likely that I have a fundamental misunderstanding or there are implicit assumptions I don't see. I fear that I have a large inferential gap here, so please be explicit and assume I'm a Martian. I genuinely feel like Gamma in A Much Better Life.
To me, all this talk about "valueing something" sounds like someone talking about "feeling the presence of the Holy Ghost". I don't mean this in a derogatory way, but the pattern "sense something funny, therefore some very specific and otherwise unsupported claim" matches. How do you know it's not just, you know, indigestion?
What is this "valuing"? How do you know that something is a "value", terminal or not? How do you know what it's about? How would you know if you were mistaken? What about unconscious hypocrisy or confabulation? Where do these "values" come from (i.e. what process creates them)? Overall, it sounds to me like people are confusing their feelings about (predicted) states of the world with caring about states directly.
To me, it seems like it's all about anticipating and achieving rewards (and avoiding punishments, but for the sake of the wireheading argument, it's equivalent). I make predicitions about what actions will trigger rewards (or instrumentally help me pursue those actions) and then engage in them. If my prediction was wrong, I drop the activity and try something else. If I "wanted" something, but getting it didn't trigger a rewarding feeling, I wouldn't take that as evidence that I "value" the activity for its own sake. I'd assume I suck at predicting or was ripped off.
Can someone give a reason why wireheading would be bad?