= 27d567bc2d48d9f40e9ccc20ea370b15)
rwallace comments on You cannot be mistaken about (not) wanting to wirehead - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (79)
It so happens that there are people working on this problem right now. See for example the current discussion taking place on Vladmir Nesov's blog.
As a preliminary step we can categorize the ways that our "wants" can change as follows (these are mostly taken from a comment by Andreas):
Can we agree that categories 1, 2, and 3 are acceptable, 5 and 6 are unacceptable, and 4, 7, and 8 are "it depends"?
The change that I suggested in my argument belongs to category 2, updating in light of new evidence. I wrote that the FAI would "try to extrapolate what your preferences would be if you knew what it felt like to be wireheaded." Does that seem more reasonable now?
If the FAI tries to extrapolate whether you'd want to be anti-wireheaded if you knew what it felt like to be anti-wireheaded, the obvious answer is no. You seem to assume that the FAI would instead try to predict whether you'd prefer to be anti-wireheaded after you were actually anti-wireheaded, but that change would be more like category 6.
No. If someone -- my next-door neighbor, my doctor, the government, a fictional genie, whoever -- is proposing to rewire my brain, my informed consent beforehand is the only thing that can make it acceptable.
Are you making this as a statement of personal preference, or general policy? What if it becomes practically impossible for a person to give informed consent, as in cases of extreme mental disability?
General policy. For example, if Wei Dai chooses the wirehead route, I might think he's missing out on a lot of other things life has to offer, but that doesn't give me the right to forcibly unwirehead him, any more than he has the right to do the reverse to me.
In other words, he and I have two separate disagreements: of value axioms, whether there should be more to life than wireheading (which is a matter of personal preference), and of moral axioms, whether it's okay to initiate the use of armed force (whether in person or by proxy) to impose one's preferred lifestyle on another (which is a matter of general policy). (And this serves as a nice pair of counterexamples to the theory I have seen floating around that there is a universal set of human values.)
In cases of extreme mental disability, we don't have an entity that is inherently capable of giving informed consent, so indeed it's not possible to apply that criterion. In that case (given the technology to do so) it would be necessary to intervene to repair the disability before the criterion can begin to apply.
rwallace, I'm not sure there is any actual disagreement between us. All I'm saying is that those who have not actually tried wireheading (or otherwise has knowledge about what it feels like to be wireheaded) perhaps shouldn't be so sure that they really prefer not to be wireheaded. And I never mentioned anything about forcibly wireheading people. (Maybe you confused my position with denisbider's?)
I took this to mean that you agreed with denisbider's position of licensing the initiation of force and justifying it based on what the altered version of the victim would prefer after the event -- was that not your intent? If not, then you're right, we don't disagree to anywhere near the extent I had thought.