= 27d567bc2d48d9f40e9ccc20ea370b15)
PhilGoetz comments on The scourge of perverse-mindedness - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (249)
No. That statement is three comments above the comment in which you said I should acknowledge my error. It was already there when you wrote that comment. And I also acknowledged my misstatement in the comment you were replying to, and elaborated on what I had meant when I made the comment.
Good! We agree.
Good! We agree again.
And we agree yet again!
And here is where we part ways.
Maybe there is no universal utility function. That's a... I won't say it's a reasonable position, but I understand its appeal. I would call it an over-reasoned position, like when a philosopher announces that he has proved that he doesn't exist. It's time to go back to the drawing board when you come up with that conclusion. Or at least to take your own advice, and stop trying to change the world when you've already said it doesn't matter how it changes.
But to believe that your utility function is nothing special, and still try to take over the universe and force your utility function on it for all time, is insane.
(Yes, yes, I know Eliezer has all sorts of disclaimers in the CEV document about how CEV should not try to take over the universe. I don't believe that it's logically possible; and I believe that his discussions of Friendly AI make it even clearer that his plans require complete control. Perhaps the theory is still vague enough that just maybe there's a way around this; but I believe the burden of proof is on those who say there is a way around it.)
It would be consistent with the theory of utility functions if, in promoting CEV, you were acting on an inner drive that said, "Ooh, baby, I'm ensuring the survival of my utility function. Oh, God, yes! Yes! YES!" But that's not what I see. I see people scribbling equations, studying the answers, and saying, "Hmm, it appears that my utility function is directing me to propagate itself. Oh, dear, I suppose I must, then."
That's just faking your utility function.
I think it's key that the people I'm speaking of who believe utility functions are arbitrary, also believe they have no free will. And it's probably also key that they assume their utility function must assign value to its own reproduction. They then use these two beliefs as an excuse to justify not following through on their belief about the arbitrariness of their utility function, because they think to do so would be logically impossible. "We can't help ourselves! Our utility functions made us do it!" I don't have a clean analysis, but there's something circular, something wrong with this picture.
Let's recap. You made a wrong claim. I responded to the wrong claim. You disputed my response. I refuted your disputation. You attempted to defend your claim. I responded to your defense. You edited your defense by replacing it with the acknowledgment of your mistake. You responded to my response still sort of defending your wrong claim, and attacking me for refuting your wrong claim. I defended my refutation, pointing out the you really did make the wrong claim and continued to defend it. And now you attack my defense, claiming that you did in fact acknowledge your mistake, and this should somehow negate your continued defense after the acknowledgement. Do you see how you are wrong here? When you acknowledge your claim is wrong, you should not at the same time criticize me for refuting your point.
I do believe my utility function is special. I don't expect the universe (outside of me, my fellow humans, and any optimizing processes we spawn off) to agree with me. But, like Eliezer says, "We'll see which one of us is still standing when this is over."
No, that isn't what happened. I'm not sure which comment the last sentence is supposed to refer to, but I'm p > .8 it didn't happen that way. If it's referring to the statement, "Okay, you don't actually need randomness," I wrote that before I ever saw your first response to that comment. But that doesn't match up with what you just described; there weren't that many exchanges before that comment. It also doesn't match up with anything after that comment, since I still don't acknowledge any such mistake made after that comment.
We're talking about 2 separate claims. The wrong claim that I made was in an early statement where I said that you "needed randomness" to explore the space of possible utility functions. The right claim that I made, at length, was that randomness is a useful tool. You are conflating my defense of that claim, with defending the initial wrong claim. You've also said that you agree that randomness is a useful tool, which suggests that what is happening is that you made a whole series of comments that I say were attacking claim 2, and that you believe were attacking claim 1.
I'm not planning to tile the universe with myself, I just want myself or something closely isomorphic to me to continue to exist. The two most obvious ways to ensure my own continued existence are avoidance of things that would destroy me, particularly intelligent agents which could devote significant resources to destroying me personally, and making redundant copies. My own ability to copy myself is limited, and an imperfect copy might compete with me for the same scarce resources, so option two is curtailed by option one. Actual destruction of enemies is just an extension of avoidance; that which no longer exists within my light-cone can no longer pose a threat.
Your characterization of my utility function as arbitrary is, itself, arbitrary. Deal with it.