= 27d567bc2d48d9f40e9ccc20ea370b15)
PhilGoetz comments on The scourge of perverse-mindedness - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (249)
That post is about a different issue. It's about whether introducing noise can help an optimization algorithm. Sounds similar; isn't. The difference is that the optimization algorithm already knows the function that it's trying to optimize.
The basic problem with CEV is that it requires reifying values in a strange way so that there are atomic "values" that can be isolated from an agent's physical and cognitive architecture; and that (I think) it assumes that we have already evolved to the point where we have discovered all of these values. You can make very general value statements, such as that you value diversity, or complexity. But a trilobite can't make any of those value statements. I think it's likely that there are even more important fundamental value statements to be made that we have not yet conceptualized; and CEV is designed from the ground up specifically to prevent such new values from being incorporated into the utility function.
The need for randomness is not because random is good; it's because, for the purpose of discovering better primitives (values) to create better utility functions, any utility function you can currently state is necessarily worse than random.
Since when is randomness required to explore the "landscape of possible worlds"? Or the possible values that we haven't considered? A methodical search would be better. How did you miss that lesson from Worse Than Random, when it included an example (the pushbutton combination lock) of exploring a space of potential solutions?
Okay, you don't actually need randomness, if you can work out a way of doing a methodical variation of all possible parameters.
(For problems of this nature, using random processes allows you to specify the statistical properties that you want the solution to have, which is often much simpler than specifying a deterministic process that has those properties. That's one reason randomness is useful.)
The point I'm trying to make is that you need not to limit yourself to "searching", meaning trying to optimize a function. You can only search when you know what you're looking for. A value system can't be evaluated from the outside. You have to try it on. Rationally, where "rational" means optimizing existing values, you wouldn't do that. So randomness (or a rationally-ordered but irrationally-pursued exploration of parameter space) will lead to places no rational agent would go.
[EDIT: Wow, the parent comment completely changed since I responded to it. WTF?]
How do you plan to map a random number into a search a space that you could not explore systematically?
According to which utility function?
I have a bad habit of re-editing a comment for several minutes after first posting it.
Suppose you want to test a program whose input variables are distributed normally. You can write a big complicated equation to sample at uniform intervals from the cumulative distribution function for the gaussian distribution. Or you can say "x = mean; for i=1 to 10 { x += rnd(2)-1 }".
Very often, the only data you know about your space is randomly-sampled data. So you look at that randomly-sampled data, and come up with some simple random model that would generate data with similar properties. The nature of the statistics you've gathered, such as the mean, variance, and correlations between observed variables, make it very hard to construct a deterministic model that would reproduce those statistics, but very easy to build a random model that does.
Some people really do have the kinds of misconceptions Eliezer was talking about; but the idea that there are hordes of scientists who attribute magical properties to randomness just isn't true. This is not a fight you need to fight. And railing against all use of randomness in the simulation or study of complex processes just puts a big sticker on your head that says "I have no experience with what I'm talking about!"
We're having 2 separate arguments here. I hope you realize that my comment that you originally responded to was not claiming that randomness has some magical power. It was about the need, when considering the future of the universe, for trying things out not just because your current utility function suggests they will have high utility. I used "random" as shorthand for "not directed by a utility function".
According to the utility function that your current utility function doesn't like, but that you will be delighted with once you try it out.
Yes, I understand you can use randomness as an approximate substitute for actually understanding the implications of your probability distributions. That does not really address my point, the randomness does not grant you access to a search space you could not otherwise explore.
If you analyze randomly-sampled data by considering the probability distribution of results for a random sampling, instead for the specific sampling you actually used, you are vulnerable to the mistake described here.
You can deterministically build a model that accounts for your uncertainty. Having a probability distribution is not the same thing as randomly choosing results from that distribution.
First of all, I am not "railing against all use of randomness in the simulation or study of complex processes". I am objecting to your claim that "randomness is required" in an epistemilogical process. Second, you should not presume to warn me about stickers on my head.
You should realize that "randomness is required" does sound very much like "claiming that randomness has some magical power", and if you mispoke, the correct response to the objection would be to admit that you made a mistake and apologize for the miscommunication, not to try to defend the wrong claim.
It appears that you don't understand the purpose of utility functions. I do not want to have a utility function U that maximizes U(U), that assigns to itself higher utility than any other utility function assigns to itself. I want to achieve states of the world that maximize my current utility function.
You mean, for instance, by saying,
I'm not defending the previous wrong claim about "needing randomness". I'm arguing against your wrong claim, which appears to be that one should never use randomness in your models.
It appears that you still don't understand what my basic point is. You can't improve your utility function by a search using your utility function. We have better utility functions than trilobites did. We could not have found them using trilobite utility functions. Trilobite CEV would, if performing optimally, have ruled them out. Extrapolate.
Wow, you are actually compounding the rudeness of abusing the edit feature to completely rewrite your comment by then analyzing my response to the original version as if it were responding to the edited version.
How did you get from "randomness is never required" to "randomness is never useful"? I acknowledge that sometimes randomness can be a good enough approximate substitute for the much harder strategy of actually understanding the implications of a probability distribution.
I understand your argument. It is wrong. You have not actually responded to my objection. To refute my objection, you would have to explain why I should want to give up my current utility function U0 in favor of some other utility function U such that
even though
Since U0 is my current utility function, and therefore (2) describes my current wants, you will not be able to convince me that I should be persuaded by (1), which is a meaningless comparison. Adopting U as my utility function does not help me maximize U0.
To the extent that trilobites can even be considered to have utility functions, my utility function is better than the trilobite utility function according to my values. The trilobites would disagree. An optimal human CEV would be a human SUCCESS and a trilobite FAIL. Likewise, an optimal trilobite CEV would be a trilobite SUCCESS and a human FAIL. There is no absolute universal utilility function that says one of these is better than the others. It is my human values that cause me to say that the human SUCCESS is better.
Unless, of course, it turns out that humans really like trilobites and would be willing to devote significant resources to keeping them alive, understanding their preferences, and carrying out those preferences (without compromising other human values). In that case, it's mutual success.
You're thinking of tribbles.
I'm breaking this out into a separate reply, because it's its own sub-thread:
If no utility function, and hence no world state, is objectively better than any other, then all utility functions are wireheading. Because the only distinction between wireheading, and not wireheading, is that the wirehead only cares about his/her own qualia, not about states of the world. If the only reason you care about states of the world is because of how your utility function evaluates them - that is to say, what qualia they generate in you - you are a wirehead.
You have it backwards. I do not care about things because of how my utility function evaluates them. Rather, my utility function evaluates things the way it does because of how I care about it. My utility function is a description of my preferences, not the source of them.
No. That statement is three comments above the comment in which you said I should acknowledge my error. It was already there when you wrote that comment. And I also acknowledged my misstatement in the comment you were replying to, and elaborated on what I had meant when I made the comment.
Good! We agree.
Good! We agree again.
And we agree yet again!
And here is where we part ways.
Maybe there is no universal utility function. That's a... I won't say it's a reasonable position, but I understand its appeal. I would call it an over-reasoned position, like when a philosopher announces that he has proved that he doesn't exist. It's time to go back to the drawing board when you come up with that conclusion. Or at least to take your own advice, and stop trying to change the world when you've already said it doesn't matter how it changes.
But to believe that your utility function is nothing special, and still try to take over the universe and force your utility function on it for all time, is insane.
(Yes, yes, I know Eliezer has all sorts of disclaimers in the CEV document about how CEV should not try to take over the universe. I don't believe that it's logically possible; and I believe that his discussions of Friendly AI make it even clearer that his plans require complete control. Perhaps the theory is still vague enough that just maybe there's a way around this; but I believe the burden of proof is on those who say there is a way around it.)
It would be consistent with the theory of utility functions if, in promoting CEV, you were acting on an inner drive that said, "Ooh, baby, I'm ensuring the survival of my utility function. Oh, God, yes! Yes! YES!" But that's not what I see. I see people scribbling equations, studying the answers, and saying, "Hmm, it appears that my utility function is directing me to propagate itself. Oh, dear, I suppose I must, then."
That's just faking your utility function.
I think it's key that the people I'm speaking of who believe utility functions are arbitrary, also believe they have no free will. And it's probably also key that they assume their utility function must assign value to its own reproduction. They then use these two beliefs as an excuse to justify not following through on their belief about the arbitrariness of their utility function, because they think to do so would be logically impossible. "We can't help ourselves! Our utility functions made us do it!" I don't have a clean analysis, but there's something circular, something wrong with this picture.
Let's recap. You made a wrong claim. I responded to the wrong claim. You disputed my response. I refuted your disputation. You attempted to defend your claim. I responded to your defense. You edited your defense by replacing it with the acknowledgment of your mistake. You responded to my response still sort of defending your wrong claim, and attacking me for refuting your wrong claim. I defended my refutation, pointing out the you really did make the wrong claim and continued to defend it. And now you attack my defense, claiming that you did in fact acknowledge your mistake, and this should somehow negate your continued defense after the acknowledgement. Do you see how you are wrong here? When you acknowledge your claim is wrong, you should not at the same time criticize me for refuting your point.
I do believe my utility function is special. I don't expect the universe (outside of me, my fellow humans, and any optimizing processes we spawn off) to agree with me. But, like Eliezer says, "We'll see which one of us is still standing when this is over."
I'm not planning to tile the universe with myself, I just want myself or something closely isomorphic to me to continue to exist. The two most obvious ways to ensure my own continued existence are avoidance of things that would destroy me, particularly intelligent agents which could devote significant resources to destroying me personally, and making redundant copies. My own ability to copy myself is limited, and an imperfect copy might compete with me for the same scarce resources, so option two is curtailed by option one. Actual destruction of enemies is just an extension of avoidance; that which no longer exists within my light-cone can no longer pose a threat.
Your characterization of my utility function as arbitrary is, itself, arbitrary. Deal with it.
That description could apply to an overwhelming majority of the possible self-consistent utility functions (which are, last I checked, infinite in number), including all of those which lead to wireheading. Please be more specific.
Utility function #311289755230920891423. Try it. You'll like it.
I have no solution to wireheading. I think a little wireheading might even be necessary. Maybe "wireheading" is a necessary component of "consciousness", or "value". Maybe all of the good places lie on a continuum between "wireheading" and "emotionless nihilism".
Fallacy of moderation. Besides, wireheading and self-destructive nihilism aren't opposite extremes on a spectrum, they're just failure states within the solution space of possible value systems.
A string of random numbers is not an explanation.
I have a simple solution to wireheading... simple for me, anyway. I don't like it, so I won't seek it out, nor modify myself in any way that might reasonably cause me to like it or want to seek it out.
The fallacy of moderation is only a fallacy when someone posits that two things that are on a continuum, that aren't actually on a continuum. (If they are on a continuum, it's only a fallacy if you have independent means for finding a correct answer to the problem that the arguing groups have made errors on, rather than simply combining their utility functions.) The question I'm raising is whether wireheading is in fact just an endpoint on the same continuum that our favored states lie.
How do you define wireheading?
I define it as valuing your qualia instead of valuing states of the world. But could something that didn't value its qualia be conscious? Could it have any fun? Would we like to be it? Isn't valuing your qualia part of the definition of what a qualia is?