= 27d567bc2d48d9f40e9ccc20ea370b15)
Vladimir_Nesov comments on The scourge of perverse-mindedness - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (249)
You never learn.
Folks. Vladimir's response is not acceptable in a rational debate. The fact that it currently has 3 points is an indictment of the Less Wrong community.
Normally I would agree, but he was responding to "Some degree of randomness is necessary". Seriously, you should know that isn't right.
That post is about a different issue. It's about whether introducing noise can help an optimization algorithm. Sounds similar; isn't. The difference is that the optimization algorithm already knows the function that it's trying to optimize.
The basic problem with CEV is that it requires reifying values in a strange way so that there are atomic "values" that can be isolated from an agent's physical and cognitive architecture; and that (I think) it assumes that we have already evolved to the point where we have discovered all of these values. You can make very general value statements, such as that you value diversity, or complexity. But a trilobite can't make any of those value statements. I think it's likely that there are even more important fundamental value statements to be made that we have not yet conceptualized; and CEV is designed from the ground up specifically to prevent such new values from being incorporated into the utility function.
The need for randomness is not because random is good; it's because, for the purpose of discovering better primitives (values) to create better utility functions, any utility function you can currently state is necessarily worse than random.
Since when is randomness required to explore the "landscape of possible worlds"? Or the possible values that we haven't considered? A methodical search would be better. How did you miss that lesson from Worse Than Random, when it included an example (the pushbutton combination lock) of exploring a space of potential solutions?
Okay, you don't actually need randomness, if you can work out a way of doing a methodical variation of all possible parameters.
(For problems of this nature, using random processes allows you to specify the statistical properties that you want the solution to have, which is often much simpler than specifying a deterministic process that has those properties. That's one reason randomness is useful.)
The point I'm trying to make is that you need not to limit yourself to "searching", meaning trying to optimize a function. You can only search when you know what you're looking for. A value system can't be evaluated from the outside. You have to try it on. Rationally, where "rational" means optimizing existing values, you wouldn't do that. So randomness (or a rationally-ordered but irrationally-pursued exploration of parameter space) will lead to places no rational agent would go.
[EDIT: Wow, the parent comment completely changed since I responded to it. WTF?]
How do you plan to map a random number into a search a space that you could not explore systematically?
According to which utility function?
I have a bad habit of re-editing a comment for several minutes after first posting it.
Suppose you want to test a program whose input variables are distributed normally. You can write a big complicated equation to sample at uniform intervals from the cumulative distribution function for the gaussian distribution. Or you can say "x = mean; for i=1 to 10 { x += rnd(2)-1 }".
Very often, the only data you know about your space is randomly-sampled data. So you look at that randomly-sampled data, and come up with some simple random model that would generate data with similar properties. The nature of the statistics you've gathered, such as the mean, variance, and correlations between observed variables, make it very hard to construct a deterministic model that would reproduce those statistics, but very easy to build a random model that does.
Some people really do have the kinds of misconceptions Eliezer was talking about; but the idea that there are hordes of scientists who attribute magical properties to randomness just isn't true. This is not a fight you need to fight. And railing against all use of randomness in the simulation or study of complex processes just puts a big sticker on your head that says "I have no experience with what I'm talking about!"
We're having 2 separate arguments here. I hope you realize that my comment that you originally responded to was not claiming that randomness has some magical power. It was about the need, when considering the future of the universe, for trying things out not just because your current utility function suggests they will have high utility. I used "random" as shorthand for "not directed by a utility function".
According to the utility function that your current utility function doesn't like, but that you will be delighted with once you try it out.
Yes, I understand you can use randomness as an approximate substitute for actually understanding the implications of your probability distributions. That does not really address my point, the randomness does not grant you access to a search space you could not otherwise explore.
If you analyze randomly-sampled data by considering the probability distribution of results for a random sampling, instead for the specific sampling you actually used, you are vulnerable to the mistake described here.
You can deterministically build a model that accounts for your uncertainty. Having a probability distribution is not the same thing as randomly choosing results from that distribution.
First of all, I am not "railing against all use of randomness in the simulation or study of complex processes". I am objecting to your claim that "randomness is required" in an epistemilogical process. Second, you should not presume to warn me about stickers on my head.
You should realize that "randomness is required" does sound very much like "claiming that randomness has some magical power", and if you mispoke, the correct response to the objection would be to admit that you made a mistake and apologize for the miscommunication, not to try to defend the wrong claim.
It appears that you don't understand the purpose of utility functions. I do not want to have a utility function U that maximizes U(U), that assigns to itself higher utility than any other utility function assigns to itself. I want to achieve states of the world that maximize my current utility function.
You mean, for instance, by saying,
I'm not defending the previous wrong claim about "needing randomness". I'm arguing against your wrong claim, which appears to be that one should never use randomness in your models.
It appears that you still don't understand what my basic point is. You can't improve your utility function by a search using your utility function. We have better utility functions than trilobites did. We could not have found them using trilobite utility functions. Trilobite CEV would, if performing optimally, have ruled them out. Extrapolate.
That description could apply to an overwhelming majority of the possible self-consistent utility functions (which are, last I checked, infinite in number), including all of those which lead to wireheading. Please be more specific.
Utility function #311289755230920891423. Try it. You'll like it.
I have no solution to wireheading. I think a little wireheading might even be necessary. Maybe "wireheading" is a necessary component of "consciousness", or "value". Maybe all of the good places lie on a continuum between "wireheading" and "emotionless nihilism".