Wei_Dai comments on What's wrong with simplicity of value? - Less Wrong

18 Post author: Wei_Dai 27 July 2011 03:09AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (37)

Comments (37)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 28 July 2011 09:58:14PM 0 points [-]

I think you agreed previously, and your response was more subtle that denying it. IIRC, you said that complex aspects of value are present, but possibly of low importance, so if we have no choice (time), we should just go the simple way, while still capturing nontrivial amount of value. This is different from saying that value could be actually simple.

Yes, this post is making a different point from that one.

But humans are not like that, they have a lot of accidental complexity in every joint, so there is no reason to expect them to implement that particular simple goal system.

But why think all that complexity is relevant? Surely at least some of the complexity is not relevant (for example a Tourette sufferer's tendency to curse at random, or the precise ease with which some people get addicted to gambling). Don't you need an additional argument to conclude that at least a substantial fraction of the complexity is relevant, and that this applies generally, even to those who think otherwise and would tend to discard such complexity when they reflect on what they really want?

Comment author: Vladimir_Nesov 28 July 2011 11:03:34PM 0 points [-]

Don't you need an additional argument to conclude that at least a substantial fraction of the complexity is relevant, and that this applies generally, even to those who think otherwise and would tend to discard such complexity when they reflect on what they really want?

Again, it's an antiprediction, argument about what your prior should be, and not an argument that takes the presumption of simple values being plausible as a starting point and then tries to convince that it should be tuned down. Something is relevant, brains are probably relevant, this is where decision-making happens. The claim that the relevant decision-theoretic summary of that has any given rare property, like simplicity, is something that needs strong evidence, if you start from that prior. I don't see why privileging simplicity is a viable starting point, why this hypothesis deserves any more consideration than the claim that the future should contain a perfect replica of myself from the year 1995.

Comment author: Will_Newsome 30 July 2011 11:58:20AM *  0 points [-]

Why is simplicity assumed to be a rare property? There are large classes of things that tend to be "simple", like much of mathematics; don't you have to argue that "brains" or minds belong to the class of things whose members do not tend to be "simple"? (I can see obvious reasons that one would think this is obvious, and non-obvious reasons that one would think this is non-obvious, which makes me think that it shouldn't be assumed, but if I'm the only person around who could make the non-obvious arguments then I guess we're out of luck.)

Powered by Reddit Powered by Reddit