Normal_Anomaly comments on Muehlhauser-Goertzel Dialogue, Part 1 - Less Wrong

28 Post author: lukeprog 16 March 2012 05:12PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (161)

You are viewing a single comment's thread. Show more comments above.

Comment author: Normal_Anomaly 17 March 2012 07:58:53PM *  1 point [-]

the idea that you put together a 3d physics module and a calculus module and a social module and a vision module and a language module and you get something that venerates Mickey Mouse shapes is... just bizarre.

Is it any less bizzare to put together a bunch of modules that would work for any goal, and get out of them something that values all four of humor, cute kittens, friendship, and movies? What I mean by this is that precisely human values are as contingent and non-special as a broad class of other values.

Comment author: Vaniver 18 March 2012 03:53:58PM *  2 points [-]

Is it any less bizzare to put together a bunch of modules that would work for any goal, and get out of them something that values all four of humor, cute kittens, friendship, and movies?

Yes. Think about it.

What I mean by this is that precisely human values are as contingent and non-special as a broad class of other values.

Human values are fragmentary subvalues of one value, which is what one would expect from a bunch of modules that each contribute to reproduction in a different way. The idea of putting together a bunch of different modules to get a single, overriding value, is bizarre. (The only possible exemption here is 'make more of myself,' but the modules are probably going to implement subvalues for that, rather than that as an explicit value. As far as single values go, that one's special, whereas things like Mickey Mouse faces are not.)