Matt_Simpson comments on Why the beliefs/values dichotomy? - Less Wrong

20 Post author: Wei_Dai 20 October 2009 04:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (153)

You are viewing a single comment's thread. Show more comments above.

Comment author: Matt_Simpson 20 October 2009 07:25:21PM *  2 points [-]

Isn't this just a case of the values the Roomba was designed to maximize being different from the values it actually maximizes? Consider the following:

We could consider simple optimizers like humans as falling into the category: "mind without a clear belief/values distinction"; they definitely do a lot of signal processing and feature extraction and control theory, but they don't really have values. The human would happily have sex with a condom thinking that it was maximizing its fitness.

i.e. Roombas are program executers, not cleanliness maximizers.

I suppose the counter is that humans don't have a clear belief/values distinction.