Matt_Simpson comments on Why the beliefs/values dichotomy? - Less Wrong

20 Post author: Wei_Dai 20 October 2009 04:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (153)

You are viewing a single comment's thread. Show more comments above.

Comment author: SforSingularity 20 October 2009 05:43:20PM 0 points [-]

One possible response here: We could consider simple optimizers like amoeba or Roomba vacuum cleaners as falling into the category: "mind without a clear belief/values distinction"; they definitely do a lot of signal processing and feature extraction and control theory, but they don't really have values. The Roomba would happily sit with wheels lifted off the ground thinking that it was cleaning a nonexistent room.

Comment author: Matt_Simpson 20 October 2009 07:25:21PM *  2 points [-]

Isn't this just a case of the values the Roomba was designed to maximize being different from the values it actually maximizes? Consider the following:

We could consider simple optimizers like humans as falling into the category: "mind without a clear belief/values distinction"; they definitely do a lot of signal processing and feature extraction and control theory, but they don't really have values. The human would happily have sex with a condom thinking that it was maximizing its fitness.

i.e. Roombas are program executers, not cleanliness maximizers.

I suppose the counter is that humans don't have a clear belief/values distinction.