SilasBarta comments on Why the beliefs/values dichotomy? - Less Wrong

20 Post author: Wei_Dai 20 October 2009 04:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (153)

You are viewing a single comment's thread. Show more comments above.

Comment author: SilasBarta 20 October 2009 11:11:35PM 0 points [-]

By the way, it seems like this exchange is re-treading my criticism of the concept of could/should/would agent: Since everything, even pebbles, has a workable decomposition into coulds and shoulds, when are they "really" separable? What isn't a CSA?

Comment author: SforSingularity 20 October 2009 11:36:27PM 0 points [-]

As I said,

With the superRoomba, the pressure that the superRoomba applies to the environment doesn't vary as much with the kind of trick you play on it; it will eventually work out what changes you have made, and adapt its strategy so that you end up with a clean floor.

This criterion seems to separate an "inanimate" object like a hydrogen atom or a pebble bouncing around the world from a superRoomba.

Comment author: SilasBarta 20 October 2009 11:43:11PM 0 points [-]

Okay, so the criterion is the extent to which the mechanism screens off environment disturbances from the final result. You used this criterion interchangeably with the issue of whether:

. It has [a] concept of "I am failing to achieve my goals".

Does that have implication for self-awareness and consciousness?

Comment author: SforSingularity 20 October 2009 11:55:54PM 2 points [-]

Does that have implication for self-awareness and consciousness?

Yes, I think so. One prominent hypothesis is that the reason that we evolved with consciousness is that there has to be some way for us to take an overview of the process of us, our goals, and the environment, and the way in which we think that our effort is producing achievement of goals. We need this so that we can do this whole "I am failing to achieve my goals?" check. Why this results in "experience" is not something I am going to attempt in this post.