Comment on The I-Less Eye

Tue, 30 Mar 2010 07:36:11 GMT

Personal identity/anticipated experience is a mechanism through which a huge chunk of preference is encoded in human minds, on an intuitive level. A lot of preference is expressed in terms of "future experience", which breaks down once there is no unique referent for that concept in the future. Whenever you copy human minds, you also copy this mechanism, which virtually guarantees lack of reflective consistency in preference in humans.

Thought experiments with mind-copying effectively involve dramatically changing the agent's values, but don't emphasize this point, as if it's a minor consideration. Getting around this particular implementation, directly to preference represented by it, and so being rational in situations of mind-copying, is not something humans are wired to be able to do.

Discuss