JulianMorrison comments on Would Your Real Preferences Please Stand Up? - Less Wrong

42 Post author: Yvain 08 August 2009 10:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (131)

You are viewing a single comment's thread. Show more comments above.

Comment author: RobinHanson 09 August 2009 02:26:46AM *  10 points [-]

Conscious minds are potentially rational, informed by morality, and qualia-laden. Unconscious minds aren't.

Your entire argument for preferring conscious over unconscious minds is this last quick throw away sentence? That's it? Come on, why can't unconscious minds be rational, informed by morality, or qualia-laden? And why are those the features that matter? Are you really implying this is so completely obvious that this one quick sentence is all that needs to be said? Declaring conscious goals to be the goals of the "person", versus unconscious goals as goals of the genome, just presupposes your answer.

Comment author: JulianMorrison 09 August 2009 11:22:43PM 0 points [-]

Drawing conclusions from what little neuroscience I've managed to overhear, the conscious mind isn't even the part where you live most of the time. It fires up when you're paying it attention. Instantaneous experience ("qualia") exists outside it except the ephemeral experience of being conscious. Playback experience exists inside but references outside it. However it's the part where your human-level intents exist. If all the rest of the brain were copied to a sim, but not the conscious mind, I think you'd have no difficulty labeling that "not me". Miss a comparable sized chunk of your unconscious, and you might not even notice.