Alicorn comments on By Which It May Be Judged - LessWrong

35 Post author: Eliezer_Yudkowsky 10 December 2012 04:26AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (934)
Tags: seq_epistemology metaethics fairness

Comments (934)

You are viewing a single comment's thread. Show more comments above.

Comment author: handoflixue 11 December 2012 09:36:46PM 5 points [-]

Ooh, good call, thank you. I suppose it might be akin to visualization, where it actually varies from person to person. Does anyone here on LessWrong have conflicting anecdotes, though? Does anyone disagree with what I said? If not, it seems like a safe generalization for now, but it's still useful to remember I'm generalizing from one example :)

Remembering that other people have genuinely alien minds is surprisingly tricky.

Comment author: Alicorn 11 December 2012 10:34:49PM 8 points [-]

The other side of this is that I would expect my brain to NOTICE it's actual goals. If my goal is to make paperclips, I will think "I should do this because it makes paperclips", instead of "I should do this because it makes people happy". My brain doesn't have a generic "I should do this" emotion, as near as I can tell - it just has ways of signalling that an activity will accomplish my goals.

Iron deficiency feels like wanting ice. For clever, verbal reasons. Not being iron deficient doesn't feel like anything. My brain did not notice that it was trying to get iron - it didn't even notice it was trying to get ice, it made up reasons according to which ice was an instrumental value for some terminal goal or other.

Powered by Reddit Powered by Reddit