Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

nshepperd comments on The Fun Theory Sequence - Less Wrong

28 Post author: Eliezer_Yudkowsky 25 January 2009 11:18AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (23)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: nshepperd 28 March 2011 10:53:51PM 6 points [-]

We're talking about giving the models subjective experience, not just "emotions". You want the AI to create conscious minds inside itself and torture them to find out whether torture is bad? And then again every time it makes a decision where torture is a conceivable outcome? I'd hope we can give the AI a model that accurately predicts how humans react to stimuli without creating a conscious observer. Humans seem to be able to do that, at least..

Beware of anthropomorphizing AIs. A Really Powerful Optimization Process shouldn't need to "suffer" for us to tell it what suffering is, and that we would like less of it.