Thinking about other people's suffering causes the emotion "concern" (a negative emotion) which is in fact "negative utility".
I agree with this, in general.
If you don't feel concern when faced with the knowledge that someone is in pain, it means that you don't experience "negative utility"
This suggests not only that concern implies negative utility, but that only concern implies negative utility and nothing else (or at least nothing relevant) does. Do you mean to suggest that? If so, I disagree utterly. If not, and you're just restricting the arena of discourse to utility-based-on-concern rather than utility-in-general, then OK... within that restricted context, I agree.
That said, I'm pretty sure you meant the former, and I disagree.
Do you think we have moral responsibilities to AI? Is it immoral to cause a Friendly AI to experience negative utility by fooling it into thinking bad things are happening and then killing it?
Maybe, but not necessarily. It depends on the specifics of the AI.
If it's true that negative affect and negative utility are roughly synonymous, it's impossible to make a being that negatively values torture and doesn't feel bad when seeing torture.
Yes, that follows. I think both claims are false.
you can't have a human which is constantly experiencing maximum positive affect, because that human would never have an incentive to act at all.
I agree that in human minds, differential affect motivates action; if we eliminate all variation in affect we eliminate that motive for action, which either requires that we find another motivation for action, or (as you suggest) we eliminate all incentives for action.
Are there other motivations?
Are there situations under which the lack of such incentives is acceptable?
If not, and you're just restricting the arena of discourse to utility-based-on-concern rather than utility-in-general, then OK... within that restricted context, I agree.
yes...we agree
If it's true that negative affect and negative utility are roughly synonymous, it's impossible to make a being that negatively values torture and doesn't feel bad when seeing torture.
Shit I'm in a contradiction. Okay, I've messed up by using "affect" under multiple definitions, my mistake.
Reformatting...
1) There are many mechanisms for creating beings that c...
People want to tell everything instead of telling the best 15 words. They want to learn everything instead of the best 15 words. In this thread, instead post the best 15-words from a book you've read recently (or anything else). It has to stand on its own. It's not a summary, the whole value needs to be contained in those words.
I'll start in the comments below.
(Voted by the Schelling study group as the best exercise of the meeting.)