You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

passive_fist comments on Open thread for December 9 - 16, 2013 - Less Wrong Discussion

5 Post author: NancyLebovitz 09 December 2013 04:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (371)

You are viewing a single comment's thread. Show more comments above.

Comment author: passive_fist 10 December 2013 11:39:57PM 4 points [-]

What you're referring to is a problem I've been thinking about and chipping away at for some time; I've even had some discussions about it here and people have generally been receptive. Maybe the reason you're being downvoted is that you're using the word 'human' to mean 'good'.

The core issue is that humans have empathy, and by this we mean that other people's utility function matters to us. More concisely, our perception of other people's utility forms a part of our utility which is conditionally independent of the direct benefits to us.

Our empathy not only extends to other humans, but also animals and perhaps even robots.

So what are examples of human beings who lack empathy? Lacking empathy is basically the definition of psychopathy. And, indeed, some psychopaths (not all, but some) have been violent criminals who e.g. killed babies for money, tortured people for amusement, etc. etc.

So you're essentially right that a game theory where the players do not have models of each other's utility functions shows aspects of psychopathy and 'inhumanity'.

But that doesn't mean game theory is wrong or 'inhuman'! All it means is that you're missing the 'empathy' ingredient. It also means that it would not be a good idea to build an AI without empathy. That's exactly what CEV attempts to solve. CEV is basically a crude attempt at trying to instill empathy in a machine.

Comment author: Bayeslisk 11 December 2013 01:12:20AM 2 points [-]

Yes, that was what I was getting at. Like I said elsewhere - game theory is not evil. It's just horrifyingly neutral. I am not using inhuman as bad; I am using inhuman as unfriendly.

Comment author: Lumifer 11 December 2013 01:17:34AM 2 points [-]

It's just horrifyingly neutral.

Then you must be horrified by all science.