You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

garabik comments on Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113 - Less Wrong Discussion

8 Post author: Gondolinian 28 February 2015 08:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (503)

You are viewing a single comment's thread.

Comment author: garabik 01 March 2015 09:41:44AM *  4 points [-]

Thinking about AI boxing - note that it is Harry who represents humanity, his core values and goals were not changed that much by the Vow, they were just formalized.

It is LV that has goals that are mostly what we'd agree about (`ensure the continuous existence of the world'), but he has very different values and no moral constraints. In short, dealing with him is like dealing with an Unfriendly AI or an Alien mind (like Sorting Hat).

So this is more like a clash between Unfriendly (or better, Indifferent) and a Friendly AI, where the goals are more or less compatible, but in addition the FAI keeps human values. And the UFAI got there first and is more powerful.

The rational way if your goals are compatible is to cooperate - however, Harry's values almost ensure that he will defect given the chance. And LV knows it, so the rational action for him is to defect (=kill) as well.

Comment author: TobyBartels 03 March 2015 03:06:04AM 1 point [-]

Unfriendly (or better, Indifferent)

Same thing.

‘The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.’