garabik comments on Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (503)
Thinking about AI boxing - note that it is Harry who represents humanity, his core values and goals were not changed that much by the Vow, they were just formalized.
It is LV that has goals that are mostly what we'd agree about (`ensure the continuous existence of the world'), but he has very different values and no moral constraints. In short, dealing with him is like dealing with an Unfriendly AI or an Alien mind (like Sorting Hat).
So this is more like a clash between Unfriendly (or better, Indifferent) and a Friendly AI, where the goals are more or less compatible, but in addition the FAI keeps human values. And the UFAI got there first and is more powerful.
The rational way if your goals are compatible is to cooperate - however, Harry's values almost ensure that he will defect given the chance. And LV knows it, so the rational action for him is to defect (=kill) as well.
Same thing.
‘The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.’