The following seems like a very obvious observation, but I don't recall seeing it made so far.
Tom Riddle Jr #1 (aka Lord Voldemort, aka Quirinus Quirrell) attempted to create an intelligent being in his own image, with the intention that it would share his values and cooperate with him in bringing about the sort of future he wanted.
This didn't go well, for reasons including (1) that Tom Riddle Jr #2 turned out not to share TR#1's values after all, and (2) that TR#2 developed new abilities TR#1 never suspected and that TR#2 was able to use against TR#1 when they happened to come into conflict.
Oh, and TR#2 might just destroy the world.
Application of all this to a problem Eliezer is known to be interested in is left as an exercise to the interested reader.
(The suggestion that Harry might be kinda like an UFAI has been made here before and my response was that Quirrelmort was a better candidate. But Quirrelmort is probably out of the picture at this point.)
This is a new thread to discuss Eliezer Yudkowsky’s Harry Potter and the Methods of Rationality and anything related to it. This thread is intended for discussing chapter 116.
There is a site dedicated to the story at hpmor.com, which is now the place to go to find the authors notes and all sorts of other goodies. AdeleneDawner has kept an archive of Author’s Notes. (This goes up to the notes for chapter 76, and is now not updating. The authors notes from chapter 77 onwards are on hpmor.com.)