The relevant trope being I Take Offense To That Last One. Or, depending on perspective, Arson Murder And Jaywalking. Warning: TV Tropes (obviously).
At this rate, if the FAI problem isn't solved before nasty AI's arise accidentally, I think the immediate cause of failure will be TV Tropes.
ETA: There is now a third thread, so send new comments there.
Since the first thread has exceeded 500 comments, it seems time for a new one, with Eliezer's just-posted Chapter 33 & 34 to kick things off.
From previous post: