All the rational arguments against it are now buried in a downvoted and deleted thread
Exactly right. The comment by CarlShuman is valuable. To the extent that it warrants a thread.
What if anyone gets killed as a result of not reading this thread because he was never exposed to the arguments of why it would be a really bad idea to violently oppose AGI research?
Passionately suppressing the conversation could also convey a message of "Shush. Don't tell anyone." as well as showing you take the idea seriously. This is in stark contrast to signalling that you think the whole idea is just silly, because reasoning like Carl's is so damn obvious.
It's probably easier to build an uncaring AI than a friendly one. So, if we assume that someone, somewhere is trying to build an AI without solving friendliness, that person will probably finish before someone who's trying to build a friendly AI.
[redacted]
[redacted]
further edit:
Wow, this is getting a rather stronger reaction than I'd anticipated. Clarification: I'm not suggesting practical measures that should be implemented. Jeez. I'm deep in an armchair, thinking about a problem that (for the moment) looks very hypothetical.
For future reference, how should I have gone about asking this question without seeming like I want to mobilize the Turing Police?