In a recent essay, Brian Tomasik argues that meme-spreading has higher expected utility than x-risk reduction. His analysis assumes a classical utilitarian ethic, but it may be generalizable to other value systems. Here's the summary:
I personally do not support efforts to reduce extinction risk because I think space colonization would potentially give rise to astronomical amounts of suffering. However, even if I thought reducing extinction risk was a good idea, I would not work on it, because spreading your particular values has generally much higher leverage than being one more voice for safety measures against extinction in a world where reducing extinction risk is hard and almost everyone has some incentives to invest in the issue.
Ok, bad wording on my part. I meant "more generally intelligent."
I was imagining people would destroy their computers, except the ones not connected to the Internet. However, if the AGI is hiding itself, it could go a long way before people realized what was going on.
Interesting scenarios. Thanks!
Exactly. On the one hand the AGI doesn't try to let humans get wind of it's plans. On the other hand it's going to produce distractions.
You have to remember how delusional some folks are. Imaging trying to convince the North Korean's that they have to destroy their computers because those computer are infested with an evil AI.
Even in the US nearly half of the population still believes in creationism. How many of them can be convinced that the evil gove... (read more)