Michael:
Underestimating the significance of superintelligence. People have a delusion that humanity is some theoretically optimum plateau of intelligence (due to brainwashing from Judeo-Christian theological ideas, which also permeate so-called “secular humanism”), which is the opposite of the truth. We’re actually among the stupidest possible species smart enough to launch a civilization.
This doesn't seem to be a part of standard Christian or Jewish theology, so blaming that attitude on this seems misguided. His last sentence is also problematic- how does he know that with a sample size of one?
Michael:
Would you rather your AI be based on Hitler or Gandhi?
Seems to understate the case. Mindspace is large. The problem isn't an AI that acts like Hitler. That's not such a bad failure as things go. The worst case scenario more resembles Cthulhu than Hitler.
Anissimov correctly calls out Goertzel on his claim that he's sure he could design a properly functioning nanny AI.
Goertzel does correctly point out that it seems likely that a nanny AI would take less understanding than a full Friendly AI with stable goals under self-modification.
We’re actually among the stupidest possible species smart enough to launch a civilization.
His last sentence is also problematic- how does he know that with a sample size of one?
You are demanding particular proof. This claim is rooted in some theoretical considerations that don't obviously rely on directly sampling things.
http://hplusmagazine.com/2011/04/20/mitigating-the-risks-of-artificial-superintelligence/