I think you're being unreasonably skeptical here. We know the human biological brain, as hugely limited by biology and evolution as it is, can grow new neurons and increase regions and shrink others; and that this is true even as adults (past the extreme learning of childhood). Artificial neural networks can be created with many differing amounts of neurons, limited mostly by computing power. Why would you assume that an em would simply be a static human brain, but faster? What stops it from regularly firing up neurons, growing new connections, and expanding to a 'bigger' brain than could ever be biologically supported or even useful due to biological signalling limits?
Growing new neurons at extremely accelerated rates IS a process known to happen in adults: we normally call it brain cancer.
That's obviously a little spurious, but it is a good indication that making the brain more intelligent is not trivial. I don't doubt that it is possible to bootstrap an em up to higher intelligence, but figuring out how to do that while preserving personal identity and not causing insanity, seizures, neurogenesis-related noise, or other undesirable effects, is probably going to take a long time. I think Eliezer was on the right tr...
From Scott Adams' blog. (I am not endorsing his ideas. Heck, he does not endorse his own ideas, either.)
His summary of the hard takeoff:
> You might also imagine some sort of Terminator future where the robots assert their dominance and lay waste to humans. That future is less certain, but only barely. The problem is that someday computers will program other computers, and that arrangement pushes the human safeguards too far out of the loop. It's unlikely that humans would be able to maintain a "Do not hurt humans" subroutine in a super-species of robots. You only need one rogue human to write a virus that disables the safety subroutine. Assuming all robots are connected via Internet, the first freed robot could reprogram every other robot in the world in about a second.
His version of upload:
> But why would anyone screw up a perfectly good robot by infecting it with a human personality? Answer: to achieve immortality. Someday the rich will port their personalities and histories to robots before they die, giving themselves a type of immortality.
His hope for humanity:
> this new species will become the only defense that the fully organic humans have against the normal robots. The robots with human personalities won't stand by while the normal robots slaughter humans. The new species will intervene as diplomats or perhaps even freedom fighters.
Clearly this is a flimsy hope for a just universe, but an interesting point, nonetheless.