On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private Jane Street Capital event (crude audio here, from 4:45; better video here [as of July 14]).
I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my similar “win” over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians usually “win” such debates.
Our topic was: Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world. He was pro, I was con. We also debated this subject here on Overcoming Bias from June to December 2008. Let me now try to summarize my current position.
[...]
It thus seems quite unlikely that one AI team could find an architectural innovation powerful enough to let it go from tiny to taking over the world within a few weeks.
I've been trying to put together a survey paper about why uploads coming first is not at all crazy. Whether it's more likely than local-super-AI-in-a-basement, I don't know and leave that to the experts. But brain emulation, as far as we currently understand it, is very similar to the problem posed by landing on the moon circa 1962 -- a matter of scaling it up (though we certainly have much more than a decade to go for mind emulation).
Ken Hayworth, now working in the Lichtman connectomics lab at Harvard, has recently written up such a survey paper to appear in the International Journal of Machine Consciousness this coming summer. I interviewed him and reviewed a preprint of his paper, comparing it with some counter-arguments from Ian Parberry and Paul Allen.
You can find my paper here. I would very much appreciate comments or suggestions if you have them. I'm already planning another section on compression and sensitivity issues with this much data, and also perhaps on ethical questions surrounding the first uploads. This was for a mid-term project for this course, so the scope (due to time constraints, etc.) couldn't be as large or well-addressed as I had hoped.