For some reason no one wants to hold Eric Drexler accountable now for the grandiose, irresponsible and frankly cringe-worthy things he wrote back in the 1980's.
Case in point. I turned 27 in 1986, the year Drexler published Engines of Creation, so I belong to the generation referred to in the following speculation:
http://e-drexler.com/d/06/00/EOC/EOC_Chapter_8.html
Imagine someone who is now thirty years old [in 1986]. In another thirty years, biotechnology will have advanced greatly, yet that thirty-year-old will be only sixty. Statistical tables which assume no advances in medicine say that a thirty-year-old U.S. citizen can now expect to live almost fifty more years - that is, well into the 2030s. Fairly routine advances (of sorts demonstrated in animals) seem likely to add years, perhaps decades, to life by 2030. The mere beginnings of cell repair technology might extend life by several decades. In short, the medicine of 2010, 2020, and 2030 seems likely to extend our thirty-year-old's life into the 2040s and 2050s. By then, if not before, medical advances may permit actual rejuvenation. Thus, those under thirty (and perhaps those substantially older) can look forward - at least tentatively - to medicine's overtaking their aging process and delivering them safely to an era of cell repair, vigor, and indefinite life-span.
I turn 54 this November, and I can assure you that no one in my generation has seen "medicine's overtaking their aging process."
Yet many cryonicists have bet their futures on this fantasy technology, when regular people can see that it has taken on the characteristics of an apocalyptic religious belief instead of a rational assessment of future capabilities. Cryonicist Thomas Donaldson warned that this would happen and not help cryonics' credibility, back around the time Drexler predicted that I would start to grow younger by now.
Apparently Drexler wants to reboot his reputation with a new book, but someone needs to remind people about the things he promised us in his 1980's-era writings which haven't come to pass.
Molecular nanotechnology, or MNT for those of you who love acronyms, seems to be a fairly common trope on LW and related literature. It's not really clear to me why. In many of the examples of "How could AI's help us" or "How could AI's rise to power" phrases like "cracks protein folding" or "making a block of diamond is just as easy as making a block of coal" are thrown about in ways that make me very very uncomfortable. Maybe it's all true, maybe I'm just late to the transhumanist party and the obviousness of this information was with my invitation that got lost in the mail, but seeing all the physics swept under the rug like that sets off every crackpot alarm I have.
I must post the disclaimer that I have done a little bit of materials science, so maybe I'm just annoyed that you're making me obsolete, but I don't see why this particular possible future gets so much attention. Let us assume that a smarter than human AI will be very difficult to control and represents a large positive or negative utility for the entirety of the human race. Even given that assumption, it's still not clear to me that MNT is a likely element of the future. It isn't clear to me than MNT is physically practical. I don't doubt that it can be done. I don't doubt that very clever metastable arrangements of atoms with novel properties can be dreamed up. Indeed, that's my day job, but I have a hard time believing the only reason you can't make a nanoassembler capable of arbitrary manipulations out of a handful of bottles you ordered from Sigma-Aldrich is because we're just not smart enough. Manipulating individuals atoms means climbing huge binding energy curves, it's an enormously steep, enormously complicated energy landscape, and the Schrodinger Equation scales very very poorly as you add additional particles and degrees of freedom. Building molecular nanotechnology seems to me to be roughly equivalent to being able to make arbitrary lego structures by shaking a large bin of lego in a particular way while blindfolded. Maybe a super human intelligence is capable of doing so, but it's not at all clear to me that it's even possible.
I assume the reason than MNT is added to a discussion on AI is because we're trying to make the future sound more plausible via adding burdensome details. I understand that AI and MNT is less probable than AI or MNT alone, but that both is supposed to sound more plausible. This is precisely where I have difficulty. I would estimate the probability of molecular nanotechnology (in the form of programmable replicators, grey goo, and the like) as lower than the probability of human or super human level AI. I can think of all sorts of objection to the former, but very few objections to the latter. Including MNT as a consequence of AI, especially including it without addressing any of the fundamental difficulties of MNT, I would argue harms the credibility of AI researchers. It makes me nervous about sharing FAI literature with people I work with, and it continues to bother me.
I am particularly bothered by this because it seems irrelevant to FAI. I'm fully convinced that a smarter than human AI could take control of the Earth via less magical means, using time tested methods such as manipulating humans, rigging elections, making friends, killing its enemies, and generally only being a marginally more clever and motivated than a typical human leader. A smarter than human AI could out-manipulate human institutions and out-plan human opponents with the sort of ruthless efficiency that modern computers beat humans in chess. I don't think convincing people that smarter than human AI's have enormous potential for good and evil is particularly difficult, once you can get them to concede that smarter than human AIs are possible. I do think that waving your hands and saying super-intelligence at things that may be physically impossible makes the whole endeavor seem less serious. If I had read the chain of reasoning smart computer->nanobots before I had built up a store of good-will from reading the Sequences, I would have almost immediately dismissed the whole FAI movement a bunch of soft science fiction, and it would have been very difficult to get me to take a second look.
Put in LW parlance, suggesting things not known to be possible by modern physics without detailed explanations puts you in the reference class "people on the internet who have their own ideas about physics". It didn't help, in my particular case, that one of my first interactions on LW was in fact with someone who appears to have their own view about a continuous version of quantum mechanics.
And maybe it's just me. Maybe this did not bother anyone else, and it's an incredible shortcut for getting people to realize just how different a future a greater than human intelligence makes possible and there is no better example. It does alarm me though, because I think that physicists and the kind of people who notice and get uncomfortable when you start invoking magic in your explanations may be the kind of people FAI is trying to attract.