Something that is not often discussed explicitly and factors into the different intuitions people have about P(Doom) is how close to optimal biology and humans are in terms of harnessing negative entropy. This consideration pertains to equally to nanobots, ASI and artifical life in general.
Let's consider grey goo first: The race to turn all resources into copies of yourself has been going for a few billion years and is quite competitive. In order to supplant organic life, nanobots would have to either surpass it in carnot efficiency or (more likely) utilise a source of negative entropy thus far untapped. Examples of this previously happening are:
- Photosynthesis
- Aerobic respiration
- Control of fire by early humans
- Agricultural revolution
- Industrial revolution
If, in the designspace of replicators, we are in a local (metastable) optimum and the ability to consume negative entropy, falls off a cliff in a place that is reachable by the synthetic but not organic life, we will get outcompeted quickly. So, are we stumbling in the dark, next to a civilisation swallowing precipice? Would the ASI need to discover new physics or are there already examples of negentropy sources that it could use better than biology?
Efficiency leads to victory only if violence is not an option. Animals are terrible at photosynthesis but survive anyways by taking resources from plants.
A species can invade and dominate an ecosystem by using a strategy that has no current counter. It doesn't need to be efficient. Intelligence allows for playing this game faster than organisms bound by evolution. Humans can make vaccines to fight the spread of a virus despite viruses being one of the fastest adapting threats.
Green goo is plausible not because it would necessarily be more efficient but because it would be using a strategy the existing ecosystem has no defenses to (IE:it's an invasive species).
Likewise AGI that wants to kill all humans could win even if it required 100x more energy per human equivalent instance if it can execute strategies we can't counter. Just being able to copy itself and work with the copies is plausibly enough to allow world takeover with enough scaling.
If we wanted to kill the ants or almost any other organism in nature we mostly have good enough biotech. For anything biotech can't kill, manipulate the environment to kill them all.
Why haven't we? Humans are not sufficiently unified+motivated+advanced to do all these things to ants or other bio life. Some of them are even useful to us. If we sterilized the planet we wouldn't have trees to cut down for wood.
Ants specifically are easy.
Gene drives allow for targeted elimination of a species. Carpet bomb their gene pool with replicating selfish genes. That's ... (read more)