To be honest I only did a brief read through. The context of the debate itself is what I object to. I find the concept of "friendly" AI itself to be terrifying. It's my life work to make sure that we don't end up in such a dystopian tyrannical future. Debating the probabilities of whether what you call AI "risk" is likely or unlikely (disjunctive or conjunctive) is rather pointless when you are ambivalent towards that particular outcome.
Now I think that you've left out a LOT of things that must happen a certain way in order for your AI risk outcomes to come to pass. You've also left out ALL of the corrective actions that could be taken by any of the human actors in the picture. It reminds me of a martial arts demonstration where the attacker throws a punch and then stands there in frozen form, unreactive while the teacher demonstrate the appropriate response at leisure. But if like me you don't see such a scenario as a bad thing in the first place, then it's an academic point. And I tire of debating things of no real world significance.
Hmm. There may have been a miscommunication here.
The context of the debate itself is what I object to. I find the concept of "friendly" AI itself to be terrifying.
This sounds like you're assuming that I'm trying to argue in favor of Friendly AI as the best solution. Now I admittedly do currently find FAI one of the most promising options for trying to navigate AI risk, but I'm not committed to that. I just want to find whatever solution works, regardless of whether it happens to be FAI or something completely else. But in order to find out wh...
Arguments for risks from general AI are sometimes criticized on the grounds that they rely on a series of linear events, each of which has to occur for the proposed scenario to go through. For example, that a sufficiently intelligent AI could escape from containment, that it could then go on to become powerful enough to take over the world, that it could do this quickly enough without being detected, etc.
The intent of my following series of posts is to briefly demonstrate that AI risk scenarios are in fact disjunctive: composed of multiple possible pathways, each of which could be sufficient by itself. To successfully control the AI systems, it is not enough to simply block one of the pathways: they all need to be dealt with.
I've got two posts in this series up so far:
AIs gaining a decisive advantage discusses four different ways by which AIs could achieve a decisive advantage over humanity. The one-picture version is:
AIs gaining the power to act autonomously discusses ways by which AIs might come to act as active agents in the world, despite possible confinement efforts or technology. The one-picture version (which you may wish to click to enlarge) is:
These posts draw heavily on my old paper, Responses to Catastrophic AGI Risk, as well as some recent conversations here on LW. Upcoming posts will try to cover more new ground.