quartz comments on Q&A with experts on risks from AI #1 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (66)
This sounds right. SIAI communications could probably be improved by acknowledging the incremental nature of AI development more explicitly. Have they addressed how this affects safety concerns?
Science of how to make an AGI is developed gradually, with many prototypes along the way, but the important threshold is where it becomes possible to make a system that can continue open-ended development on its own (if left undisturbed and provided with moderate amount of computing resources). Some time after that point, it may become impossible to stop such a system, and if it ends up developing greater and greater advantage over time, without holding beneficial values, humanity eventually loses. It's the point where the process starts becoming more and more dangerous on its own, until it "explodes" in our faces, like supercritical mass of fissile material.