Jordan comments on Be a Visiting Fellow at the Singularity Institute - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (156)
EDIT: I've realized that some misinterpretation of my arguments has been due to disagreements in terminology. I define "expert systems" as systems designed to address a specific class of well-defined problems, capable of logical reasoning and probabilistic inference given a set of "axiom-like" rules, and updating their knowledge database with specific kinds of information.
AGI I define specifically as AI which has human or extra-human level capabilities, or the potential to reach those capabilities.
Now my response to the above:
"Expert AI systems are already used in hospitals, and will surely be used more and more as the technology progresses. There isn't a single point where AI is suddenly better than humans at all aspects of a field. Current AIs are already better than doctors in some areas, but worse in many others. As the range of AI expertise increases doctors will shift more towards managerial roles, understanding the strengths and weakness of the myriad expert systems, refereeing between them and knowing when to overrule them."
I agree with all of these.
"By the time true AGI arrives narrow AI will probably be pervasive enough that the line between the two will be too fuzzy to allow for a naive ban on AGI."
To me it seems the greatest enabler of AI catastrophe is ignorance. But by the time narrow AI becomes pervasive, it's also likely that people will possess much more of the technical understanding needed to comprehend the threat that AGI possesses.
"Moreover, I highly doubt people are going to vote to save jobs (especially jobs of the affluent) at the expense of human life."
You are being too idealistic here.
Or perhaps it's the contrary: pervasive narrow AI fosters an undue sense of security. People become comfortable via familiarity, whether it's justified or not. This morning I was peering down a 50 foot cliff, half way up, suspended by nothing but a half inch wide rope. No fear, no hesitation, perfect familiarity. Luckily, due to knowledge of numerous deaths of past climbers I can maintain a conscious alertness to safety and stave off complacency. But in the case of AI, what overt catastrophes will similarly stave off complacency toward existential risk short of an existential catastrophe itself?