= 27d567bc2d48d9f40e9ccc20ea370b15)
Vladimir_Nesov comments on Q&A with experts on risks from AI #2 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (28)
XiXiDu: You should clarify this "human-level intelligence" concept, it seems to be systematically causing trouble. For example:
The idea of "human-level intelligence" inspired by science fiction or naive impressions from AI that refers to somewhat human-like AIs is pervasive enough that when better-informed people hear a term like "human-level intelligence", they round up to this cliche and proceed with criticizing it.
Agreed. But not all respondents trash the question just because it's poorly phrased. Nils Nilsson writes:
I really like this guy.
I agree. I think the most important use of the concept is in question 3, and so for timeline purposes we can rephrase "human-level intelligence" as "human-level competence at improving its source code, combined with a structure that allows general intelligence."
Question 3 would then read "What probability do you assign to the possibility of an AGI with human-level competence at improving its source code being able to self-modify its way up to massively superhuman skills in many areas within a matter of hours/days/< 5 years?"
I don't think most AI researchers think of "improving its source code" as one of the benchmarks in an AI research program. Whether or not you think it is, asking them to identify a benchmark that they've actually thought about (I really like Nilson's 80% of human jobs, especially since it jives well with a Hansonian singularity) seems more likely to get an informative response.
Might be worth specifying whether "human-level competence at improving its source code" here means "as good at improving source code as an average professional programmer," "as good at improving source code as an average human," "as good at improving source code as the best professional programmer," or something else.