Jordan comments on Be a Visiting Fellow at the Singularity Institute - Less Wrong

26 Post author: AnnaSalamon 19 May 2010 08:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (156)

You are viewing a single comment's thread. Show more comments above.

Comment author: Jordan 25 May 2010 08:53:13PM 2 points [-]

But by the time narrow AI becomes pervasive, it's also likely that people will possess much more of the technical understanding needed to comprehend the threat that AGI possesses.

Or perhaps it's the contrary: pervasive narrow AI fosters an undue sense of security. People become comfortable via familiarity, whether it's justified or not. This morning I was peering down a 50 foot cliff, half way up, suspended by nothing but a half inch wide rope. No fear, no hesitation, perfect familiarity. Luckily, due to knowledge of numerous deaths of past climbers I can maintain a conscious alertness to safety and stave off complacency. But in the case of AI, what overt catastrophes will similarly stave off complacency toward existential risk short of an existential catastrophe itself?