The very popular blog Wait But Why has published the first part of a two-part explanation/summary of AI risks and superintelligence, and it looks like the second part will be focused on Friendly AI. I found it very clear, reasonably thorough and appropriately urgent without signaling paranoia or fringe-ness. It may be a good article to share with interested friends.
Update: Part 2 is now here.
Agree 100%. Wait But Why is very accessible. Previous posts have focused on the Fermi Paradox, procrastination, sentience/consciousness, religion, and immortality. It reads like a very friendly, very accessible Less Wrong.
My one data point:
I don't have much of a formal background in Math, CS or AI. I know a bit from reading LessWrong and Kurzweil though. There weren't any core ideas in the article that were new to me... but Wait But Why has a way of explaining things that is really helpful, and my understanding has definitely been solidified after reading that. Plus it was enjoyable.
There is some debate about how soon AI will reach human-level general intelligence—the median year on a survey of hundreds of scientists was 2040
This is probably a mistake in the article, because the 2040 median is most likely coming from this survey (www.nickbostrom.com/papers/survey.pdf), and specifically coming from when AI experts were asked what year they could be 50% sure of human level AI coming about. For comparison, the 90% interval was 2075, which paints a very different picture.
Edit: the author has fixed the phrasing on that line
This is probably a mistake in the article, because the 2040 median is most likely coming from this survey (www.nickbostrom.com/papers/survey.pdf), and specifically coming from when AI experts were asked what year they could be 50% sure of human level AI coming about. For comparison, the 90% interval was 2075, which paints a very different picture.
Edit: the author has fixed the phrasing on that line