Where can I find some precise, well thought out reasons why the risk of human extinction from strong AI is not just small, but for practical purposes equal to 0? I am interested in both arguments from people who believe the risk is zero, and people who do not believe this, but still attempt to "steel man" the argument.
The primary disagreement, in the steel man universe, is over urgency. If one knew that we would make AGI in 2200, then one would be less worried about solving the friendliness problem now. If one knew that we would make AGI in 2020, then one would be very worried about solving the friendliness problem now.
For many people who work on AI, it's hard to believe that it will 'just start working' at that high level of ability soon, because of how optimistic AI proponents have been over the years and how hard it is to wring a bit more predictive accuracy out of the algorithms for their problems.
But if one takes the position not that one is certain that it will happen soon, but that one is uncertain when it will happen, the uncertainty implies that it will happen sooner or later, and that means we need to do some planning for the sooner case. (That is, uncertainty does not imply it can only happen a long time from now.) This is, it seems, the most effective way to communicate with people who aren't worried about Strong AI.
There is also the question of what should this type of research actually look like.
A few notes about the site mechanics
A few notes about the community
If English is not your first language, don't let that make you afraid to post or comment. You can get English help on Discussion- or Main-level posts by sending a PM to one of the following users (use the "send message" link on the upper right of their user page). Either put the text of the post in the PM, or just say that you'd like English help and you'll get a response with an email address.
* Normal_Anomaly
* Randaly
* shokwave
* Barry Cotter
A list of some posts that are pretty awesome
I recommend the major sequences to everybody, but I realize how daunting they look at first. So for purposes of immediate gratification, the following posts are particularly interesting/illuminating/provocative and don't require any previous reading:
More suggestions are welcome! Or just check out the top-rated posts from the history of Less Wrong. Most posts at +50 or more are well worth your time.
Welcome to Less Wrong, and we look forward to hearing from you throughout the site!
Note from Clarity: MBlume and other contributors wrote the original version of this welcome post, and orthonormal edited it a fair bit. If there's anything I should add or update please send me a private message or make the change by making the next thread—I may not notice a comment on the post. Finally, once this gets past 500 comments, anyone is welcome to copy and edit this intro to start the next welcome thread.