You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

nyan_sandwich comments on Politics Discussion Thread February 2013 - Less Wrong Discussion

1 Post author: OrphanWilde 06 February 2013 09:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (146)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 09 February 2013 02:48:28AM 1 point [-]

This could be the difference of an entire year that it takes an FAI to implement whatever changes to make society better.

You are not taking AI seriously. Is this intentional?

A superintelligence could likely take over the world in a matter of days, no matter what people thought. (They would think it was great, because the AI could manipulate them better than the best current marketing tactics, even if it couldn't just rewrite their brains with nano.)

It may not do this, for the sake of our comfort, but if anything was urgent, it would be done.

Comment author: Jack 11 February 2013 06:44:35PM 1 point [-]

A superintelligence could likely take over the world in a matter of days, no matter what people thought. (They would think it was great, because the AI could manipulate them better than the best current marketing tactics, even if it couldn't just rewrite their brains with nano.)

While I wouldn't dismiss this possibility at all you seem a little overconfident. The best current marketing tactics can shift market share a percentage point or two or maybe make a half-percentage-point difference in a political campaign. Obviously better than the best is better. But assuming ethical limitations on persuasion tactics and general human suspicion of new things "days" seems pretty optimistic (and twenty-years pessimistic). No good reason to think the persuasive power of marketing is at all linear with the intelligence of the creator. We ought to have very large error bars on this kind of thing and while the focus on these fast take-over scenarios makes sense for emphasizing risk that focus will make them appear more likely to us than they actually are.