You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Galap comments on Stupid Questions, 2nd half of December - Less Wrong Discussion

2 Post author: Bound_up 23 December 2015 05:31AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (186)

You are viewing a single comment's thread.

Comment author: Galap 24 December 2015 07:46:03AM 1 point [-]

Why do people believe that AI is dangerous? What direct evidence is there that this is likely to be the case?

Comment author: Manfred 24 December 2015 04:39:56PM *  2 points [-]

It's because computers do what you program them to do. If you build an AI with superhuman intelligence and creativity, and the way it makes decisions is to best fulfill some objective, that objective might get fulfilled but everything else might get fubar.

Suppose the objective is "protect the people of Sweden from threats." This AI will almost certainly kill everyone outside Sweden, to eliminate potential threats. As for the survivors, well - what's a "threat?" Does skin cancer or the flu or emotional harm count? What state would you say truly minimizes these threats - that sounds like a coma or a sensory deprivation tank to me.

Comment author: MrMind 24 December 2015 01:01:19PM 2 points [-]

Since we have no AI, we do not have any direct evidence.
The argument though goes like this: AI is orthogonal to purpose, so any sufficiently advanced AI could self-improve and have an exponential impact on our society. But human values are fragile and complex, and if we do not carefully design said AI purpose carefully, it could trample all over them uncaringly.

Comment author: ChristianKl 24 December 2015 09:29:45AM *  3 points [-]

Read Bostrom's Superintelligence. It summarizes all of the main arguments.

Comment author: polymathwannabe 28 December 2015 12:48:46PM 0 points [-]

This is one of MIRI's pivotal papers on the subject.