Students asked to defend AGI danger update in favor of AGI riskiness

lukeprog

In the Spring semester of 2011, I decided to see how effectively I could communicate the idea of a threat from AGI to my undergraduate classes. I spent three sessions on this for each of my two classes. My goal was to convince my students that all of us are going to be killed by an artificial intelligence. My strategy was to induce the students to come up with the ideas themselves. I gave out a survey before and after. An analysis of the survey responses indicates that the students underwent a statistically significant shift in their reported attitudes. After the three sessions, students reported believing that AGI would have a larger impact1 and also a worse impact2 than they originally reported believing.

Not a surprising result, perhaps, but the details of how Geoff taught AGI danger and the reactions of his students are quite interesting.

From Geoff Anders of Leverage Research:

In the Spring semester of 2011, I decided to see how effectively I could communicate the idea of a threat from AGI to my undergraduate classes. I spent three sessions on this for each of my two classes. My goal was to convince my students that all of us are going to be killed by an artificial intelligence. My strategy was to induce the students to come up with the ideas themselves. I gave out a survey before and after. An analysis of the survey responses indicates that the students underwent a statistically significant shift in their reported attitudes. After the three sessions, students reported believing that AGI would have a larger impact1 and also a worse impact2 than they originally reported believing.

Not a surprising result, perhaps, but the details of how Geoff taught AGI danger and the reactions of his students are quite interesting.

Some computer programs crash - just as some possible superintelligences would kill all humans.

No *most" computer programs crash [...]

By "no", you apparently mean "yes".

With a self-modifying AI this is a lot harder to do.

Well, that is a completely different argument - and one that would appear to be in need of supporting evidence - since automated testing, linting and the ability to program in high-level languages are all improving simultaneously.

I am not aware of any evidence that real computer programs are getting more crash-prone with the passage of time.

With a self-modifying AI this is a lot harder to do.

Well, that is a completely different argument - and one that would appear to be in need of supporting evidence - since automated testing, linting and the ability to program in high-level languages are all improving simultaneously.

The point is that the first time you run the seed AI it will attempt to take over the world, so you don't have the luxury of debugging it.

7

Students asked to defend AGI danger update in favor of AGI riskiness

7

7

7

Students asked to defend AGI danger update in favor of AGI riskiness

7

7