Hanson Debating Yudkowsky, Jun 2011

XiXiDu

On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private Jane Street Capital event (crude audio here, from 4:45; better video here [as of July 14]).

I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my similar “win” over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians usually “win” such debates.

Our topic was: Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world. He was pro, I was con. We also debated this subject here on Overcoming Bias from June to December 2008. Let me now try to summarize my current position.

[...]

It thus seems quite unlikely that one AI team could find an architectural innovation powerful enough to let it go from tiny to taking over the world within a few weeks.

Link: overcomingbias.com/2011/07/debating-yudkowsky.html

On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private Jane Street Capital event (crude audio here, from 4:45; better video here [as of July 14]).

I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my similar “win” over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians usually “win” such debates.

Our topic was: Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world. He was pro, I was con. We also debated this subject here on Overcoming Bias from June to December 2008. Let me now try to summarize my current position.

[...]

It thus seems quite unlikely that one AI team could find an architectural innovation powerful enough to let it go from tiny to taking over the world within a few weeks.

Link: overcomingbias.com/2011/07/debating-yudkowsky.html

In particular, you said here that you believe...

Yes, I said that I believe that even sub-human level AI pose an existential risk. At the same time I am highly skeptical of FOOM. So why don't I agree with Eliezer outright anyway? Because the risks from AI that I perceive to be a possibility are not something you can solve by inventing provable "friendliness". How are you going to make a sophisticated monitoring system friendly? Why would people want to make it friendly? How are you going to make a virus with sub-human level general intelligence friendly? Why would one do that? Risks from AI are a broad category that need meta-solutions that involve preemptive political and security measures. You need to make sure that the first intelligent surveillance systems are employed transparently and democratically so that everyone can monitor the world for the various risks ahead. We need a global immune system that keeps care that nowhere someone gets ahead of everyone else.

Have you taken your own survey and published the results somewhere? Or is it only for AI researchers? It seems like there are a great deal of hidden assumptions on all sides which make these discussions go off the tracks very quickly. Some kind of basic survey with standard probability estimates might easily show where views differ.

So why don't I agree with Eliezer outright anyway? Because the risks from AI that I perceive to be a possibility are not something you can solve by inventing provable "friendliness".

I agree that friendliness is a l... (read more)

2hairyfigment15y

Please convince me that your Roboto Protocol could work. I don't observe politics ever producing results like the ones you seem to require.

21

Hanson Debating Yudkowsky, Jun 2011

21

21

21

Hanson Debating Yudkowsky, Jun 2011

21

21