Hanson Debating Yudkowsky, Jun 2011

XiXiDu

On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private Jane Street Capital event (crude audio here, from 4:45; better video here [as of July 14]).

I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my similar “win” over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians usually “win” such debates.

Our topic was: Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world. He was pro, I was con. We also debated this subject here on Overcoming Bias from June to December 2008. Let me now try to summarize my current position.

[...]

It thus seems quite unlikely that one AI team could find an architectural innovation powerful enough to let it go from tiny to taking over the world within a few weeks.

Link: overcomingbias.com/2011/07/debating-yudkowsky.html

On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private Jane Street Capital event (crude audio here, from 4:45; better video here [as of July 14]).

I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my similar “win” over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians usually “win” such debates.

Our topic was: Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world. He was pro, I was con. We also debated this subject here on Overcoming Bias from June to December 2008. Let me now try to summarize my current position.

[...]

It thus seems quite unlikely that one AI team could find an architectural innovation powerful enough to let it go from tiny to taking over the world within a few weeks.

Link: overcomingbias.com/2011/07/debating-yudkowsky.html

Have you taken your own survey and published the results somewhere? Or is it only for AI researchers? It seems like there are a great deal of hidden assumptions on all sides which make these discussions go off the tracks very quickly. Some kind of basic survey with standard probability estimates might easily show where views differ.

So why don't I agree with Eliezer outright anyway? Because the risks from AI that I perceive to be a possibility are not something you can solve by inventing provable "friendliness".

I agree that friendliness is a long shot. If you know of a better solution, please let me know.

How are you going to make a sophisticated monitoring system friendly?

By developing a theory of friendliness and implementing it in software.

Why would people want to make it friendly?

Because unfriendly things are bad.

How are you going to make a virus with sub-human level general intelligence friendly?

By developing a theory of friendliness and implementing it in software.

Why would one do that?

Because unfriendly things are bad.

Risks from AI are a broad category that need meta-solutions that involve preemptive political and security measures. You need to make sure that the first intelligent surveillance systems are employed transparently and democratically so that everyone can monitor the world for the various risks ahead. We need a global immune system that keeps care that nowhere someone gets ahead of everyone else.

Sounds like a job for CEV and a friendly AI to run it on.

Have you taken your own survey and published the results somewhere?

Yes I have done so. But I don't trust my ability to make correct probability estimates, don't trust the overall arguments and methods and don't know how to integrate that uncertainty into my estimates. It is all too vague.

There sure are a lot of convincing arguments in favor of risks from AI. But do arguments suffice? Nobody is an expert when it comes to intelligence. Even worse, I don't think anybody knows much about artificial general intelligence.

My problem is that I fear that some co... (read more)

21

Hanson Debating Yudkowsky, Jun 2011

21

21

21

Hanson Debating Yudkowsky, Jun 2011

21

21