Hanson Debating Yudkowsky, Jun 2011

XiXiDu

On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private Jane Street Capital event (crude audio here, from 4:45; better video here [as of July 14]).

I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my similar “win” over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians usually “win” such debates.

Our topic was: Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world. He was pro, I was con. We also debated this subject here on Overcoming Bias from June to December 2008. Let me now try to summarize my current position.

[...]

It thus seems quite unlikely that one AI team could find an architectural innovation powerful enough to let it go from tiny to taking over the world within a few weeks.

Link: overcomingbias.com/2011/07/debating-yudkowsky.html

On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private Jane Street Capital event (crude audio here, from 4:45; better video here [as of July 14]).

I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my similar “win” over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians usually “win” such debates.

Our topic was: Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world. He was pro, I was con. We also debated this subject here on Overcoming Bias from June to December 2008. Let me now try to summarize my current position.

[...]

It thus seems quite unlikely that one AI team could find an architectural innovation powerful enough to let it go from tiny to taking over the world within a few weeks.

Link: overcomingbias.com/2011/07/debating-yudkowsky.html

Favorable? I don't know why you'd think that. Seems to me the charitable interpretation of Hanson's view has him thinking of ems as naturally Friendly, or near-Friendly

You would have to tell me what friendly and unfriendly means in this context. Hanson expects ems to be very numerous and very poor. I doubt he expects any one of them to have the resources available to what's usually called an fai. Is a human being running at human speeds F or UF?

If we somehow get trillions of uFAIs with practically different goals, then of course the expected value of killing humanity goes way down. But it still sounds greater than the expected value of cooperating with us, by Hanson's analysis.

I don't think the notion of "cooperating with us" is coherent. Just as the trillions of ems might have practically different goals, so might the billions of live humans.

I also don't understand how you get that much memory and processing power without some designer that seems awfully close to an AI-programming AI.

Possibly, being poor, they would not have that much memory and processing power.

Taking the last part first for context: this layman thinks that just simulating a conscious brain (experiencing something other than pure terror or slow insanity) would take a lot of resources using the copy-an-airplane-with-bullet-holes approach where you don't know what the parts actually do, at least not well enough to make a self-reflective programming AI from scratch.

As to the rest, I'm assuming my previous claims hold for the case of a single AGI because you seemed to argue that simply introducing a lot more AGIs changes the argument. ("Cooperat... (read more)

21

Hanson Debating Yudkowsky, Jun 2011

21

21

21

Hanson Debating Yudkowsky, Jun 2011

21

21