JoshuaZ comments on Tallinn-Evans $125,000 Singularity Challenge - Less Wrong

27 Post author: Kaj_Sotala 26 December 2010 11:21AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (369)

You are viewing a single comment's thread. Show more comments above.

Comment author: JoshuaZ 29 December 2010 05:44:06AM 2 points [-]

This doesn't address the most controversial aspect, which is that AI would go foom. If extreme fooming doesn't occur this isn't nearly as big an issue. That is an issue where many people have discussed it and not all have come away convinced. Robin Hanson had a long debate with Eliezer over this and Robin was not convinced. Personally, I consider fooming to be unlikely but plausible. But how likely one thinks it is matters a lot.

Comment author: Rain 29 December 2010 06:04:42PM 4 points [-]

This doesn't address the most controversial aspect, which is that [nuclear weapons] would [ignite the atmosphere]. If extreme [atmospheric ignition] doesn't occur this isn't nearly as big an issue.

Even without foom, AI is a major existential risk, in my opinion.

Comment author: shokwave 29 December 2010 07:47:17AM 0 points [-]

Foom is included in that proof concept. Human intelligence has made faster and faster computation; a human intelligence sped up could reasonably expect to increase the speed and amount of computation available to it; resulting in faster speeds, and so on.

Comment author: JoshuaZ 29 December 2010 01:53:27PM 4 points [-]

You are repeating what amounts to a single cached thought. The claim in question is that there's enough evidence to convince a skeptic. Giving a short line of logic for that isn't at all the same. Moreover, the claim that such evidence exists is empirically very hard to justify given the Yudkowsky-Hanson debate. Hanson is very smart. Eliezer did his best to present a case for AI going foom. He didn't convince Hanson.

Comment author: shokwave 29 December 2010 02:59:16PM 1 point [-]

You are repeating what amounts to a single cached thought.

I'm not allowed to cache thoughts that are right?

You seem to be taking "Hanson disagreed with Eliezer" as proof that all evidence Eliezer presented doesn't amount to FOOM.

I'd note here that I started out learning from this site very skeptical, treating "I now believe in the Singularity" as a failure mode of my rationality, but something tells me you'd be suspicious of that too.

Comment author: JoshuaZ 29 December 2010 07:10:59PM *  3 points [-]

I'm not allowed to cache thoughts that are right?

You are. But when people ask for evidence it is generally more helpful to actually point to the evidence rather than simply repeating a secondary cached thought that is part of the interpretation of the evidence.

You seem to be taking "Hanson disagreed with Eliezer" as proof that all evidence Eliezer presented doesn't amount to FOOM.

No. I must have been unclear. I'm pointing to the fact that there are people who are clearly quite smart and haven't become convinced by the claim after looking at it in detail. Which means that when someone like XiXiDu asks where the evidence is a one paragraph summary with zero links is probably not going to be sufficient.

I'd note here that I started out learning from this site very skeptical, treating "I now believe in the Singularity" as a failure mode of my rationality, but something tells me you'd be suspicious of that too.

I'm not suspicious of it. My own estimate for fooming has gone up since I've spent time here (mainly due to certain arguments made by cousin_it), but I don't see why you think I'd be suspicious or not. Your personal opinion or my personal opinion just isn't that relevant when someone has asked "where's the evidence?" Maybe our personal opinions with all the logic and evidence drawn out in detail might matter. But that's a very different sort of thing.