You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

steven0461 comments on against "AI risk" - Less Wrong Discussion

24 Post author: Wei_Dai 11 April 2012 10:46PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (89)

You are viewing a single comment's thread. Show more comments above.

Comment author: steven0461 12 April 2012 12:21:55AM 1 point [-]

Nanotech x-risk would seem to come out of mass-producing weapons that kill survivors of an all out war (which leaves neither side standing), like systems that could replicate in the wild and destroy the niche of primitive humans, really numerous robotic weapons that would hunt down survivors over time, and such like.

What about takeover by an undesirable singleton? Also, if nanotechnology enables AI or uploads, that's an AI risk, but it might still involve unique considerations we don't usually think to talk about. The opportunities to reduce risk here have to be very small to justify LessWrong's ignoring the topic almost entirely, as it seems to me that it has. The site may well have low-hanging conceptual insights to offer that haven't been covered by CRN or Foresight.

Comment author: CarlShulman 12 April 2012 12:34:21AM *  6 points [-]

to justify LessWrong's ignoring the topic

That's a much lower standard than "should Luke make this a focus when trading breadth vs speed in making his document". If people get enthused about that, they're welcome to. I've probably put 50-300 hours (depending on how inclusive a criterion I use for relevant hours) into the topic, and saw diminishing returns. If I overlap with Eric Drexler or such folk at a venue I would inquire, and I would read a novel contribution, but I'm not going to be putting much into it given my alternatives soon.

Comment author: steven0461 12 April 2012 01:09:22AM 3 points [-]

I agree that it's a lower standard. I didn't mean to endorse Wei's claims in the original post, certainly not based on nanotech alone. If you don't personally think it's worth more of your time to pay attention to nanotech, I'm sure you're right, but it still seems like a collective failure of attention that we haven't talked about it at all. You'd expect some people to have a pre-existing interest. If you ever think it's worth it to further describe the conclusions of those 50-300 hours, I'd certainly be curious.

Comment author: CarlShulman 12 April 2012 01:12:12AM 2 points [-]

I'll keep that in mind.