Roko comments on Changing accepted public opinion and Skynet - Less Wrong

15 [deleted] 22 May 2009 11:05AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (58)

You are viewing a single comment's thread. Show more comments above.

Comment author: derekz 22 May 2009 12:49:39PM 6 points [-]

One thing that might help change the opinion of people about friendly AI is to make some progress on it. For example, if Eliezer has had any interesting ideas about how to do it in the last five years of thinking about it, it could be helpful to communicate them.

A case that is credible to a large number of people needs to be made that this is a high-probability near-term problem. Without that it's just a scary sci-fi movie, and frankly there are scarier sci-fi movie concepts out there (e.g. bioterror). Making an analogy with a nuclear bomb is simply not an effective argument. People were not persuaded about global warming with a "greenhouse" analogy. That sort of thing creates a sort of dim level of awareness, but "AI might kill us" is not some new idea; everybody is already aware of that -- just like they are aware that a meteor might wipe us out, aliens might invade, or an engineered virus or new life form could kill us all. Which of those things get attention from policy-makers and their advisers, and why?

Besides the weakness of relying on analogy, this analogy isn't even all that good -- it takes concerted and advanced targeted technical dedication to make a nuclear FOOM fast enough to "explode". It's a reasonably simple matter to make it FOOM slowly and provide us with electrical power to enhance our standard of living.

If the message is "don't build Skynet", funding agencies will say "ok, we won't fund Skynet" and AI researchers will say "I'm not building Skynet". If somebody is working on a dangerous project, name names and point fingers.

GIve a chain of reasoning. If some of you rationalists have concluded a significant probability of an AI FOOM coming soon, all you have to do is explicate the reasoning and probabilities involved. If your conclusion is justified, if your ratiocination is sound, you must be able to explicate it in a convincing way, or else how are you so confident in it?

This isn't really an "awareness" issue -- because it's scary and in some sense reasonable it makes a great story, thus hour after hour of TV, movie blockbusters stretching back through decades, novel after novel after novel.

Make a convincing case and people will start to be convinced by it. I know you think you have already, but you haven't.

Comment deleted 22 May 2009 05:55:09PM [-]
Comment author: whpearson 22 May 2009 08:06:51PM 0 points [-]

Personally I think that governmental support for reduction in consumption in fossil fuels is at least partly due to energy supply concerns, both in terms of abundance (oil discovery is not increasing) and political concerns (we don't want to be reliant on russia gas),

From this view we should still try to transition away from most fossil fuel consumption, apart from perhaps coal... and it makes sense to ally with the people concerned with global warming to get the support of the populace.

Comment deleted 22 May 2009 08:12:09PM [-]
Comment author: whpearson 22 May 2009 08:39:13PM *  2 points [-]

Coal can be nasty for other reasons apart from greenhouse gases. How much of the coal is low sulphur?

I don't see tar-sand as a total option, part of the energy mix sure. But we still need to pursue alternatives.