Wei_Dai comments on Safety Culture and the Marginal Effect of a Dollar - Less Wrong

23 Post author: jimrandomh 09 June 2011 03:59AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (105)

You are viewing a single comment's thread.

Comment author: Wei_Dai 09 June 2011 06:10:02AM *  5 points [-]

As a result of this calculation, I will be thinking and writing about AI safety, attempting to convince others of its importance, and, in the moderately probable event that I become very rich, donating money to the SIAI so that they can pay others to do the same.

Surely the most existential-risk-reduction-per-buck at this point is not "thinking and writing about AI safety", but thinking up more strategies like it in order to possibly find even better ones? Shouldn't SIAI (or perhaps FHI, depending on the comparative advantage between them) fund and publish some sort of systematic search-and-comparison of existential risk reduction strategies in order to have high confidence that the strategies it ends up pursuing are the optimal ones?

ETA: To be more constructive, has anyone done a similar analysis for "pushing for world-wide safety regulations on AI research" or "spending money directly on building FAI"?

Comment author: timtyler 12 June 2011 09:40:44AM *  1 point [-]

has anyone done a similar analysis for "pushing for world-wide safety regulations on AI research" or "spending money directly on building FAI"?

The number one point of comparison for safety regulations is the cryptography export regulations. I am pretty sceptical about something similar being attempted for machine intelligence. It is possible to imagine the export of smart robots to "bad" countries being banned - for fear that they will reverse-engineer their secrets - but not easy to imagine that anyone will bother. Machine intelligence will ultimately be more useful than cryptography was. It seems pretty difficult to imagine an effective ban. So far, I haven't seen any serious proposals to do that.

Governments seem likely to continue promoting this kind of thing, not banning it.