Nothing. The arguments towards any course of action have very low external probabilities (which I assign when I see equally plausible but contradicting arguments), resulting in very low expected utilities even if the bad AI is presumed to do some drastically evil stuff vs good AI doing some drastically nice stuff. There are many problems for which efforts have larger expected payoff.
edit:
I do subscribe to the school of thought that the irregular connectionist AIs (neural networks, brain emulations of various kind and the like) are the ones least likely to engage in highly structured effort like maximization of some scary utility to the destruction of everything else. I'm very dubious that the agent can have foresight so good as to decide humans are not worth preserving, as part of general "gather more interesting information" heuristics.
While the design space near the FAI is a minefield of monster AIs and a bugged FAI represents a worst case scenario. There is a draft of my article on the topic. Note: I am a software developer, and I am very sceptical about our ability to write FAI that is not bugged, as well as of ability to detect substantial problems in FAI goal system, as regardless of the goal system the FAI will do all it can to pretend to be working correctly.
There is a draft of my article on the topic.
I can't see this draft. I think only those who write them can see drafts.
This is directed at those who agree with SIAI but are not doing everything they can to support their mission.
Why are you not doing more?
Comments where people proclaim that they have contributed money to SIAI are upvoted 50 times and more. 180 people voted for 'unfriendly AI' to be the most fearsome risk.
If you are one of those people and are not fully committed to the cause, I am asking you, why are you not doing more?