"Don't be evil" is just not good enough!
Because all except (so it is claimed) one company and all governments bar none do not even pretend to embrace this attitude.
I think reputation systems would help a lot.
OK, so my government gets low karma. So what? How does that stop them for doing whatever they want to for years to come?
If you suggest it would cause members of parliament to vote no confidence and cause early elections - something which only applies in a many-parties democracy - then I suggest that in such a situation no government could remain stable for long. There'd be a new cause celebre every other week, and a government only has to lose one vote to fall.
And if the public, through karma, could force government to act in certain ways without going through elections, then we'd have direct democracy with absolute-majority-rule. A system that's even worse than what we have today.
Of course, governments and companies have reputations today.
There are few enough countries that people can keep track of their reputations reasonably easily when it comes to trade, travel and changing citizenship.
It is probably companies where reputations are needed the most. You can search - and there's resources like:
http://www.dmoz.org/Society/Issues/Business/Allegedly_Unethical_Firms/
...but society needs more.
Robin criticizes Eliezer for not having written up his arguments about the Singularity in a standard style and submitted them for publication. Others, too, make the same complaint: the arguments involved are covered over such a huge mountain of posts that it's impossible for most outsiders to seriously evaluate them. This is a problem for both those who'd want to critique the concept, and for those who tentatively agree and would want to learn more about it.
Since it appears (do correct me if I'm wrong!) that Eliezer doesn't currently consider it worth the time and effort to do this, why not enlist the LW community in summarizing his arguments the best we can and submit them somewhere once we're done? Minds and Machines will be having a special issue on transhumanism, cognitive enhancement and AI, with a deadline for submission in January; that seems like a good opportunity for the paper. Their call for papers is asking for submissions that are around 4000 to 12 000 words.
The paper should probably
Devote the second half to discussing the question of FAI, with references to e.g. Joshua Greene's thesis and other relevant sources for establishing this argument.Carl Shulman says SIAI is already working on a separate paper on this, so it'd be better for us to concentrate merely on the FOOM aspect.I have created a wiki page for the draft version of the paper. Anyone's free to edit.