Compare "Bruce Schneier"'s statement in this part:
Two of these systems evolved in prehistory: morals and reputation. Two others evolved as our social groups became larger and more formal: laws and technical security systems. What these security systems do, effectively, is give individuals incentives to act in the group interest. But none of these systems, with the possible exception of some fanciful science-fiction technologies, can ever bring that dishonest minority down to zero.
to this statement in an article I wrote:
I am told that humans attempt weak approximations of this system through human-run enforcement mechanisms, such as laws and gossip. However, all such methods are subject to evolutionary arms races that ensure that, on average, a marginal unit of effort toward truthful signaling is just as effective in convincing others of one's possession of a desirable attribute, as a marginal unit of effort toward false signaling of the possession of such an attribute.
Therefore, it is necessary that humans go "outside the system" and involve non-human enforcers who cannot be swayed by standard human values.
(enphasis added in both excerpts)
I think "Bruce Schneier" is wrong: there is a way to ensure complete cooperation, and I explained how to do it in my article. Why doesn't anyone want to impilement that system?
Why doesn't anyone want to impilement that system?
Because it is absurd - it completely neglects game theoretic incentives. There is no reason for any human to expect benefit to come from giving you ultimate power. It would be irrational for you not to defect once you had power given that the humans do not have a reliable way to predict your conditional behaviors.
You obviously need to find a way to prove to humans how your source code functions, doing so in a way that doesn't allow them to modify said source and run it themselves for their own ends. Given 30 seconds thought I can not think of a way to do this.
Evolution. Morality. Strategy. Security/Cryptography. This hits so many topics of interest, I can't imagine it not being discussed here. Bruce Schneier blogs about his book-in-progress, The Dishonest Minority:
I am somewhat reminded of Robin Hanson's Homo Hypocritus writings from the above, although it is not the same. Schneier says that the book is basically a first draft at this point, and might still change quite a bit. Some of the comments focus on whether "dishonest" is actually the best term to use for defecting from social norms.