since the Magna Carta was issued? (In my mind, the reference is to English Common Law
Limiting to the common law tradition makes it even more dubious. Today, the population of England and Wales is around 60 million. Wikipedia says:
March 2006 there were 1,825 judges in post in England and Wales, most of whom were Circuit Judges (626) or District Judges (572)
On the number of solicitors (barristers are much less numerous):
The number of solicitors qualified to work in England and Wales has rocketed over the past 30 years, according to new figures from the Law Society. The number holding certificates - which excludes retired lawyers and those no longer following a legal career - are at nearly 118,000, up 36% on ten years ago.
Or this:
There were 2,500 barristers and 32,000 solicitors in England and Wales in the early 1970s. Now there are 15,000 barristers and 115,000 solicitors.
And further in the past the overall population was much smaller, as well as poorer and with fewer lawyers (who were less educated, and more impaired by lead, micronutrient deficiencies, etc):
1315 – Between 4 and 6 million.[3] 1350 – 3 million or less.[4] 1541 – 2,774,000 [note 1][5] 1601 – 4,110,000 [5] 1651 – 5,228,000 [5] 1701 – 5,058,000 [5] 1751 – 5,772,000 [5] 1801 – 8,308,000 at the time of the first census. Census officials estimated at the time that there had been an increase of 77% in the preceding 100 years. In each county women were in the majority.[6] Wrigley and Schofield estimate 8,664,000 based on birth and death records.[5] 1811 – 9,496,000
"More than a million fairly intelligent individuals have put in substantial amounts of work
If we count litigating for particular clients on humdrum matters (the great majority of cases) in all legal systems everywhere, I would agree with this.
"have put in substantial amounts of work to make the legal system capable of solving social problems decently well"
It seems almost all the work is not directed at that task, or duplicative, or specialized to particular situations in ways that obsolesce. I didn't apply much of this filter in the initial comment, but it seems pretty intense too.
Ok, you've convinced me that millions is an overestimate.
Summing the top 60% of judges, top 10% of practicing lawyers, and the top 10% of legal thinkers who were not practicing lawyers - since 1215, that's more than 100,00 people. What other intellectual enterprise has that commitment for that period of time? The military has more people total, but far fewer deep thinkers. Religious institutions, maybe? I'd need to think harder about how to appropriately play reference class tennis - the whole Catholic Church is not a fair comparison because it covers more...
-- Nick Szabo
Nick Szabo and I have very similar backrounds and interests. We both majored in computer science at the University of Washington. We're both very interested in economics and security. We came up with similar ideas about digital money. So why don't I advocate working on security problems while ignoring AGI, goals and Friendliness?
In fact, I once did think that working on security was the best way to push the future towards a positive Singularity and away from a negative one. I started working on my Crypto++ Library shortly after reading Vernor Vinge's A Fire Upon the Deep. I believe it was the first general purpose open source cryptography library, and it's still one of the most popular. (Studying cryptography led me to become involved in the Cypherpunks community with its emphasis on privacy and freedom from government intrusion, but a major reason for me to become interested in cryptography in the first place was a desire to help increase security against future entities similar to the Blight described in Vinge's novel.)
I've since changed my mind, for two reasons.
1. The economics of security seems very unfavorable to the defense, in every field except cryptography.
Studying cryptography gave me hope that improving security could make a difference. But in every other security field, both physical and virtual, little progress is apparent, certainly not enough that humans might hope to defend their property rights against smarter intelligences. Achieving "security against malware as strong as we can achieve for symmetric key cryptography" seems quite hopeless in particular. Nick links above to a 2004 technical report titled "Polaris: Virus Safe Computing for Windows XP", which is strange considering that it's now 2012 and malware have little trouble with the latest operating systems and their defenses. Also striking to me has been the fact that even dedicated security software like OpenSSH and OpenSSL have had design and coding flaws that introduced security holes to the systems that run them.
One way to think about Friendly AI is that it's an offensive approach to the problem of security (i.e., take over the world), instead of a defensive one.
2. Solving the problem of security at a sufficient level of generality requires understanding goals, and is essentially equivalent to solving Friendliness.
What does it mean to have "secure property rights", anyway? If I build an impregnable fortress around me, but an Unfriendly AI causes me to give up my goals in favor of its own by crafting a philosophical argument that is extremely convincing to me but wrong (or more generally, subverts my motivational system in some way), have I retained my "property rights"? What if it does the same to one of my robot servants, so that it subtly starts serving the UFAI's interests while thinking it's still serving mine? How does one define whether a human or an AI has been "subverted" or is "secure", without reference to its "goals"? It became apparent to me that fully solving security is not very different from solving Friendliness.
I would be very interested to know what Nick (and others taking a similar position) thinks after reading the above, or if they've already had similar thoughts but still came to their current conclusions.