Organizations are highly disanalogous to potential AIs, and suffer from severe diminishing returns: http://www.nytimes.com/2010/12/19/magazine/19Urban_West-t.html?reddit=&pagewanted=all&_r=0
As West notes, Hurricane Katrina couldn’t wipe out New Orleans, and a nuclear bomb did not erase Hiroshima from the map. In contrast, where are Pan Am and Enron today? The modern corporation has an average life span of 40 to 50 years. This raises the obvious question: Why are corporations so fleeting? After buying data on more than 23,000 publicly traded companies, Bettencourt and West discovered that corporate productivity, unlike urban productivity, was entirely sublinear. As the number of employees grows, the amount of profit per employee shrinks. West gets giddy when he shows me the linear regression charts. “Look at this bloody plot,” he says. “It’s ridiculous how well the points line up.” The graph reflects the bleak reality of corporate growth, in which efficiencies of scale are almost always outweighed by the burdens of bureaucracy. “When a company starts out, it’s all about the new idea,” West says. “And then, if the company gets lucky, the idea takes off. Everybody is happy and rich. But then management starts worrying about the bottom line, and so all these people are hired to keep track of the paper clips. This is the beginning of the end.” The danger, West says, is that the inevitable decline in profit per employee makes large companies increasingly vulnerable to market volatility. Since the company now has to support an expensive staff — overhead costs increase with size — even a minor disturbance can lead to significant losses. As West puts it, “Companies are killed by their need to keep on getting bigger.”
Yet they rule the world anyway.
But then management starts worrying about the bottom line and so all these people are hired to keep track of the paper clips. This is the beginning of the end.
And so LessWrong has been proved correct that paperclips will be the end of us all.
If I understand the Singularitarian argument espoused by many members of this community (eg. Muehlhauser and Salamon), it goes something like this:
I'm in danger of getting into politics. Since I understand that political arguments are not welcome here, I will refer to these potentially unfriendly human intelligences broadly as organizations.
Smart organizations
By "organization" I mean something commonplace, with a twist. It's commonplace because I'm talking about a bunch of people coordinated somehow. The twist is that I want to include the information technology infrastructure used by that bunch of people within the extension of "organization".
Do organizations have intelligence? I think so. Here's some of the reasons why:
I talked with Mr. Muehlhauser about this specifically. I gather that at least at the time he thought human organizations should not be counted as intelligences (or at least as intelligences with the potential to become superintelligences) because they are not as versatile as human beings.
...and then...
I think that Muehlhauser is slightly mistaken on a few subtle but important points. I'm going to assert my position on them without much argument because I think they are fairly sensible, but if any reader disagrees I will try to defend them in the comments.
Mean organizations
* My preferred standard of rationality is communicative rationality, a Habermasian ideal of a rationality aimed at consensus through principled communication. As a consequence, when I believe a position to be rational, I believe that it is possible and desirable to convince other rational agents of it.