sbenthall comments on Intelligence explosion in organizations, or why I'm not worried about the singularity - Less Wrong

13 Post author: sbenthall 27 December 2012 04:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (187)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 28 December 2012 12:00:12AM 6 points [-]

What is the relevance of profit per employee to the question of the power of organizations?

Corporations exist, if they have any purpose at all, to maximize profit. So this presents a sort of dilemma: their diminishing returns and fragile existence suggest that either they do intend to maximize profit but just aren't that great at it; or they don't have even that purpose which is evolutionarily fit and which they are intended to by law, culture, and by their owners, in which case how can we consider them powerful at all or remotely similar to potential AIs etc?

And why would a machine intelligence not suffer similar coordination problems as it scales up?

For any of the many disanalogies one could mention. I bet organizations would work a lot better if they could only brainwash employees into valuing nothing but the good of the organization - and that's just one nugatory difference between AIs (uploads or de novo) and organizations.

Comment author: sbenthall 28 December 2012 05:08:29PM 1 point [-]

Corporations exist, if they have any purpose at all, to maximize profit. So this presents a sort of dilemma: their diminishing returns and fragile existence suggest that either they do intend to maximize profit but just aren't that great at it; or they don't have even that purpose which is evolutionarily fit and which they are intended to by law, culture, and by their owners, in which case how can we consider them powerful at all or remotely similar to potential AIs etc?

Ok, let's recognize some diversity between corporations. There are lots of different kinds.

Some corporations fail. Others are enormously successful, commanding power at a global scale, with thousands and thousands of employees.

It's the latter kind of organization that I'm considering as a candidate for organizational superintelligence. These seem pretty robust and good at what they do (making shareholders profit).

As HalMorris suggests, that there are diminishing returns to profit with number of employees doesn't make the organization unsuccessful in reaching its goals. It's just that they face diminishing returns on a certain kind of resource. An AI could face similar diminishing returns.

I bet organizations would work a lot better if they could only brainwash employees into valuing nothing but the good of the organization - and that's just one nugatory difference between AIs (uploads or de novo) and organizations.

I agree completely. I worry that in some cases this is going on. I've heard rumors of this sort of thing happening in the dormitories of Chinese factory workers, for example.

But more mundane ways of doing this involve giving employees bonuses based on company performance, or stock options. Or, for a different kind of organization, by providing citizens with a national identity. Organizations encourage loyalty in all kinds of ways.

Comment author: gwern 28 December 2012 05:41:45PM 0 points [-]

It's the latter kind of organization that I'm considering as a candidate for organizational superintelligence. These seem pretty robust and good at what they do (making shareholders profit).

As far as I know, large corporations are almost as ephemeral as small corporations.

But more mundane ways of doing this involve giving employees bonuses based on company performance, or stock options. Or, for a different kind of organization, by providing citizens with a national identity. Organizations encourage loyalty in all kinds of ways.

Which tells you something about how valuable it is, and how ineffective each of the many ways is, no?