I have come to realize I have been posting these too rapidly. The motivation was to provide feedback and extra motivation to keep me on track to finish writing, but lots of people are away for the holidays, and posting the most abstract (and in many ways, least interesting) parts of the sequence one after another probably wasn't ideal. The next two posts are ready to go, but I will not post part 5 until January 5.
Meanwhile, I still will try to respond to comments on posts 3 and 4.
They are placed in severely toxic situations. To succeed, they transform their entire selves into maximizers of what get you ahead in middle corporate management.
Is the claim that this applies narrowly to corporate management, or that this is a fair description of the human condition? I'd think aboriginal hunters have roughly the same description - hungry and afraid is pretty toxic. Becoming better at killing when necessary, or climbing the group hierarchy by rather callous behaviors is the only way to succeed.
My claim certainly isn't that it doesn't apply to anyone outside corporate management or outside mazes (mazes need not be corporate). Certainly there are lots of types of severely toxic situations.
However, I do think that an aboriginal hunter does not engage in what I am describing here.
Nor has most of my life been spent in such situations. At no point other than my job on Wall Street did I feel under anything like this level of pressure - and even then, while it got pretty bad by the end, it was nothing like what is described here. Same goes for everyone I know who isn't in something that is recognizably a maze.
You say that a corporation is not an agent in its own right, only:
> an abstraction of what happens as the result of the preferences of individuals, and choices made by individual people, and their interactions.
But the human mind is an abstraction of what happens as the result of the preferences of individual subagents, and "choices" made by individual subagents, and their interactions. With a multi-agent model of mind, it seems indefensible to claim that an agent cannot be formed out of smaller agents - even when they are (like schemas in the mind) competing with one another. Corporations look like agents so much because that's what they are, even though it is also a direct result of the complex patterns of individual choice that you are talking about here.
Thanks. Fixed in original (correct link is to Mitt Romney's speech: https://www.youtube.com/watch?v=E2h8ujX6T0A )
Previously in Sequence: Moloch Hasn’t Won, Perfect Competition, Imperfect Competition
The book Moral Mazes describes the world of managers in several large American corporations.
They are placed in severely toxic situations. To succeed, they transform their entire selves into maximizers of what get you ahead in middle corporate management.
This includes active hostility to all morality and to all other considerations not being actively maximized. Those considerations compete for attention and resources.
Those who refuse or fail to self-modify in this way fail to climb the corporate ladder. Those who ‘succeed’ at this task sometimes rise to higher corporate positions, but there are not enough positions to go around, so many still fail.
Reminder: Sufficiently Powerful Optimization Of Any Known Target Destroys All Value
This is a default characteristic of all sufficiently strong optimizers. Recall that when we optimize for X but are indifferent to Y, we by default actively optimize against Y, for all Y that would make any claims to resources. See The Hidden Complexity of Wishes.
Perfect competition destroys all value by being a sufficiently powerful optimizer.
The competition for success as a middle manager described in Moral Mazes destroys the managers (among many other things it destroys) the same way, by putting them and the company’s own culture and norms under sufficiently powerful optimization pressure towards a narrow target.
Does Big Business Hate Your Family?
Consider this comment, elevated to a post at Marginal Revolution, responding to Tyler Cowen’s noting that the National Conservatism Conference had a talk called “Big Business Hates Your Family”:
Big business, like the AGI, does not hate your family. Big business thinks your family possesses capital, preferences and other assets that could be used for something else. The effects of this on your family are a side effect. Big business also notes that your family is attempting to optimize the world for something other than the profits of big business, and would like to prevent you from doing this, since it would tend to reduce its profits. This is all doubly true if you work for the Big Business in question, since your family is now asserting its preferences and claims to resources in an even more directly competitive way. For morality (or anything else that might have a claim to resources, so basically anything anyone cares about), same thing. Replace “your family” with “morality.”
No, wait. That’s wrong.
The above paragraph, like the quoted text, is misleadingly treating Big Business in general, or a given business in particular, like it is an agent.
We need to fully appreciate that corporations are not agents. There is no agent called Big Business. Nor are any of the individual big businesses agents as such.
Corporations are people. Not only in the ‘corporations are people, my friend‘ sense, but in the sense that corporations don’t act or have preferences, but rather are composed of people that act and have preferences. There’s just a bunch of guys.
The CEO is an individual representing their own interests, like everyone else at the company. The profits they care about are their own. Occasionally they will make some efforts to maximize corporate profits. Often they won’t, or will focus primarily on other things.
English makes it very hard not to make the agent mistake. I will no doubt keep saying that corporations want things and prefer things and think things, because I don’t know of another reasonable and compact way of saying the same thing. But understand that I mean that as an abstraction of what happens as the result of the preferences of individuals, and choices made by individual people, and their interactions.
Let’s reformulate the question.
Do The Managers of Big Businesses Hate Your Family?
When I say these organizations are immoral, it’s not necessarily that the people running major corporations are mustache-twirling villains who hate morality.
Most don’t have mustaches.
Morality causes choices and optimization towards the moral and against the immoral. That interferes with choices and optimization away from the uncomfortable and towards the comfortable. It interferes with choices away from bad for your boss towards good for your boss. Or, in less broken situations and/or with a less cynical perspective, also, from less profitable towards more profitable.
Do enough of that, and some of those involved for long enough do become mustache-twirling villains, because they are humans who are trying to live with what they are doing. You become what you continuously do and say. Eventually one becomes the mask. Mustaches become more common as people get older.
The first-level model says this stays rare. Being a mustache-twirling villain is to have preferences (if nothing else, you prefer having a mustache), and thus is bad for business the same way having morals is bad for business. You want to be completely indifferent, and be seen as completely indifferent.
It is worth taking five minutes here to think about how it might become seen as advantageous by the managers themselves, under these conditions, to become actively immoral, and prefer doing the wrong thing over the right thing because it is wrong.
Lacking all Preferences
None of this has anything to do with morality as such. What they are against is what Moloch is against. Having preferences at all. They are against caring about anything at all other than climbing the corporate (or academic, or government, or other as appropriate) ladder.
It doesn’t matter whether you care about the laws of accounting, wearing the color red, eating meat, cheating on your wife, seeing hit movies on opening weekend, overthrow of third world governments, falsifying scientific data sets, your favorite prime number, different brands of olive oil, genocide, or having a life outside the corporation, or time to watch your kids grow up. Caring about anything not chosen to help your career is a liability. Being seen or thought of as caring about something else can be even worse than actually doing so, as you are preemptively punished slash seen as not having a future, and potential allies want nothing to do with you.
In particular, any sign of, or even worse defense of, an outside life is deadly.
Thus what is called “Pournelle’s iron law of bureaucracy“: In any bureaucracy, the people devoted to the benefit of the bureaucracy itself always get in control and those dedicated to the goals the bureaucracy is supposed to accomplish have less and less influence, and sometimes are eliminated entirely.
In order to get an instinctive sense of all this, if you have not yet done so, I encourage you to read or at least browse Quotes from Moral Mazes. Some of those quotes will be reiterated later in the sequence. You also may wish to see Moral Mazes and Short Termism.
If you are interested enough to power through it (not the author’s fault, but it’s a tough read) and have the time, even better would be to pause here and read the whole book.