The Upward Scaling Importance of Rationality
I've read about a quarter of the sequences, but I'm not sure if this topic has been addressed on LessWrong before. If it has, let me know.
The Upward Scaling Importance of Rationality goes like this:
The more influence your thought process and decisions have, the more important it is that you're rationalist. In the grand scheme of things, it is relatively unimportant that a barback at a restaurant is a rationalist, and I say this having done that. It is extremely important that a leader of a highly influential company, or a president of a university or country is a rationalist. Their decisions affect thousands if not millions of people.
The more influential you are, the more your decisions have potential to screw over other people. Influence doesn't necessarily have to be in a management position: elementary school teachers and police officers are highly influential, even though they aren't in control of an organization. Influence can even be by virtue of the people you reach out to. A famous person with a large fanbase or a parent of a child prodigy, both have the capacity to influence the world with their decisions.
Though arguably, this can be extended to anyone who votes.
So rationality scales upward: the more influential someone is, the more important is it they're rationalists. Neglecting this can have bad consequences.
The Wrongness Iceberg
As soon as I got out of college I got a job at a restaurant. At the time I had never had a job at a restaurant, but my mom had known the owners and I felt obligated to avoid performing badly. Yet inevitably I did perform badly, and how this performance was evaluated would greatly affect my way of perceiving my mistakes.
If you're entrenched in an organization, there's a good chance you have an idea of what it is you're supposed to do and what mistakes you will or will not be making. But suppose you're in a position like this one: by way of your ignorance you know you're going to make a lot of mistakes, and it's just a question of when and how much. Further, you know that if you make too many mistakes, you make people you care about look bad. And finally, there are a lot of unknown unknowns: you don't know what possible mistakes and acts of ignorance exist to begin with, so many mistakes you've made you will be blind to.
The proactive thing to do, naturally, is to try to minimize how many mistakes you make.
There are two key ways to gauge the depth of being told you have made a mistake. The first way is to take mistakes literally, as if no other mistake exists, and any other mistake would be pointed out to you. So if you correct this mistake, everything else should be fine. This is how you'd expect to take mistakes if you were, say, under the supervision of an editor.
But the second kind is where the title of this writeup comes in. Not everyone is literal, or critical enough to notice every mistake. Much of the time, you'll only receive news of a mistake if many other mistakes are already afoot, and this mistake just happens to stand out from the set of mistakes you've already made. And since you don't know what mistakes you could be making, you don't know if there are many more mistakes under your level of awareness that you could be correcting for, but aren't.
In short, you're tasked with avoiding a wrongness iceberg: a mistake indicative of a nautical mile of mistakes below the surface and your level of awareness.
This is a debilitating position to be in, because your mental map of your performance prior to discovering the iceberg needs to be completely rewritten; in addition to accounting for all of the new areas you need to work on, you will likely account for the embarrassment of realizing that you have opened up a new frontier of mistakes to reflect on from your period of unaware incompetence.
While I don't think it's impossible that people exist who have never been in a situation like this, I think anyone who dives into a new field or skill is familiar, at least, with this feeling of brief yet total incompetence. And if you're in a field with enough depth and subjective calls to allow for a wrongness iceberg scenario, there might not be much you can do to prevent it. The most you can do is provide adequate resistance for the inevitable.
That's why I've created this mental model to think about it constructively. In every situation where I've faced a wrongness iceberg, the anxiety has been catastrophic. If you can at least deal with it, you can realize why it is you're anxious and what's going on with your assessment of your own mistakes. From experience, knowing that I'm worried about making this kind of iceberg-revealing mistake is helpful for mitigating my stress. And if you can somehow preempt an iceberg, that's even better.
side note: I've extended this concept to other domains, and it works well. A "dishonesty iceberg" is when one person's lie reveals a nautical mile of lies below the surface, and an "attraction iceberg" is when one person's expression of attraction toward you are indicative of a much greater level of internal attraction.
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)