Follow-up to: Fake Frameworks, Kenshō

Related to: Slack, Newcomblike Problems are the Norm

Previously titled: The Real-World Omega (see these comments)

I’d like to offer a fake framework here. It’s a little silly, and not fully justified, but it keeps producing meaningful results in my life when I use it. Some of my own personal examples are:

• Overcoming a crushing depression
• Learning how to set aside my “performance mode” and be more authentic and vulnerable when I want to be
• Shifting my attachment style from anxious-preoccupied to mostly secure
• Fixing a lifelong problem where I love athletics but I kept badly damaging my body almost any time I tried anything athletic
• Setting myself up to experience kenshō

I recognize that this doesn’t address Oli’s hesitation stemming from his sense of a lemons problem. I’m afraid I don’t know how to, at least not yet. So in the meantime, please take these as my reports of my experience and how it seems to me they came about, rather than as an attempt to persuade.

Though I do hope y’all will benefit from this. If nothing else, I think this framework is fun. I’ve sometimes described one use of it as “overcoming personally meaningful challenges by living an epic life.” It’s an awesome framework to play with — especially when others join in while remembering that the framework is fake.

Here I’ll outline a general theory I’ll want to call on in some upcoming posts. Tomorrow I’ll separately post an application. (Originally they were written as one post, but I got some feedback on an earlier draft that convinced me they should be separate.) At that point we should have some meaningful tools for wrestling with self-deception.

So with that, let’s get started.

When you walk into an improv scene, you usually have no idea what role you’re playing. All you have is some initial prompt — something like:

“You three are in a garden. The scene has to involve a stuffed bear somehow. Go!”

So now you’re looking to the other people there. Then someone jumps forward and adds to the scene: “Oh, there it is! I’m glad we finally found it!” Now you know a little bit about your character, and about the character of the person who spoke, but not enough to fully define anyone’s role.

You can then expand the scene by adding something: “It’s about time! We’re almost late now.” Now you’ve specified more about what’s going on, who you are, and who the other players are. But it’s still the case that none of you knows what’s going on.

In fact, if you think you know, you’ll often quickly be proven wrong. Maybe you imagine in that scene you’re an uptight punctual person. And then the third person in the scene says to you, “What do you care, Alex? You’re always late to everything anyway!” Surprise! Now you need to flush who you thought you were from your mind, accept the new frame, and run with it as part of your newly evolving identity. Otherwise the scene sort of crashes.

It would go more smoothly if you didn’t hold any preconceptions about who you are or what’s going on. The scene tends to work better if you stay in the present moment and just jump in with the first thing that comes to mind (as long as it’s shaped by what has happened so far). Then the collection of interactions and emerging roles spontaneously guides your behavior, which in turn help guide others’ behavior, all of which recursively defines the “who” and “what” of the scene. Your job as a player isn’t to play a character; it’s to co-create a scene.

We can sort of pretend that there’s a “director”: it’s the intelligence that emerges between the players via their interactions. It’s a distributed system that computes relationships and context by guiding each node in its network to act freely within constraints. From this vantage point, the network guides players, and the job of each player is to be guidable but not purely passive (since a passive node is just relaying information rather than aiding in the computation). As long as everyone involved is plugged into and responsive to this network, the scene will usually play out well.

I suspect that improv works because we’re doing something a lot like it pretty much all the time. The web of social relationships we’re embedded in helps define our roles as it forms and includes us. And that same web, as the distributed “director” of the “scene”, guides us in what we do.

A lot of (but not all) people get a strong hit of this when they go back to visit their family. If you move away and then make new friends and sort of become a new person (!), you might at first think this is just who you are now. But then you visit your parents… and suddenly you feel and act a lot like you did before you moved away. You might even try to hold onto this “new you” with them… and they might respond to what they see as strange behavior by trying to nudge you into acting “normal”: ignoring surprising things you say, changing the topic to something familiar, starting an old fight, etc.

In most cases, I don’t think this is malice. It’s just that they need the scene to work. They don’t know how to interact with this “new you”, so they tug on their connection with you to pull you back into a role they recognize. If that fails, then they have to redefine who they are in relation to you — which often (but not always) happens eventually.

I’m basically taking as an axiom of this framework that people need the “scene” to work — which is to say, they need to be able to play out their roles in relation to others’ roles within a coherent context. I don’t think why this is the case is relevant for using this framework… but I’ll wave my hands at a vague just-so story anyway for the sake of pumping intuition: human beings’ main survival strategy seems to be based on coordinating in often complex ways in tribes. For the individual, this means that fitting in becomes paramount. For the group, this means knowing what to expect from each person is critical. So a trade becomes possible: the individual can fit into and benefit from the group as long as they’re playing a role that fits well with the collective.

This can result in some pretty strange roles. From this vantage point, a person who repeatedly leaves one abusive relationship only to get into another roughly similar one actually makes a lot of sense: this is a role that this person knows how to play. It’s horrible, but it’s still better than not fitting into the social scene. It creates a coherent relationship with someone who’s willing to (or has to) play an “abuser” role, and often with people in “rescuer” roles too. The trap they’re in isn’t (just) that their current abusive partner is gaslighting or threatening them; it’s that they don’t have another role they can see how to play. Unless and until that person finds a different one that fits into the social web, the strands of that web will tug them back into their old role. They don’t have enough slack in the web around them to change their fate.

The same kind of web/slack dynamics show up in more pleasant-to-play roles too. The privilege of a middle-class American white man by default has him playing out some kind of roughly known story-like path (probably involving college and having kids and maybe a divorce) that, in the end, will probably still leave him being one of the richest people on Earth. And all the while, he might well have no clue that he has other options or even that he’s on a path — but he’ll still know, somehow, not to step off that path (“I have to go to college; are you crazy?”). Never mind that his lack of slack here is awfully convenient for him.

I’ve watched religious conversions and deconversions happen via basically the same mechanism. I knew a fellow many years ago (unattached to this community) who was a proud atheist. Then he started dating a Christian girl. Something like a month later, he started quoting the Bible — but “only because they’re handy metaphors” and not because he really believed any of that stuff, you see. It later turned out he’d been going to church with her. He kept offering reasons that seemed vaguely plausible (“It’s a neat group of people, and it matters to her, and I can take the time to read”), but there’s a pattern here that was obvious. A few months later he told me he’d converted. Last I heard they had moved to Utah.

The great part is, I knew this was going to happen when they started dating. Why? Because when I warned him that he might find himself wanting to believe her religion once they started having sex, his reaction was to reassure me by acting confident that he was immune to this. That meant he was more focused on managing my perception of him than he was in noticing how the social web was tugging him toward a transition of roles. I didn’t know if they’d stay together, but I was pretty sure that if they did, he’d convert.

I could give literally hundreds of examples like this. From where I’m standing, it looks like one of the great challenges of rationality is that people change their minds about meaningful things mostly only when the web tugs them into a new role. Actually thinking in a way that for real changes your mind in ways that defy your web-given role is socially deviant, and therefore personally dangerous, and therefore something you’re motivated not to learn how to do.

Ah, but if we’re immersed in a culture where status and belonging are tied to changing our minds, and we can signal that we’re open to updating our beliefs, then we’re good… as long as we know Goodhart’s Demon isn’t lurking in the shadows of our minds here. But surely it’s okay, right? After all, we’re smart and we know Bayesian math, and we care about truth! What could possibly go wrong?

Another challenge here is that the part of us that feels like it’s thinking and talking is (usually) analogous to a character in an improv scene. The players know they’re in a scene, but the characters they’re playing don’t. The characters also aren’t surprised about who or what they are: the not-knowing of identity and context is something only the players experience, to open themselves up to the guidance of the distributed “director”. This means that (a) the characters are actively wrong about why they do what they do, and (b) they are also deeply confused about how much sense everything makes and don’t know they’re confused.

I claim that most of us, most of the time, are playing out characters as defined by the surrounding web — and we usually haven’t a clue how to Look at this fact, much less intentionally use our web slack to change our stories.

I think this is also part of why improv is challenging: you have to set aside the character you would normally play in order to create room for something new.

There’s a way in which the social web holds the position of Omega in an ongoing set of Newcomblike problems. The web as a whole wants to know what kind of role you’re playing, and how well you’re going to play it, so that it can know what to expect of you. So, a lot of its distributed resources go into computing a model of you.

One of the more obvious transmission methods is chat — idle gossip, storytelling, speculation, small talk. People sync up their impressions of someone they’ve met, and try to make sense of surprising events in conversation. If a lover brings their partner some flowers and the recipient freaks out and runs off, suddenly there’s a need to understand, and the flower-giver might try asking a mutual friend for some help understanding. And even if they do come to understand (“Oh, that’s because their last partner brought them flowers to break up with them”), there’s often an impulse to share the story with friends, so that the web as a whole can hold everyone in sensible roles and make the scene work. (“Oh, we had a funny misunderstanding earlier, poor Sam….”)

A lot of this is transmitted more subtly too, in body language and facial expressions and vocal tone and so on. If Bob is “creepy” (i.e., is playing a “creepy” role in the web), then it speaks volumes if everyone who meets Bob then cringes just a tiny bit when he’s later mentioned even if they say only good things about him. This means that someone who has never met Bob can get a “vibe” about him from multiple people in a way that shapes how they interpret what Bob says and does when they finally do meet him.

Sometimes, some people with enough web-savvy weaponize this. It doesn’t mean anything for someone to “be creepy” except that they have a web-like impact on others — which is to say, they have a “creepy” role. In a healthy network, this correlates with something actually meaningfully bad that’s worth tracking. But because perceived roles shape what people expect of a person, it’s enough for a rumor to echo through the web in order for someone to be interpreted as “creepy”. So a sufficiently cunning person could actually cause someone to be slowly isolated and distrusted without there being any facts at all to justify this as Omega’s stance.

(And yes, I’ve seen this happen. Many times.)

The same kind of thing can happen with “positive” labels, too. What it means for someone to be fit for a leadership role, in Omega’s eyes, is that they are seen as compatible with that role. So if someone is tall, attractive, and either vicious or strong depending on how you choose to see it, it might be enough to have the “strong” interpretation echo more powerfully than the “vicious” one in order for the web to conspire to put them in a leadership position.

…which means that even people who are seen as good leaders might not, in fact, be good leaders in the sense of making good leadership decisions. But they are by definition good leaders in the sense of playing the role well. After all, if the general consensus is that Abraham Lincoln was a great President, then there’s a sense in which that makes it true, since that’s what “great” means here. The “explanations” thereafter are often stories to justify one’s holding of a popular opinion.

The same thing holds for when someone seems “rational”. This is one reason to worry deeply when members of subgroups internally agree with each other on who is a top-notch clear thinker or “really a rationalist” but disagree with people in other subgroups. This looks less to me like people seeking truth, and a lot more like groups engaging in a subtle memetic battle over what “rational” gets to mean.

From where I’m standing, it looks to me like we’re all immersed in not-knowing, while our “characters” keep talking as though they know what’s going on, implicitly following some hidden-to-them script.

The web encodes a lot of its guidance about what to expect and how to behave via the structure of stories. Or rather, story structures are what expectations about roles and scenes are.

The trouble is, a lot of the stories we talk about have the structure of what our characters are supposed to say rather than of what actually happens. Imagine a movie where the new kid at a school gets bullied by the popular kids and then makes friends with quirky outcasts. What happens to the bullies in the end? In real life, bullies often don’t get their comeuppance — but having this fictional story in our hearts lets us play out the indignant versions of our characters in the real-life version. Because the bullies aren’t supposed to get away with it, right? That wouldn’t be fair!

Some parts of our story-like intuitions are scripts. Some are things our scripts say we should think or talk about. Some are merely incidental details. Sussing out which parts are which is part of the trick of getting this framework to work for you. For instance, the stereotypical story of the worried nagging wife confronting the emotionally distant husband as he comes home really late from work… is actually a pretty good caricature of a script that lots of couples play out, as long as you know to ignore the gender and class assumptions embedded in it.

But it’s hard to sort this out without just enacting our scripts. The version of you that would be thinking about it is your character, which (in this framework) can accurately understand its own role only if it has enough slack to become genre-savvy within the web; otherwise it just keeps playing out its role. In the husband/wife script mentioned above, there’s a tendency for the “wife” to get excited when “she” learns about the relationship script, because it looks to “her” like it suggests how to save the relationship — which is “her” enacting “her” role. This often aggravates the fears of the “husband”, causing “him” to pull away and act dismissive of the script’s relevance (which is “his” role), driving “her” to insist that they just need to talk about this… which is the same pattern they were in before. They try to become genre-savvy, but there (usually) just isn’t enough slack between them, so the effort merely changes the topic while they play out their usual scene.

If you know how to Look at lived stories, then I think a way out can start becoming a lot more obvious. Unfortunately, I don’t think I can describe it very well for a general audience, because how anyone receives what I say is itself subject to the influence of lived stories. But if you can: try Looking in the present moment at your own sense of not-knowing, notice that the same thing is alive in others, and watch as the story arises and plays out across all of you.

Tomorrow I’ll share a weaker but easier-to-use partial solution that I think doesn’t require Looking.

This was long, so I’ll try to summarize:

• You can choose to see social groups at all scales as running a distributed computation across the social web. If you view that process as generating an intelligent agent, you can think of this web-agent as the real-world Omega as it tries to predict and guide each person’s behavior.
• Omega offers each person a trade: prioritize making the scene work, and you’ll be included in it. In fact, Omega is the aggregate efforts of all the people who have accepted that trade. And basically everyone we know about accepts this trade.
• Everything about yourself that you have conscious access to is subject to your role as part of Omega. If you try to defy this, then your fate will play out through your defiance.
• Room for interpretation in your role in the scene means your script has room to change. This is slack in the social web.
• There’s a way of directly seeing how to change your fate by Looking. That’s not helpful unless and until you learn how to Look, though.

I’ll close this post by noting that there’s a meta-level to track here. In the story The Emperor’s New Clothes, the child’s utterance wasn’t enough on its own to pop the illusion:

"But the Emperor has nothing at all on!" said a little child.

"Listen to the voice of innocence!" exclaimed his father; and what the child had said was whispered from one to another.

"But he has nothing at all on!" at last cried out all the people. The Emperor was vexed, for he knew that the people were right; but he thought the procession must go on now! And the lords of the bedchamber took greater pains than ever, to appear holding up a train, although, in reality, there was no train to hold.

What if the father had instead responded “No, child, you’re just too foolish to see his fine garments”? He might have, out of fear of what those who were standing nearby might think of him and his kid. Then the child’s simple voice of reason would not be heard.

Or what if the people near the father/child pair had felt too uneasy to pass along what the child had said?

What if the Emperor could have instilled this kind of nervousness in his people ahead of time? He might have thought that there will be innocent children in the parade, and it might have occurred to some part of him that they had best not be taken seriously — to spare others their embarrassment, of course. Then, oh then what strange propaganda they all would see.

Some of the scripts Omega assigns work less well if they’re known. Because of this, Omega will often move to silence people who threaten to speak those fragile truths. This can show up, for instance, as people trying to dismiss and discredit the person saying the idea rather than just the idea. The arguments usually sound sensible on the surface, but the underlying tone ringing through the strands of the web is “Don’t listen to this one.”

If it’s not clear why I’m mentioning this, then I imagine it’ll become really obvious quite soon.