My research into animal mimicry, which closely resembles Baudrillardian simulacra, makes me think the slide in language/signaling from the first to second step is a potentially intractable problem. Once some association in information-space develops a reputation among situated actors, and is recognized as open to manipulation which benefits some of those actors at the cost of others... well, there's no way to break the freeriders of dishonest signaling.
Let's say that a black and red phenotype on a butterfly develops a reputation among predators as inedible (the butterfly releases toxins on being eaten). Now it's protected, great! What used to be a lose-lose (predator eats toxins, butterfly gets eaten) is transformed to a win-win (predator avoids toxins, butterfly survives) by the power of information: honest signaling benefits everyone. This is "step 1."
Unfortunately, the next step is other, non-toxic butterflies "noticing" (which is to say, evolution exploiting) this statistical association, and protecting themselves by dishonestly signaling the black and red, protected phenotype. This works alright at first, but it's driven by frequency-dependent selection: the more dishonest signalers, the less protection for everyone, toxic or not. This is "step 2."
But the actually toxic butterflies—the original honest signalers—they can't go anywhere. They're just stuck. One might happen to evolve a new phenotype, but that phenotype isn't protected by reputational association, and it's going to take a very long time for the new signal-association to take hold in predators. Once other insects have learned how to replicate the proxy-association or symbol that protected them, they can only wait it out until it's no longer protective.
You may have noticed this is a very similar mechanism to Goodhart's Law; the mechanism's the same far as I can tell. It's all about a publicly visible signal proxies for a hidden quality which outsiders do not have access to. (E.g. the lemon problem in used car sales, or size/confidence as a proxy for fighting ability in macaque hierarchies.) It can be easier and more reliable to just learn and copy the proxy than to evolve the hidden quality and hope other people catch on. (Think how many black and red butterflies got munched before the predators learned). It's a bleak problem; I haven't been able to make much progress on it, though I'd be super curious to hear if you think I've made errors in my premises, or if there's literature in game theory on this problem.
Signaling frontier moves. Movement speed depends, as you note, on how rapidly the noisy channel becomes a sexual or survival impediment. There is some research on this in game theory but only rudimentary simulations of populations forming high trust networks using unfakeably costly signals to out compete the free riders since they can internalize the benefits of their network.
Like posting a picture of yourself in a luxury resort. There are now companies offering to fake this for you since it is so commonly used as currency on social media/dating apps.
High trust networks usually use shibboleths of shared vocabulary plus some basic litmus tests.
Any chance you could point me to some keywords/authors/texts on this topic? I'd love to learn more.
I'd chase citations+check connected papers from these two high level reviews:
https://royalsocietypublishing.org/doi/pdf/10.1098/rsif.2012.0997
Great analogy.
Do you have examples of equilibria around these dynamics in the animal world? Do you have a sense of how stable these equilibria are?
e.g. do toxic black-and-red butterflies persist after their non-toxic lookalikes arrive?
Thanks Zvi, I'd read a bunch of posts on simulacra before but didn't really get it, nor the usefulness of it, until now. The thing that helped the most was laying out the different kinds of people (Oracle, Sage, Lawyer, Drone, etc.).
I found this post really helpful for crystallizing a lot of concepts in the simulacra discussion. Thanks!
I read an early draft of this, which ended up (potentially) clarifying something that had been nagging me about Simulacra. I think this is something of a disagreement with Zvi and/or Benquo, which I mentioned in the draft comments but decided to post as a comment on LW rather than debating it beforehand.
A key clarification in this piece is "In Level 3 vs Level 4, 3 is authentically trying to represent their group affiliation and position in social reality. Level 4 deceives people about it's group affiliation, and/or holds social reality as a thing to be manipulated."
In the draft-comments, Zvi said (paraphrased) "That's part of it, but not all of it. Level 4 has a whole different mindset, which can't form longterm physical plans, which accumulates power rather than pursuing goals."
And... this feels inelegant to me.
Certainly this type of level 4 person exists. But it feels like an additional claim, beyond the simulacrum model, to argue that all (or most?) level 4 people have this mindset. (I'm not arguing whether this claim is true or false, just it feels unnatural to me to group it in with the model)
It feels much simpler to define level 4 as "willing to lie about level 3", full stop. And then any additional considerations about 4-type-people are additional claims, outside the model (though perhaps derived from it)
And I think this maybe relates to an early objection I had to the original simulacrum model: the model implies there is a progression, from level 1 to 2 to 3 to 4. And it's not obvious to me that that's true. It seems plausible to me that ancient hunter-gatherers developed an understanding of physical-level-reality and social reality in tandem with each other. I agree that in the very beginning, physical-level reality had to have come first. But it seems plausible to me that social-reality predates language. (I.e. maybe early rats or wolves or primates first developed internal respect for object level reality, but by the time they were communicating with words, they may have already developed some primordial social reality, and using object-level-communications as coordination mechanisms.)
(This is not a claim that social reality does predate language, only that I wouldn't be surprised if it were true)
So the more elegant model, to me, looks less like levels 1-4, and more like a 2x2 grid, where there is physical reality, social reality, and the propensity to lie/manipulate beliefs about either of them.
This feels like a useful way to carve up reality to me, independent of what order the levels were developed, or how common level 4 implies Zvi's hard-to-grok-level-4-gestalt-of-attributes.
(I do believe that the hard-to-grok-level-4-combination-of-attributes is a significant and common pattern in the world, that we need to contend with in some way.)
Addressing the full substance of this would be (at least) another long post, and things are really weird and I'm still trying to fully understand them, so this is at best a very quick sketch, but basically...
You're not going to give a good gears-level understanding of the level-4 mindset, be able to predict the actions of those acting with level-4 orientation, or model the actions of a level-4 group/organization/civilization, if you view 4 simply as 4::3 as 2::1, start treating that reduction/abstraction as the thing, and deny the need to think about any of these other dynamics.
What you *can* do, I think (only 80% confident this actually works but 95%+ that it is a good quick heuristic?) is use "are they willing to mislead about *and/or take action to reshape meanings and associations in* level 3, and treat that as necessary and sufficient to identify someone operating at least partly at level 4. Iff they do that, they're at least partly level 4. Note that you can't ask whether they do this *on purpose* if you want the right answer, you only get to ask if they're actually doing it (but same with identifying other levels of action too).
As I said throughout MM, these are things that our brains actively don't want to look at or see. That's true even if we ourselves are on level 4 - it is an instinct of level 4 players to prevent coherent models of all kinds, and in particular of level 4, until the *overall civilization's level* reaches level 4 and then all hell breaks loose and predictions get very hard, before/while all literal hell actually does break loose and you get some form of collapse. So the instinct is constantly to round everything down.
That doesn't mean you can't carve reality very usefully using the 2x2. This post is doing its best to *not rely on* the progression order (beyond keeping the names 1, 2, 3 and 4) and to *not rely on* the True Form of level 4 or what not.
The thing where it looks like I/we are ascribing 'good' things to level 1, and 'bad' things to other levels, is probably not *entirely* objective, but it's mostly because, well, that's the way it is and I'm/we're trying to tell it like it is best we can and understand best we can. If we treat levelism as an ism then that's a good way to *not even look at* most of what Level 4 behavior actually is in any useful way, possibly 2 and/or 3 as well.
Progression is real too, in multiple ways/levels, but yes the system can be useful without them. Note that level 1 proceeded level 2 and level 3 even if all three (or four) pre-date language.
Certainly this type of level 4 person exists. But it feels like an additional claim, beyond the simulacrum model, to argue that all (or most?) level 4 people have this mindset. (I'm not arguing whether this claim is true or false, just it feels unnatural to me to group it in with the model)
It feels much simpler to define level 4 as "willing to lie about level 3", full stop. And then any additional considerations about 4-type-people are additional claims, outside the model (though perhaps derived from it)
I think this captures one of my issues with the original simulacra levels post. It feels like there's an aesthetic bias going on in a lot of these posts where truth-oriented rationalists are ascribing all sorts of positive attributes to the truth-oriented levels, and negative attributes to the non-truth oriented levels.
So the more elegant model, to me, looks less like levels 1-4, and more like a 2x2 grid, where there is physical reality, social reality, and the propensity to lie/manipulate beliefs about either of them.
This gets at one of the other things I've been uncomfortable with the model - it's conflating being able to see any given level with being unable to see the other levels.
For instance, I know people who basically aren't tracking reality at all, they're just tracking tribal affiliations. when they talk. Meanwhile, I know people who are intimately aware of reality in their minds, INCLUDING the tribal affiliations they're signaling, and take both into account when talking.
This implies then that rather than a 2x2, where you're either a liar or a truthteller along two separate axes, there's actually four separate skills.
- The ability to track the truth of object-level reality, and communicate it.
- The ability to lie about object-level reality
- The ability to track the truth of social reality, and signal it.
- The ability to create dishonest signals about social reality.
I can think of a few people who are barely able to keep track of the truth, but are great at lying about the truth to get object-level benefit. However, I think this is relatively rare, and your 2x2 may be more elegant.
Edit:
The final things that's going on here is the conflation of: Being able to see the level, with Being able to play at the level, with Choosing to play at the level.
There are people who can see the social reality being manipulated, but couldn't manipulate it themselves. There are people who can are able to see other's tracking the truth, and who can play at the level of finding and communicating the truth, but choose not to. There are people who can see social reality being manipulated, and can play at that level if they need to, but choose to stay at the level of communicating the truth if they can.
Nod.
I do think Zvi's post here does a good job of breaking down what it looks like for people to have different relationships with different levels. (He may not have quite broken down the "able to see a level" and "ability/propensity for interacting on that level" aspect, but it feels like a natural extension of the post)
I ended up writing some more thoughts on how the concept of Simulacra Levels seem to have evolved on LessWrong, over on Chris Leong's post. It was sort of relevant to other things I brought up in this comment thread.
I do have some sense that the entire simulacrum model was kinda backchaining from "try to articulate something about how bad level 4 is." (I think I have a greater sense of this in Zvi's portrayal of simulacra, partly because I expect it to be more directly downstream of Zvi's work on Moral Mazes)
I'm not sure if that's quite accurate, but, it'd make sense if Zvi or Benquo replied with something like "if you get rid of the nuanced-comprehensiveness-of-level-4, you're throwing out a motivating piece of the whole thing."
But, even if that were the case, I think it'd still make sense to break out the nuanced-gestalt-of-how-level-4-often-operates into a separate model, that builds on a simpler, more streamlined version of the simulacrum model.
I don't understand why the word "bad" needs to be involved. The motivation of trying to find words that actually describe level 4 even to ourselves, and hopefully to others, to create common knowledge, is a huge motivation. But that has nothing to do with whether level 4 is *bad*. I notice that when I or Benquo (or Jessica or Zack etc etc) describe things while carefully not using "good" or "bad" people constantly ascribe them to us anyway. I understand why, but it's frustrating nonetheless.
Two things I notice that makes me want to slow down here a bit is:
From this comment:
while carefully not using "good" or "bad" people constantly ascribe them to us anyway.
And from the other comment
and deny the need to think about any of these other dynamics.
First, I do think your layout of the levels here was fair (and seemed to go out of it's way to do so, especially with the different groupings). A significant chunk of why I think you think level 4 is "bad" is associations from the Moral Mazes sequence, and an assumption that you see level 4 as tightly entwined with the sort of person who thrives in a moral maze. That involved reading into some things you may not have intended, in which case, sorry.
(That said, I'm honestly pretty surprised if you wouldn't characterize level 4 as "bad", even if you went out of your way to avoid doing so in this particular post)
Second, man, I went out of my out of my way to say "It's important to think about the pattern that you're pointing to with your characterization of level 4, I just don't think it makes sense to cluster in the simulacrum abstraction. The abstraction seems less useful if it's trying to do too many things at once." I'm very much not saying you don't need to think about these other dynamics.
(a while later I'll respond in more depth to the object level discussion, right now just wanted to express some worry like, if we're both reading things into each other other's comments we didn't say, that's a warning side to slow down and be more careful or something)
The 'bad' word is just not useful in such situations, I think you even noted that a bunch of people wish I hadn't used it in Complexity is Bad and Choices are Bad.
We need some amount of level 4 awareness. We need to be able to change social reality not only communicate inside it. And the level 4 effects happen whether we intend or notice them, or not.
What I'd be tempted to call bad is when the general simulacrum level gets to 4. Or when someone gets into the patterns of inability to think about reality on the object level or even to realize a reality exists. It's still a poor atom blaster that won't point both ways.
Nod, makes sense.
I think we are (at least mostly) in agreement about this aspect of the territory, and the disagreement is just over what sort of maps are most useful.
Also, I don't think my original comment here was intended to focus on the "badness" characterization.
What I meant to be saying is "It seems like the overall simulacrum model was invented, in large part, to some particular failure modes that happen when society or individuals operate primarily on the level 4 level." Thus, I realize my suggestion to factor out the implicit "level 4 is complicated" claims probably flies against the original intended-use-case of the model. But, nonetheless, I think it'll be easier to talk about "societal level 4" and it's pathologies with a different model that builds off a simpler simulacrum model.
I was just using "bad" as shorthand. It wasn't meant to be a cruxy element of my argument.
We don't really have a lot of language to talk about aesthetic nuance. I know that my use of "bad" is just trying to say "it seems like you're framing this as ugly/unpleasing/aesthetically unpleasing", but the only words that sort of point at that without a bunch of inferential distance is "bad" and "good".
The 2x2 grid idea reminded me of the AQAL chart, developed by Ken Wilber. It is in the upper-right corner of this image. The levels of simulacra are also similar to the ego development stages as it explores how the ego progresses when adding more layers of societal cooperation or tension, though it does not map 1-1. For example, the diplomat at level 3 is concerned with social norms and in-group behavior, whereas the expert at level 3/4 may override these norms in pursuit of a different goal. From my understanding, each level in the stages below is part of a spectrum and each are meant to have an unevolved and evolved form as they progress.
Because I am a human, I will likely not write a bunch of simulations to show what happens to individuals and groups when they speak at various levels and act predicated on believing that others are speaking at various levels.
Right now I really really want to though. I think it'd be fascinating to put together sims with enough gears to show interesting behaviors and I think it'd be fascinating to read a well written report of someone doing so and summarizing their experiences/results/etc.
This is the post that first spelled out how Simulacra levels worked in a way that seemed fully comprehensive, which I understood.
I really like the different archetypes (i.e. Oracle, Trickster, Sage, Lawyer, etc). They showcased how the different levels blend together, while still having distinct properties that made sense to reason about separately. Each archetype felt very natural to me, like I could imagine people operating in that way.
The description Level 4 here still feels a bit inarticulate/confused. This post is mostly compatible with the 2x2 grid version, but it makes the additional claim that Level 4 don't know how to make plans, and are 'particularly hard to grok.' It bundles in some worldview from Immoral Mazes / Raoian Sociopaths.
For me, a big outstanding question re: Simulacra is "does it actually make sense to bundle the Kafkaesque sociopath who can't make plans as an explicit part of Level 4?"
I think this is a kinda empirical question. An example of the sort of evidence that'd persuade me are "among politicians or middle managers who spend most of their time optimizing for power, interacting with facts and tribal affiliations as a game, what proportion of them actually lose their ability to make plans, or otherwise become more... lovecraftian or whatever?" Is it more like "70%", "50%", "10%"?. It's plausible to me that there's a relatively small number of actors who stand out as particularly extreme (and then get focused on for toxoplasma of rage reasons)
Or, rather: if I simply describe Primarily Level 4 people as "holding social-signaling as object", am I actually missing anything? Do they tend to have any attributes? What?
...
I do this post is among the best intro to the Simulacra Levels concept, and think it's worth polishing up slightly. I assume Zvi has thought a bit more about Level 4 by now. If it still seems like there's something Importantly, Confusingly Up With Them, I'm hoping that can be spelled out a bit more. (I think my favorite version of the Level 4 section would include explicitly the "holding social reality as object" thing, since I think that part's relatively straightforward, and then go on to say what else is going on with them)
...
[Addenda]
A followup question I found myself thinking is: "What do Simulacrum levels actually add to the conversation? We knew that sometimes people lie. We already knew about Beliefs as Attire, Belief-in-Belief, Professing and Cheering, etc. We knew that social reality was pervasive and politics make us go funny in the head.
After thinking a bit, here are my answers:
First, Baudrillard-style simulacrum levels point at a particular progression/mechanism that's interesting You have object level truth that becomes distorted, then masking, then uncoupled completely from reality. This seems relevant to some subsets of Social Reality Woes (the original example of "Vice President of Drudgery" seemed to capture a real phenomenon of a thing that happened to business titles), but it's not obvious to me that this is usually what's going on. Zvi argues that the Baudrillard definition is still fairly intertwined with the Lion definition, but I didn't personally find it that persuasive. (I also don't find it cruxy for anything other than 'should we still call these simulacrum levels?' so I'm not that worried about it).
Second, Simulacrum Level 4 exists. This is mabe vaguely alluded to in the original LW sequences and HPMOR, but I don't think it's really been spelled out. "Belief in Belief" covers Level 3, but it doesn't really address the sort the level of cynicism that goes into someone who is dishonest about their beliefs-in-belief, and what that dishonesty feels like from the inside. This increases my desire to hear Zvi, Benquo or others weigh in about how they think about level 4 these days.
(I'm copying this into my original review comment so it's easily viewable on the /reviewVoting page, but posting here since it's a new set of thoughts)
A followup question I found myself thinking is: "What do Simulacrum levels actually add to the conversation?" We knew that sometimes people lie. We already knew about Beliefs as Attire, Belief-in-Belief, Professing and Cheering, etc. We knew that social reality was pervasive and politics make us go funny in the head. Do Simulacra add anything?
After thinking a bit, here are my answers:
First, Baudrillard-style simulacrum levels point at a particular progression/mechanism that's interesting You have object level truth that becomes distorted, then masking, then uncoupled completely from reality. This seems relevant to some subsets of Social Reality Woes (the original example of "Vice President of Drudgery" seemed to capture a real phenomenon of a thing that happened to business titles), but it's not obvious to me that this is usually what's going on. Zvi argues that the Baudrillard definition is still fairly intertwined with the Lion definition, but I didn't personally find it that persuasive. (I also don't find it cruxy for anything other than 'should we still call these simulacrum levels?' so I'm not that worried about it).
Second, Simulacrum Level 4 exists. This is mabe vaguely alluded to in the original LW sequences and HPMOR, but I don't think it's really been spelled out. "Belief in Belief" covers Level 3, but it doesn't really address the sort the level of cynicism that goes into someone who is dishonest about their beliefs-in-belief, and what that dishonesty feels like from the inside. This increases my desire to hear Zvi, Benquo or others weigh in about how they think about level 4 these days.
I understood this better when I made it more concrete, though not with the same phrase at all levels.
- W says "There’s not a pandemic headed our way from China" simply because W believes there’s not a pandemic headed our way from China. W is making a claim, turns out to be wrong, and is surprised.
- X says "Masks aren't effective against COVID-19" because X is a government agency who wants to reserve masks for frontline workers, and hasn't thought up any strategies besides lying to the nation. X knows they were probably lying.
- Y reflexively shares a meme saying "The anti-mask folks are definitely getting sick from their protests" because Y's friends are all pro-mask. Y's not actually trying to reach anyone who might be dissuaded from going to those protests. Y will switch on a dime once Y's friends identify with the next set of protests, and share stories about how outdoor protests are actually safe - Y never remembers that they were wrong before.
- Journalist Z writes an article mocking the tech weirdos who stopped shaking hands in February out of fear that coronavirus was coming. Z isn't following a crowd, Z's making a step to shift the crowd- making a bet that Z, and tech journalism in general, will look good to the crowd for dunking on the tech nerds. Z will be very surprised, later, to find out that there were actual real-world events at stake.
Whereas, if things are too forsaken, one loses the ability to communicate about the lion at all. There is no combination of sounds one can make that makes people think there is an actual lion across an actual river that will actually eat them if they cross the river.
Hm. This sounds like a challenge.
How about this:
Those "popular kids" who keep talking about fictitious "lions" on the other side of the river are actually losers. They try to pretend that they're simply "the safe and responsible people" and pat themselves on the back over it, but really they're just a bunch of cowards who wouldn't know what to do if there were a lion, and so they can't even look across the river and will just shame you for being "reckless" if you doubt the existence of lions that they "just know" are there. I hate having to say something that could lump me with these deplorable fools, and never before has there actually been a lion on the other side of the river, but this time there is. This time it's real, and I'm not saying we can't cross if need be, but if we're going to cross we need to be armed and prepared.
I can see a couple potential failure modes. One is if "Those guys are just crying wolf, but I am legit saving you [and therefore am cool in the way they pretend they are]" itself becomes a cool kid thing to say. The other is if your audience is motivated to see you as "one of them" to the point of being willing to ignore the evidence in front of them, they will do so despite you having credibly signaled that this is not true. Translating to actual issues I can think of, I think it would mostly actually work though.
It becomes harder if you think those guys are actually cool, but that shouldn't really be a problem in practice. Either a) there actually has been a lion every single time it is claimed, in which case it's kinda hard for "there's a lion!" to indicate group membership because it's simply true. Or b) they've actually been wrong, in which case you have something to distance yourself from.
If the truth is contentious and even though there has always been a lion, they've never believed you, then you have a bigger problem than simply having your assertions mistaken for group membership slogans; you simply aren't trusted to be right. I'd still say there's things that can be done there, but it does become a different issue.
Minor presentational quibble:
The original intent of this post was to go on to analysis of other issues surrounding Covid-19. I was hoping to make clear what I meant by the more disputed statements in my Covid-19 model summary from two weeks ago, and also how and why I believe those dynamics occurred, and what dynamics one can expect going forward. But this post is long enough, so I’ve pushed that into future posts.
It'd be nice if the title were updated to match this, since this means it's no longer an accurate summary of the content.
… that said, I also notice that in a universe in which this were intentional, it could also be a nice medium-level demonstration of Level 2 thinking: “If I name my post with something important-sounding in the title, more people will read it, and I will (probabilistically) gain points.”
It does still use the arrival of the Covid-19 pandemic as its motivating example, and I didn't have an obviously better title. Would be happy to change to a good alternative title if one was found.
Nice write up, I believe I have a better grasp of simulacra levels after this post.
(Think this is missing 1-3 additional roles. Discussion question, what is The Idealist?)
I'll take this as an exercise for the readers.
I'll start with a definition of how I see The Idealist: The Idealist is someone with an ideal of how people should act to have the best consequences in the world. In the most simple case this could simply be someone that believes the truth to be most important and that everyone should stay in Level 1. However, this type of idealist could ironically be seen as a Level 2 move: "I'm telling the truth so that you will stay on Level 1 and tell the truth as well".
It becomes more complicated when the implications of the ideal affects higher simulacra levels. Consider an idealist with the ideal that intelligence is the most important and that advanced communication and progressively higher level reasoning is the way to achieve this. This way you value the complexity arising in Level 4 and you want people to see your group as good and that they move to a higher level.
In common, the types of idealist I can imagine want to affect the map so that people act in accordance with their ideal (Level 2 move), but they also want their group to be perceived as the cool group and will say things that will make undecided people move towards their group (Level 4 move). Therefore I would say that The Idealist is a Level 2 + Level 4 player. Furthermore, The Idealist only says something if it improves both how others perceive the map and how other value the in group (relative to the out group).
I don't know if this is roughly what you had in mind when you thought about The Idealist, but this is my stab at the problem.
First of all, I really appreciate this article! It helps me conceptualize cleanly some vocabulary that's flying around in the rationalist community that I previously didn't really understand.
To me, the most obvious missing archetype is The Truth-Giver or perhaps The Teacher.
The Teacher is concerned with only conveying truthful messages. She will usually tell the truth, but she may occasionally omit truthful things, or even (rarely) tell half-truths if she thinks it's easier to convey truthful messages via half-truth. Importantly, she's different from the Sage or the Pragmatist in that she's not concerned with other object-level consequences, only in conveying truthful messages.
Consider the claim:
There’s a pandemic headed our way from China
Suppose that The Teacher believes that the following is more correct:
There's a pandemic headed our way from Italy.
The Teacher will choose usually to clarify and say the full message, however if she only has one bit of response, she'll say "yes" to "Is there a pandemic headed our way from China?" Importantly (unlike the Pragmatist) she'll do this even if the perceived consequences are negative, as long as the subject gets more truthful information than they otherwise would have.
Against Level 1 and Level 2 players, The Teacher will never see a need to resort to Level 3. However, against a fully Level 3 player, she will (begrudgingly) issue utterances correctly conveying the ideological faction she's on, as that's the most relevant/only bit to transfer to fully Level 3 players.
I think Teacher roles are incredibly important in practical everyday communication, since all information is lossy, inferential gaps are common, attention and text is limited, etc. Indeed, I would go so far as argue that Teacher roles are often preferable to Oracle roles in murky situations if the purpose is to collectively seek truth.
Curated. I learned a lot about the higher levels in this post, and the roles helped me understand the psychology associated with each level quite a bit. This was a clearly valuable contribution on top of the existing conversation on this (and quite nicely self-contained).
I'm not sure Level 3 is actually less agentic than Level 1. The Oracle does not choose which truths to speak in order to pursue goals; if they did, they'd be the Sage.
That's fair, but I think a nearby concept is The Scientist who not only speaks all truth but is trying to learn as much truth as possible. And I think Zvi is imagining that level 1 is motivated to find out the truth, as well as always reporting it.
It's always funny when you say there's a lion across the river, everyone knows there is a lion across the river, and everyone knows everyone is actually speaking about the social reality, because who cares about the lion, anyway.
(Think this is missing 1-3 additional roles. Discussion question, what is The Idealist?)
Let me take a shot at describing an Idealist.
I think the Idealist can be described as a deranged pragmatist, that bases most of his endeavor on hope (whether he reasons otherwise or not). he cares mostly about outcomes, but is also usually an altruist, so he wants collectively good outcomes. he is prone to not see the possible unintented consequences of his actions, and focuses on the hoped consequences of them.
I think the idealist can be further classified into 3.
One is close to the ideologue, which i also need to define.
The ideologue resides primarily in L-2 & L-3. They're disconnected from reality, but not so disconnected from it to fit L-4. they will use L-1 when it fits, but it's no requirement (it's usually a restraint and getting free from it gives them more power).
The first type of idealist has decided the ideological strategy is optimal for him.
the second type of Idealist resides mostly in L-1, but is the type of person to suggest we should lie about AI to get more funding for MIRI.
The third type has gone of the rails and visits L-4 far too often. He's so captivated by hope that he cannot see any negative consequences, He's sure L-4 would bring the consequences he wishes in a straightforward manner, without much added trouble.
What do you think of this model?
P.S: I also posted a sketch i made of the above article
group the first three levels together and contrast them with the fourth. This thinking is that there is ordinary decent interaction, humans being human, as represented by the first three levels. Then there are the schemers who prey upon us, twist everything in their sick games and play us against each other. We vote for the lizards, as Douglas Adams reminds us, because if we don’t, the wrong lizard might win.
Certainly this could be turned around with a different set of aesthetics.
There are the true humans who use their free-will free from arbitrary restrictions, have true freedom and choice to take any action they deem best, vs. the "sheep" who let themselves be hampered by arbitrary restrictions.
It might be a fun/useful exercise to do an aesthetic reversal for each of the groups you list here, seeing how "bad guy" in the dichotomy is actually beautiful, and in what ways the "good guy" is ugly.
“Good Morning!” said Bilbo, and he meant it. The sun was shining, and the grass was very green. But Gandalf looked at him from under long bushy eyebrows that stuck out further than the brim of his shady hat.
“What do you mean?” he said. “Do you wish me a good morning, or mean that it is a good morning whether I want it or not; or that you feel good this morning; or that it is a morning to be good on?”
-- End exact quote from "The Hobbit".
Bilbo reflected that it was a great shame that Gandalf had violated literary norms by offering 4 options (in place of the traditional 3), and that this number corresponded to the number of simulacra levels, but that only 1 or 2 of the options corresponded cleanly to levels. (1 : a good morning whether I want it or not), (4 : wish me a good morning).
This topic increases the weight I'm putting on the importance of Brain Machine Interfacing as a necessary step for our society to continue existing.
Re-reading this, it strikes me that an entity communicating purely on the first level is himself a drone, not an agent. He is a slave to the territory, and can only report its condition, even when it may harm him. (See Kant's thought experiment about an ax murderer who enters your home and demands knowledge of where your friend is hidden.)
I had trouble initially understanding the level 2 vs 4 distinction. It read as if level 2 was a willingness to lie about object-reality to bring about specific consequences, while level 4 involved similarly lying about object-reality to bring about specific, selfish consequences.
This didn’t seem like the most meaningful distinction, so I wondered what I was missing. These comments seem to describe level 4 as also being concerned with lying about social-reality, which feels elegant? However, we should be thinking of level 4 as being concerned with both object- and social-level reality?
In either case, this piece felt easier to digest to a total newbie than your initial links, so thanks for that!
Excellent work! Look forward to your next post. COVID-19 was such a good illustration of much of this.
Previously: Covid-19: My Current Model, On Negative Feedback and Simulacra
This post aims to unpack and explain simulacra levels of action using the threat of covid-19 as its central example. My intention is for future posts to then apply this model to many covid-related dynamics.
In Elizabeth’s Negative Feedback and Simulacra, she examined several example situations on which information was being processed on multiple simulacra levels at once. On Negative Feedback and Simulacra was my take on those examples.
To re-familiarize ourselves with the simulacra levels, here’s the introduction Elizabeth offered to them in her post:
If that doesn’t make sense, try this anonymous comment on the post
Almost everyone would rather not be eaten by a lion. I certainly would rather not be eaten by a lion.
Whether or not I am eaten by a lion still does not drive much of my decision making. It is highly implausible that I will be eaten by a lion.
I hope that if in the future it becomes plausible that I may get eaten by a lion, how to not get eaten by a lion would then drive much of my decision making.
If the presence of lions in various places would not put anyone in any danger, that makes it much less expensive for me to be wrong about where they are. The less people are concerned about the consequences of having or inflicting incorrect object-level models of the world, the less concerned they will be with Level 1 and with the Level 1 accuracy of their statements.
The prioritization of various simulacra levels becomes a habit. If you are used to interpreting “There’s a lion across the river” almost entirely as “I’m with the popular kids who are too cool to go across the river,” because that’s what it almost always means in your village, it may be very difficult for someone to say “No, really, I’m not associating with the cool kids right now. There’s literally an actual lion across the actual river and if you cross the river you will die.”
There is no good way to sacrifice the cool points in order to communicate the presence of a lion. Even if it works at first, soon there will be a tendency for the new wording to become the canonical form of “I’m with the popular kids who are too cool to go across the river.”
If everyone’s instinct is to interpret “There’s a lion across the river” as both “There is an actual lion across the actual river” and also “I’m with the kids who are too cool to cross the river” then there is a chance.
There is still a barrier. Whoever wants to share knowledge of the lion will become less cool by doing so. Ideally, for high enough stakes, this stops being a problem in multiple ways. If lives are at stake, especially one’s own or one’s loved ones, being cool looks less important than avoiding the lion. Ideally, being the person who saved us from the lion is also considered kind of cool, allowing one to both starve lions and look cool. That only works if everyone realizes the lion was there. But the payoff could be very large. So there’s a chance.
Whereas, if things are too forsaken, one loses the ability to communicate about the lion at all. There is no combination of sounds one can make that makes people think there is an actual lion across an actual river that will actually eat them if they cross the river.
I’m not trying to be subtle here. You can guess where this is going.
There’s a Virus Across The Ocean
Level 1
“There’s a pandemic headed our way from China” means “there’s a pandemic headed our way from China.”
“There’s no pandemic headed our way from China” means “there’s no pandemic headed our way from China.”
Level 2
“There’s a pandemic headed our way from China” means “I want you to act as if you think there might be a pandemic on our way from China” while hoping to still be interpreted by the listener as meaning “There’s a pandemic headed our way from China.”
The speaker hopes to be interpreted as still meaning “There’s a pandemic headed our way from China.”
This may involve any of the following:
“I want you to click on this headline about a pandemic headed our way from China and/or subscribe to and share my newsletter, website or channel.”
“I want you to have more negative affect towards China and/or other foreigners.”
“I want you to be afraid.”
“I want to shut down economic activity.”
“I want to sell you toilet paper, masks and hand sanitizer at premium prices.”
“I want you to thank me later for any or all of the above.”
“There is no pandemic headed our way from China” means “I want you to act as if you think there is not a pandemic headed our way from China.”
The speaker hopes to be interpreted as still meaning “There is not a pandemic headed our way from China.”
This may include any of the following:
“I want you to click on this headline about there not being a pandemic headed our way from China and/or subscribe to and share my newsletter, website or channel.”
“I want you to have more positive affect towards China and/or other foreigners.”
“I do not want you to be afraid.”
“I want to stop you from hoarding toilet paper, masks and hand sanitizer.”
“I want to avoid shutting down economic activity.”
“I want you to thank me later for any or all of the above.”
Level 1 vs. Level 2
The key difference is in the intended effect of the statement made, and to what extent the statement is correlated with the true state of the physical world.
Level 2 statements do not have to be untrue. Nor need it be something you do not believe. It is often said that “the truth is the best lie.”
Nor need the statement be selfish. Many Level 2 statements really are intended for the subject’s ‘own good.’
What makes a statement Level 2 rather than Level 1 is that you don’t care whether or not it is true. Instead, you care about what actions it causes people to take, and whether or not you like those actions.
If there is a tiger across the river, but the group is insufficiently afraid of tigers, you might claim there is a lion across the river instead.
If there is a tiger across the river, but you have a shotgun and are itching to get a tiger’s head as a trophy, you might claim there is not a lion across the river, so that we will cross the river. Whether or not you are aware of a lion across the river doesn’t matter. A lion would prevent you from hunting tigers. You want to hunt a tiger.
The Four Direct Communicators
Sticking to the first two levels for now, we can divide into quadrants and see five types of communication strategies.
Let V = “There is a virus across the ocean that is likely to cause a pandemic here.”
Let ~V = “There is not a virus across the ocean that is likely to cause a pandemic here.”
Let C = The consequences of being told V.
Let ~C = The consequences of being told ~V.
Let N = The consequences of being told nothing. For simplicity assume everyone either believes that C>N, or that ~C>N.
The Oracle only looks at Level 1. The Oracle says V if and only if The Oracle believes V with sufficient confidence, says ~V if they believe ~V with sufficient confidence, and says nothing or “I don’t know” otherwise. They care not whether C>~C or ~C>C.
The Trickster only looks at Level 2. The Trickster says V if they think C>~C. They say ~V if they think ~C>C. Otherwise they say nothing. They care not whether V is true.
The Nihilist cares about neither Level 1 nor Level 2. They say whatever they feel like saying, then eat at Arby’s.
The Sage looks at both Level 1 and Level 2 and avoids actions that violate either principle. They say V if both they believe V and they believe C>~C. They say ~V if both they believe ~V and they believe ~C>C. If they believe that V but ~C>C, or they believe that ~V but C>~C, they say nothing.
The Pragmatist looks at both Level 1 and Level 2 and assigns some value to each, then takes action that balances both concerns. They recognize that there is a cost to saying that which is not, but that cost is not infinite. Thus, if The Sage would talk, they talk. If The Sage would not talk, they may talk anyway. Up to a point, they’ll speak the truth and take the direct consequences even if those consequences are bad. Past a certain point, they’ll be willing to lie.
There’s obviously a continuum all around, especially between Sage and Pragmatists. Almost everyone has some breaking point where they would lie for a sufficiently powerful cause. Most people place value on telling the truth beyond known specific and direct consequences, and thus it takes some threshold of bad other consequences to get them to be quiet.
Once you know which of these types a person is, you can trust them in some sense.
It’s easy to interpret an Oracle or a Sage. When a Sage is silent, you can trust that either they don’t know the answer, or they believe telling you the answer would be bad. Often, absent strong glomarization, this lets you figure out the answer.
But it’s also easy, once you know they’re a trickster, to trust a Trickster in their own way. You simply interpret their statements as manipulations rather than observations.
Level 3
“There’s a pandemic headed our way from China” means “I wish to associate with the group that claims there is a pandemic headed our way from China.”
This may include any of the following:
“I want to affirm my membership in my in-group, and that I dislike the out-group.”
“I want to be part of the group of people with power, and/or who are winning.”
“I want to be seen serving those with power and spreading their messages.”
“I want to be seen as smart, on the ball, ahead of the curve, scientific and other neat stuff like that.”
“I want to be part of the group that cares about people.”
“I want to be part of the group that believes/defies experts.”
“I want to be part of the group of so-called ‘responsible experts’ on this.”
“I want to be part of the group that isn’t afraid to tell you hard truths.”
“I want to be part of the group that doesn’t have bad traits like racism.”
“The high status move is to endorse this position at this time.”
“The publication I write for wants to hear this.”
And so on.
“There’s no pandemic headed our way from China” means “I wish to associate with the group that claims there is not a pandemic headed our way from China.”
It may include any or all of exactly the same things, depending on your local situation, and where you are in the timeline.
The polarity of many of these motivations has changed. In some cases, it has changed multiple times.
Level 4
“There’s a pandemic headed our way from China” means “It is advantageous for me to say there is a pandemic headed our way from China.”
“This set of verbal incantations focuses attention on things where focused attention helps me, and away from places where focused attention hurts me.”
“This set of verbal incantations will make people think I am associated and allied with those who it is advantageous for them to think I am associated and allied with.”
“This set of verbal incantations associates me with good words and emotions, or my enemies with bad words and emotions.”
“It would be advantageous for me if the group I am associated with is viewed as advocating the claim that there is a pandemic headed our way from China.”
“This claim fits the heuristics of claims that will make me look responsible.”
“This claim fits the heuristics of claims that will make me look strong and/or powerful and/or in the know and/or a winner.”
“This claim fits the heuristics of claims that give one leverage and/or power.”
“This claim fits the heuristics of claims that will make me someone others think that others will view as a valuable ally, especially others also operating at Level 4.”
“This claim fits the heuristics of claims that will stop me from being scapegoated.”
“This claim fits the heuristics of creating optionality, and/or putting rivals and opponents into situations where they will look bad.”
“There’s no pandemic headed our way from China” means the same thing, except in this situation the additional incantation ‘no’ seems appropriate.
Note that the Level 4 actor has in an important sense lost the ability to think or plan.
It might or might not impact this calculation whether or not your statement is true (Level 1), or whether it will be believed (Level 2), or what coalitions this statement signals your membership in or support of (Level 3). The primary way in which Level 1+2 considerations impact this decision is indirectly, through their impact on Level 3.
That’s all part of a calculation, and matters only to the extent that it effects the consequences of saying the thing.
I find Level 4 is the hardest to grok, by far. It does not come naturally to me.
A potentially easier way to understand Level 4 is to think about how it fits into the contrasts between the first three levels, as will now be discussed.
Level 1 vs. Level 2 vs. Level 3
Consider the first three levels of action and consequence. Level 1 cares about the object level. Level 2 cares about the consequences of changing perception of the object level. Level 3 cares about which coalition your statement associates you with, and which coalitions would approve or disapprove of your statement.
We can think of each of the possible three contrasts. Is Oceania at war with East Asia, Eurasia or both?
We can then add Level 4 into the mix, wherever it belongs based on that division.
Level 1 vs. Levels 2+3+4: Truth versus Untruth
This division feels the most natural to me and people similar to me. Here, Level 4 actions definitely fall into the 2+3 camp, with this full division being level 1 vs. levels 2+3+4.
Level 1 statements correspond to object-level truth. If everyone makes level 1 statements, everyone’s map improves. Decisions get better. Action that has good object-level consequences can be taken. People can be trusted in a pure sense.
Higher level statements corrupt all that. Trust is destroyed. People’s beliefs no longer converge towards the truth. Actions taken are what those with power, those who manipulate and form alliances more effectively, think they want to happen. Since they don’t have the true picture of the situation either, they often don’t like those consequences, and regret their choices to the extent that they care about those consequences.
What difference, this perspective asks, does it make whether you said “Don’t prepare, there’s nothing to worry about” because you wanted to save masks for health care workers, or be able to buy up all the hand sanitizer for resale, or because you were worried about prejudice against Asians, or you wanted to keep the economy open, or you wanted to look responsible and calming rather than irresponsible and alarmist, or that’s what all the authoritative media and government sources were saying and you wanted to be seen as someone who holds the party line?
(Or, at Level 4, if you believe that it will increase the power of your group to be seen as advocating for this position, because it will improve your image or it is popular?)
From this perspective, the only important thing is, you didn’t care if what you said was true. You said it because it was useful to you to say it. That’s what matters.
Thus, we can repeat the 2×2 from above. The only difference is now ‘the consequences’ include coalition politics and other more abstract things.
It is easy to see why primarily Level 1 activities are helpful in dealing with Covid-19. It is also easy to see why one might view all primarily Level 2, 3 and 4 activities as assumed to be unhelpful.
Level 1+3 vs. Levels 2+4: Authentic vs. Inauthentic
This division can definitely be weird when first pointed out or considered, but it makes a decent amount of sense.
If I say that V because I want to show that I am a member of the group that believes V, then that is a signal of group membership rather than evidence for V. But one can see this as an authentic signal of group membership. I really do wish to associate with the V-advocates and not with the anti-Vs.
The risk when sending this signal honestly is that one can confuse my statement of group membership with a claim of V.
I can be making an ‘honest’ statement about which coalition I am supporting, and you can get the wrong impression that V is true.
Or, I could be making an honest statement that V, and you could get the impression that I wish to belong to the V-advocating coalition. This also distorts my map of reality in a potentially dangerous way.
If you can tell when someone is engaging in Level 3 actions versus Level 1 actions, then one can preserve the sanctity and trustworthiness of the Level 1 actions.
The Oracle who only cares about Level 1 is easy to interpret.
So is The Drone, who only cares about Level 3 and will be discussed below. The drone's claim tells you their belief of what their side is currently advocating. No more, no less.
Alternatively, one can be an advocate for a side that presents the case in the best possible light, while only making true statements. This is a variation on the 1+2 strategy of The Sage. One can call this communication strategy The Lawyer. The Lawyer will say only things that have positive impact on level 1 and positive impact on level 3. Alternatively, they might say only things that have positive impact on level 1 and positive combined impact on levels 2 and 3. Or they might need to fulfill all three requirements.
By contrast, statements on Level 2 or Level 4 can be entirely false.
Level 2 does this to manipulate your Level 1 map, and therefore your actions.
Level 4 does this as part of a system that manipulates your Level 3 map, and therefore your actions, believing that this is what drives human action.
If such considerations can dominate, or frequently do, then everything becomes a game.
In this perspective, primarily Level 3 actions are a positive driving force. Such considerations can motivate humans to align with accurate maps and helpful behaviors, under at least some conditions where pure object-level considerations would not work. This is one way we coordinate around washing our hands, locking down or wearing a mask.
Whereas Level 2 and Level 4 actions distort that, and are what lead to inaccurate maps and unhelpful actions.
Levels 1+2 vs. Levels 3+4: Facts vs. Politics
This is natural in the sense that it’s higher levels versus lower levels. One can view the first two levels as caring about the object level at all.
You might lie. But you’re lying because truth matters, beliefs determine physical actions and actions have consequences. Your desire for actions that follow from bad maps of the underlying territory is unfortunate. But at least there are maps of the territory involved, however flawed, and people are trying to cause actions that have consequences they themselves want to occur, even if those consequences are not good for others.
One can view Level 2 statements and actions as a sort of corruption of Level 1, but one still grounded in reality. They’re fighting dirty, but they’re still fighting. One can still speak one’s mind, there is still a marketplace of ideas and over time truth retains a competitive advantage.
Whereas Level 3 is an entirely different thing that has nothing but contempt for the idea that facts matter, or actions have consequences distinct from how they are viewed by others.
Hence the claim that “Facts Don’t Matter.”
Facts Don’t Matter signifies that it does not matter if it is common knowledge that someone is lying.
Thus, Facts Don’t Matter is the dominance of level 3+4 considerations over level 1+2 considerations. Why should I care if the words I say correspond accurately to the physical world’s past, present or future? What matters is their impact on my membership in my coalition, and the success of myself and that coalition in playing political games.
An embodiment of this distinction is the resonance of the statement “I Demand A Plausible Lie.” This is a request to cease purely Level 3+4 behavior and at least adapt some Level 2 considerations. It insists that one be allowed to maintain a map of reality at all outside of politics, and that political considerations be bound at least a little by reality.
To the purely political actor, the implausible lie is better. If the lie is implausible, then those repeating it have sent a costly signal of loyalty, and cut ties with lower levels. You don’t have to worry they repeated the statement because it happens to match the physical world, or that they will refuse to repeat the next one if it fails to match.
Note also that if you only are playing politics, you might be able to act directly in a way that has a direct effect. What you cannot do is make or carry out physical plans involving multiple steps. In the best of times actually planning is very hard. This makes it impossible.
Levels 1+2+4 vs. Level 3: The Drone vs. the Agent
When grouping levels 1+2 against level 3, it feels natural to me to put level 4 with level 3. When talking about this with Ben Hoffman, it became clear there was also a view, where it is level 3 rather than level 4 that feels alien and hard to grok, that naturally groups level four instead with the first two levels.
In this view, the Drone, who cares only about level 3 considerations, is the odd one out.
Anyone acting on any other level is an agent. They are acting on systems. Level 1 acts upon the physical world. Level 2 acts upon other people’s models of the physical world. Level 4 acts upon people’s models of other people and their dynamics.
Whereas the Drone lacks agency and free will entirely. The Drone does what they see others in their group are doing, says what others in their group are saying. More than anyone else, the Drone is a dead player.
Levels 1+2+3 vs. Level 4: The People vs. The Lizards
One could also group the first three levels together and contrast them with the fourth. This thinking is that there is ordinary decent interaction, humans being human, as represented by the first three levels. Then there are the schemers who prey upon us, twist everything in their sick games and play us against each other. We vote for the lizards, as Douglas Adams reminds us, because if we don’t, the wrong lizard might win.
The lizards do not care about Covid-19. It would be a category error to say that the lizards care about things at all. That implies they believe in the existence of things, or prefer one state of those things to another state, and act upon that in some way. That’s not their jam.
Instead of having goals and trying to achieve them, the lizards have systems of power accumulation. They follow habits of behavior that move away from potentially blameworthy actions towards ones that will be seen as good, to sculpt perception of them as powerful and their opponents as weak, and so on. On multiple levels they have a complete inability to plan, even more so than in the last section. They are the politician who prepares two speeches, one pro and one anti, and gives whichever sounds better.
This perspective says the problem is mainly the lizards – or, alternatively, that you want to make sure you are one of the lizards. Get rid of the lizards, and you won’t get a paradise, but you’ll get systems that move towards truth and justice, and that can plan and do useful things.
A Gentle Glossary of Strategies
From here, L-1 is level 1, L-2 is level 2, L-3 is level 3, L-4 is level 4.
Let’s summarize the players. This is what they would say. What they would do is similar.
These players are roles that individuals take in situations. Few people will embody one of them at all times in all circumstances. Sometimes you see a lion across the river.
Nothing: The Nihilist says some things, then eats at Arby’s.
L-1: The Oracle speaks the truth, even if their voice trembles.
L-2: The Trickster says that which causes beliefs that cause the actions they want.
L-1 and L-2: The Sage says only true things that don’t have bad consequences.
L-3: The Drone sings songs and carries signs, mostly saying hurray for our side.
L-1 and L-3: The Lawyer says the true things that comprise the best argument for their position.
L-3 and L-4: The Politician ignores the object level and only considers politics.
L-4: The Lizard trusts their instincts and does that which creates or captures power.
L-All: The Pragmatist balances impact at all levels they are aware of slash care about when deciding what to say.
Several of these roles have important divisions into two or more related but distinct approaches. A key question is whether considerations act as veto points, or if they are weighed against each other. Further discussion is beyond scope here but I hope it happens in the future.
(Think this is missing 1-3 additional roles. Discussion question, what is The Idealist?)
Paths Forward
All actions and statements operate on all four levels at once, to the extent that they have implications on those levels.
The intent of a statement is often entirely on one level. That’s not how humans or Bayesians interpret an action. If you want to improve the physical world without having any higher-level side effects, that’s going to require extra effort. Avoiding meaningful implications on levels 2, 3 and 4 is hard work. The same bleeding effect occurs when aiming for higher levels, as well. For concrete discussion of this, see my previous post on simulacra levels.
As I say in the previous section, many of these roles have multiple variations, and have a lot of complexity inside them. Posts that explore them in detail would be worthwhile.
There is also the issue of the overall simulacra level of a group, organization or civilization. What is the default interpretation of information or action? What is the assumed motivation? What does that say about the group’s dynamics and its ability to do things? I’m still trying to work through these things. There’s clearly a somewhat distinct way of thinking about 3rd and 4th level simulacra that is built around these questions rather than thinking about individual actions. I suspect that the two are fully compatible and describe aspects of the same thing, but I’m still working that out and will talk about it more when I better understand it.
It’s also possible there are two or more different models that are using the same four-level language structure, that share their concepts of levels one and two but disagree about how level four works, and to some extent about level three. The more we talk about it, and the more concrete we can make our examples, the better we can sort all this out. The important thing is to get models that are useful.
The original intent of this post was to go on to analysis of other issues surrounding Covid-19. I was hoping to make clear what I meant by the more disputed statements in my Covid-19 model summary from two weeks ago, and also how and why I believe those dynamics occurred, and what dynamics one can expect going forward. But this post is long enough, so I’ve pushed that into future posts.