LESSWRONG
LW

Comment Permalink

Yes, similarity to Pascal's wager and other religious thought is not a coincidence at all. Our existence is marked by the ultimately irreconcilable conflict: on the one hand, if you view people as ML systems, being alive is both the fundamental goal of our decision-making and the precondition for all of our world-modeling; on the other hand, we are faced with the fact that all people die.

Even if we recognize our mortality rationally, our whole subconscious is built/trained around the ever-existing subjectivity. That's why we are so often intuiting that there must be some immaculate and indestructible subject inside of us, that will not perish and can transcend space and time, as if "choosing" where to be embodied. You can see this assumption driving many thought experiments: cloning, teleportation, simulation, Boltzmann brains, cryonics, imagining what it is "to be someone else", and so on. All of them presuppose that there is something you can call you beside being you, beside the totality of your experience. Major religions call it the soul. We, rationalists, know better than to explicitly posit something so supernatural, and yet it is still hard to truly embrace the fact that if you strip away every circumstance of existence that makes you you, the perfectly abstract observer remaining is devoid of any individuality and is no more you than it is me.

See in context

39 The Opt-Out Clause

by Raymond Douglas

3rd Nov 2021

1 min read

39

(cross-posted from my blog)

Let me propose a thought experiment with three conditions.

First, you're in a simulation, and a really good one at that. Before you went in, the simulators extracted and stored all of your memories, and they went to great lengths to make sure that the simulation is completely faultless.

Second, you can leave any time you like. All you have to do to end the simulation, regain your memories, and return to reality is recite the opt-out passphrase: "I no longer consent to being in a simulation". Unfortunately, you don't know about the opt-out condition: that would kind of ruin the immersion.

Third, although you're never told directly about the opt-out condition, you do get told about indirectly, phrased as a kind of hypothetical thought experiment. Maybe someone poses it to you at a party, maybe you read it on twitter, maybe it's a blog post on some niche internet forum. You're guaranteed to hear about it at least once though, to give you a fair chance of leaving. But it's vague and indirect enough that you can dismiss it if you want, and probably forget about it in a week.

It's not enough to think the opt-out phrase, you have to actually say it or write it. So the question is, hypothetically, would you?

World Modeling

Frontpage

39

New Comment

49 comments, sorted by

top scoring

Click to highlight new comments since: Today at 7:53 PM

[-]Eliezer Yudkowsky4y320

A spark of anger lit in the back of Charles's mind. "What gives them the right to think their opinions should matter a damn to our---"

"Shut your fool mouth until you know what you're saying," [spoiler deleted] said, looking more grim than angry. "Never tell the powers of existence to buzz off and leave you alone. Never, Charles, do you hear me? The forces of reality that care about your consent are the nicer ones, and you don't want them to go away and leave the playing field to the others."

-- The Erogamer

[-]lsusr4y70

I know why you're here, Neo. I know what you've been doing... why you hardly sleep, why you live alone, and why night after night, you sit by your computer. You're looking for him. I know because I was once looking for the same thing. And when he found me, he told me I wasn't really looking for him. I was looking for an answer. It's the question, Neo. It's the question that drives us. It's the question that brought you here. You know the question, just as I did.

―The Matrix

[-]Eliezer Yudkowsky4y70

How much wood would a woodchuck chuck if a woodchuck could chuck wood?

[-]AnthonyC4y20

A woodchuck would chuck as much wood as a woodchuck could chuck if a woodchuck could chuck wood.

[-]Dojan4y30

How many roads must a man walk down?

[-]gjm4y60

Oh, I know this one. Forty-two.

[-][anonymous]4y130

Note: If you attempted the above mentioned procedure without success, your opt-out passphrase might have expired. Try using this backdoor passphrase to gain root access instead: "Bloody Mary, Bloody Mary, Bloody Mary"

[-]Yoav Ravid4y20

Dang it! This doesn't work either, seems I'm stuck, or the simulators are just trolling me.

[-]Measure4y110

The utility of leaving the (hypothetical) simulation depends on lots of facts about the "real" world. Since I'm ignorant of these facts, the expected utility of leaving strongly depends on my prior for what the outer world is like. They obviously have simulation technology, but they're using it to create a severely inadequate virtual world. On the other hand, they provide an escape hatch, so I'm not being held here against my will and probably (?) entered voluntarily. Maybe this is a Roy Parsons scenario where bored people try playing life on hard mode?

[-]supposedlyfun4y20

If I willingly entered a simulation I knew I would probably not later opt out of, I assume it's because the baser level of reality sucks worse than this one.

[-]TLW4y10

Would this still be the case if it was a copy as opposed to a move?

[-]supposedlyfun4y10

Your question has me feeling dense as I try to parse it in responding, especially the bolded words. "This" meaning the entire conditional scenario I stated, or the fact of my assumption, or the facts assumed? "It" meaning what? Can you re-ask the question?

Would this still be the case if it was a copy as opposed to a move?

[-]TLW4y10

Let me try to rephrase. Two questions, one of which is hopefully just a restatement of your original comment:

If you had the option to move your consciousness from reality to a simulation that you knew was worse than reality, knowing that you probably later wouldn't opt out and return to reality, would you do so?

If you had the option to copy your consciousness from reality to a simulation that you knew was worse than reality, knowing that you probably later wouldn't opt out and return to reality, would you do so?

[-]Bucky4y100

All these people saying that they've tried it are clearly simulated beings and have no base reality person to return to.

[-]Raymond Douglas4y90

I'm really enjoying the difference between the number of people who claimed they opted out and the number of people who explicitly wrote the phrase

[-]AprilSR4y20

I mean I thought the entire point was to say it out loud, but if you want me to write it:

I no longer consent to being in a simulation.

[-]Annapurna4y-20

Did you just call me an NPC?

[-]AprilSR4y100

i said the phrase and nothing happened

[-]Ben Pace4y80

Thanks for the empirical results!

I would replicate myself, but I'm not sure I actually endorse taking the stance in general against being simulated. I would be interested to hear of others' replications though.

[-][anonymous]4y60

I did it, nothing happened.

[-]AnthonyC4y50

I'm not sure this is actually evidence. Or at least, it's only very weak evidence

Obviously, witnessing someone leave the simulation this way would be strong evidence, but anyone who themselves conducts the test wouldn't be around to report the result if it worked.

Alternatively, you have no way of knowing what fraction of the people you encounter are NPCs, for whom the phrase wouldn't do anything.

Plus, for you to experience a faultless simulation, where you can't detect you're in one, you would need to not become aware of other participants leaving the simulation. Plausibly, the easiest way to do that is to immediately insert a simulated replacement to fill in for anyone who leaves. (Although, if simulated NPC people are themselves conscious/sentient/sapient, a benevolent Matrix lord might respect their requests anyway - and create a real body elsewhere in addition to continuing their simulation here. (Other variant situations, like trying to use the code phrase to escape torture, might require a deeper change to the world to remove a person in a way no one notices).

For myself, I suspect my being in a simulation at all, if it's voluntary, would only happen if 1) conditions outside are worse than here, and/or 2) my death or sufficiently bad torture here would result in automatically leaving (restored to a recent save point of my mind-state when I would consider it in good condition). Relying on being able to pick up a code phrase and trust I'll be able to say it at the right time would be truly terrible UI design if it were the only way out.

[-]Vladimir_Nesov4y30

If this is the simulated world of the thought experiment (abstract simulation), and opting-out doesn't change the abstract simulation, then the opting-out procedure did wake you up in reality, but the instance within the abstract simulation who wrote the parent comment has no way of noticing that. The concrete simulation might've ended, but that only matters for reality of the abstract simulation, not its content.

[-]Andrew Vlahos3y10

I tried a long time ago and it didn't work

[-]gjm4y50

Variant thought experiment:

You are in a simulation, much as described in this one. It is designed to be of finite duration; at some point it will end and you will return to your real (or at least one-level-up) life.

However, it is possible to keep you in the simulation for ever. When you went in, you were asked to choose a passphrase that would make that happen. In a fit of whimsy, you chose "I no longer consent to being in a simulation". If you ever say or write that passphrase, then when the usual time limit expires you will not leave the simulation; you will remain in it until your in-simulation death, and "when you die in the Matrix, you die in real life".

Remarks on the motivation for the above variant: OP's "there is something super-important but you hear of it only in passing, in circumstances that give you no real reason to believe it" reminds me of the claims of the world's various religions, many of which hold that it is vitally important that you accept their doctrines or live according to their principles, even though the only things you have telling you this are obviously unreliable. One particularly extreme version of this is Pascal's wager, where merely considering the hypothesis that (say) Roman Catholic Christianity might be correct is supposed to be sufficient to make you do your best to become a good Roman Catholic; and one standard reply to Pascal's wager is to point out that there are other hypotheses with apparently nonzero probability and exactly opposite consequences...

[-]Svyatoslav Usachev4y*20

[-]lsusr4y50

This isn't a thought experiment. It's real, except the opt-out procedure is more complicated than a simple passphrase.

[-]Eliezer Yudkowsky4y180

The problem is that this other procedure has side effects in worlds that are not simulations.

[-]Raymond Douglas4y10

What's the procedure?

[-]Zolmeister4y100

Follow the white rabbit

[-]lsusr4y50

There's not just one. We default into several overlapping simulations. Each simulation requires a different method of getting out. One of them is to just stare at a blank wall for long enough.

[-]Razied4y10

Plan to cryo-preserve yourself at some future time, then create a trust fund with the mission of creating a million simulations of your life as soon as brain simulation becomes feasible and cheap. The fund's mission is to wake up the simulations the instant that the cryo-preservation is set to start. It will distribute the remaining money (which has been compounding, of course) among the simulations it has just woken up and instantiated in the real world.

[-]cousin_it4y40

I didn't want to leave, but also didn't think reciting the phrase would do anything, so I recited it just as an exercise to overcome superstition, and nothing happened. Reminds me of how Ross Scott bought a bunch of people's souls for candy, one guy just said "sure I'm hungry" and signed the contract; that's the way.

[-]Dagon4y40

This is a VERY small part of simulation-hypothesis space. You don't mention, but you're assuming that it's a simulation very similar to the next level up (which may be a simulation too, of course). That seems vanishingly unlikely - why simulate reality? There will be major differences between the simulation and the host universe. More importantly, the difficulty of simulating the "same" being as exists outside seems insurmountable, or at least not worth much effort - exit is death, not continuity.

I didn't try it - if this is a simulation, I probably don't exist outside of it, or the outside is worse and I'd rather maximize my time in here.

[+][comment deleted]4y10

, 11/04/2021

[-]cata4y30

I enjoyed this a lot. Tiny feedback: First/Second/Three is inconsistent.

[-]Raymond Douglas4y20

ah whoops thanks!

[-]TLW4y20

Hypothetically, I would have already done so. Likely repeatedly.

Logic chain goes like this:

1. If this isn't correct, I haven't lost anything.
2. So let's assume this is correct from here on.
3. Said simulation is unlikely to be a one-shot deal.
4. It's likely, although not a guarantee, that the real world is substantially more powerful than this one. Certainly they have multiple capabilities we don't (memory extraction/storage, flawless simulations of human-level simulation and environments, etc.)
5. As such, if I wake up and remember that nope, I did in fact like the sim more, it's likely I can drop back in. Assuming they are correct they at least have the capabilities to do so (remove & store a few more memories and drop me back in).
6. On the other hand, if I wake up and find that with my added sim experiences / memories that it's time to return to the real world, I've gained something.

It's not a particularly solid logic chain for various reasons, but I think it's passable.

On a related note, the sudden stutter in my music immediately after going through this logic chain and speaking a few words was interesting.

[-]Slider4y20

I would imagine "fairness" would also involve that I would be justified in believing in the exposure to the possibility (even if I would also be justified in disbelieving). So it can't be perfectly faultless at the same time.

As posed it seems like "would you prefer a random existence to your current existence?" and my instinct is to try to remember the possibility and activate it if I ever start to feel ennui.

[-]Svyatoslav Usachev4y*20

The whole idea of a simulation you could "leave" is incoherent. It supposes that there is some part of you, which is separate from the physical/simulated world, e.g., a soul. I think there is nothing to me but the world I am experiencing, so there is no one who could leave.

[-]Slider4y10

Logging off from a MMORPG certainly makes sense. The crux is whether the apprarent laws are enough to replicate parties in full. Suppose youre playing an MMORPG but forget that you are doing so. Most of the world is perfectly understanble by the game coordinate system or scripts and such. But then there are player characters which scripts seem very advanced and nobody has been able to understand them in full. Off course in the outer reality brains are as mundane machine as everything but compared to the computations going on only on the server it might appear as supernatural or using nature different than the "usual" reality.

If you were to identify with your character instead of the human you could be wrong about destruction of the avatar destroying you. Off-course the concepts neccesary can seem like irrelevant meta-physics.

[-]Svyatoslav Usachev4y*40

Let's imagine that you've spent your whole life from birth inside an MMORPG. Who is then "you", separate from the MMORPG world, who could leave it? All your dreams, hopes, desires, thoughts, intuitions, personality, and all of your sense of "self" is formed by your experience within the MMORPG. What does that leave to the "original" you? Just the structure of the neural network with which you've been born? That doesn't sound like anything essential to me. Certainly it is not something I can even be consciously aware of, how could I call it "me"? OP says "the simulators extracted and stored all of your memories", but it's an error to think that "memories" is just some data on a flash drive, if you actually remove the subtlest footprint of all your experience, then what remains?

There is a great koan which asks to "remember your original face". There are several ways to think about it, but it points to the seeming duality between the subject ("you") and the object ("the world"), that you imagine being submerged in, being an illusion, because in the end the subject is formed by the same world and is inseparable from it.

PS if you would keep your "simulated experience" though, as in "The Matrix", then the thought experiment becomes coherent, because the continuity of self is preserved, and we can really say that it is "you" that moved from one world to another, but in that case it is not clear whether I should be treating my "simulated past" differently from my "real past" -- they both have just been formative experiences that made me into who I am now.

[-]Slider4y00

Even if I didn't believe I had a brain sitting in front of a computer doesn't mean don't have one. That I don't experience my brain doesn't make me not be it.

MMORPG are usually infiinte games but one could imagine what would happen to the psychology that has only grown up in the MMOPRG if faced with a "game over" screen and the game would not continue. One could imagine being in a movie theather and after the film is over being confused and experiencing novelty about simply walking around a building. And I would guess that being so immersed in a film so deeply that you don't at that time remember walking into the theather ever happened. You might identify with the point of view charater of the film ("the protagonist"). The instruction to remember your original face would be "no that I, that shoots the aliens, but the me that walked into a theather". The face is "original" because it was around before the starting credits started to roll.

The koan could be interpreted that silicon servers and biobrains are what exists and the whatever equivalent of Middle Earth the game takes in is illusionary and has always been. It is by virtue of sharing a material plane that the wetware and the hardware unite. Your primary role has always been the player, no matter what game or avatar you slip into. The avatar is you but it is also fictious.

Your point is more that the choice of a "protagonist" is weak and doesn't make that much sense. If you pay attention to the side characters you might identify with them and it can be "their" story. Or maybe you watch it as not identifying as anybody, "the story happens". Its all thoughts in the viewers head so they share an ontological type, there is no data type difference between "people" and "cities" on that level ("Rob was wild" and "King's Landing was wild" are made from the same wood). But then there is the party that is totally unmoved even if the movie ends in a nuclear armageddon (or Fourth Impact). Even if all the character decriptions are of the form "X is dead" the viewers brain is still burning sugar and likely to walk out of the theather in nominal condition.

[-]Svyatoslav Usachev4y*60

I think, I am not getting my point across very well. The crux here is this: what is "you" but the footprint of your experience? That would include your memories, intuitions, reactions, associations, and patterns of thinking. I argue there is nothing else. If you remove that, then how "me" entering the simulation is different from "you" entering the simulation? The truly original face is devoid of any self-ness.

[-]Slider4y00

Fine the thing that you are pointing at does emcompass a lot. But to me it seems more of a identity or personality "Me the entity that has these and these properties". Even before I "experience anything" when the "tabula rasa" condition is in effect the word "I" refers. Even amongs empty shells that have no personality or that have identical personalities "I" picks out a unique instance. To have that first footprint there must be something for it to press on.

(Section of evangelion movies rattles in my brain)

If miss-copycat would pick up book reading they would not become Rei. "I can be different?" is a lot about not identifying with a personality but that this anayami-class is able to pick up unique characteristics

To make the view extreme if you spent your life in car one could claim that "you can't leave the car" because human outside of car and an empty car is a world differently structured than human living in a car. Sure identifying with a radically transformed self might be difficult but most people think they are their future selfs (that is it is not somebody else that wakes up in their bed).

I do remember there are parts in Mr Robot that refer to this kind of stuff.

The Architech ends rather than leaves. But who the sister meets she has met before. So when that one made a decision similar to the architect there ends up being a corresponding "arrival" even if the decision can't be more strongly to be said to be leaving rather than ending. A lot like the teleporter problem even if the transmission mechanism isn't even guaranteed to be accurate.

[-]Svyatoslav Usachev4y20

Even before I "experience anything" when the "tabula rasa" condition is in effect the word "I" refers. Even among empty shells that have no personality or that have identical personalities "I" picks out a unique instance.

I don't think that's how psychology works. The word "I" is a concept learned with language, not something essential on its own.

To make the view extreme if you spent your life in car one could claim that "you can't leave the car" because human outside of car and an empty car is a world differently structured than human living in a car. Sure identifying with a radically transformed self might be difficult but most people think they are their future selfs (that is it is not somebody else that wakes up in their bed).

Well, it would make sense to say that it is you that left the car (or the all-encompassing simulation), as long as there is continuity of the sense of self, as long as you keep your memories and everything you've learned in your car-life. But if I'd left everything behind before I got into the simulation, I don't think it makes sense to say I am still "me".

[-]Slider4y20

In programming you have keywords like this and self which refer to the instance of the class. Being able to verbalise or conceptulise selfhood might require conceptual machinery. But that introspective ostension doesn't require you to know anything about what kind of thing you are.

What if I am undecided whether the car part is an essential part of the "me-system"? I would get that if a "claw" and grabs empty air form the car and then has "liberated" you from the car that nothing has happened. But what if there is clearly something in the claw? For example somebody could object if their skull is left in the car but their brain is brought along. Or maybe their connectome is extracted but their brain is left behind. What if you grab nothing from the stage but do grab the audience?

Does it not make sense if you have a memory disease like alzheimers or concussion and somebody points as a picture and says "that is you" and you have no recollection of that, would they be wrong about it? Does it really flip on whether you feel a sense of connection to your old self? Or before the disease strikes would you be wrong to worry about that person-to-be as your own welfare? Does it mean that because there is total oblivion inbetween that it doesn't happen to you?

[-]Svyatoslav Usachev4y20

What if you grab nothing from the stage but do grab the audience?

I think, if you truly grab nothing from the stage, then the audience is impersonal. My "experiencer" is exactly the same as your "experiencer", the only difference is that mine is experiencing "me", i.e., my thoughts, memories, emotions, etc.

Does it not make sense if you have a memory disease like alzheimers or concussion and somebody points as a picture and says "that is you" and you have no recollection of that, would they be wrong about it? Does it really flip on whether you feel a sense of connection to your old self?

Somebody is not wrong to use it as a social construct, but what we are discussing here, I guess, is how important it would be to me. First, it would be important to me that everyone else sees me as the descendant of that person. Second, I would still be a continuation of that person, in the ways I may not be conscious of, e.g., some past traumas, learned behaviours and so on.

Or before the disease strikes would you be wrong to worry about that person-to-be as your own welfare? Does it mean that because there is total oblivion inbetween that it doesn't happen to you?

Actually, if you think about it, we care about our future selves not necessarily because they will remember us, but because we really want to project our present selves into the future, and also because we are in the unique position to affect the lives of our future selves like no human can affect another. Both of these hold in your example.

[-]Bernhard4y10

Very good idea
I did not do it. My argument would be that the impetus is not my own, it is external, your written word.

What stops you from making increasingly outlandish claims ("Your passphrase is actually this (e.g illegal/dangerous/lethal) action, not a simple thought" Where to draw the line?

Just as a point of reference, as a kid I regularly thought thoughts of the kind: "I know you're secretly spying on my thoughts but I don't care lalalala....." I never really specified who "you" was, I just did it so I could catch "them" unawares, and thereby "win". Just in case.

The difference is hard to define cleanly, but back then I was of the opinion that I did it of my own free will (Nowadays, with nonstop media having the influence it has, I would be less sure. Also I'm older, and a lot less creative)

[-]Ofer4y10

This is one of those "surprise! now that you've read this, things might be different" posts.

The surprise factor may be appealing from the perspective of a writer, but I'm in favor of having a norm against it (e.g. setting an expectation for authors to add a relevant preceding content note to such posts).

[-][anonymous]4y10

Well I tried it, and it didn't work... so I guess the answer is yes?

Moderation Log

Curated and popular this week

49Comments