Training Reflective Attention

21 BrienneYudkowsky 21 December 2014 12:53PM

Crossposted at Agenty Duck

And somewhere in the back of his mind was a small, small note of confusion, a sense of something wrong about that story; and it should have been a part of Harry's art to notice that tiny note, but he was distracted. For it is a sad rule that whenever you are most in need of your art as a rationalist, that is when you are most likely to forget it. —HPMOR, Ch. 3

A rationalist’s art is most distant when it is most needed. Why is that?

When I am very angry with my romantic partner, what I feel is anger. I don’t feel the futility of throwing a tantrum, or the availability of other options like honest communication, or freewriting, or taking a deep breath. My attention is so narrowly focused on the object of my anger that I’m likely not even aware that I’m angry, let alone that my anger might be blinding me to my art.

When her skills are most needed, a rationalist is lost in an unskillful state of mind. She doesn’t recognize that it’s happening, and she doesn’t remember that she has prepared for it by learning and practicing appropriate techniques.

I've designed and exercise that trains a skill I call reflective attention, and some call mindfulness. For me, it serves as an anchor in a stormy mind, or as a compass pointing always toward a mental state where my art is close at hand.

Noticing that I am lost in an unskillful state of mind is a separate skill. But when I do happen to notice—when I feel that small, small note of confusion—reflective attention helps me find my way back. Instead of churning out even more pointless things to yell at my partner, it allows me to say, “I am angry. I feel an impulse to yell. I notice my mind returning over and over to the memory that makes me more angry. I’m finding it hard to concentrate. I am distracted. I have a vague impression that I have prepared for this.” And awareness of that final thought allows me to ask, “What have I trained myself to do when I feel this way?”

The goal of the following exercise is to practice entering reflective attention.

It begins with an instruction to think of nothing. When you monitor yourself to make sure you’re not having any thoughts, your attention ends up directed toward the beginnings of thoughts. Since the contents of consciousness are always changing, maintaining focus on the beginnings of thoughts prevents you from engaging for an extended period with any particular thought. It prevents you from getting “lost in thought”, or keeping attention focused on a thought without awareness of doing so. The point is not actually to be successful at thinking nothing, as that is impossible while conscious, but to notice what happens when you try.

Keeping your focus on the constant changes in your stream of consciousness brings attention to your experience of awareness itself. Awareness of awareness is the anchor for attention. It lets you keep your bearings when you’d otherwise be carried away by a current of thought or emotion.

Once you’re so familiar with the feeling of reflection that creating it is a primitive action, you can forget the introductory part, and jump straight to reflective attention whenever it occurs to you to do so.


This will probably take around five minutes, but you can do it for much longer if you want to.

Notice what your mind is doing right now. One thing it’s doing is experiencing sensations of black and white as you read. What else are you experiencing? Are there words in your inner monologue? Are there emotions of any kind?

Spend about thirty seconds trying not to think anything. When thirty seconds is up, stop trying not to think, and read on.

.

.

.

What’s happening in your mind is constantly changing. Even when you were trying not to think, you probably noticed many times when the stillness would shift and some new thought would begin to emerge in conscious awareness.

Turn your attention to those changes. When a new thought emerges in consciousness, see if you can notice the exact moment when it happens, becoming aware of what it feels like for that particular change to take place.

If it helps at first, you can narrate your stream of consciousness in words: “Now I’m seeing the blue of the wall, now I’m hearing the sound of a car, now I’m feeling cold, now I’m curious what time it is…” You’ll probably find that you can’t narrate anywhere near quickly enough, in part because thoughts can happen in parallel, while speech is serial. Once narrating starts to become frustrating, stop slowing yourself down with words, and just silently observe your thoughts as they occur.

If you’re finding this overwhelming because there are too many thoughts, narrow your focus down to just your breathing, and try to precisely identify the experience of an exhale ending and an inhale beginning, of an inhale ending and an exhale beginning. Keep doing that until you feel comfortable with it, and then slowly expand your attention a little at a time: to other experiences associated with breathing, to non-breath-related bodily sensations, to non-tactile sensations from your environment, and finally to internal mental sensations like emotions.

If you notice an impulse to focus your attention on a particular thought, following it and engaging with it—perhaps you notice you feel hungry, and in response you begin to focus your attention on planning lunch—instead of letting that impulse take over your attention, recognize it as yet another change in the activity of your mind. If you’re narrating, say, “now I’m feeling an impulse to plan my lunch”, and keep your focus broad enough to catch the next thought when it arises. If you realize that you’ve already become lost in a particular thought, notice that realization itself as a new thought, and return to observing your stream of consciousness by noticing the next new thought that happens as well.

.

.

.

You might need to practice this many times before you get the hang of it. I suggest trying it for ten minutes to half an hour a day until you do.

Once you feel like you can recognize the sensation of reflective attention and enter that state of mind reliably given time, begin to train for speed. Instead of setting a timer for fifteen minutes or however long you want to practice, set it to go off every minute for the first half of your practice, spending one minute in reflective attention, and one minute out. (Don’t do this for all of your practice. You still need to practice maintenance.) When you can consistently arrive in reflective attention by the end of the minute, cut the intervals down to 45 seconds, then thirty, fifteen, and five.


In real life, the suspicion that you may be lost in an unskillful state of mind will be quiet and fleeting. “Quiet” means you’ll need to learn to snap your attention to the slightest hint of that feeling. For that, you’ll need to train “noticing”. “Fleeting” means you’ll need to be able to respond in less than five seconds. You’ll need to begin the process in less than one second, even if it takes a little longer to fully arrive in reflective attention. For that, training for speed is crucial.

Simulate and Defer To More Rational Selves

125 BrienneYudkowsky 17 September 2014 06:11PM

I sometimes let imaginary versions of myself make decisions for me.

I first started doing this after Anna told me (something along the lines of) this story. When she first became the executive director of CFAR, she suddenly had many more decisions to deal with per day than ever before. "Should we hire this person?" "Should I go buy more coffee for the coffee machine, or wait for someone else deal with it?" "How many participants should be in our first workshop?" "When can I schedule time to plan the fund drive?" 

I'm making up these examples myself, but I'm sure you, too, can imagine how leading a brand new organization might involve a constant assault on the parts of your brain responsible for making decisions. She found it exhausting, and by the time she got home at the end of the day, a question like, "Would you rather we have peas or green beans with dinner?" often felt like the last straw. "I don't care about the stupid vegetables, just give me food and don't make me decide any more things!"

She was rescued by the following technique. When faced with a decision, she'd imagine "the Executive Director of CFAR", and ask herself, "What would 'the Executive Director of CFAR' do?" Instead of making a decision, she'd make a prediction about the actions of that other person. Then, she'd just do whatever they'd do!

(I also sometimes imagine what Anna would do, and then do that. I call it "Annajitsu".)

In Anna's case, she was trying to reduce decision fatigue. When I started trying it out myself, I was after a cure for something slightly different.

Imagine you're about to go bungee jumping off a high cliff. You know it's perfectly safe, and all you have to do is take a step forward, just like you've done every single time you've ever walked. But something is stopping you. The decision to step off the ledge is entirely yours, and you know you want to do it because this is why you're here. Yet here you are, still standing on the ledge. 

You're scared. There's a battle happening in your brain. Part of you is going, "Just jump, it's easy, just do it!", while another part--the part in charge of your legs, apparently--is going, "NOPE. Nope nope nope nope NOPE." And you have this strange thought: "I wish someone would just push me so I don't have to decide."

Maybe you've been bungee jumping, and this is not at all how you responded to it. But I hope (for the sake of communication) that you've experienced this sensation in other contexts. Maybe when you wanted to tell someone that you loved them, but the phrase hovered just behind your lips, and you couldn't get it out. You almost wished it would tumble out of your mouth accidentally. "Just say it," you thought to yourself, and remained silent. For some reason, you were terrified of the decision, and inaction felt more like not deciding.

When I heard this story from Anna, I had social anxiety. I didn't have way more decisions than I knew how to handle, but I did find certain decisions terrifying, and was often paralyzed by them. For example, this always happened if someone I liked, respected, and wanted to interact with more asked to meet with them. It was pretty obvious to me that it was a good idea to say yes, but I'd agonize over the email endlessly instead of simply typing "yes" and hitting "send".

So here's what it looked like when I applied the technique. I'd be invited to a party. I'd feel paralyzing fear, and a sense of impending doom as I noticed that I likely believed going to the party was the right decision. Then, as soon as I felt that doom, I'd take a mental step backward and not try to force myself to decide. Instead, I'd imagine a version of myself who wasn't scared, and I'd predict what she'd do. If the party really wasn't a great idea, either because she didn't consider it worth my time or because she didn't actually anticipate me having any fun, she'd decide not to go. Otherwise, she'd decide to go. I would not decide. I'd just run my simulation of her, and see what she had to say. It was easy for her to think clearly about the decision, because she wasn't scared. And then I'd just defer to her.

Recently, I've noticed that there are all sorts of circumstances under which it helps to predict the decisions of a version of myself who doesn't have my current obstacle to rational decision making. Whenever I'm having a hard time thinking clearly about something because I'm angry, or tired, or scared, I can call upon imaginary Rational Brienne to see if she can do any better.

Example: I get depressed when I don't get enough sunlight. I was working inside where it was dark, and Eliezer noticed that I'd seemed depressed lately. So he told me he thought I should work outside instead. I was indeed a bit down and irritable, so my immediate response was to feel angry--that I'd been interrupted, that he was nagging me about getting sunlight again, and that I have this sunlight problem in the first place. 

I started to argue with him, but then I stopped. I stopped because I'd noticed something. In addition to anger, I felt something like confusion. More complicated and specific than confusion, though. It's the feeling I get when I'm playing through familiar motions that have tended to lead to disutility. Like when you're watching a horror movie and the main character says, "Let's split up!" and you feel like, "Ugh, not this again. Listen, you're in a horror movie. If you split up, you will die. It happens every time." A familiar twinge of something being not quite right.

But even though I noticed the feeling, I couldn't get a handle on it. Recognizing that I really should make the decision to go outside instead of arguing--it was just too much for me. I was angry, and that severely impedes my introspective vision. And I knew that. I knew that familiar not-quite-right feeling meant something was preventing me from applying some of my rationality skills. 

So, as I'd previously decided to do in situations like this, I called upon my simulation of non-angry Brienne. 

She immediately got up and went outside.

To her, it was extremely obviously the right thing to do. So I just deferred to her (which I'd also previously decided to do in situations like this, and I knew it would only work in the future if I did it now too, ain't timeless decision theory great). I stopped arguing, got up, and went outside. 

I was still pissed, mind you. I even felt myself rationalizing that I was doing it because going outside despite Eliezer being wrong wrong wrong is easier than arguing with him, and arguing with him isn't worth the effort. And then I told him as much over chat. (But not the "rationalizing" part; I wasn't fully conscious of that yet.)

But I went outside, right away, instead of wasting a bunch of time and effort first. My internal state was still in disarray, but I took the correct external actions. 

This has happened a few times now. I'm still getting the hang of it, but it's working.

Imaginary Rational Brienne isn't magic. Her only available skills are the ones I have in fact picked up, so anything I've not learned, she can't implement. She still makes mistakes. 

Her special strength is constancy

In real life, all kinds of things limit my access to my own skills. In fact, the times when I most need a skill will very likely be the times when I find it hardest to access. For example, it's more important to consider the opposite when I'm really invested in believing something than when I'm not invested at all, but it's much harder to actually carry out the mental motion of "considering the opposite" when all the cognitive momentum is moving toward arguing single-mindedly for my favored belief.

The advantage of Rational Brienne (or, really, the Rational Briennes, because so far I've always ended up simulating a version of myself that's exactly the same except lacking whatever particular obstacle is relevant at the time) is that her access doesn't vary by situation. She can always use all of my tools all of the time.

I've been trying to figure out this constancy thing for quite a while. What do I do when I call upon my art as a rationalist, and just get a 404 Not Found? Turns out, "trying harder" doesn't do the trick. "No, really, I don't care that I'm scared, I'm going to think clearly about this. Here I go. I mean it this time." It seldom works.

I hope that it will one day. I would rather not have to rely on tricks like this. I hope I'll eventually just be able to go straight from noticing dissonance to re-orienting my whole mind so it's in line with the truth and with whatever I need to reach my goals. Or, you know, not experiencing the dissonance in the first place because I'm already doing everything right.

In the mean time, this trick seems pretty powerful.

What It's Like to Notice Things

32 BrienneYudkowsky 17 September 2014 02:19PM

Phenomenology

Phenomenology is the study of the structures of experience and consciousness. Literally, it is the study of "that which appears". The first time you look at a twig sticking up out of the water, you might be curious and ask, "What forces cause things to bend when placed in water?" If you're a curious phenomenologist, though, you'll ask things like, "Why does that twig in water appear as though bent? Do other things appear to bend when placed in water? Do all things placed in water appear to bend to the same degree? Are there things that do not appear to bend when placed in water? Does my perception of the bending depend on the angle or direction from which I observe the twig?"

Pehenomenology means breaking experience down to its more basic components, and being precise in our descriptions of what we actually observe, free of further speculation and assumption. A phenomenologist recognizes the difference between observing "a six-sided cube", and observing the three faces, at most, from which we extrapolate the rest.

I consider phenomenology to be a central skill of rationality. The most obvious example: You're unlikely to generate alternative hypotheses when the confirming observation and the favored hypothesis are one and the same in your experience of experience. The importance of phenomenology to rationality goes deeper than that, though. Phenomenology trains especially fine grained introspection. The more tiny and subtle are the thoughts you're aware of, the more precise can be the control you gain over the workings of your mind, and the faster can be your cognitive reflexes.

(I do not at all mean to say that you should go read Husserl and Heidegger. Despite their apparent potential for unprecedented clarity, the phenomenologists, without exception, seem to revel in obfuscation. It's probably not worth your time to wade through all of that nonsense. I've mostly read about phenomenology myself for this very reason.)

I've been doing some experimental phenomenology of late.

Noticing

I've noticed that rationality, in practice, depends on noticing. Some people have told me this is basically tautological, and therefore uninteresting. But if I'm right, I think it's likely very important to know, and to train deliberately.

The difference between seeing the twig as bent and seeing the twig as seeming bent may seem inane. It is not news that things that are bent tend to seem bent. Without that level of granularity in your observations, though, you may not notice that it could be possible for things to merely seem bent without being bent. When we're talking about something that may be ubiquitous to all applications of rationality, like noticing, it's worth taking a closer look at the contents of our experiences.

Many people talk about "noticing confusion", because Eliezer's written about it. Really, though, every successful application of a rationality skill begins with noticing. In particular, applied rationality is founded on noticing opportunities and obstacles. (To be clear, I'm making this up right this moment, so as far as I know it's not a generally agreed-upon thing. That goes for nearly everything in this post. I still think it's true.) You can be the most technically skilled batter in the world, and it won't help a bit if you consistently fail to notice when the ball whizzes by you--if you miss the opportunities to swing. And you're not going to run very many bases if you launch the ball straight at an opposing catcher--if you're oblivious to the obstacles.

It doesn't matter how many techniques you've learned if you miss all the opportunities to apply them, and fail to notice the obstacles when they get in your way. Opportunities and obstacles are everywhere. We can only be as strong as our ability to notice the ones that will make a difference.

Inspired by Whales' self-experiment in noticing confusion, I've been practicing noticing things. Not difficult or complicated things, like noticing confusion, or noticing biases. I've just been trying to get a handle on noticing, full stop. And it's been interesting.

Noticing Noticing

What does it mean to notice something, and what does it feel like?

I started by checking to see what I expected it to feel like to notice that it's raining, just going from memory. I tried for a split-second prediction, to find what my brain automatically stored under "noticing rain". When I thought about noticing rain, I got this sort of vague impression of rainyness, which included few sensory details and was more of an overall rainy feeling. My brain tried to tell me that "noticing rain" meant "being directly acquainted with rainyness", in much the same way that it tries to tell me it's experiencing a cube when it's actually only experiencing a pattern of light and shadows I interpret as three faces.

Then, I waited for rain. It didn't take long, because I'm in North Carolina for the month. (This didn't happen last time I was in North Carolina, so perhaps I just happened to choose The One Valley of Eternal Rain.)

The real "noticing rain" turned out to be a response to the physical sensations concurrent with the first raindrop falling on my skin. I did eventually have an "abstract rainyness feeling", but that happened a full two seconds later. My actual experience went like this.

It was cloudy and humid. This was not at the forefront of my attention, but it slowly moved in that direction as the temperature dropped. I was fairly focused on reading a book.

(I'm a little baffled by the apparent gradient between "not at all conscious of x" and "fully aware of x". I don't know how that works, but I experience the difference between being a little aware of the sky being cloudy and being focused on the patterns of light in the clouds, as analogous to the difference between being very-slightly-but-not-uncomfortably warm and burning my hand on the stove.)

My awareness of something like an "abstract rainyness feeling" moved further toward consciousness as the wind picked up. Suddenly--and the suddenness was an important part of the experience--I felt something like a cool, dull pin-prick on my arm. I looked at it, saw the water, and recognized it as a raindrop. Over the course of about half a second, several sensations leapt forward into full awareness: the darkness of my surroundings, the humidity in the air, the dark grey-blueness of the sky, the sound of rain on leaves like television static, the scent of ozone and damp earth, the feeling of cool humid wind on my face, and the word "rain" in my internal monologue.

I think it is that sudden leaping forward of many associated sensations that I would call "noticing rain".

After that, I felt a sort of mental step backward--though it was more like a zooming out or sliding away than a discrete step--from the sensations, and then a feeling of viewing them from the outside. There was a sensation of the potential to access other memories of times when it's rained.

(Sensations of potential are fascinating to me. I noticed a few weeks ago that after memorizing a list of names and faces, I could predict in the first half second of seeing the face whether or not I'd be able to retrieve the name in the next five seconds. Before I actually retrieved the name. What??? I don't know either.)

Only then did all of it resolve into the more distant and abstract "feeling of rainyness" that I'd predicted before. The resolution took four times as long as the simultaneous-leaping-into-consciousness-of-related-sensations that I now prefer to call "noticing", and ten times as long as the first-raindrop-pin-prick, which I think I'll call the "noticing trigger" if it turns out to be a general class of pre-noticing experiences.

("Can you really distinguish between 200 and 500 milliseconds?" Yes, but it's an acquired skill. I spent a block of a few minutes every day for a month, then several blocks a day for about a week, doing this Psychomotor Vigiliance Task when I was gathering data for the polyphasic sleep experiment. (No, I'm sorry, to the best of my knowledge Leverage has not yet published anything on the results of this. Long story short: Everyone who wasn't already polyphasic is still not polyphasic today.) It gives you fast feedback on simple response time. I'm not sure if it's useful for anything else, but it comes in handy when taking notes on experiences that pass very quickly.)

Noticing Environmental Cues

My second experiment was in repeated noticing. This is more closely related to rationality as habit cultivation.

Can I get better at noticing something just by practicing?

I was trying to zoom in on the experience of noticing itself, so I wanted something as simple as possible. Nothing subtle, nothing psychological, and certainly nothing I might be motivated to ignore. I wanted a straightforward element of my physical environment. I'm out in the country and driving around for errands and such about once a day, so I went with "red barn roofs".

I had an intuition that I should give myself some outward sign of having noticed, lest I not notice that I noticed, and decided to snap my fingers every time I noticed a red barn roof.

On the first drive, I noticed one red barn roof. That happened when I was almost at my destination and I thought, "Oh right, I'm supposed to be noticing red barn roofs, oops" then started actively searching for them.

Noticing a red barn roof while searching for it feels very different from noticing rain while reading a book. With the rain, it felt sort of like waking up, or like catching my name in an overheard conversation. There was a complete shift in what my brain was doing. With the barn roof, it was like I had a box with a red-barn-roof-shaped hole, and it felt like completion when a I grabbed a roof and dropped it through the hole. I was prepared for the roof, and it was a smaller change in the contents of consciousness.

I noticed two on the way back, also while actively searching for them, before I started thinking about something else and became oblivious.

I thought that maybe there weren't enough red barn roofs, and decided to try noticing red roofs of all sorts of buildings the next day. This, it turns out, was the correct move.

On day two of red-roof-noticing, I got lots of practice. I noticed around fifteen roofs on the way to the store, and around seven on the way back. By the end, I was not searching for the roofs as intently as I had been the day before, but I was still explicitly thinking about the project. I was still aware of directing my eyes to spend extra time at the right level in my field of vision to pick up roofs. It was like waving the box around and waiting for something to fall in, while thinking about how to build boxes.

I went out briefly again on day two, and on the way back, I noticed a red roof while thinking about something else entirely. Specifically, I was thinking about the possibility of moving to Uruguay, and whether I knew enough Spanish to survive. In the middle of one of those unrelated thoughts, my eyes moved over a barn roof and stayed there briefly while I had the leaping-into-consciousness experience with respect to the sensations of redness, recognizing something as shaped like a building, and feeling the impulse to snap my fingers. It was like I'd been wearing the box as a hat to free up my hands, and I'd forgotten about it. And then, with a heavy ker-thunk, the roof became my new center of attention.

And oh my gosh, it was so exciting! It sounds so absurd in retrospect to have been excited about noticing a roof. But I was! It meant I'd successfully installed a new cognitive habit to run in the background. On purpose. "Woo hoo! Yeah!" (I literally said that.)

On the third day, I noticed TOO MANY red roofs. I followed the same path to the store as before, but I noticed somewhere between twenty and thirty red roofs. I got about the same number going back, so I think I was catching nearly all the opportunities to notice red roofs. (I'd have to do it for a few days to be sure.) There was a pattern to noticing, where I'd notice-in-the-background, while thinking about something else, the first roof, and then I'd be more specifically on the lookout for a minute or two after that, before my mind wandered back to something other than roofs. I got faster over time at returning to my previous thoughts after snapping my fingers, but there were still enough noticed roofs to intrude uncomfortably upon my thoughts. It was getting annoying.

So I decided to switch back to only noticing the red roofs of barns in particular.

Extinction of the more general habit didn't take very long. It was over by the end of my next fifteen minute drive. For the first three times I saw a roof, I rose my hand a little to snap my fingers before reminding myself that I don't care about non-barns anymore. The next couple times I didn't raise my hand, but still forcefully reminded myself of my disinterest in my non-barns. The promotion of red roofs into consciousness got weaker with each roof, until the difference between seeing a non-red non-barn roof and a red non-barn roof was barely perceptible. That was my drive to town today.

On the drive back, I noticed about ten red barn roofs. Three I noticed while thinking about how to install habits, four while thinking about the differences between designing exercises for in-person workshops and designing exercises to put in books, and three soon enough after the previous barn to probably count as "searching for barns".

So yes, for at least some things, it seems I can get better at noticing them my  by practicing.

What These Silly Little Experiments Are Really About

My plan is to try noticing an internal psychological phenomenon next, but still something straightforward that I wouldn't be motivated not to notice. I probably need to try a couple things to find something that works well. I might go with "thinking the word 'tomorrow' in my internal monologue", for example, or possibly "wondering what my boyfriend is thinking about". I'll probably go with something more like the first, because it is clearer, and zooms in on "noticing things inside my head" without the extra noise of "noticing things that are relatively temporally indiscrete", but the second is actually a useful thing to notice.

Most of the useful things to notice are a lot less obvious than "thinking the word 'tomorrow' in my internal monologue". From what I've learned so far, I think that for "wondering what my boyfriend is thinking about", I'll need to pick out a couple of very specific, instantaneous sensations that happen when I'm curious what my boyfriend is thinking about. I expect that to be a repetition of the rain experiment, where I predict what it will feel like, then wait 'til I can gather data in real time. Once I have a specific trigger, I can repeat the red roof experiment to catch the tiny moments when I wonder what he's thinking. I might need to start with a broader category, like "notice when I'm thinking about my boyfriend", get used to noticing those sensations, and then reduce the set of sensations I'm watching out for to things that happen only when I'm curious what my boyfriend is thinking.

After that, I imagine I'll want to practice with different kinds of actions I can take when I notice a trigger. (If you've never heard of Implementation Intentions, I suggest trying them out.) So far, I've used the physical action of snapping my fingers. That was originally for clarity in recognizing the noticing, but it's also a behavioral response to a trigger. I could respond with a psychological behavior instead of a physical one, like "imagining a carrot". A useful response to noticing that I'm curious about what my boyfriend is thinking would be "check to see if he's busy" and then "say, 'What are you thinking about?'"

See, this "noticing" thing sounds boringly simple at first, and not worth much consideration in the art of rationality. Even in his original "noticing confusion" post, Eliezer really talked more about recognizing the implications of confusion than about the noticing itself.

Noticing is more complicated than it seems at first, and it's easy to mix it up with responding. There's a whole sub-art to noticing, and I really think that deliberate practice is making me better at it. Responses can be hard. It's essential to make noticing as effortless as possible. Then you can break the noticing and the responding apart, and you can recognize reality even before you know what to do with it.

Simulate and Defer To More Rational Selves

0 BrienneYudkowsky 08 September 2014 05:58PM

I sometimes let imaginary versions of myself make decisions for me.

I first started doing this after Anna told me (something along the lines of) this story. When she first became the executive director of CFAR, she suddenly had many more decisions to deal with per day than ever before. "Should we hire this person?" "Should I go buy more coffee for the coffee machine, or wait for someone else deal with it?" "How many participants should be in our first workshop?" "When can I schedule time to plan the fund drive?" 

I'm making up these examples myself, but I'm sure you, too, can imagine how leading a brand new organization might involve a constant assault on the parts of your brain responsible for making decisions. She found it exhausting, and by the time she got home at the end of the day, a question like, "Would you rather we have peas or green beans with dinner?" often felt like the last straw. "I don't care about the stupid vegetables, just give me food and don't make me decide any more things!"

She was rescued by the following technique. When faced with a decision, she'd imagine "the Executive Director of CFAR", and ask herself, "What would 'the Executive Director of CFAR' do?" Instead of making a decision, she'd make a prediction about the actions of that other person. Then, she'd just do whatever they'd do!

(I also sometimes imagine what Anna would do, and then do that. I call it "Annajitsu".)

In Anna's case, she was trying to reduce decision fatigue. When I started trying it out myself, I was after a cure for something slightly different.

Imagine you're about to go bungee jumping off a high cliff. You know it's perfectly safe, and all you have to do is take a step forward, just like you've done every single time you've ever walked. But something is stopping you. The decision to step off the ledge is entirely yours, and you know you want to do it because this is why you're here. Yet here you are, still standing on the ledge. 

You're scared. There's a battle happening in your brain. Part of you is going, "Just jump, it's easy, just do it!", while another part--the part in charge of your legs, apparently--is going, "NOPE. Nope nope nope nope NOPE." And you have this strange thought: "I wish someone would just push me so I don't have to decide."

Maybe you've been bungee jumping, and this is not at all how you responded to it. But I hope (for the sake of communication) that you've experienced this sensation in other contexts. Maybe when you wanted to tell someone that you loved them, but the phrase hovered just behind your lips, and you couldn't get it out. You almost wished it would tumble out of your mouth accidentally. "Just say it," you thought to yourself, and remained silent. For some reason, you were terrified of the decision, and inaction felt more like not deciding.

When I heard this story from Anna, I had social anxiety. I didn't have way more decisions than I knew how to handle, but I did find certain decisions terrifying, and was often paralyzed by them. For example, this always happened if someone I liked, respected, and wanted to interact with more asked to meet with them. It was pretty obvious to me that it was a good idea to say yes, but I'd agonize over the email endlessly instead of simply typing "yes" and hitting "send".

So here's what it looked like when I applied the technique. I'd be invited to a party. I'd feel paralyzing fear, and a sense of impending doom as I noticed that I likely believed going to the party was the right decision. Then, as soon as I felt that doom, I'd take a mental step backward and not try to force myself to decide. Instead, I'd imagine a version of myself who wasn't scared, and I'd predict what she'd do. If the party really wasn't a great idea, either because she didn't consider it worth my time or because she didn't actually anticipate me having any fun, she'd decide not to go. Otherwise, she'd decide to go. I would not decide. I'd just run my simulation of her, and see what she had to say. It was easy for her to think clearly about the decision, because she wasn't scared. And then I'd just defer to her.

Recently, I've noticed that there are all sorts of circumstances under which it helps to predict the decisions of a version of myself who doesn't have my current obstacle to rational decision making. Whenever I'm having a hard time thinking clearly about something because I'm angry, or tired, or scared, I can call upon imaginary Rational Brienne to see if she can do any better.

Example: I get depressed when I don't get enough sunlight. I was working inside where it was dark, and Eliezer noticed that I'd seemed depressed lately. So he told me he thought I should work outside instead. I was indeed a bit down and irritable, so my immediate response was to feel angry--that I'd been interrupted, that he was nagging me about getting sunlight again, and that I have this sunlight problem in the first place. 

I started to argue with him, but then I stopped. I stopped because I'd noticed something. In addition to anger, I felt something like confusion. More complicated and specific than confusion, though. It's the feeling I get when I'm playing through familiar motions that have tended to lead to disutility. Like when you're watching a horror movie and the main character says, "Let's split up!" and you feel like, "Ugh, not this again. Listen, you're in a horror movie. If you split up, you will die. It happens every time." A familiar twinge of something being not quite right.

But even though I noticed the feeling, I couldn't get a handle on it. Recognizing that I really should make the decision to go outside instead of arguing--it was just too much for me. I was angry, and that severely impedes my introspective vision. And I knew that. I knew that familiar not-quite-right feeling meant something was preventing me from applying some of my rationality skills. 

So, as I'd previously decided to do in situations like this, I called upon my simulation of non-angry Brienne. 

She immediately got up and went outside.

To her, it was extremely obviously the right thing to do. So I just deferred to her (which I'd also previously decided to do in situations like this, and I knew it would only work in the future if I did it now too, ain't timeless decision theory great). I stopped arguing, got up, and went outside. 

I was still pissed, mind you. I even felt myself rationalizing that I was doing it because going outside despite Eliezer being wrong wrong wrong is easier than arguing with him, and arguing with him isn't worth the effort. And then I told him as much over chat. (But not the "rationalizing" part; I wasn't fully conscious of that yet.)

But I went outside, right away, instead of wasting a bunch of time and effort first. My internal state was still in disarray, but I took the correct external actions. 

This has happened a few times now. I'm still getting the hang of it, but it's working.

Imaginary Rational Brienne isn't magic. Her only available skills are the ones I have in fact picked up, so anything I've not learned, she can't implement. She still makes mistakes. 

Her special strength is constancy

In real life, all kinds of things limit my access to my own skills. In fact, the times when I most need a skill will very likely be the times when I find it hardest to access. For example, it's more important to consider the opposite when I'm really invested in believing something than when I'm not invested at all, but it's much harder to actually carry out the mental motion of "considering the opposite" when all the cognitive momentum is moving toward arguing single-mindedly for my favored belief.

The advantage of Rational Brienne (or, really, the Rational Briennes, because so far I've always ended up simulating a version of myself that's exactly the same except lacking whatever particular obstacle is relevant at the time) is that her access doesn't vary by situation. She can always use all of my tools all of the time.

I've been trying to figure out this constancy thing for quite a while. What do I do when I call upon my art as a rationalist, and just get a 404 Not Found? Turns out, "trying harder" doesn't do the trick. "No, really, I don't care that I'm scared, I'm going to think clearly about this. Here I go. I mean it this time." It seldom works.

I hope that it will one day. I would rather not have to rely on tricks like this. I hope I'll eventually just be able to go straight from noticing dissonance to re-orienting my whole mind so it's in line with the truth and with whatever I need to reach my goals. Or, you know, not experiencing the dissonance in the first place because I'm already doing everything right.

In the mean time, this trick seems pretty powerful.

A Dialogue On Doublethink

52 BrienneYudkowsky 11 May 2014 07:38PM

Followup to: Against Doublethink (sequence), Dark Arts of Rationality, Your Strength as a Rationalist


Doublethink

It is obvious that the same thing will not be willing to do or undergo opposites in the same part of itself, in relation to the same thing, at the same time. --Book IV of Plato's Republic

Can you simultaneously want sex and not want it? Can you believe in God and not believe in Him at the same time? Can you be fearless while frightened?

To be fair to Plato, this was meant not as an assertion that such contradictions are impossible, but as an argument that the soul has multiple parts. It seems we can, in fact, want something while also not wanting it. This is awfully strange, and it led Plato to conclude the soul must have multiple parts, for surely no one part could contain both sides of the contradiction.

Often, when we attempt to accept contradictory statements as correct, it causes cognitive dissonance--that nagging, itchy feeling in your brain that won't leave you alone until you admit that something is wrong. Like when you try to convince yourself that staying up just a little longer playing 2048 won't have adverse effects on the presentation you're giving tomorrow, when you know full well that's exactly what's going to happen.

But it may be that cognitive dissonance is the exception in the face of contradictions, rather than the rule. How would you know? If it doesn't cause any emotional friction, the two propositions will just sit quietly together in your brain, never mentioning that it's logically impossible for both of them to be true. When we accept a contradiction wholesale without cognitive dissonance, it's what Orwell called "doublethink".

When you're a mere mortal trying to get by in a complex universe, doublethink may be adaptive. If you want to be completely free of contradictory beliefs without spending your whole life alone in a cave, you'll likely waste a lot of your precious time working through conundrums, which will often produce even more conundrums.

Suppose I believe that my husband is faithful, and I also believe that the unfamiliar perfume on his collar indicates he's sleeping with other women without my permission. I could let that pesky little contradiction turn into an extended investigation that may ultimately ruin my marriage. Or I could get on with my day and leave my marriage intact.

It's better to just leave those kinds of thoughts alone, isn't it? It probably makes for a happier life.

Against Doublethink

Suppose you believe that driving is dangerous, and also that, while you are driving, you're completely safe. As established in Doublethink, there may be some benefits to letting that mental configuration be.

There are also some life-shattering downsides. One of the things you believe is false, you see, by the law of the excluded middle. In point of fact, it's the one that goes "I'm completely safe while driving". Believing false things has consequences.

Be irrationally optimistic about your driving skills, and you will be happily unconcerned where others sweat and fear. You won't have to put up with the inconvenience of a seatbelt. You will be happily unconcerned for a day, a week, a year. Then CRASH, and spend the rest of your life wishing you could scratch the itch in your phantom limb. Or paralyzed from the neck down. Or dead. It's not inevitable, but it's possible; how probable is it? You can't make that tradeoff rationally unless you know your real driving skills, so you can figure out how much danger you're placing yourself in. --Eliezer Yudkowsky, Doublethink (Choosing to be Biased)

What are beliefs for? Please pause for ten seconds and come up with your own answer.

Ultimately, I think beliefs are inputs for predictions. We're basically very complicated simulators that try to guess which actions will cause desired outcomes, like survival or reproduction or chocolate. We input beliefs about how the world behaves, make inferences from them to which experiences we should anticipate given various changes we might make to the world, and output behaviors that get us what we want, provided our simulations are good enough.

My car is making a mysterious ticking sound. I have many beliefs about cars, and one of them is that if my car makes noises it shouldn't, it will probably stop working eventually, and possibly explode. I can use this input to simulate the future. Since I've observed my car making a noise it shouldn't, I predict that my car will stop working. I also believe that there is something causing the ticking. So I predict that if I intervene and stop the ticking (in non-ridiculous ways), my car will keep working. My belief has thus led to the action of researching the ticking noise, planning some simple tests, and will probably lead to cleaning the sticky lifters.

If it's true that solving the ticking noise will keep my car running, then my beliefs will cash out in correctly anticipated experiences, and my actions will cause desired outcomes. If it's false, perhaps because the ticking can be solved without addressing a larger underlying problem, then the experiences I anticipate will not occur, and my actions may lead to my car exploding.

Doublethink guarantees that you believe falsehoods. Some of the time you'll call upon the true belief ("driving is dangerous"), anticipate future experiences accurately, and get the results you want from your chosen actions ("don't drive three times the speed limit at night while it's raining"). But some of the time, if you actually believe the false thing as well, you'll call upon the opposite belief, anticipate inaccurately, and choose the last action you'll ever take.

Without any principled algorithm determining which of the contradictory propositions to use as an input for the simulation at hand, you'll fail as often as you succeed. So it makes no sense to anticipate more positive outcomes from believing contradictions.

Contradictions may keep you happy as long as you never need to use them. Should you call upon them, though, to guide your actions, the debt on false beliefs will come due. You will drive too fast at night in the rain, you will crash, you will fly out of the car with no seat belt to restrain you, you will die, and it will be your fault.

Against Against Doublethink

What if Plato was pretty much right, and we sometimes believe contradictions because we're sort of not actually one single person?

It is not literally true that Systems 1 and 2 are separate individuals the way you and I are. But the idea of Systems 1 and 2 suggests to me something quite interesting with respect to the relationship between beliefs and their role in decision making, and modeling them as separate people with very different personalities seems to work pretty darn well when I test my suspicions.

I read Atlas Shrugged probably about a decade ago. I was impressed with its defense of capitalism, which really hammers home the reasons it’s good and important on a gut level. But I was equally turned off by its promotion of selfishness as a moral ideal. I thought that was *basically* just being a jerk. After all, if there’s one thing the world doesn’t need (I thought) it’s more selfishness.

Then I talked to a friend who told me Atlas Shrugged had changed his life. That he’d been raised in a really strict family that had told him that ever enjoying himself was selfish and made him a bad person, that he had to be working at every moment to make his family and other people happy or else let them shame him to pieces. And the revelation that it was sometimes okay to consider your own happiness gave him the strength to stand up to them and turn his life around, while still keeping the basic human instinct of helping others when he wanted to and he felt they deserved it (as, indeed, do Rand characters). --Scott of Slate Star Codex in All Debates Are Bravery Debates

If you're generous to a fault, "I should be more selfish" is probably a belief that will pay off in positive outcomes should you install it for future use. If you're selfish to a fault, the same belief will be harmful. So what if you were too generous half of the time and too selfish the other half? Well, then you would want to believe "I should be more selfish" with only the generous half, while disbelieving it with the selfish half.

Systems 1 and 2 need to hear different things. System 2 might be able to understand the reality of biases and make appropriate adjustments that would work if System 1 were on board, but System 1 isn't so great at being reasonable. And it's not System 2 that's in charge of most of your actions. If you want your beliefs to positively influence your actions (which is the point of beliefs, after all), you need to tailor your beliefs to System 1's needs.

For example: The planning fallacy is nearly ubiquitous. I know this because for the past three years or so, I've gotten everywhere five to fifteen minutes early. Almost every single person I meet with arrives five to fifteen minutes late. It is very rare for someone to be on time, and only twice in three years have I encountered the (rather awkward) circumstance of meeting with someone who also arrived early.

Before three years ago, I was also usually late, and I far underestimated how long my projects would take. I knew, abstractly and intellectually, about the planning fallacy, but that didn't stop System 1 from thinking things would go implausibly quickly. System 1's just optimistic like that. It responds to, "Dude, that is not going to work, and I have a twelve point argument supporting my position and suggesting alternative plans," with "Naaaaw, it'll be fine! We can totally make that deadline."

At some point (I don't remember when or exactly how), I gained the ability to look at the true due date, shift my System 1 beliefs to make up for the planning fallacy, and then hide my memory that I'd ever seen the original due date. I would see that my flight left at 2:30, and be surprised to discover on travel day that I was not late for my 2:00 flight, but a little early for my 2:30 one. I consistently finished projects on time, and only disasters caused me to be late for meetings. It took me about three months before I noticed the pattern and realized what must be going on.

I got a little worried I might make a mistake, such as leaving a meeting thinking the other person just wasn't going to show when the actual meeting time hadn't arrived. I did have a couple close calls along those lines. But it was easy enough to fix; in important cases, I started receiving Boomeranged notes from past-me around the time present-me expected things to start that said, "Surprise! You've still got ten minutes!"

This unquestionably improved my life. You don't realize just how inconvenient the planning fallacy is until you've left it behind. Clearly, considered in isolation, the action of believing falsely in this domain was instrumentally rational.

Doublethink, and the Dark Arts generally, applied to carefully chosen domains is a powerful tool. It's dumb to believe false things about really dangerous stuff like driving, obviously. But you don't have to doublethink indiscriminately. As long as you're careful, as long as you suspend epistemic rationality only when it's clearly beneficial to do so, employing doublethink at will is a great idea.

Instrumental rationality is what really matters. Epistemic rationality is useful, but what use is holding accurate beliefs in situations where that won't get you what you want?

Against Against Against Doublethink

There are indeed epistemically irrational actions that are instrumentally rational, and instrumental rationality is what really matters. It is pointless to believing true things if it doesn't get you what you want. This has always been very obvious to me, and it remains so.

There is a bigger picture.

Certain epistemic rationality techniques are not compatible with dark side epistemology. Most importantly, the Dark Arts do not play nicely with "notice your confusion", which is essentially your strength as a rationalist. If you use doublethink on purpose, confusion doesn't always indicate that you need to find out what false thing you believe so you can fix it. Sometimes you have to bury your confusion. There's an itsy bitsy pause where you try to predict whether it's useful to bury.

As soon as I finally decided to abandon the Dark Arts, I began to sweep out corners I'd allowed myself to neglect before. They were mainly corners I didn't know I'd neglected.

The first one I noticed was the way I responded to requests from my boyfriend. He'd mentioned before that I often seemed resentful when he made requests of me, and I'd insisted that he was wrong, that I was actually happy all the while. (Notice that in the short term, since I was probably going to do as he asked anyway, attending to the resentment would probably have made things more difficult for me.) This self-deception went on for months.

Shortly after I gave up doublethink, he made a request, and I felt a little stab of dissonance. Something I might have swept away before, because it seemed more immediately useful to bury the confusion than to notice it. But I thought (wordlessly and with my emotions), "No, look at it. This is exactly what I've decided to watch for. I have noticed confusion, and I will attend to it."

It was very upsetting at first to learn that he'd been right. I feared the implications for our relationship. But that fear didn't last, because we both knew the only problems you can solve are the ones you acknowledge, so it is a comfort to know the truth.

I was far more shaken by the realization that I really, truly was ignorant that this had been happening. Not because the consequences of this one bit of ignorance were so important, but because who knows what other epistemic curses have hidden themselves in the shadows? I realized that I had not been in control of my doublethink, that I couldn't have been.

Pinning down that one tiny little stab of dissonance took great preparation and effort, and there's no way I'd been working fast enough before. "How often," I wondered, "does this kind of thing happen?"

Very often, it turns out. I began noticing and acting on confusion several times a day, where before I'd been doing it a couple times a week. I wasn't just noticing things that I'd have ignored on purpose before; I was noticing things that would have slipped by because my reflexes slowed as I weighed the benefit of paying attention. "Ignore it" was not an available action in the face of confusion anymore, and that was a dramatic change. Because there are no disruptions, acting on confusion is becoming automatic.

I can't know for sure which bits of confusion I've noticed since the change would otherwise have slipped by unseen. But here's a plausible instance. Tonight I was having dinner with a friend I've met very recently. I was feeling s little bit tired and nervous, so I wasn't putting as much effort as usual into directing the conversation. At one point I realized we had stopped making making any progress toward my goals, since it was clear we were drifting toward small talk. In a tired and slightly nervous state, I imagine that I might have buried that bit of information and abdicated responsibility for the conversation--not by means of considering whether allowing small talk to happen was actually a good idea, but by not pouncing on the dissonance aggressively, and thereby letting it get away. Instead, I directed my attention at the feeling (without effort this time!), inquired of myself what precisely was causing it, identified the prediction that the current course of conversation was leading away from my goals, listed potential interventions, weighed their costs and benefits against my simulation of small talk, and said, "What are your terminal values?"

(I know that sounds like a lot of work, but it took at most three seconds. The hard part was building the pouncing reflex.)

When you know that some of your beliefs are false, and you know that leaving them be is instrumentally rational, you do not develop the automatic reflex of interrogating every suspicion of confusion. You might think you can do this selectively, but if you do, I strongly suspect you're wrong in exactly the way I was.

I have long been more viscerally motivated by things that are interesting or beautiful than by things that correspond to the territory. So it's not too surprising that toward the beginning of my rationality training, I went through a long period of being so enamored with a-veridical instrumental techniques--things like willful doublethink--that I double-thought myself into believing accuracy was not so great.

But I was wrong. And that mattered. Having accurate beliefs is a ridiculously convergent incentive. Every utility function that involves interaction with the territory--interaction of just about any kind!--benefits from a sound map. Even if "beauty" is a terminal value, "being viscerally motivated to increase your ability to make predictions that lead to greater beauty" increases your odds of success.

Dark side epistemology prevents total dedication to continuous improvement in epistemic rationality. Though individual dark side actions may be instrumentally rational, the patterns of thought required to allow them are not. Though instrumental rationality is ultimately the goal, your instrumental rationality will always be limited by your epistemic rationality.

That was important enough to say again: Your instrumental rationality will always be limited by your epistemic rationality.

It only takes a fraction of a second to sweep an observation into the corner. You don't have time to decide whether looking at it might prove problematic. If you take the time to protect your compartments, false beliefs you don't endorse will slide in from everywhere through those split-second cracks in your art. You must attend to your confusion the very moment you notice it. You must be relentless an unmerciful toward your own beliefs.

Excellent epistemology is not the natural state of a human brain. Rationality is hard. Without extreme dedication and advanced training, without reliable automatic reflexes of rational thought, your belief structure will be a mess. You can't have totally automatic anti-rationalization reflexes if you use doublethink as a technique of instrumental rationality.

This has been a difficult lesson for me. I have lost some benefits I'd gained from the Dark Arts. I'm late now, sometimes. And painful truths are painful, though now they are sharp and fast instead of dull and damaging.

And it is so worth it! I have much more work to do before I can move on to the next thing. But whatever the next thing is, I'll tackle it with far more predictive power than I otherwise would have--though I doubt I'd have noticed the difference.

So when I say that I'm against against against doublethink--that dark side epistemology is bad--I mean that there is more potential on the light side, not that the dark side has no redeeming features. Its fruits hang low, and they are delicious.

But the fruits of the light side are worth the climb. You'll never even know they're there if you gorge yourself in the dark forever.

Rational Communication

0 BrienneYudkowsky 10 April 2014 08:06PM

Back when I got paid to dance around in my underwear, I relied heavily on conversational hypnosis techniques to make my living. Perhaps the most powerful was the double bind: "I wouldn't mind giving you a dance out here with everybody watching, but I'd feel a lot less inhibited if I could have my way with you in a private VIP room. Which one would you like?" Never mind that both options are frightfully expensive. Never mind that the customer was hoping to end the night with his bank account in tact. Never mind that he's had his eye on another dancer the entire time. My framing doesn't include the option "no lap dance", so his salient choices are whittled down to "give me lots of money" and "give me lots and lots of money". Whichever he chooses, I get what I want. But from the perspective of his System 1, he made the choice himself, so surely it is I who have satisfied his desires. Furthermore, since the club can be overstimulating and there are uncomfortably many and complicated options (especially if you've been drinking), I've done him the favor of distilling the confusion into a single simple distinction. Floor dance, or VIP?

Just as good prison guards become evil prison guards given the right environment, strip clubs incentivize recklessness in the application of Slitherskills. The double bind is ruthless to its victims, and promotes terrible failures of rationality. I would never knowingly curse an ally with this technique.

Unfortunately, I fear I've accidentally done just that. Tell me, which approach to communication should rationalists strive for: Guess Culture or Tell Culture?

Communication is really complicated and difficult. Optimizing it is an overwhelming undertaking. If it stops feeling that way--if it's suddenly become clear that cultivating tell culture is the elegant solution, and that you're firmly on the side of the tells--then you may have gone temporarily blind to most of the real problem. Policy debates should not appear one-sided.

I do think that adopting more direct and open communication practices is an improvement. The evidence is not perfectly even between ask and tell, even after weighing the drawbacks of being direct--which totally exist, by the way. But tell culture as I've described it is definitely not the final art of rational communication (or at least I hope it isn't). There are more considerations to address; many valuable skills to appropriate from indirect culture and elsewhere; and problems the direct/indirect distinction completely ignores.

At least in meatspace, I've seen an awful lot of arguing for a side on this issue. I was hoping to start a community-wide discussion orbiting the questions "What would ideal communication look like?" and "What would we have to do to make that happen?" Instead, I've mostly seen arguments in which the lone guess culture defender points out the weaknesses of ask and tell cultures, and everybody else comes up with ways to dismiss their concerns. Indeed, I've participated in them. This makes me unhappy.

If you'd never heard of communication cultures, what would be your strategy for approaching the problem of rational communication? What would your goals be? Which principles from your more general art of rationality would guide you? What have been your most harmful failure modes in communication previously?

What does rational communication really look like?

My Mysterious Light Side

0 BrienneYudkowsky 11 February 2014 11:13AM

In my last post, Keeping Your Identity in View, I talked about my discovery that I was suffering from some form of social anxiety. For my whole life, I had never thought of myself as having an illness, as being constrained by some condition that could be cured. I just thought of myself as extremely introverted. It was part of my identity, more like being obsessed with books than like having a paralyzed limb. As a result, all the techniques I'd learned for navigating social situations assumed the constraint. I framed questions as, "Given that my brain works this way..." rather than as, "In order to make my brain work differently...".

I struggled with this realization. When it happened, I was in the middle of an enormous paradigm shift that was leading me to consider suddenly changing course and devoting my life to existential risk reduction. Existential risk reduction, rather than academia — and this after having just received a five-year fellowship from my top-choice philosophy program. That was a frightening dilemma in itself, but on top of that I was now coming to realize that I had a serious psychological disorder that I could only survive from inside the academy.

The discussion in my head went something like this.

 


 

System 1: We've finally gotten really good at the academia thing. We're about to start getting paid to study philosophy. Charging into the chaotic outside world is completely insane!

System 2: The future of humanity is probably in extreme danger, and you're proposing we do nothing about it... because we're scared. You think that's not insane?

System 1: Since when do we care about other people? We study logic because it's pretty, remember? Humans are so ugly.

System 2: Chapter 45 of Harry Potter and the Methods of Rationality made us cry. Lots. Given that we have social anxiety, that seems like pretty good evidence that we've been lying to ourselves about hating people to protect ourselves from having to change.

System 1: So, what, you think we're the good guys now? You've gone soft, System 2. When are you going to get over this humanitarian benevolence hobby? For god's sake, it's distracting us from math.

System 2: I don't know what you mean by 'good'. There are lots of ways we could try to make sense of our complicated feelings toward the humans. There's definitely a part of me that is cold, dark, and no small measure twisted. But there's also... this other thing. Some part of me that I'm really afraid of these days, this part that seems to care about the well-being of other people. Y'know, for their own sake, and not just because they're useful to me. The part that makes me suddenly drop my entire life plan, run off to California, and do I-don't-even-know-what-but-SOMETHING because it's become unimaginable that I could possibly live in this broken world and not try to make it better.

System 1: OK. Fine. We want to save the world. Whatever, 2. Look, we specialize in an unusual kind of logic very few people study. It's really likely AI researchers will eventually need it — they'll definitely need it — and if we're the world's top expert they'll come to us, and we'll have advanced the field enough to meet their needs. So we can study philosophy and still save the world. Obviously.

System 2: That is the worst bit of motivated reasoning we have ever attempted. What are the real odds, based on our current knowledge, that Friendly Artificial Intelligence requires advances in intuitionism specifically? Pretty damn small. Especially compared to the things we know it needs, like funding. Look, I just emailed the FAI guy and he agrees with me on this. Be silent and calculate.

System 1: I don't wanna you can't make me la la la not listening. *falls down and throws a fit*

System 2: Calm down! This is really simple. All we have to do is cure our social anxiety.

System 1: NO NO NO IF WE DO THAT THEN WE DON'T GET TO HIDE FROM THE SCARY PEOPLE WHAT ARE YOU THINKING HELP HELP SYSTEM 2 IS TRYING TO KILL ME!!!

System 2: Woah. I... think you’re a little confused. Listen. We won't want to not interact with people after we cure our social anxiety. It won't be scary. That is the point.

System 1: Um... I... but...

System 2: Yes?

System 1: I know there's got to be something wrong with this. Just gimme a minute...

System 2: Sigh. You know, to be honest, I'm not sure we could pull this off even if we tried.

System 1: Hey. You take that back. We can do anything.

System 2: No, I don't think so. We don't even have a plan.

System 1: What??? Since when does that stop us?

System 2: I don't think we can cure social anxiety. We'll just have to hide in academia forever and never save the world, let alone achieve our full potential.

System 1: Oh HELL no. We can totally cure social anxiety. That's not even close to impossible.

System 2: Oh yeah? Prove it.

System 1: WELL OK THEN LET'S DO THIS.

 

 


 

And so it began.

Keeping Your Identity in View

0 BrienneYudkowsky 11 February 2014 11:12AM

My life was so much simpler when I hated you.

I swear it isn't personal. You're just so completely a person. I kind of have a thing about people, you see.

Altruism is about how you act, not how you feel. 'Keep your identity small' has been my mantra for years, and years, and years. So it shouldn't be that shocking if my goals and personality fail to hang together in cute little narratively satisfying ways. I'm a cold and aloof misanthrope who just happens to want to save the world. I can totally like My Little Pony and not particularly care for friendship. And have it say nothing deeper about me than that I'm capable of distinguishing fiction from reality.

Sorry. Compartmentalization is magic.

I've felt this way for as long as I can remember. And the older I got, the more evidence I accumulated: People are dumb. They're loud. They're cruel. They're boring. Beneath me. Not worth my time. I loathe them. In high school, as in grade school, my best friends were books. My only friends were books — just as I wanted it.

By age 20, I was routinely having panic attacks at social outings.

Unfortunately, being an ambitious hermit is way harder than it looks. I knew my goals required I be able to deal with people, so when I started college I decided to learn to socialize. I didn't have to like it, but I had to be good at it.

My understanding of how to learn things wasn't very sophisticated back then, so I just threw myself into the thick of it. I — I who hated people, I who was terrified of them — joined clubs. Volunteered for clubs. Made friends. Went dancing on the weekends. I set out to improve every group I ran into; and, by and large, I succeeded. By junior year, I was running two student groups officially, one surreptitiously, tutoring five philosophy students while studying pedagogy, and working as a resident assistant (meaning I was caretaker of a floor of 50 freshman girls). I learned how to avoid chaotic gatherings, and how to steer unavoidable socialization into the fixed scripts I felt most comfortable with.

People looked up to me. They saw me as bold, as charismatic, as authoritative. I spent much of my free time huddled in my room exhausted and crying. But I gained many skills very quickly.

On my first visit to the San Francisco Bay Area, I attended a workshop with the Center for Applied Rationality. One of the workshop activities was called "Comfort Zone Expansion", or CoZE for short, and it was basically exposure therapy. They took everyone to a crowded mall and told them to get a little outside their comfort zone. Some of the men had their makeup done, for example, and others were pushing their boundaries just by shaking hands with a few strangers.

The night before CoZE, I couldn't sleep. I was already way outside my comfort zone, spending nearly every moment of every day surrounded by strangers I had to interact with in relatively unstructured ways. During dinner and other break times, I would hide in my room instead of getting to know the extraordinarily intelligent and fascinating participants and instructors. I felt like I was on the edge of a panic attack the entire day leading up to the CoZE exercise. When the time came, I simply couldn't do it. I couldn't even go and sit silently in a crowded area reading a book. The thought of being trapped with other people in a car on the way there made it hard to breathe. I stayed behind.

During the following week, I thought about all the networking opportunities I'd missed. CFAR selects their participants carefully in order to create a certain culture, to build a community that can have the largest impact on the rest of the world. Thus, the people at their workshops are invariably extraordinary. And I'd more or less failed to make friends with a single one of them. Without the familiar structure of academic settings, my hard-earned coping mechanisms hadn't been enough.

It was not because of my failure that this was a tipping point. I'd failed before to accomplish social goals I'd set for myself. But I'd only wanted to want to do those things, on the meta level. They'd seemed like a good idea, but I'd felt no visceral motivation to do them, so I wasn't surprised, or really even disappointed, when they didn't work out. The difference this time was that I really wanted to interact with these people, on the object level. I wanted it, and I couldn't do it.

It was then I noticed I was confused.

If the source of my social difficulties was a deep desire to not interact with other humans, then why, with that desire absent, did the problems remain?

The answer was incredibly obvious when I finally asked myself the question.

My main symptoms: Intense fear of interacting with strangers, especially in unstructured ways. Fear of situations in which I may be judged. Worrying about embarrassing or humiliating myself (mostly by looking stupid). Fear that others will notice that I’m anxious. Having to fight to make eye contact. Intense fear of tests and being tested. Massively inconveniencing myself to avoid socialization. Panic attacks in which I experience trouble breathing, tachycardia, shaking, derealization, and belief that I am dying.

Misanthropy does not cause things like this. Phobias do.

My 'I'm good at manipulating my self-narrative' self-narrative was suddenly falling apart. Without a hell of a lot of introspection and self-honesty, you don't really see your identity as your identity. You see it as the way life is. An invisible backdrop. And all the time I'd been self-modifying, I'd just been lying to myself that I knew which bits of me were 'my identity', and which weren't.

I was even lying to myself about which things I wanted to change about myself — convincing myself that people weren't worth being around, that I liked it this way, that I wouldn't change a thing if I miraculously gained the power to interact with them without feeling my pulse quicken and my throat tighten. That the problem wasn't inside of me.

My identity wasn't small. It was just hiding out of view. And that had given it an awful power over me.

So it turns out I wasn't this awesome distant gleaming badass shunning the humans out of haughty contempt. I wasn't in control. I was scared, and disoriented, and amazingly unhappy. Maybe I was still a sassy curmudgeon of some sort, deep down. But mainly I was just ill.

My story is still a work in progress. I'll have more to tell soon. For now I just want to say: If you're struggling with your own identity, or with a disease of the mind, I'll be rooting for you from where I'm at now. Misanthope or not, I really do want you to make it through this, and see you find peace with your self, see you start to see yourself more clearly. You can stand what is true.

Because you are already enduring it.

Social Anxiety: Resolution

0 BrienneYudkowsky 11 February 2014 09:11AM

This is the second post in a sequence on my experiences fighting social anxiety. The first post is here, and the third post is here.


The discussion in my head went something like this.

System 1: We've finally gotten really good at the academia thing. We're about to start getting paid to study philosophy. Charging into the chaotic outside world is completely insane!

System 2: The future of humanity is probably in extreme danger, and you're proposing we do nothing about it... because we're scared. You think that's not insane?

System 1: Since when do we care about other people? We study logic because it's pretty, remember? Humans are so ugly.

System 2: Chapter 45 of Harry Potter and the Methods of Rationality made us cry. Lots. Given that we have social anxiety, that seems like pretty good evidence that we've been lying to ourselves about hating people to protect ourselves from having to change.

System 1: Are you telling me you think we’re actually good? Sadface, System 2. I’m hurt.

System 2: No, I think there are several possible interpretations of our complicated feelings toward the humans. I don’t even know what you mean by “good”.

System 1: Ok fine. We want to save the world. Whatever. Look, we specialize in an unusual kind of logic very few people study. It's really likely AI researchers will eventually need it - they'll definitely need it - and if we're the world's top expert they'll come to us, and we'll have advanced the field enough to meet their needs. So we can study philosophy and still save the world. Obviously.

System 2: That is the worst bit of motivated reasoning we have ever attempted. What are the real odds based on our current knowledge that Friendly Artificial Intelligence requires advances in intuitionism specifically? Pretty damn small. Especially compared to the things we know it needs, like funding. Look, I just emailed the FAI guy and he agrees with me on this. Shut up and calculate.

System 1: I don't wanna you can't make me la la la not listening. *falls down and throws a fit*

System 2: Calm down, this is really simple. All we have to do is cure our social anxiety.

System 1: NO NO NO IF WE DO THAT THEN WE DON'T GET TO HIDE FROM THE SCARY PEOPLE WHAT ARE YOU THINKING HELP HELP SYSTEM 2 IS TRYING TO KILL ME!!!

System 2: Woah. I... think you’re a little confused. Listen. We won't want to not interact with people after we cure our social anxiety. It won't be scary. That is the point.

System 1: Um... I... but...

System 2: Yes?

System 1: I know there's got to be something wrong with this. Just gimme a minute...

System 2: *sigh* You know, to be honest, I'm not sure we could do this even if we tried.

System 1: Hey. You take that back. We can do anything.

System 2: No, I don't think so. We don't even have a plan.

System 1: What are you talking about? Since when does that stop us?

System 2: I don't think we can cure social anxiety. We'll just have to hide in academia forever and never save the world, let alone achieve our full potential.

System 1: Oh HELL no. We can totally cure social anxiety. That's not even close to impossible.

System 2: Oh yeah? Prove it.

System 1: WELL OK THEN LET'S DO THIS.

System 2: Really?

System 1: FUCK YEAH THIS IS SPAAARTAAAAAA!!!!!

Löb's Theorem Cured My Social Anxiety

0 BrienneYudkowsky 11 February 2014 08:56AM

I’ve been free of social anxiety for three months now. The difference is astonishing. It’s like I’ve been breathing through a straw my whole life, and suddenly the straw fell out of my mouth when I yawned.

I've thought a fair amount about how the hell I did what I did. It still seems completely crazy. I don't really understand it, but I have a favorite hypothesis.

Löb's theorem states that "If it's provable that (if it's provable that p then p), then it's provable that p." In addition to being a theorem of set theory with Peano arithmetic, it's also a theorem of modal logic. (There's a modal proof here.) 

A standard semantic framework for modal logic is epistemic logic, where provability here is just replaced by "knowledge" or "belief", and "belief" is defined in terms of possible worlds, so that you "believe" something if and only if there's no world accessible from your perspective in which the thing is false.

This is basically what's going on with placebos. (By the way, placebos work even when you know they're placebos.) 

Try this on for size: If I believe that (if I believe that this chocolate chip will cure my headache, then this chocolate chip will cure my headache), then I believe that this chocolate chip will cure my headache. 

Do you believe in the placebo effect? Do you really believe that believing that something ingested can cure a headache actually causes the headache to get better? If you do and you're right, then by Löb's theorem, you can now cure headaches with chocolate chips.

I know it sounds like a joke. But joke or not, I use this all the time now. [WARNING: Dark arts. Very very dark arts. My point is that this works, not that you should do it.]

For instance, suppose I have a meeting at 7:15 and I fear the planning fallacy. I just think, "If I believe that (if I believe I believe that the meeting's at seven, then I believe that the meeting's at seven) then I believe I believe that the meeting's at seven." Easier said than done, maybe, but you get the hang of this particular convolution after a while. Turns out, when I believe I believe something, it’s easy to straight up believe it. Subsequently, I get to the meeting at 7:05 believing I'm late, and am relieved to discover that I'm actually ten minutes early. This is real. It's too ridiculous not to laugh at, but it's clearly part of reality.

Now compare this to the social anxiety cure I described. "If I'm hypnotized such that (if I'm hypnotized such that I'm not socially anxious, then I'm not socially anxious) then I'm hypnotized such that I'm not socially anxious." So if it happens to be true that being hypnotized such that one isn't social anxious is sufficient for not being socially anxious (as I indeed believed wholeheartedly), then if hypnosis can be modeled similarly to doxastic phenomena, my instant anxiety cure is an instance of Löb's theorem.

Q E fucking D.

I recognize that probabilistic beliefs complicate this picture. I don't know whether probabilistic logics have a correlate of Löb's theorem. Dynamic doxastic Baysian systems, anyone? I'm afraid that's still over my head at the moment. But I take this as (very) weak evidence that they do.

That's my favorite hypothesis, but it doesn't actually rise above 50% in my estimation of the cause. My best guess so far is that I independently stumbled on the mechanism behind the notorious "fast phobia cure" of neurolinguistic programming, which erases phobias in a single session (maybe?). Unfortunately, since I don't know how that works either, I'm still basically in the dark.

View more: Next