Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: BrienneYudkowsky 22 January 2016 11:01:53PM 5 points [-]

I wrote the first story I've ever truly been proud of.

Comment author: BrienneYudkowsky 20 December 2015 01:42:11AM *  2 points [-]

So, by way of analogy, what might an error in attempting to (say) consider the opposite look like, and what would a good “mental posture” be that would make the error matter less?

(written before reading on)

Outward orientation. Focus on features of the external world. Seeking harmony with the movements of reality.

Here are some motions I might make if I discover I've failed to consider the opposite:

  • Oh no, I've broken a rule! That was bad and I am bad. wallows What do I need to do now to atone?
  • Can I retroactively save myself from fully acknowledging that I've made the error by finding a convincing argument showing that I didn't actually need to consider the opposite in the first place?
  • This is evidence that I'm intrinsically not the better version of myself I like to imagine.
  • I'm not as good as the people who wouldn't have made this mistake.

If you startle a cat that's preparing to pounce, it might suddenly jump, whereas if you startle it wile it's lazing about, it might just twitch and look at you suddenly. When it's preparing to pounce, its posture makes jumping the default reaction to anything that happens.

If any of these mental motions is my reflexive response to discovering an error, I must be posed for self assessment, as though I'm prepared to pounce on myself - "What do my experiences mean about me as a person?" - and for judgement of my relationship with other people, with imaginary versions of myself, or with a system of rules.

Some motions I'd rather make upon discovering I've failed to consider the opposite:

  • Consider the opposite. (Better late than never.)
  • What was the nature of my mistake, what damage have I done, and how can I repair it?
  • How would I like to respond next time I encounter an experience like the one that happened just before I made this mistake?

(Preferably in that order.)

What posture would make these thoughts the sort I'd have as an automatic reflex if a failure to consider the opposite were to sneak up on me and yell "boo"?

There are probably some more specific good answers to this, but the one that comes to mind - and my current best answer to the more general question "what posture is good for rationality?" - is something like "seeking harmony with external reality".

This is a feeling I'm familiar with from partner dance. When I'm not dancing very well, I tend to have a strong inward focus. I'm concerned about what I am doing, whether the thing I did was what the lead meant for me to do, and how I look to other people.

When I'm dancing my best, my focus is always outward: on the lead, on the music, on the patterns of movement we're creating together. My focus is on the dance, not on myself. It's a kind of being in love, an intense selfless attentiveness to the phenomenon of dancing.

Similarly, when I'm trying to make good decisions in the midst of uncertainty and frequent error, I move more effectively if my attention is on the world, instead of on myself.

Excuse me for getting all poetical, but: Just as a master dancer must be in love with the dance, so must a master rationalist be in love with the truth. Maintaining a posture of selfless attentiveness to accuracy is what it means to be in love with the truth.

When I've fallen - say, by failing to consider the opposite - and may have damaged my model, this kind of outward-facing, world-aligned mental posture helps me spring right back up to rejoin the dance and make things right again.

[Edit: "Maintaining selfless attentiveness" is most of how I personally be in love. I am aware of having an unusual way of being in love. This might be closer to what most people experience as parental love than romantic love. Anyway, it's probably a bad phrasing for most, and just a good handle for me.]

In response to Tell Culture
Comment author: BrienneYudkowsky 12 June 2015 04:30:28PM 2 points [-]

For the record, I mostly regret posting this.

Comment author: Swimmer963 10 January 2015 04:34:19PM 17 points [-]

Personally, I feel that it's healthy to work as an assistant to someone (and stop thinking about work when you leave the office at 6pm), but it's unhealthy to be the assistant of someone (and treat them as a fantasy hero 24/7 and possibly sleep with them).

I think this is exactly what Brienne is talking about when she points out that society doesn't look kindly on people who want to serve others. And... I think maybe you're pointing at something real. It does seem possible that when "being" an assistant breaks, it breaks harder than when "working as" an assistant breaks. So it's a higher-stakes situation to put yourself in. (Both for the leader and for their assistant).

I don't think that negates any of what I said in the post though. Half of my point is basically just "some people are the kind of people who want to be nurses, no, really." Like, it seems to be really hard for people who aren't those kind of people to understand that for me, roles that aren't especially high-status but involve being really useful to other people hit all of my happiness buttons. That people are actually different and that their dream job might be one I'd hate, and vice versa.

The other part probably only makes sense when aimed at people who have taken the concept of "heroes" on board...which large portions of this community have. And that point is mainly: if you're going to accept that heroes and people who want to be heroes are a thing, you've got to have the concept of sidekicks too, otherwise you have a broken unhealthy community. It sounds like you're arguing that it's best not to take either concept on board. Maybe. You can argue that point.

That's also the advice I often give to programmers, to think of themselves as working for a company (in exchange for money) rather than at a company (as part of a common cause).

I'm not sure I have that switch? I've developed strong feelings of loyalty towards every job I've had. As a nurse, this loyalty is felt only a little bit towards the hospital where I work; I feel more of it for my immediate colleagues, and the rest of it towards some abstract "Profession of Nursing." I'm not sure how to stop feeling that way, or honestly why I'd want to stop.

We have all kinds of ancient tribal instincts, which are amplified by reading fantasy and bad (hero-based) sci-fi. I feel that such instincts are usually harmful in the long run, although they seem to make sense in the moment.

This comes across a little bit as saying "hey, don't have emotions!" Which...yeah, maybe emotions cause a lot of problems, but not having them isn't an option. And I'm not sure that not having narratives is an option either. It seems to me that I'm going to think of my life as a narrative in any case; I might as well try to understand and analyze and shape it. (Just as I shape my emotions, trying to lean away from the emotions that seem net-negative...but the way to do that is to guide yourself towards different emotions.)

Comment author: BrienneYudkowsky 10 February 2015 08:34:40PM 5 points [-]

I'm not sure why we're focusing in on narratives here, but I suspect it's for not very good reasons. Whether it's good for some people to "think of themselves as sidekicks" seems less important than whether it's good for people to actually perform the actions of a "sidekick". We can talk about how to promote or discourage the set of actions once that's settled. I'd much rather present a breakdown of what I actually do day to day and why, and then have people point out what precisely it is that I'm doing wrong.

Comment author: RomeoStevens 09 January 2013 10:23:47PM 0 points [-]

No, didn't realize this got so upvoted. The fad is a bit past. I still want cartoons on the level of RSA Animate for the sequences.

Comment author: BrienneYudkowsky 03 January 2015 04:34:53PM 1 point [-]

I think this should still happen.

Comment author: gothgirl420666 23 December 2014 06:35:33PM *  0 points [-]

Sorry, I was being kind of snarky, I should have explained further. My point is that the other meditation instructions I've seen have said that it is in fact possible (but very difficult) to be successful at thinking nothing while conscious, and to a certain extent that is the point. So I'm not sure where you're getting the idea that it is impossible. I think Eliezer has written a lot about prematurely concluding that things are impossible, when in fact they are merely very difficult.

Comment author: BrienneYudkowsky 24 December 2014 09:11:02PM 0 points [-]

I said it because of how I think about thoughts. When i say "thought", I mean anything that is happening in consciousness. Any sensation, any mental event that you're subjectively experiencing. When I say "conscious", I mean "you're experiencing things" (and maybe also you're awake). So if you're not experiencing things, you're not conscious. So if I taboo "thought" and "conscious", then I'd express this bit as "Try to stop having mental events. (You can't actually do that while in a state that affords trying, of course. Trying is a mental event.)"

Comment author: gothgirl420666 22 December 2014 01:08:17AM 0 points [-]

This is interesting to me. It seems like you are using meditation to more frequently engage in self-reflection, meta-cognition, introspection, etc. I'm trying to meditate (in part) to do the exact opposite - I think I'm far too self-reflective to my own detriment, and the only way to stop the endless cycle of thought loops is to get better at clearing out my head.

The point is not actually to be successful at thinking nothing, as that is impossible while conscious

[citation needed]

Comment author: BrienneYudkowsky 23 December 2014 01:52:39PM 0 points [-]

It doesn't need citation. How would that help? It just needs clarification. Which will be easier if you'd tell me what you think might be wrong about it.

Comment author: Stefan_Schubert 22 December 2014 01:47:33PM 1 point [-]

This series of posts on noticing, attention, metacognition, and so on, is really great and smart. I think it's profoundly important stuff. I hope you keep posting on this.

To what extent are you including this material in CFAR classes?

Comment author: BrienneYudkowsky 23 December 2014 01:50:34PM 1 point [-]

Not at all yet, though some of it is inspired by CFAR material. I'm not a CFAR staff member, just an occasional guest instructor. I'm in Chile for four months developing this stuff so I've had almost no contact with them for that time. But who knows, maybe they'll find some of it useful and pick it up.

Comment author: brazil84 21 December 2014 04:10:48PM 1 point [-]

When her skills are most needed, a rationalist is lost in an unskillful state of mind. She doesn’t recognize that it’s happening, and she doesn’t remember that she has prepared for it by learning and practicing appropriate techniques.

Would you mind providing a couple examples of some of the worst decisions and/or actions taken while in such a state of mind that you later regretted?

Comment author: BrienneYudkowsky 21 December 2014 06:56:23PM 1 point [-]

If I answer that question honestly, it means I'm telling you the decisions I've made that I regret most. Mistakes like that pretty much don't happen when my mind is in good condition. I'm pretty sure I'm willing to do that, but I'd at least like to make sure first that trivial mistakes won't do.

Training Reflective Attention

21 BrienneYudkowsky 21 December 2014 12:53PM

Crossposted at Agenty Duck

And somewhere in the back of his mind was a small, small note of confusion, a sense of something wrong about that story; and it should have been a part of Harry's art to notice that tiny note, but he was distracted. For it is a sad rule that whenever you are most in need of your art as a rationalist, that is when you are most likely to forget it. —HPMOR, Ch. 3

A rationalist’s art is most distant when it is most needed. Why is that?

When I am very angry with my romantic partner, what I feel is anger. I don’t feel the futility of throwing a tantrum, or the availability of other options like honest communication, or freewriting, or taking a deep breath. My attention is so narrowly focused on the object of my anger that I’m likely not even aware that I’m angry, let alone that my anger might be blinding me to my art.

When her skills are most needed, a rationalist is lost in an unskillful state of mind. She doesn’t recognize that it’s happening, and she doesn’t remember that she has prepared for it by learning and practicing appropriate techniques.

I've designed and exercise that trains a skill I call reflective attention, and some call mindfulness. For me, it serves as an anchor in a stormy mind, or as a compass pointing always toward a mental state where my art is close at hand.

Noticing that I am lost in an unskillful state of mind is a separate skill. But when I do happen to notice—when I feel that small, small note of confusion—reflective attention helps me find my way back. Instead of churning out even more pointless things to yell at my partner, it allows me to say, “I am angry. I feel an impulse to yell. I notice my mind returning over and over to the memory that makes me more angry. I’m finding it hard to concentrate. I am distracted. I have a vague impression that I have prepared for this.” And awareness of that final thought allows me to ask, “What have I trained myself to do when I feel this way?”

The goal of the following exercise is to practice entering reflective attention.

It begins with an instruction to think of nothing. When you monitor yourself to make sure you’re not having any thoughts, your attention ends up directed toward the beginnings of thoughts. Since the contents of consciousness are always changing, maintaining focus on the beginnings of thoughts prevents you from engaging for an extended period with any particular thought. It prevents you from getting “lost in thought”, or keeping attention focused on a thought without awareness of doing so. The point is not actually to be successful at thinking nothing, as that is impossible while conscious, but to notice what happens when you try.

Keeping your focus on the constant changes in your stream of consciousness brings attention to your experience of awareness itself. Awareness of awareness is the anchor for attention. It lets you keep your bearings when you’d otherwise be carried away by a current of thought or emotion.

Once you’re so familiar with the feeling of reflection that creating it is a primitive action, you can forget the introductory part, and jump straight to reflective attention whenever it occurs to you to do so.

This will probably take around five minutes, but you can do it for much longer if you want to.

Notice what your mind is doing right now. One thing it’s doing is experiencing sensations of black and white as you read. What else are you experiencing? Are there words in your inner monologue? Are there emotions of any kind?

Spend about thirty seconds trying not to think anything. When thirty seconds is up, stop trying not to think, and read on.




What’s happening in your mind is constantly changing. Even when you were trying not to think, you probably noticed many times when the stillness would shift and some new thought would begin to emerge in conscious awareness.

Turn your attention to those changes. When a new thought emerges in consciousness, see if you can notice the exact moment when it happens, becoming aware of what it feels like for that particular change to take place.

If it helps at first, you can narrate your stream of consciousness in words: “Now I’m seeing the blue of the wall, now I’m hearing the sound of a car, now I’m feeling cold, now I’m curious what time it is…” You’ll probably find that you can’t narrate anywhere near quickly enough, in part because thoughts can happen in parallel, while speech is serial. Once narrating starts to become frustrating, stop slowing yourself down with words, and just silently observe your thoughts as they occur.

If you’re finding this overwhelming because there are too many thoughts, narrow your focus down to just your breathing, and try to precisely identify the experience of an exhale ending and an inhale beginning, of an inhale ending and an exhale beginning. Keep doing that until you feel comfortable with it, and then slowly expand your attention a little at a time: to other experiences associated with breathing, to non-breath-related bodily sensations, to non-tactile sensations from your environment, and finally to internal mental sensations like emotions.

If you notice an impulse to focus your attention on a particular thought, following it and engaging with it—perhaps you notice you feel hungry, and in response you begin to focus your attention on planning lunch—instead of letting that impulse take over your attention, recognize it as yet another change in the activity of your mind. If you’re narrating, say, “now I’m feeling an impulse to plan my lunch”, and keep your focus broad enough to catch the next thought when it arises. If you realize that you’ve already become lost in a particular thought, notice that realization itself as a new thought, and return to observing your stream of consciousness by noticing the next new thought that happens as well.




You might need to practice this many times before you get the hang of it. I suggest trying it for ten minutes to half an hour a day until you do.

Once you feel like you can recognize the sensation of reflective attention and enter that state of mind reliably given time, begin to train for speed. Instead of setting a timer for fifteen minutes or however long you want to practice, set it to go off every minute for the first half of your practice, spending one minute in reflective attention, and one minute out. (Don’t do this for all of your practice. You still need to practice maintenance.) When you can consistently arrive in reflective attention by the end of the minute, cut the intervals down to 45 seconds, then thirty, fifteen, and five.

In real life, the suspicion that you may be lost in an unskillful state of mind will be quiet and fleeting. “Quiet” means you’ll need to learn to snap your attention to the slightest hint of that feeling. For that, you’ll need to train “noticing”. “Fleeting” means you’ll need to be able to respond in less than five seconds. You’ll need to begin the process in less than one second, even if it takes a little longer to fully arrive in reflective attention. For that, training for speed is crucial.

View more: Next