tl;dr: I have a more compelling argument for why a generalized "noticing metacognition" skill is useful, than I've previously been able to articulate. And, a simple exercise framework that will probably work decently for most people.

This is basically the parts of Tuning your Cognitive Strategies that were most useful to me, and hopefully a smoother onramp for people who found it confusing.

I'm extremely Noticing-pilled. 

I think the ability to notice subtle things in your mind, or in the world around you, is one of the foundational rationality skills. It's necessary for building habits^[1], for seeing clues that help you solve confusing problems, for diagnosing how you could have thought that faster, and for generally building a mechanistic model of your mind that lets you become a power-user for your brain.

Despite this, I didn't include Noticing in my recent workshops, because it takes awhile to pay off, and many people initially find it disorienting, or overwhelming, or pointless. Instead of teaching people to "notice their metacognition", I just tell them to do "Livelogging", where you write out your train-of-thought in a google doc while you solve a difficult puzzle, and then afterwards can ask questions like "how could I have thought-those-thoughts faster" and "how can I generalize lessons from that?", while reviewing the Livelog record. 

Livelogging is maybe 40% as good as generalized Noticing in the workshop context, but it's enough to get the job done, and requires little training.

But, during last weekend's workshop, I ended up giving a spontaneous lecture about Noticing which I think help motivate it much more clearly. So, if you've been skeptical why you should invest in noticing subtle things, I'm curious if this changes your mind.

Seven cognitive states worth noticing

Ultimately, I think it's worthwhile to gain "generalized noticing", where you have a passive ability to:

• a) notice a wide variety of subtle phenomena,
• and b) develop a taste for when those phenomena are important enough to actually promote to your attention.

This takes awhile to learn. But there's a particular cluster of cognitive states I find worth paying attention to, which make sense as a unit. And it should make sense why you'd want to notice all of them and do stuff with them.

Moreover, it's relatively straightforward to practice them in the context of solving puzzles, because most of the states reliably turn up. (Not only that, but they often turn up in the same order, so you can start by practicing noticing just one thing at a time)

This works best with some kind of puzzle that you can interact with, which has the possibility to surprise you. (I recommend the videogame Baba is You for a giant source of well designed puzzles)

Some states that stand out as particularly worth noticing are:

1. Surprise
2. Confusion
3. "Nagging feeling you're off-track."
4. Feeling Stuck
5. Disengaged / bored
6. "Huh, that's a little odd"
7. Having an insight (or, suddenly becoming "unstuck")

Each of those states has some corresponding skills, that are often useful to employ when you notice yourself in said state. I'll go over each state – both what it means in a philosophical sense and what sort of feelings might go along with it, that you might look for.^[2]

For each state, I'll list the techniques I currently apply. I'm erring on the side of being somewhat opinionated because I think people often bounce off and do weak/lame versions of the skills that aren't that helpful. But, you can substitute your own techniques, and it's hopefully intuitive why you'd want some kind of similar technique in this slot. (If it's not intuitive, argue with me about it in the comments!)

Surprise

Being surprised means "something happened, which you assigned low probability to." There are a few reasons that might have happened. 

Maybe you had an implicit (wrong) model of the situation, which didn't remotely consider the possibility of this event. Now that the event's happened, maybe a new model immediately occurred to you, which fully explains the surprise. Or, maybe you realize you don't have a model at all (and, now you are confused. See below) 

Maybe you already explicitly accounted for a possibility, it was just a low probability event. (i.e. if you get struck by lightning in a thunderstorm, that's not confusing but it's surprising).

How I might notice: I feel my eyes open wide. A jolt of adrenaline. I involuntarily say "oh!".

What to do: Check if you're also feeling confused. Check if you just had an insight. Check if the surprise has implications you should propagate through your model.

Confusion

Confusion is the state of not having a model of a situation. 

Two ways to be confused include:

1. you know you don't have an accurate model of the situation
2. you think you have an accurate model, but are wrong.

When you "notice that you're confused," you're moving from state #2 to state #1.  

Confusion is particularly subtle and hard to notice (especially compared to surprise, which is often more "loud"). But, often, when you are surprised, you are also confused, which is why it's useful to have the habit "notice surprise -> check if confused."

A related aspect is that confusion often feels murky and uncomfortable, and people tend to slide off it by default.

How I might notice: A subtle fuzzy feeling when I think about something; I notice one thought doesn't quite connect to another thought, or that I can't spell out exactly what's going on; Thinking about it feels sluggish and I find myself bouncing off to more tractable parts of the puzzle.

What to do: Explicitly note that you're confused, and decide what to do about it. Write down what you're confused about, as precisely as you can, or explain it to a friend or rubber duck. See if the confusion dissolves once you get explicit about it. Consider doing a deep dive, recursing to make "resolve this confusion" the sub-puzzle you are now explicitly focused on.

"Nagging off-track feeling"

You have a niggling doubt, that maybe your current approach to solving the puzzle either won't work, or will take a really long time.

How I might notice: I've scrunched my face muscles in a slightly disappointed frown; I feel a little squirm in your chest; I mutter out loud "this is gonna take forever"; My inner monologue says 'eh, I can ignore that it'll be fine' and a little quiet voice knows in it's heart that it's not fine."

What to do: When you notice this feeling, you might explicitly start brainstorming alternate plans. Or, at least make a note to consider doing so if the feeling persists.

"Being Stuck"

You haven't had a new thought in a while. Maybe you're thoughts go in loops. Maybe you haven't had any thoughts at all.

How I might notice: I think a thought, and notice I've thought that thought twice recently; I might feel frustrated, bored or disengaged; I might just feel a blank nothingness.

What to do: When you've been stuck awhile, I recommend brainstorming and reviewing your metastrategies. That is, don't directly try to directly solve the problem. Instead, brainstorm approaches that might help you solve the problem (including cognitive steps like "break it into smaller pieces", tools like "get a pen and paper" or physiological things like "get a drink of water"). 

Don't stop at just a few metastrategies (unless you've found one that feels really promising. See also: "having an insight"). Try to generate a lot, to force yourself past the initial obvious thing. Try to generate at least one new one you haven't thought about recently.

Bored / disengaged

You don't really care about what's going on. Maybe you're explicitly stuck, maybe you're being carried by momentum and you still have some traction but nothing feels at stake and it doesn't feel worthwhile.

How I might notice: I have trouble focusing; My eyes keep sliding around but not sticking to anything; I suddenly find myself on facebook, mysteriously. I'm tired. I'm yawning. I feel slightly annoyed.

What to do: In this state, you might ask "what do I want right now? what do I care about? what are my goals?" Probably take at least a short break (or perhaps, be done for the day, coming back tomorrow, or maybe just deciding this isn't worth it). 

Ask yourself why you originally started on the puzzle – if you still believe in it, think through what originally felt motivating and see if you can reconnect with it. 

"Huh, odd."

This is technically just a variant of "surprise" or "confusion", but I list it here separately because it feels phenomenologically distinct when it happens to me.

Often I notice a little quirk of a puzzle videogame, or a codebase I'm debugging, or a social situation, and then awhile later I'll realize this was an important clue I could have picked up on. 

This could be confusion ("huh, that piece doesn't quite make sense"), or slight surprise ("oh, that's an odd thing for the puzzle designer to have included", "that's an odd expression on her face.")

How I might notice: I blink a couple times; My face scrunches up in a way that tilts my eyebrow and compresses one side of my mouth; I say the word "huh", or think the thought "weird."

What to do: Real life is filled with tons of these moments, so you can't necessarily track them all. If you're in an explicitly crafted puzzle-exercise, it's almost certainly worth writing down, and ask "what might this imply?"

Insight

You have an idea! Maybe you were just stuck, and now you are unstuck. Maybe you were following a train of thought and two things clicked together and now something feels ALIVE WITH PROMISE.

This might turn out to be important, or it might be a false positive that doesn't pan out. Either way, congratulations – you connected some ideas together and at least generated something interesting.

How I might notice: I jolt upright with excitement; I gasp, or say "oh!".

What to do: The previous "what to dos" hopefully made intuitive sense, reinforcing ideas you were already familiar with. This one is a bit weirder and opinionated, but it's kinda the whole point of the essay.

When you notice an insight, I think it's worth:

1. Making a brief note you will remember later to investigate the insight. (maybe just write an exclamation point in your notebook, or bold the sentence you just wrote)
2. Finish your current train of thought (you're probably in the middle of having an insight – don't want to lose it!)
3. When your train of thought is at a reasonable stopping point (i.e. the rate of generating new important-feeling thoughts has slowed down enough), write down your current understanding of the insight as clearly as you can.
4. Then, immediately try to Unpack the Insight while it's fresh in your mind (which is complicated enough to be it's own section, see below)

How it fits together

Before we get to Insight Unpacking, let's recap and clarify the frame a bit. We have a bunch of cognitive states you can notice, and habits that are useful to have when you notice them. How's this process work?

Noticing Subtler Variants

When you first start training noticing, you might only be able to notice states when they're particularly "loud" (i.e. jolting upright in surprise, being extremely tired and realizing you're bored). It might also be too much to fit into your head, to both practice noticing surprise/confusion/etc and practice doing any particular techniques while trying to solve a hard puzzle.

So I recommend, when starting, to simply try solving a puzzle, and make a tallymark for each state when it comes up, and then go back to solving the puzzle.

Over time, you'll learn to notice subtler variations of each state, and you might start 

"Precognition?"

Logan Strohl says that eventually, you can even learn to notice the precursor moments to a given state, so you might start noticing it before it even happens, precognitive style. For example, before you get defensive, you might learn to notice conversational moves, or physiological cues, that are likely to make you defensive in a few seconds.

 I haven't experience this myself at this point, but it seems plausible and interesting if you really doubled down on the skill (I think Logan has invested massively more hours of practice into noticing than me)

Integrating "noticing -> skill" TAPS

Once you've gotten the hang of noticing states while puzzlesolving, you can begin to layer in explicit additional skills. You'll get better at managing your working memory such that pausing to do additional metacognition feels more natural.

Building up to "Generalized Noticing"

These seven cognitive-states aren't the only things worth noticing – there's tons of subtle stuff like defensiveness, and particular felt senses, flinching away from uncomfortable things, clinging to a particular rabbit hole or sense that you need to justify yourself to an imaginary authority figure.

I think starting with these seven is reasonable because "Puzzle Solving Videogames" are a format that's easy to replicate them in, and is fun enough to be rewarding on it's own. But I think once you get a handle on each of them, you can hopefully start extrapolating to other experiences. ("I feel something going on that's not any of those seven")

While you practice noticing them, you can also start to develop a feeling of when they are "important" and when they aren't especially. The sense of "this emotion seems like it's causing a problem, or is entangled with something important" is something I've learned to generally track whenever I notice a new micro-state that I might have otherwise let slide.

The generalized Rationality Tap is:

Notice something feels off, or feels important in some way -> Stop, Drop, and Ask Yourself "Should I wake up, become sapient, and do something about it?'"

Supercharging "credit assignment"

All the previous content fits into a hopefully ordinary seeming frame, where when you're solving problems, it's useful to notice particular situations and do particular things in them.

This next part is more speculative – I believe I've gotten benefits from this (I believe i'm better at debugging code, and I think generally better at thinking and strategizing because of these sorts of exercises). But it's a bit hard to prove, paid off on the timescale of years, and it's easy to delude yourself about this sort of thing.

The two additional mechanisms I'm interested in here are:

• Supercharging your ability to accurately "postmortem" problems (or, accurately reward yourself when things go well)
  • i.e. accurately diagnosing "How could I have thought that faster?"
• Shortening your feedbackloops for problem solving
  • i.e. Purposeful Practice and Tuning your Cognitive Strategies

Both of these employ the same mechanism of "you get to look at your thought process in highly grained detail. You can uncover which precise moments were most responsible for your most important updates." Then you can pay special attention to those specific cognitive motions, think about how to generalize them, and practice doing "just the most important loadbearing parts."

Or, look at the parts that look like wasted motion, and do less of those.

But if you want to reward the most important updates, you might want highly detailed models of exactly the moment when you unlocked the answer to a confusing problem.

This is difficult, because right at the moment you have an insight, you are busy having an insight, and disrupting your train of thought to look more closely at it is kinda counterproductive. (You might lose the insight, and then have a harder time figuring out if it was even a good insight worth studying)

Thus, you might want the skill of...

Insight Unpacking

I'll briefly recap the instructions from earlier. When you notice "aha! I just had an insight!", you can then:

The idea is to write down a detailed list of the steps from the earliest point in the insight-chain, to the very end.

Typically, when I was first starting out, this would look like me writing down a train of thought like:

1. "I'm stuck. This sucks"
2. ...
3. ???
4. "Oh, maybe I can try X!"

The jump from "I'm stuck" to "oh, maybe I can try X!" might be less than a second.

It's not that useful to write down "and then I magically jumped from 'stuck' to 'a solution'" in your retrospective. 

But a lot can happen in that second. 

After writing down that initial list, I'd ask myself "okay, but, like, what made me think 'I can try X!". I don't have very specific advice here other than "try real hard to remember", alas. But, when I "try real hard to remember", eventually I'll say:

"Well, in between the 'I'm stuck' and 'oh I'll try X", I just remember being very frustrated."

So I add to the list:

1. "I'm stuck. This sucks"
2. ...
3. <frustrated>
4. ???
5. "Oh, maybe I can try X!"

Then I stare at it some more, and maybe I remember "I was specifically frustrated about how I'd moved my Baba is You character around the same loop 3 times, around that annoying part of the map."

List becomes:

1. "I'm stuck. This sucks"
2. ...
3. <frustrated>
3.5 <frustrated at walking in a loop doing the same thing around that annoying part of the map>
4. ???
5. "Oh, maybe I can try X!"

Then I suddenly remember viscerally saying "that doesn't make any fucking sense"

1. "I'm stuck. This sucks"
2. ...
3. <frustrated>
3.5 <frustrated at walking in a loop doing the same thing around that annoying part of the map>
4. "That doesn't make any fucking sense."
5. "Oh, maybe I can try X!"

And then I remember that right after thinking "that doesn't make any fucking sense", I thought "unless... wait", and then had some kind of imagery flash:

1. "I'm stuck. This sucks"
2. ...
3. <frustrated>
3.5 <frustrated at walking in a loop doing the same thing around that annoying part of the map>
4. "That doesn't make any fucking sense."
5. "...unless... wait"
6. <imagery flash>
7. "Oh, maybe I can try X!"

What was that imagery flash? I very briefly saccaded my eyes to That Annoying Part of the Game Map, which was actually sort of odd, come to think of it.  And then there was another flash. And then I realized that That Annoying Part of the Map implied that actually, it DID fucking make sense.

1. "I'm stuck. This sucks"
2. ...
3. <frustrated>
3.5 <frustrated at walking in a loop, doing the same thing, around that annoying part in the center of the game map>
4. "That doesn't make any fucking sense."
5. "...unless... wait"
6. <look at That Part of Map>
7. <another imagery flash>
8. ???
9. "Oh, maybe I can try X!"

When I reflect on the second imagery flash, I remember it was some kind of slidey "shape-rotate-y" disorientation. And that in the process of having that disorienting felt sense, it clicked together that:

a) the annoyingly convoluted part in the middle of the map I was looping around wasn't just an annoying shape for arbitrary reasons. It was a clue.

b) a different part of the level map corresponded to the weird shape

c) maybe if I toggle some levers in both parts of the map, something interesting will happen.

Note: this is unfortunately a made up example which I confabulated as I went along. If you're having a hard time visualizing the imaginary level, the important bits are:

• the level is shaped like a maze
• two parts of the maze have structural similarity (but one rotated upside-down)
• there are many levers throughout the maze
• it turns out the levers in the two similar-shaped parts need to be toggled in the same configuration.

Insights don't always matter

Of course, sometimes you realize you could toggle some levers that seem to correspond and... nothing happens. Or, something happens, but a different thing than you expected. 

My experience is that the visceral feeling of PROMISING INSIGHT corresponds to something "at least somewhat useful" maybe 10%-30% of the time. 

(A subset of my cruxy predictions agenda has been to try to get calibrated about how often a "visceral sense of PROMISING INSIGHT!" is actually useful. It was hard, because stopping to make a fatebook prediction during a complicated insight is pretty annoying. I eventually realized I could just write down "!!!", and vaguely keep track of how often that panned out)

Detailed Retrospectives and Generalization

The most obvious way to apply this is towards detailed retrospectives. When I look at the unpacked insight process above, I can think about what actually happened, I can see that that the active ingredients were:

1. noticing that two parts of a game level corresponded to each other
2. investigating what that correspondence might imply
3. hypothesizing that one of the additional features of the correspondence might be important
4. realizing that the levers are the only additional feature of correspondence, and that I should toggle them.

There might have been lots of other stuff in this imaginary game map, so I might not have been able to immediately jump to those four steps. But, an optimal play of this level would involve pruning the possibility space as efficiently as possible until I had those four thoughts.

When I first (imaginarily) played this imaginary level, I probably did a whole of messy, useless thinking, following red herrings and coming up with excuses to give up. What sort of thoughts might have been most useful for getting to the end of the map?

Probably something like:

1. fully explore the map
2. note down everything that seemed odd or interesting about the map
3. one of those things would be "parts of the maze are symmetrical"
4. I might not have explored the symmetry first, if there were other noteworthy things, but "symmetry" does seem like a phenomena that is often important.

When I make notes to myself for things to do in future puzzle solvings (either toy exercises or real world puzzles), the general takeaways to reinforce for later might be:

• explore and note down everything interesting in a breadth-first way.
• upweight "symmetry" as a particular clue that I look for in the future.

On-the-fly Purposeful Practice^[3]

Retrospectives happen at the end. But can you do even better, and apply metacognition on-the fly?

Good practice has as short a feedback-loop as possible, so you can clearly see the connection between micro-level actions that you take, and results. If you are good at noticing microstates, you can notice in realtime "I'm going in circles and feel frustrated. I should maybe be pursuing a different strategy." 

You can then intentionally employ some kind of metastrategy.

You can then (hopefully) see that metastrategy pay off within a few minutes or maybe even seconds (where the payoff is a new idea, even if it turns out to not help).

You can get even more granular: while you're employing a metastrategy, you can notice little fluctuations in your thought process or physiology, and sometimes it'll be obvious that they are going down a wrong track on the second-to-second level (or even subsecond level), and you can course correct, and do a little mini version of the Retrospective process at the end.

Noticing "outside world stuff"

I briefly want to note: this essay has focused on noticing metacognition. It's also super valuable to notice various outside world stuff. We had one example here of "noticing symmetry" as something you might want to be able to do. There are other things like noticing other people's microexpressions as you navigate a tricky situation.

A thorough treatment of "generally, what kind of outside world stuff is worth noticing?" is a whole broader topic that deserves it's own post.

Though, I want to note: my shoulder Logan Strohl would say something like "Your access to all external phenomena still ultimately routes through your own perceptions." Noticing the perception of "My eye is twitching in a way that maybe means I'm annoyed' isn't that different from noticing the perception of "I perceive visually that my romantic partner has a microexpression that usually means she's sad." In some sense they have the same "type".

(I don't know if Real @LoganStrohl would agree with that summary)

Summary

Okay. So that was a lot. Here's a quick recap:

• There are tons of microstates worth noticing.
• Notice -> Do Habit.
  • You can chain "notice that microstate" into habits, where you apply particular skills. i.e. "I notice I'm confused" -> "take specific actions that tend to help you resolve confusion."
• Seven particular puzzle-solving states.
  • There's a particular cluster of microstates that come up a lot in puzzlesolving, and you can cause yourself to experience them a lot and practice noticing them in a repeated fashion.
• Generalized "noticing importance."
  • Those microstates are diverse enough that, if you've gotten a handle on noticing all of them, you can start to generalize the "notice arbitrary things that seem important."
• Retrospectives and generalization.
  • You can use this noticing to make detailed maps of what went into a thought process, identify the important parts, and generalize them.
• Fine-grained Purposeful Practice
  • You can also use this noticing to improve your purposeful practice, noticing micro details on the fly that you can use to construct and evaluate little mini exercises, with very short feedback loops.

Noticing "Potential Importance"

A question that "generalized noticing" skeptics ask me is "how the hell am I supposed to know which things are important?" There's tons of clues that maybe are significant happening to me all the time. There's tons of little microstates going on inside me all the time. If tracked all of that, I'd be overwhelmed.

Something that's occurring to me literally just now as I write this, is that I think one particular thing to notice is "the sensation that something might be important."

I think there are maybe a few different flavors of this. Each of them has a different phenomenological cue. Some of them have a feeling of wrongness. Some of them, I associate with "I have explicitly tagged this type of experience as useful to track". Some of them feel beautiful and good.

I think, when I say "generalized noticing", I mean:

• the ability to track a wide variety of subtle phenomenological cues
• the ability to loudly notice and promote to attention, in particular, some phenomenological cues that you've historically tracked as "turned out to be important."

Most of the time, I am not particularly noticing everything going on around me. But I am particularly attuned to noticing various flavors of "seems important." Those are the things I promote to attention, and chain into a habit of "start tracking my metacognition in more detail."

Yesterday I was arguing with someone (ironically, about whether "generalized noticing" was useful), and they briefly muttered an argument that seemed off to me, and I noticed in the moment "the fact that they said that argument seems indicative of something important."

And I said "hey, that argument you just made seems indicative that your thought-process is off in some way. ALSO, hey, the fact that I just noticed that little micro-moment on the fly, despite not having trained to notice that particular type of micro-moment, is an example of why I think 'generalized noticing' is worthwhile."

When I think about it now, I realized: I had a little visceral sense of... I'm not sure how to describe it, but a feeling "causality", that the fact that they had just chosen to mutter that particular argument was connected to other important decisions they were making.

I unfortunately do not know how to articulate this better, and having just identified this frame five minutes ago I'm not sure I'll endorse it a year from now. But, I think "learn to notice the feeling 'potentially important'" is perhaps the most valuable part of this whole thing.

1. ^{^}

   For example, you might want "to get less defensive." What's that mean? Well, it means "when I am feeling defensive, I need to notice that, and then do things like 'ask what I'm defensive about' or 'take a few deep breaths'."

2. ^{^}

   Obvious caveat: people are different. You might need to discover your own phenomenology for each cognitive state.

3. ^{^}

   Note: you probable are familiar with "Deliberate Practice." Most practice is "naive" practice. "Purposeful Practice" and "Deliberate Practice" are technical terms, where Purposeful Practice means "you have an explicit goals and some kind of feedback loop" and Deliberate Practice means "and experts have tested this practice regimen, and it works." 
   
   I don't currently know of Deliberate Practice regimens for the general art of "solving confusing problems", and am still in the process of validating my own suggested regimens, so probably you are usually doing Purposeful Practice.

4. ^{^}

   This is actually occuring to me live, in realtime.