"What hidden obstacle could be causing my failures?"
My mental shorthand for this is the following experience: I try to pull open the silverware drawer. It jams at an inch open. I push it shut and try again, same result. I pull harder, it opens a tiny bit more before stopping.
Reflection: Some physical object is getting in the way of the motion. Something could be on the drawer track, but more likely it is inside the drawer. It is a rigid object, because I always stop at the same place, although slightly squashable because I was able to yank and pull a little harder. It is probably striking the inner wall of the cabinet in which the drawer is mounted. It is on an angle because I can't see it when I look through the inch gap. There is a fork or knife angled up and poking against the inner wall. Digging around with my finger quickly finds a fork.
Since then, I've brought up this question by asking myself "what is the fork in the drawer"?
For example, my linear algebra students generally seem smart and attentive, but they become confused whenever I do a detailed computation with inner products. After some thought about which computations confuse them, hypothesize that whoever taught them basic matrix manipulations didn't teach the "transpose" operator, and particularly didn't teach the rule (AB)^T = B^T A^T. Fixed very quickly. (Of course, I also try to encourage them to ask questions about what confuses them, but I think that it is impossible to ever get a class comfortable enough questioning you to not need to think on your own about what is the underlying difficulty causing confusion.)
See also: Boring Advice Repository, Solved Problems Repository, Grad Student Advice Repository, Useful Concepts Repository, Bad Concepts Repository
I just got back from the July CFAR workshop, where I was a guest instructor. One useful piece of rationality I started paying more attention to as a result of the workshop is the idea of useful questions to ask in various situations, particularly because I had been introduced to a new one:
"What skill am I actually training?"
This is a question that can be asked whenever you're practicing something, but more generally it can also be asked whenever you're doing something you do frequently, and it can help you notice when you're practicing a skill you weren't intending to train. Some examples of when to use this question:
Many of the lessons of the sequences can also be packaged as useful questions, like "what do I believe and why do I believe it?" and "what would I expect to see if this were true?"
I'd like to invite people to post other examples of useful questions in the comments, hopefully together with an explanation of why they're useful and some examples of when to use them. As usual, one useful question per comment for voting purposes.