That said, I don't typically ask myself this question. Is this a question you routinely ask yourself? If so, do you find that it's actually helpful to ask it, as distinct from the benefits you would get from e.g. a general cue like "think harder about this problem"?
Noticing is one of the basic skills of rationality, though not much mentioned here, other than the mantra, "I notice that I am confused." Noticing things both outside you, and inside you. You cannot update on evidence you have not noticed, search for evidence you have not noticed you need, think about what you have not noticed you need to think about, examine beliefs you have not noticed you are holding, or resolve a confusion you have not noticed you are in.
It is a useful question to ask from time to time, and especially when experiencing any perplexity.
See also: Boring Advice Repository, Solved Problems Repository, Grad Student Advice Repository, Useful Concepts Repository, Bad Concepts Repository
I just got back from the July CFAR workshop, where I was a guest instructor. One useful piece of rationality I started paying more attention to as a result of the workshop is the idea of useful questions to ask in various situations, particularly because I had been introduced to a new one:
"What skill am I actually training?"
This is a question that can be asked whenever you're practicing something, but more generally it can also be asked whenever you're doing something you do frequently, and it can help you notice when you're practicing a skill you weren't intending to train. Some examples of when to use this question:
Many of the lessons of the sequences can also be packaged as useful questions, like "what do I believe and why do I believe it?" and "what would I expect to see if this were true?"
I'd like to invite people to post other examples of useful questions in the comments, hopefully together with an explanation of why they're useful and some examples of when to use them. As usual, one useful question per comment for voting purposes.