You teach students math by putting them in a classroom and having them take notes while a lecturer talks about math. What skill are you actually training? How to take notes.
Also 'how to follow mathematical reasoning,' which can be even more insidious: they feel like they understood all the lectures but then see the exam and have no idea where to start.
In a different scenario they learn to solve problems by doing textbook exercises, spend four years acing courses, then get to grad school and realize they have no idea how to go about research.
See also: Boring Advice Repository, Solved Problems Repository, Grad Student Advice Repository, Useful Concepts Repository, Bad Concepts Repository
I just got back from the July CFAR workshop, where I was a guest instructor. One useful piece of rationality I started paying more attention to as a result of the workshop is the idea of useful questions to ask in various situations, particularly because I had been introduced to a new one:
"What skill am I actually training?"
This is a question that can be asked whenever you're practicing something, but more generally it can also be asked whenever you're doing something you do frequently, and it can help you notice when you're practicing a skill you weren't intending to train. Some examples of when to use this question:
Many of the lessons of the sequences can also be packaged as useful questions, like "what do I believe and why do I believe it?" and "what would I expect to see if this were true?"
I'd like to invite people to post other examples of useful questions in the comments, hopefully together with an explanation of why they're useful and some examples of when to use them. As usual, one useful question per comment for voting purposes.