There's a type of advice I've observed which I'm having trouble categorising into the above: advice which is valuable, but which has a prerequisite level of competence or understanding before you can use it.
I've been swing dancing for about four and a half years, and I've taken a lot of classes and workshops in it. There are several common pieces of advice that get thrown around by teachers: "keep your feet under you", "dance into the floor", "engage your core"... they're generally referring to how something feels when you're doing it, which has a reasonable margin for subjectivity, so some people will hear advice like that and think "oh, yeah, that makes sense", while others will hear it and think "well where else would my feet be? Over me?"
Good instructors find a way to give you the bodily experience without giving you misinterpretable verbal advice. "Engage your core" can mean different things to different people. There's a reasonable amount of crossover between the swing dance community and the circus skills community, and when an aerial acrobat hears "engage your core", they tense up like they're about to be thrown, which is not what's meant in the dance context. But if someone says "imagine you've got a sword stuck point-first in your navel, and you have to keep that sword horizontal", that's quite a specific piece of advice which directly addresses what is meant.
Sometimes you're just not ready for a piece of advice. Earlier this year at a weekend-long event, a well-respected and reputable instructor said something in a workshop. I can't remember his exact wording, but in my notes for the class I wrote "the tension and compression you feel in your hands is a consequence of your bodies moving, not a cause". I thought about this over the rest of the weekend and it completely blew my mind and made me revise huge chunks of how I thought the dance mechanically worked. If I'd heard it two years ago, my response to it would probably have been a lot less productive.
Sometimes you'll be working on something, and you'll notice a physical experience you'd previously not paid attention to, and a piece of advice you heard in a class months or even years ago will suddenly make sense. This doesn't necessarily just happen once with any given piece of advice. The simple and seemingly-obvious imperative of "dance to the music" is one that you can get a lot of repeated use out of at different levels of experience. When you're starting out, it means "just move in time to the music, OK?" A little later on, it means "fit the phrasing of your movements to the structure of the music". A little later still, it means "take inspiration from the features of the music. If the clarinet does something twiddly, maybe you can do something twiddly to complement it in your movements." At the moment, for me, it means something quite ridiculously art-wanky that I'd be mildly embarrassed about sharing, but in another six months I'll probably peel away another layer of "dance to the music", and it'll give me a whole extra take on the words.
I seem to have taken this opportunity to talk at length about swing dancing, (a perennial hazard), but this general idea of advice which sits on prerequisite knowledge or experience is one I see elsewhere. Less-Wrong-flavoured advice like "your strength as a rationalist is to be more confused by fiction than by reality" feels a lot like "keep your feet under you". "Go meta", much like "dance to the music", has different levels of subtletly and meaning. Less Wrong buzzword expressions sometimes feel like thing people are throwing around because they're available to do so, but when someone uses one to get to an important central point of an issue, when they nail it, it's genuinely illuminatory.
This category of advice is usually what I run in to when I'm evaluating things at all (people rarely trust me with secret wisdom, but they often trust me with merely advanced wisdom :))
As a metric, I'd suggest a variant on #3 works: is there some reason to suspect that YOU have a special level of competence that grants you unusual insight? And then ask equally why is it being shared? Is it because it's useful at all levels, or because your instructor trusts that YOU are clearly an advanced student, who can understand these things?
Or is it simply because it...
Lukeprog has done some great posts talking about how scholarship has benefited him personally and how to do it. However, he hasn't talked much about how to tell the good from the bad, and I think this is an especially big problem with it comes to "how to"s, rather than standard academic subjects.
The thing is this: writing on academic subjects tends to be dominated by academics seeking to gain status from their work. Not all fields have a strong correlation between status and making actual discoveries, but many (perhaps most) do. So for academic subjects, the big challenge when it comes to making sure you're not reading nonsense is to try to evaluate the soundness of the field in question.
When it comes to seeking out "how to" advice, though, writing on the subjects seems to be dominated by people looking to make money selling advice. That means that if bad advice will sell better than good advice, they'll give the bad advice. Even if their writing includes legitimate advice, they have incentives to give a distorted picture of the subject they're writing on if it will sell better (for example, making whatever they're talking about sound easier than it really is). In some cases, advice books end up trying to nudge you towards buying some more expensive thing the author is selling.
So: when you're researching a "how to" topic, how do you tell the good advice from the bad advice?