That's a good point--if a type of question on an IQ test shows variability from year to year, do psychologists say it's a bad type of question and remove it from the test?
Yes. A high test-retest correlation is one of the most basic criteria for an IQ test question or any inventory/item intended to measure something which is considered reasonably stable. (If it can't even measure itself, how is it going measure anything else?)
It's great to make people more aware of bad mental habits and encourage better ones, as many people have done on LessWrong. The way we deal with weak thinking is, however, like how people dealt with depression before the development of effective anti-depressants:
The only "anti-stupidity drugs" we have are nootropics. But the nootropics we have weren't developed as nootropics. Piracetam was, I think, developed to treat seizures. L-DOPA was developed to treat Parkinson's. No one knows who started using ginkgo biloba or what they used it for; it was used to treat asthma 5000 years ago. Adderall derives from drugs used to keep soldiers awake in World War 2.
And none of them are very good against stupidity. AFAIK, to date, not one drug has been developed by understanding and targeting the causes of different types of stupidity. We have the tools to do this--we could, for instance, sequence a lot of peoples' DNA, give them all IQ tests, and do a genome-wide association study, as a start.
We don't research these things because society doesn't want to research them. People don't conceive of stupidity as a disease that can be cured. We need, somehow, to promote thinking of stupidity as a mental illness. As something drug companies could make billions of dollars off of.