Related: Meta Addiction

Says Eliezer in LessWrong Q&A (16/30):

In one sense, a whole chunk of LessWrong is more or less my meta-thinking skills.

When we become more rational, it's usually because we invent a new cognitive rule that:

1. Explains why certain beliefs and actions lead to winning in a set of previously observed situations that all share some property; and,
2. Leads to winning in some, if not all, heretofore unforeseen situations that also share this property.

When you learn the general rule of not-arguing-over-definitions, you obtain a general understanding of why humans on a desert island will draw lines in the sand to communicate if necessary instead of, say, mutually drawing lines that are naively intended to communicate the fact that they are dissatisfied with their respective companions' line-drawing methods. You will foresee future instances of the general failure mode as well.

You might say that one possible statement of the problem of human rationality is obtaining a complete understanding of the algorithm implicit in the physical structure of our brains that allows us to generate such new and improved rules.

Because there is some such algorithm. Your new cognitive rules are output, and the question is: "What algorithm generates them?" If you explicitly understood that algorithm, then many, if not all, other insights about human rationality would simply fall out of it as consequences.

You know, there exists a science of metacognition that has scarcely been mentioned in seven years of LessWrong.

And if it was mentioned, it was almost always in reference to the relationship between meditation and metacognition. It seems like there would be more to say than just that.

But enough about that, let's get back to the far more interesting matter of the rationalist movement's addiction to meta-level solutions.

Abstract of Spada, Zandvoort, & Wells (2006):

The present study examined metacognitions in problem drinkers and a community sample. A sample of 60 problem drinkers and 84 individuals from the general population were compared on the following measures: Hospital Anxiety and Depression Scale, Meta-Cognitions Questionnaire 30, Quantity Frequency Scale and Alcohol Use Disorders Identification Test. Mann–Whitney U-tests, logistic regression analysis and hierarchical regression analyses were performed on the data. Mann–Whitney U-tests revealed that metacognitions, anxiety, depression and drinking scores were significantly higher for problem drinkers than for the general population. The logistic regression analysis indicated that beliefs about cognitive confidence and beliefs about the need to control thoughts were independent predictors of a classification as a problem drinker over and above negative emotions. Finally, hierarchical regression analyses on the combined samples showed that beliefs about cognitive confidence, and beliefs about the need to control thoughts, independently predicted both alcohol use and problem drinking scores. These results add to the argument that metacognitive theory is relevant in understanding excessive and problematic alcohol use.

It might be that problem drinkers aren't avoiding punishment signals by drinking, as one might initially think, and that they don't start and continue drinking because they're anxious. It might be that they are rewarded for using a strategy that allows them to regulate their cognition. They revisit the alcohol over and over again because the need for a solution to cognitive self-regulation led them to try drinking in the first place, and, in a most limited and hardly sustainable sense, it's consistently solved the problem before.

Problem drinkers stop being problem drinkers when they find a better reward; i.e. when they find a more rewarding cognitive self-regulation solution than drinking. This is rare because it takes time to obtain the feedback necessary for something other than drinking to be a more rewarding solution, and it's more rewarding to directly maximize the reward signal (find ways to keep drinking instead of stop drinking) instead of directly maximizing the external thing in the world that the reward signal correlates with (cognitive self-regulation).

Going meta works sometimes, and probably more often than you think, considering that you've been taught that meta is dangerous. And when it works and you know it works, it's highly rewarding.

I don't have evidence, but I nevertheless predict that intelligent humans are more likely to develop high metacognitive ability independently; that is, without being primed into doing so.

You'd imagine then that many LessWrong users would have started being rewarded very early in their lives for choosing meta-level solutions over object-level ones. How would you even make your way across the Internet all the way to LessWrong unless you were already far along the path of looking for meta-solutions?

(One way is that you happened upon an object-level solution that was mentioned here. But you know, not all LessWrong users are addicts.)

I also predict that the sort of process described in the abstract above is the same thing that separates rationalists who stave off their addiction to meta-solutions from rationalists who relapse or never get unhooked in the first place.

The obverse error is overvaluing object-level solutions. It's also possible to straddle the line between the two types of solutions in the wrong way; otherwise there would be an old LessWrong post with the same content as this one.