Related: Meta Addiction

Says Eliezer in LessWrong Q&A (16/30):

In one sense, a whole chunk of LessWrong is more or less my meta-thinking skills.

When we become more rational, it's usually because we invent a new cognitive rule that:

1. Explains why certain beliefs and actions lead to winning in a set of previously observed situations that all share some property; and,
2. Leads to winning in some, if not all, heretofore unforeseen situations that also share this property.

When you learn the general rule of not-arguing-over-definitions, you obtain a general understanding of why humans on a desert island will draw lines in the sand to communicate if necessary instead of, say, mutually drawing lines that are naively intended to communicate the fact that they are dissatisfied with their respective companions' line-drawing methods. You will foresee future instances of the general failure mode as well.

You might say that one possible statement of the problem of human rationality is obtaining a complete understanding of the algorithm implicit in the physical structure of our brains that allows us to generate such new and improved rules.

Because there is some such algorithm. Your new cognitive rules are output, and the question is: "What algorithm generates them?" If you explicitly understood that algorithm, then many, if not all, other insights about human rationality would simply fall out of it as consequences.

You know, there exists a science of metacognition that has scarcely been mentioned in seven years of LessWrong.

And if it was mentioned, it was almost always in reference to the relationship between meditation and metacognition. It seems like there would be more to say than just that.

But enough about that, let's get back to the far more interesting matter of the rationalist movement's addiction to meta-level solutions.

Abstract of Spada, Zandvoort, & Wells (2006):

The present study examined metacognitions in problem drinkers and a community sample. A sample of 60 problem drinkers and 84 individuals from the general population were compared on the following measures: Hospital Anxiety and Depression Scale, Meta-Cognitions Questionnaire 30, Quantity Frequency Scale and Alcohol Use Disorders Identification Test. Mann–Whitney U-tests, logistic regression analysis and hierarchical regression analyses were performed on the data. Mann–Whitney U-tests revealed that metacognitions, anxiety, depression and drinking scores were significantly higher for problem drinkers than for the general population. The logistic regression analysis indicated that beliefs about cognitive confidence and beliefs about the need to control thoughts were independent predictors of a classification as a problem drinker over and above negative emotions. Finally, hierarchical regression analyses on the combined samples showed that beliefs about cognitive confidence, and beliefs about the need to control thoughts, independently predicted both alcohol use and problem drinking scores. These results add to the argument that metacognitive theory is relevant in understanding excessive and problematic alcohol use.

It might be that problem drinkers aren't avoiding punishment signals by drinking, as one might initially think, and that they don't start and continue drinking because they're anxious. It might be that they are rewarded for using a strategy that allows them to regulate their cognition. They revisit the alcohol over and over again because the need for a solution to cognitive self-regulation led them to try drinking in the first place, and, in a most limited and hardly sustainable sense, it's consistently solved the problem before.

Problem drinkers stop being problem drinkers when they find a better reward; i.e. when they find a more rewarding cognitive self-regulation solution than drinking. This is rare because it takes time to obtain the feedback necessary for something other than drinking to be a more rewarding solution, and it's more rewarding to directly maximize the reward signal (find ways to keep drinking instead of stop drinking) instead of directly maximizing the external thing in the world that the reward signal correlates with (cognitive self-regulation).

Going meta works sometimes, and probably more often than you think, considering that you've been taught that meta is dangerous. And when it works and you know it works, it's highly rewarding.

I don't have evidence, but I nevertheless predict that intelligent humans are more likely to develop high metacognitive ability independently; that is, without being primed into doing so.

You'd imagine then that many LessWrong users would have started being rewarded very early in their lives for choosing meta-level solutions over object-level ones. How would you even make your way across the Internet all the way to LessWrong unless you were already far along the path of looking for meta-solutions?

(One way is that you happened upon an object-level solution that was mentioned here. But you know, not all LessWrong users are addicts.)

I also predict that the sort of process described in the abstract above is the same thing that separates rationalists who stave off their addiction to meta-solutions from rationalists who relapse or never get unhooked in the first place.

The obverse error is overvaluing object-level solutions. It's also possible to straddle the line between the two types of solutions in the wrong way; otherwise there would be an old LessWrong post with the same content as this one.

In a recent post Gwerley covered Constructive Developmental Theory, and Subject-Object Notation.  I'll be going through a basic description of the ideas, as well as adding related ideas from the Four Player Model.

---

Constructive Developmental Theory:

Constructive Developmental Theory is a Theory of Mind that splits the development of people into five levels, though the levels each have a unique set of advantages/disadvantages, not being "better" or "worse" than one another.¹  This theory is largely based on if the individual is subject to something or able to hold it as an object using meta cognition, such that each level holds the previous levels as special cases.²  This progress makes it so a higher order mind will notice things a lower order cannot.

• First Order/The Impulsive Mind

  • An impulsive mind has only it's reflexes as an object.
  • At this level an organism³ is still sorting out sensory input, and does not yet have a theory of mind.

• Second Order/The Instrumental Mind

  • As an Instrumental Mind the being is able to understand the difference between self and other.
  • Though second order minds understand the concept of self they are subject to their wants, needs and desires.⁴
  • At this level the person has only one viewpoint. (Solipsism)

• Third Order/The Socialized Mind

  • The Socialized mind is able to hold as an object their emotions, needs and desires.
  • This level is subject to cultural in-group/out-group pressures.
  • They are defined by their relation to society.
    1. This relation makes them susceptible to the wants and needs of others, making them likely to try and do/say what [they think] others want.
    2. Their reliance on society and authority makes them good followers.
  • 58% of adults are on this level.

*With the bulk of people being on this level it's important to keep status with them.  Failure to do so risks loosing momentum on any movement you're working on. (Trans humanism, Cryonics or FAI being the three that jump to mind with this community.)

• Fourth Order/The Self-Authoring Mind

  • The Self-Authoring Mind is able to hold as an object the environment it belongs to. 
  • Self-Authors are still subject to their own ideologies.
  • They are defined by what they think of themselves in relation to their ideologies as a static unchanging state dependent on their ideology. (The "self" can change if the ideology does.)
    1. This makes them able thinkers. (Provided they have knowledge of the subject matter.) allowing them to oppose things they think are wrong as their sense of self is not dependent on their relation to the community.
    2. This freethinking also makes them hesitant followers unless they reach the same conclusions on their own. 
  • 35% of adults are on this level

*While less essential than Socialized Minds, Self-Authoring Minds are a good indicator that your movement is healthy and still able to adapt to changes.  Being the primary source of said changes fourth order minds are important in order to avoid things like an Ann Rand cult.

• Fifth Order/The Self-Transforming Mind
  

  • The Self-Transforming Mind is able to hold as an object the relation between ideologies, including their own. 
  • This subjects them to the relation between the ideologies, and the search for a solution to their contradictions.
    
    1. The Self-Transforming Mind is the point at which much of the advice on this site start to make sense as something other than just "it just works."
    2. It allows changes to self akin to harry's occlumancy training in HPMOR "Anyone you can imagine you can be." 
  • 1% of adults are on this level

 *The most useful and the least essential of the groups.  They are able to fill any role needed, but are made fully redundant by a enough lower order minds in the necessary roles.

⁵I was unable to find the six percent not accounted for above.

---

Subject Object Notation:

Subject-Object Notation is a way of showing where relative to two incompatible ideas you are.  For example:

The Instrumental Mind (2) and The Socialized Mind (3)

• 2: At this level one will view the world as self and other, unable to make further differentiation when trying to understand motivations and information. (Solipsism)

 

• 2(3): They understand that others have thoughts and feelings, but they are unable to understand them.

 

• 2/3: Those they are close to can be partially understood, but much is lost in the primitive understanding of others.

 

• 3/2: This is the tipping point between other people think things differently from me, and this person thinks this, that person thinks that, and I think this.  This change also incurs the shift from self-centered to belonging to the tribe.

 

• 3(2): This level allows generalization across large groups and focus on the individual.  While still occasionally self-centered an individual at this level will be aware of and bend to social pressure.

 

• 3: At this level a person is able to understand the various different motivations of others, however they are subject to tribal status, being defined by what others think of them, and treating an entire out-group as homogenous.  (Republicans are so.../Democrats are so.../Religious people are so.../atheists are so...)

 

 

Using Subject-Object Notation on Constructive Developmental Theory yields 21 unique "levels" of development.

 

1 1(2) 1/2 2/1 2(1)

2 2(3) 2/3 3/2 3(2)

3 3(4) 3/4 4/3 4(3)

4 4(5) 4/5 5/4 5(4)

5

---

Four Player Model:

Movers: The ones making changes to the current group behaviour.

Follower: Those who are continuing the current move.

Opposers: Those correcting the current move.

Bystanders: The ones watching for anything else the group should be looking out for.

	Socialized Mind	Self-Authoring Mind	Self-Transforming Mind
Moving	This a rare state for a Socialized Mind.  The inherent risk to status makes even potentially large gains less appealing.	A natural role for a Self-Authoring Mind, being independent of the group allows them to propose changes, though that is limited by their beliefs and ideologies.	Much the same as Self-Authors Self-Transformers are suitable for filling the role of mover, though with larger amounts of resources to draw from.
Following	At this level people are defined by tribal status making them excellent followers.	Following is not a role a level 4 will fall into unless they arrive at the conclusion on their own.	While not as difficult as it was as a Self-Author following is still the weakest point of the higher levels due largely to the absence of cultural influence in personal thought.
Opposing	Individuals at this level do not oppose without prompting, and will likely try to smooth over any mover/Opposer conflict.	At least as much as Moving, Opposing suits a 4th order mind, because even if they agree they are able to play Devil's Advocate as disagreement doesn't undermine their sense of self.	Opposing is a role Transforming Minds fill with little work.  The vast amount of viewpoints they can hold allow them to freely choose a good response to Movers.
Bystanding	Similar to opposing third order minds don't make very good bystanders, as that would necessitate leaving the group thought process that defines them.	A Self-Author is a suitable, if slightly biased, Bystander for much the same reason they are good Opposers.	This is the role that this level truly excels in due to the shear number of viewpoints they are able to use.

 

---

Authors Notes:

¹The lack of "better" levels seems to indicate that each level is a local optima with at least a few required for a stable society.

²This would seem to indicate that higher orders are capable of everything that a lower order is, motivation not withstanding.

³This level includes both human babies and animals.

⁴In addition to children some animals have pack/herd/pod mentalities that would appear to be at least 2(3).

⁵I would predict 5+ percent in level 2, and only the wild children in level 1, (those children who are raised by wild animals) with even some of them as level 2.

*This is the relation to Four Player Model

---

Attributions:

http://developmentalobserver.blog.com/2010/06/09/an-overview-of-constructive-developmental-theory-cdt/ - Three highest levels of CDT

http://sustainabilitythinking.wordpress.com/2012/06/09/constructive-developmental-theory/ - less detailed description of all five

http://developmentalobserver.blog.com/2011/06/08/additional-resources-on-adult-development/ - Assorted links

http://developmentalobserver.blog.com/2011/02/21/kantors-four-player-model-through-the-lens-of-cdt/ - Kantor's Four Player Model

http://malcolmocean.com/2014/10/subject-object-notation/ - Subject-Object Notation

http://www.ccl.org/leadership/pdf/landing/constructivedevtheory.pdf - CDT more in depth