In a recent post Gwerley covered Constructive Developmental Theory, and Subject-Object Notation.  I'll be going through a basic description of the ideas, as well as adding related ideas from the Four Player Model.

Constructive Developmental Theory:

Constructive Developmental Theory is a Theory of Mind that splits the development of people into five levels, though the levels each have a unique set of advantages/disadvantages, not being "better" or "worse" than one another.¹  This theory is largely based on if the individual is subject to something or able to hold it as an object using meta cognition, such that each level holds the previous levels as special cases.²  This progress makes it so a higher order mind will notice things a lower order cannot.

• First Order/The Impulsive Mind

  • An impulsive mind has only it's reflexes as an object.
  • At this level an organism³ is still sorting out sensory input, and does not yet have a theory of mind.

• Second Order/The Instrumental Mind

  • As an Instrumental Mind the being is able to understand the difference between self and other.
  • Though second order minds understand the concept of self they are subject to their wants, needs and desires.⁴
  • At this level the person has only one viewpoint. (Solipsism)

• Third Order/The Socialized Mind

  • The Socialized mind is able to hold as an object their emotions, needs and desires.
  • This level is subject to cultural in-group/out-group pressures.
  • They are defined by their relation to society.
    1. This relation makes them susceptible to the wants and needs of others, making them likely to try and do/say what [they think] others want.
    2. Their reliance on society and authority makes them good followers.
  • 58% of adults are on this level.

*With the bulk of people being on this level it's important to keep status with them.  Failure to do so risks loosing momentum on any movement you're working on. (Trans humanism, Cryonics or FAI being the three that jump to mind with this community.)

• Fourth Order/The Self-Authoring Mind

  • The Self-Authoring Mind is able to hold as an object the environment it belongs to. 
  • Self-Authors are still subject to their own ideologies.
  • They are defined by what they think of themselves in relation to their ideologies as a static unchanging state dependent on their ideology. (The "self" can change if the ideology does.)
    1. This makes them able thinkers. (Provided they have knowledge of the subject matter.) allowing them to oppose things they think are wrong as their sense of self is not dependent on their relation to the community.
    2. This freethinking also makes them hesitant followers unless they reach the same conclusions on their own. 
  • 35% of adults are on this level

*While less essential than Socialized Minds, Self-Authoring Minds are a good indicator that your movement is healthy and still able to adapt to changes.  Being the primary source of said changes fourth order minds are important in order to avoid things like an Ann Rand cult.

• Fifth Order/The Self-Transforming Mind
  

  • The Self-Transforming Mind is able to hold as an object the relation between ideologies, including their own. 
  • This subjects them to the relation between the ideologies, and the search for a solution to their contradictions.
    
    1. The Self-Transforming Mind is the point at which much of the advice on this site start to make sense as something other than just "it just works."
    2. It allows changes to self akin to harry's occlumancy training in HPMOR "Anyone you can imagine you can be." 
  • 1% of adults are on this level

 *The most useful and the least essential of the groups.  They are able to fill any role needed, but are made fully redundant by a enough lower order minds in the necessary roles.

⁵I was unable to find the six percent not accounted for above.

Subject Object Notation:

Subject-Object Notation is a way of showing where relative to two incompatible ideas you are.  For example:

The Instrumental Mind (2) and The Socialized Mind (3)

• 2: At this level one will view the world as self and other, unable to make further differentiation when trying to understand motivations and information. (Solipsism)

• 2(3): They understand that others have thoughts and feelings, but they are unable to understand them.

• 2/3: Those they are close to can be partially understood, but much is lost in the primitive understanding of others.

• 3/2: This is the tipping point between other people think things differently from me, and this person thinks this, that person thinks that, and I think this.  This change also incurs the shift from self-centered to belonging to the tribe.

• 3(2): This level allows generalization across large groups and focus on the individual.  While still occasionally self-centered an individual at this level will be aware of and bend to social pressure.

• 3: At this level a person is able to understand the various different motivations of others, however they are subject to tribal status, being defined by what others think of them, and treating an entire out-group as homogenous.  (Republicans are so.../Democrats are so.../Religious people are so.../atheists are so...)

Using Subject-Object Notation on Constructive Developmental Theory yields 21 unique "levels" of development.

1 1(2) 1/2 2/1 2(1)

2 2(3) 2/3 3/2 3(2)

3 3(4) 3/4 4/3 4(3)

4 4(5) 4/5 5/4 5(4)

5

Four Player Model:

Movers: The ones making changes to the current group behaviour.

Follower: Those who are continuing the current move.

Opposers: Those correcting the current move.

Bystanders: The ones watching for anything else the group should be looking out for.

Authors Notes:

¹The lack of "better" levels seems to indicate that each level is a local optima with at least a few required for a stable society.

²This would seem to indicate that higher orders are capable of everything that a lower order is, motivation not withstanding.

³This level includes both human babies and animals.

⁴In addition to children some animals have pack/herd/pod mentalities that would appear to be at least 2(3).

⁵I would predict 5+ percent in level 2, and only the wild children in level 1, (those children who are raised by wild animals) with even some of them as level 2.

*This is the relation to Four Player Model

Attributions:

http://developmentalobserver.blog.com/2010/06/09/an-overview-of-constructive-developmental-theory-cdt/ - Three highest levels of CDT

http://sustainabilitythinking.wordpress.com/2012/06/09/constructive-developmental-theory/ - less detailed description of all five

http://developmentalobserver.blog.com/2011/06/08/additional-resources-on-adult-development/ - Assorted links

http://developmentalobserver.blog.com/2011/02/21/kantors-four-player-model-through-the-lens-of-cdt/ - Kantor's Four Player Model

http://malcolmocean.com/2014/10/subject-object-notation/ - Subject-Object Notation

http://www.ccl.org/leadership/pdf/landing/constructivedevtheory.pdf - CDT more in depth

After having viewed a recent post by Gworley I noticed that the material was deliberately opaque*.  It's not as complicated as it seems, and should be able to be taught to people at lower than "Level 4" on Kegan's Constructive Developmental Theory.  The only serious block I saw was the ridiculous gap in inferential distance. 
 
With that in mind I was hoping someone might have recommendations on even tangentially related material as what I have now appears to be insufficient.  (Simplifying CDT appears to be manageable, but not particularly useful without further material like Kantor's Four Player Model and Subject-Object Notation.)

*edit: Not Gworley's, it was Kegan's material that was opaque.

Finding a good decision theory is hard. Previous attempts, such as Timeless Decision Theory, work, it seems, in providing a stable, effective decision theory, but are mathematically complicated. Simpler theories, like CDT or EDT, are much more intuitive, but have deep flaws. They fail at certain problems, and thus violate the maxim that rational agents should win. This makes them imperfect.

But it seems to me that there is a relatively simple fix one could make to them, in the style of TDT, to extend their power considerably. Here I will show an implementation of such an extension of CDT, that wins on the problems that classic CDT fails on. It quite possibly could turn out that this is not as powerful as TDT, but it is a significant step in that direction, starting only from the naivest of decision theories. It also could turn out that this is nothing more than a reformulation of TDT or a lesser version thereof. In that case, this still has some value as a simpler formulation, easier to understand. Because as it stands, TDT seems like a far cry from a trivial extension of the basic, intuitive decision theories, as this hopes to be.

We will start by remarking that when CDT (or EDT) tries to figure out the expected value or a action or outcome, the naive way which it does so drops crucial information, which is what TDT manages to preserve. As such, I will try to calculate a CDT with this information not dropped. This information is, for CDT, the fact that Omega has simulated you and figured out what you are going to do. Why does a CDT agent automatically assume that it is the "real" one, so to speak? This trivial tweak seems powerful. I will, for the purpose of this post, call this tweaked version of CDT "Simulationist Causal Decision Theory", or SCDT for short.

Let's run this tweaked version though Newcomb's problem. Let Alice be a SCDT agent. Before the problem begins, as is standard in Newcomb's problem, Omega looks at Alice and calculates what choice Alice will make in the game. Without to much loss of generality, we can assume that Omega directly simulates Alice, and runs the simulation through the a simulation of the game, in order make the determination of what choice Alice will make. In other formulations of Newcomb's problem, Omega figures in out some other way what Alice will do, say by doing a formal analysis of her source code, but that seems intuitively equivalent. This is a possible flaw, but if the different versions of Newcomb's problem are equivalent (as they seem to be) this point evaporates, and so we will put it aside for now, and continue.

We will call the simulated agent SimAlice. SimAlice does not know, of course, that she is being simulated, and is an exact copy of Alice in all respects. In particular, she also uses the same SCDT thought processes as Alice, and she has the same utility function as Alice.

So, Alice (or SimAlice, she doesn't know which one she is) is presented with the game. She reasons thusly:

There are two possible cases: Either I am Alice or I am SimAlice. 

• If I am Alice: Choosing both boxes will always get me exactly $1000 more then choosing just one. Regardless of whether or not there is $1,000,000 in box 2, by choosing box 1 as well, I am getting an extra $1000. (Note that this is exactly the same reasoning standard CDT uses!)
• If I am SimAlice: Then "I" don't actually get any money in this game, regardless of what I choose. But my goal is not SimAlice getting money it is is Alice getting money, by the simple fact that this is what Alice wants, and we assumed above that SimAlice uses the same utility function as Alice.And depending what I choose now, that will affect the way Omega sets up the boxes, and so affects the amount of money Alice will get. Specifically, if I one box, Omega will put an extra $1,000,000 in box 2, and so Alice will get an extra $1,000,000, no matter what she chooses. (Because in both the choices Alice could make (taking either box 2 or boxes 1&2), she takes box 2, and so will wind up with a bonus $1,000,000 above what she would get if box 2 was empty, which is what would happen if SimAlice didn't two box.)

So, as I don't know whether I am Alice or SimAlice, and as there is one of each, there is a 0.5 probability of me being either one, so by the law of total expectation,

E[money|I one box]=0.5 * E[money|(I one box)&(I am Alice)] + 0.5 * E[money|(I one box)&(I am SimAlice)]

So my expected return off one boxing (above what I would get by two boxing) is 0.5 * -$1000 + 0.5 * $1,000,000 = $450,000, which is positive, so I should one box.

A technical report of the Future of Humanity Institute (authored by me), on why anthropic probability isn't enough to reach decisions in anthropic situations. You also have to choose your decision theory, and take into account your altruism towards your copies. And these components can co-vary while leaving your ultimate decision the same - typically, EDT agents using SSA will reach the same decisions as CDT agents using SIA, and altruistic causal agents may decide the same way as selfish evidential agents.

Anthropics: why probability isn't enough

This paper argues that the current treatment of anthropic and self-locating problems over-emphasises the importance of anthropic probabilities, and ignores other relevant and important factors, such as whether the various copies of the agents in question consider that they are acting in a linked fashion and whether they are mutually altruistic towards each other. These issues, generally irrelevant for non-anthropic problems, come to the forefront in anthropic situations and are at least as important as the anthropic probabilities: indeed they can erase the difference between different theories of anthropic probability, or increase their divergence. These help to reinterpret the decisions, rather than probabilities, as the fundamental objects of interest in anthropic problems.

I stumbled upon this paper by Andy Egan and thought that its main result should be shared. We have the Newcomb problem as counterexample to CDT, but that can be dismissed as being speculative or science-fictiony. In this paper, Andy Egan constructs a smoking lesion counterexample to CDT, and makes the fascinating claim that one can construct counterexamples to CDT by starting from any counterexample to EDT and modifying it systematically.

The "smoking lesion" counterexample to EDT goes like this:

• There is a rare gene (G) that both causes people to smoke (S) and causes cancer (C). Susan mildly prefers to smoke than not to - should she do so?

EDT implies that she should not smoke (since the likely outcome in a world where she doesn't smoke is better than the likely outcome in a world where she does). CDT correctly allows her to smoke: she shouldn't care about the information revealed by her preferences.

But we can modify this problem to become a counterexample to CDT, as follows:

• There is a rare gene (G) that both causes people to smoke (S) and makes smokers vulnerable to cancer (C). Susan mildly prefers to smoke than not to - should she do so?

Here EDT correctly tells her not to smoke. CDT refuses to use her possible decision as evidence that she has the gene and tells her to smoke. But this makes her very likely to get cancer, as she is very likely to have the gene given that she smokes.

The idea behind this new example is that EDT runs into paradoxes whenever there is a common cause (G) of both some action (S) and some undesirable consequence (C). We then take that problem and modify it so that there is a common cause G of both some action (S) and of a causal relationship between that action and the undesirable consequence (S→C). This is then often a paradox of CDT.

It isn't perfect match - for instance if the gene G were common, then CDT would say not to smoke in the modified smoker's lesion. But it still seems that most EDT paradoxes can be adapted to become paradoxes of CDT.

I have read lots of LW posts on this topic, and everyone seems to take this for granted without giving a proper explanation. So if anyone could explain this to me, I would appreciate that.

This is a simple question that is in need of a simple answer. Please don't link to pages and pages of theorycrafting. Thank you.

Edit: Since posting this, I have come to the conclusion that CDT doesn't actually play Newcomb. Here's a disagreement with that statement:

If you write up a CDT algorithm and then put it into a Newcomb's problem simulator, it will do something. It's playing the game; maybe not well, but it's playing.

And here's my response:

The thing is, an actual Newcomb simulator can't possibly exist because Omega doesn't exist. There are tons of workarounds, like using coin tosses as a substitution for Omega and ignoring the results whenever the coin was wrong, but that is something fundamentally different from Newcomb.

You can only simulate Newcomb in theory, and it is perfectly possible to just not play a theoretical game, if you reject the theory it is based on. In theoretical Newcomb, CDT doesn't care about the rule of Omega being right, so CDT does not play Newcomb.

If you're trying to simulate Newcomb in reality by substituting Omega with someone who has only empirically been proven right, you substitute Newcomb with a problem that consists of little more than simple calculation of priors and payoffs, and that's hardly the point here.

Edit 2: Clarification regarding backwards causality, which seems to confuse people:

Newcomb assumes that Omega is omniscient, which more importantly means that the decision you make right now determines whether Omega has put money in the box or not. Obviously this is backwards causality, and therefore not possible in real life, which is why Nozick doesn't spend too much ink on this.

But if you rule out the possibility of backwards causality, Omega can only make his prediction of your decision based on all your actions up to the point where it has to decide whether to put money in the box or not. In that case, if you take two people who have so far always acted (decided) identical, but one will one-box while the other one will two-box, Omega cannot make different predictions for them. And no matter what prediction Omega makes, you don't want to be the one who one-boxes.

Edit 3: Further clarification on the possible problems that could be considered Newcomb:

There's four types of Newcomb problems:

1. Omniscient Omega (backwards causality) - CDT rejects this case, which cannot exist in reality.
2. Fallible Omega, but still backwards causality - CDT rejects this case, which cannot exist in reality.
3. Infallible Omega, no backwards causality - CDT correctly two-boxes. To improve payouts, CDT would have to have decided differently in the past, which is not decision theory anymore.
4. Fallible Omega, no backwards causality - CDT correctly two-boxes. To improve payouts, CDT would have to have decided differently in the past, which is not decision theory anymore.

That's all there is to it.

Edit 4: Excerpt from Nozick's "Newcomb's Problem and Two Principles of Choice":

Now, at last, to return to Newcomb's example of the predictor. If one believes, for this case, that there is backwards causality, that your choice causes the money to be there or not, that it causes him to have made the prediction that he made, then there is no problem. One takes only what is in the second box. Or if one believes that the way the predictor works is by looking into the future; he, in some sense, sees what you are doing, and hence is no more likely to be wrong about what you do than someone else who is standing there at the time and watching you, and would normally see you, say, open only one box, then there is no problem. You take only what is in the second box. But suppose we establish or take as given that there is no backwards causality, that what you actually decide to do does not affect what he did in the past, that what you actually decide to do is not part of the explanation of why he made the prediction he made. So let us agree that the predictor works as follows: He observes you sometime before you are faced with the choice, examines you with complicated apparatus, etc., and then uses his theory to predict on the basis of this state you were in, what choice you would make later when faced with the choice. Your deciding to do as you do is not part of the explanation of why he makes the prediction he does, though your being in a certain state earlier, is part of the explanation of why he makes the prediction he does, and why you decide as you do.

I believe that one should take what is in both boxes. I fear that the considerations I have adduced thus far will not convince those proponents of taking only what is in the second box. Furthermore I suspect that an adequate solution to this problem will go much deeper than I have yet gone or shall go in this paper. So I want to pose one question. I assume that it is clear that in the vaccine example, the person should not be convinced by the probability argument, and should choose the dominant action. I assume also that it is clear that in the case of the two brothers, the brother should not be convinced by the probability argument offered. The question I should like to put to proponents of taking only what is in the second box in Newcomb's example (and hence not performing the dominant action) is: what is the difference between Newcomb's example and the other two examples which make the difference between not following the dominance principle, and following it?

By orthonormal's suggestion, I take this out of comments.

Consider a CDT agent making a decision in a Newcomb's problem, in which Omega is known to make predictions by perfectly simulating the players. Assume further that the agent is capable of anthropic reasoning about simulations. Then, while making its decision, the agent will be uncertain about whether it is in the real world or in Omega's simulation, since the world would look the same to it either way.

The resulting problem has a structural similarity to the Absentminded driver problem¹. Like in that problem, directly assigning probabilities to each of the two possibilities is incorrect. The planning-optimal decision, however, is readily available to CDT, and it is, naturally, to one-box.

Objection 1. This argument requires that Omega is known to make predictions by simulation, which is not necessarily the case.

Answer: It appears to be sufficient that the agent only knows that Omega is always correct. If this is the case, then a simulating-Omega and some-other-method-Omega are indistinguishable, so the agent can freely assume simulation.

[This is a rather shaky reasoning, I'm not sure it is correct in general. However, I hypothesise that whatever method Omega uses, if the CDT agent knows the method, it will one-box. It is only a "magical Omega" that throws CDT off.]

Objection 2. The argument does not work for the problems where Omega is not always correct, but correct with, say, 90% probability.

Answer: Such problems are underspecified, because it is unclear how the probability is calculated. [For example, Omega that always predicts "two-box" will be correct in 90% cases if 90% of agents in the population are two-boxers.] A "natural" way to complete the problem definition is to stipulate that there is no correlation between correctness of Omega's predictions and any property of the players. But this is equivalent to Omega first making a perfectly correct prediction, and then adding a 10% random noise. In this case, the CDT agent is again free to consider Omega a perfect simulator (with added noise), which again leads to one-boxing.

Objection 3. In order for the CDT agent to one-box, it needs a special "non-self-centered" utility function, which when inside the simulation would value things outside.

Answer: The agent in the simulation has exactly the same experiences as the agent outside, so it is the same self, so it values the Omega-offered utilons the same. This seems to be a general consequence of reasoning about simulations. Of course, it is possible to give the agent a special irrational simulation-fearing utility, but what would be the purpose?

Objection 4. CDT still won't cooperate in the Prisoner's Dilemma against a CDT agent with an orthogonal utility function.

Answer: damn.

¹ Thanks to Will_Newsome for pointing me to this.

I'd like to present a couple thoughts. While I am somewhat confident in my reasonning, my conclusions strongly contradict what I perceive (possibly incorrectly) to be the concensus around decision theory on LessWrong. This consensus has been formed by people who have spent more time than me thinking about it, and are more intelligent than I am. I am aware of that, this is strong evidence that I am mistaken or obvious. I believe nonetheless the argument I'm about to make is valuable and should be heard. 

It is argued that the key difference between Newcomb's problem and Solomon's problem is that precommitment is useful in the former and useless in the latter. I agree that the problems are indeed different, but I do not think that is the fundamental reason. The devil is in the details.

Solomon's problem states that

 - There is a gene that causes people to chew gum and to develop throat cancer
 - Chewing gum benefits everyone

It is generally claimed that EDT would decide not to chew gum, because doing so would place the agent in a state where its expected utility is reduced. This seems incorrect to me. The ambiguity is in what is meant by "causes people to chew gum". If the gene really causes people to chew gum, then that gene by definition affects that agent's decision theory, and the hypothesis that it is also following EDT is contradictory. What is generally meant is that having this gene induces a preference to chew gum, which is generally acted upon by whatever decision algorithm is used. An EDT agent must be fully aware of its own preferences, otherwise it could not calculate its own utility, therefore, the expected utility of chewing gum must be calculated conditional on having a preexisting or non preexisting taste for gum. In a nutshell, an EDT agent updates not on his action to chew gum, but on his desire to do so.

I've established here a distinction between preferences and decision theory. In fact, the two are interchangeable. It is always possible to hard code preferences in the decision theory, and vice versa. The distinction is very similar to the one drawn between code and data. It is an arbitrary but useful distinction. Intuitively, I believe hard coding preferences in the decision algorithm is poor design, though I do not have a clear argument why that is.

If we insist on preferences being part of the decision algorithm, the best decision algorithm for solomon's problem is the one that doesn't have a cancer causing gene. If the algorithm is EDT, then liking gum is a preference, and EDT makes the same decision as CDT.

Let's now look at Newcomb's problem. Omega's decision is clearly not based on a subjective preference for one box or two box (let's say an aesthetic preference for example). Omega's decision is based on our decision algorithm itself. This is the key difference between the two problems, and this is why precommitment works for Newcomb's and not Solomon's.

Solomon's problem is equivalent to this problem, which is not Newcomb's

- If Omega thinks you were born loving Beige, he puts $1,000 in box Beige and nothing in box Aquamarine.
- Otherwise, he puts $1,000 in box Beige and nothing in box Aquamarine.

In this problem, both CDT and EDT (correctly) two box. Again, this is because EDT knows that it loves beige.

Now the real Newcomb's problem. I argue that an EDT agent should integrate his own decision as evidence. 

 - If EDT's decision is to two-box, then Omega's prediction is that EDT two boxes and EDT should indeed two-box.
 - If EDT's decision is to one-box, then Omega's prediction is that EDT one box, and EDT should two-box. 

Since EDT reflects on his own decision, it can only be the only fixed point which is to two box.

Both CDT and EDT decide to chew gum and to two box.

If we're out shopping for decision algorithms (TDT, UDT...), we might as well shop for a set of preferences, since they can be interchangeable. It is clear that some preferences allow winning, when variable sum games are involved. This has been implemented by evolution as moral preferences, not as decision algorithms. One useful preference is the preference to keep one's word. Such a preference allows to pay Parfit's hitchiker without involving any preference reversal. Once you're safe, you do not try not to pay, because you genuinely prefer not breaking your promise than keeping the money. Yes, you could have preferences to two box, but there is no reason why you should catter in advance to crazy cosmic entities rewarding certain algorithms or preferences. Omega is no more likely than the TDT and UDT minimizer, evil entities known for torturing TDT and UDT practionners.

Edit: meant to write EDT two-boxes, which is the only fixed point.