= 27d567bc2d48d9f40e9ccc20ea370b15)
metacognition is certainly a related thing, but i think discussion of it is generally too abstract to be directly useful for modeling human thinking without first applying it to real brains. this doesn't mean it's inappropriate, only that i think it needs to be more concrete than what is typically discussed under the topic of metacognition to be useful.
This comes off as purely bragging and applause lights. And several of the applause lights aren't relevant. (that chapter of HPMoR and the 12th virtue of rationality)
Additionally, I disagree with one of the premises, which is that CDT is meaningful and useful but can only be well-understood by people who have reached a certain level in its hierarchy. I feel competent to make this judgment, because by its own description, I uncomfortably sit in level 4 (and have a very poor model of what it means to be level 3; I suspect some unpleasant circumstances and mild neuroatypicality made me leapfrog 3). And the theory does not seem to have explanatory power,
Or in short: Scrap this post and come back when you can explain it better, and don't use excuses like 'this needs higher-order thinking' ; I'll believe you have an insight but you're not conveying it now, and those excuses are just excuses.
I mean no insult to you, but if you don't understand level 3, you're almost certainly then actually spending your time thinking at levels 2 and 3 but mistaking it for level 4. This seems to be rather strongly backed up by assessment data.
Understanding Agency
Note: In this article I refer to "constructive developmental theory" as "constructive development theory", however the former is more common and should be used instead. I changed it in the version of this on my own blog, but because I think it would add some confusion to the comments if I changed it here, I'll leave it as is but just note it so you can use the more common terminology.
I used to get frustrated with myself. I'd say existential risk was an important problem or that I wanted to live an awesome life, but then I took no action to mitigate existential risks or make my life more awesome. For a long time I had no good way to explain this, often blaming it on things like akrasia, but in late 2011 I changed. I started acting to make the world have more of what I valued in it.
I've spent a lot of the past year trying to understand what happened and how I might tell other people about it. I would probably still be searching for the right framing if not for a party a few months ago. There, Malcolm Ocean and Ethan Dickinson introduced me to Constructive Development Theory, also known as Subject-Object Theory, a cognitive development theory first described by Robert Kegan et al.. Since then I've been ruminating on the idea, and after reading Malcolm's introduction to constructive development, I realize that constructive development is the concept I need to explain my 2011 mind-shift.
In short, in late 2011 I started to spend more of my time thinking at constructive development level 4 than 3, and level 4 thinking is the minimum required to stand a real chance of making the world the way you want it.
Since that sounds like utter nonsense without context, go read Malcolm's article on constructive development. Right now. Go do it. I'll still be here when you're done. Don't even bother trying to go any further until you have read it.
In fact, you should also read the links he links before you come back, and maybe do a little research on your own, because I'm not going to bother explaining constructive development theory here: I'm just going to use it.
Before we continue, one more warning. If you're not already doing most of your thinking at least half-way along the 3 to 4 transition (which I will hereon refer to as reaching 4/3), you will probably also not fully understand what I've written below because that's unfortunately also about how far along you have to be before constructive development theory makes intuitive sense to most people. I know that sounds like an excuse so I can say whatever I want, but before reaching 4/3 people tend to find constructive development theory confusing and probably not useful, and this is admittedly a weakness. My intentions must therefore be naturally limited to convincing other folks who have reached 4/3 that constructive development theory is useful for understanding what makes them different and suggests how they can help others attain a similar level of cognitive development.
Once you reach 4/3 it becomes possible to reliably apply abstract concepts to satisfy your values because you now have the ability to spend most of your time thinking about yourself from a sufficiently distant outside view that you can manipulate the concept of "you" in a way that allows you to figure out how to apply said concepts. Since that's a bit abstract, let's see what that looks like with an example.
Consider two persons in almost any given profession, but for salience let's choose teachers. Alice and Bob both value their students' learning highly and know many techniques that will successfully help their students learn. When Alice prepares for a class, she thinks mostly about the kind of teacher she needs to be in order to help her students learn. When Bob prepares for a class, he thinks mostly about what he needs to do in order to help his students learn. Both have the same goal, yet Alice is thinking mostly at level 4 while Bob is thinking mostly at level 3. Alice is trying to solve the problem of how to be a better teacher, while Bob is trying to solve the problem of how to teach better. Both are important, and Alice must also solve the problem of how to teach better, but she now views that problem as incidental to becoming a better teacher.
To complicate matters, Bob doesn't really understand that Alice is doing something different from him, nor does their colleague Carol, who spends most of her time thinking at level 2 and trying to solve the problem of how to better perform various teaching techniques. But Carol will believe she is doing the same thing as Alice and Bob, and Bob will believe he's doing the same thing as Alice (viz. thinking about how to be a better teacher) and if you try to explain this to Bob or Carol they will likely fail to appreciate that there is any real difference.
But the difference is important: at constructive development level 4, you can be the object of your own thoughts, not just the subject. At level 3 you can be the subject but not the object of your thinking, which can be incredibly frustrating, and at level 2 you can't even fully model yourself. So level 4 thinking is the minimum required to fully reason about yourself, which is why reaching 4/3 is an important inflection point in cognitive development.
If reaching 4/3 is important and actually explains different levels of achievement in satisfying values, we should find existing discussions of reaching 4/3 but with different terminology. Eliezer seems to obliquely get at something related to reaching 4/3 in his twelfth and last virtue of rationality. CFAR talks about core skill growth, which seems to include many things related to constructive development level 4 thinking. But most concretely, we see it around chapter 65 of HPMOR when other characters realize that Harry has gained agency, something talked about widely both within and outside the Less Wrong community.
But core skill growth and agency are opaque. When a person has agency we mean something like "they make their own decisions". But of course everyone trivially makes their own decisions: their brains are not directly controlled by some outside force, no matter the pressures placed upon them. What we really mean is something more like "they think, come to decisions about what to do, and then act on those decisions in ways that may be counter to the 'default' actions they would have otherwise taken". But for someone who lacks agency this is not very helpful because it frames agency like a property one either has or doesn't, not as a thought process that can be developed. Thinking of agency as a consequence of reaching 4/3 solves this problem. Similarly, understanding core skill growth as increasing time spent thinking at higher constructive development levels makes its meaning clearer.
Perhaps unsurprisingly, agency is the thing you need to make the world what you want. You can know many techniques for increasing productivity, forming friendships, earning trust, having fun, and otherwise better satisfying your values, but without agency you will be unable to reliably apply them. This makes reaching 4/3 the most important step in your cognitive development, and the faster you can get there the better off you will be.
The challenge now is to find ways of helping people constructively develop. I think we have already made some good strides here with comfort zone expansion exercises and framing rationality as the skills that help you better optimize the world for what you value, but I also think we can do better because I know many folks who have been part of the Less Wrong community for a long time yet have thus far won very little. I anticipate better progress is possible now, though, thanks to having a useful model for understanding the most fundamental aspect of becoming stronger.
Thanks to Ethan Dickinson for offering suggestions on an early draft.
I suspect boredom to be another thing that can result in willpower depletion: it's hard to stay engaged in something when it's boring. It may be possible on less difficult tasks to keep going longer, but it eventually begins to drain on you (although maybe this is covered by wanting to do some other specific thing, but I suspect it was distinct in that you can be bored without having something you would rather be doing).
Digging the Bull's Horn
Some time ago I learned of the metaphor of 'digging the bull's horn'. This might sound a little strange, since horns are mostly hollow, but imagine a bull's horn used to store black powder. In the beginning the work is easy and you can scoop out a lot powder with very little effort. As you dig down, though, each scoop yields less powder as you dig into the narrow part of the horn until the only way you can get out more powder is to turn the horn over a dump it out.
It's often the same way with learning. When you start out in a subject there is a lot to be learned (both in quantity of material you have not yet seen and in quantity of benefits you have to gain from the information), but as you dig deeper into a subject the useful insights come less often or are more limited in scope. Eventually you dig down so far that the only way to learn more is to discover new things that no one has yet learned (to stretch the metaphor, you have to add your own powder back to dig out).
It's useful to know that you're digging the bull's horn when learning because, unless you really enjoy a subject or have some reason to believe that contributing to it is worthwhile, you can know in advance that most of the really valuable insights you'll gain will come early on. If you want to benefit from knowing about as much stuff as possible, you'll often want to stop actively pursuing a subject unless you want to make a career out of it.
But, for a few subjects, this isn't true. Sometimes, as you continue to learn the last few hard things that don't seem to provide big, broadly-useful insights, you manage to accumulate a critical level of knowledge about the subject that opens up a whole new world of insights to you that were previously hidden. To push the metaphor, you eventually dig so deep that you come out the other side to find a huge pile of powder.
The Way seems to be one of those subjects you can dig past the end of: there are some people who have mastered The Way to such an extent that they have access to a huge range of benefits not available to those still digging the horn. But when it comes to other subjects, how do you know? Great insights could be hiding beyond currently obscure fields of study because no one has bothered to dig deep enough. Aside from having clear examples of people who came out the other side to give us reason to believe it's worth while to deep really deep on some subjects, is there any way we can make a good prediction about what subjects may be worth digging to the end of the bull's horn?
Let me just toss out some caution here.
I'm all for getting excited and making stuff happen. Maybe it really is that there have not yet been any LW startups because we all just failed to coordinate on it and in hindsight we'll all say "why the hell did we all wait for so long". That said, let's not forget a few key things here.
- Most startups fail
- even when the principals are smart and motivated
- even when the idea is really good
- even when [x] is [y]
And, as I already said, for some reason we haven't already had a bunch of successful LW startups. It's certainly not for lack of smart people, entrepreneurs, or technical skills.
If a LW startup is going to succeed, I think we would benefit from understanding first why we don't already have successful LW startups (not even one).
Just to toss in my own strongest suspicion. Among LWers under 25, they probably see themselves as young and still learning and not yet brave enough to throw themselves all in to something. For those over 25, they (myself included) probably see themselves as already busy doing something and would need some pretty strong motivation to do something else, even if it does align with core values.
Let me just toss out some caution here.
I'm all for getting excited and making stuff happen. Maybe it really is that there have not yet been any LW startups because we all just failed to coordinate on it and in hindsight we'll all say "why the hell did we all wait for so long". That said, let's not forget a few key things here.
- Most startups fail
- even when the principals are smart and motivated
- even when the idea is really good
- even when [x] is [y]
And, as I already said, for some reason we haven't already had a bunch of successful LW startups. It's certainly not for lack of smart people, entrepreneurs, or technical skills.
If a LW startup is going to succeed, I think we would benefit from understanding first why we don't already have successful LW startups (not even one).
This post addresses the subject of the appropriate human data compression format. Though an interesting idea, I think that the proposed method is too low in resolution. You acknowledge the lossiness, but I think it's just going to be too much.
Although the method you advocate is probably the best thing short of cryonics, I doubt there is any satisfactory compression method that can make anyone that's more similar to you than a best friend or a family member who gets stuck with your memories. It's better to have too much data than too much.
Because we share the same evolutionary past as all of our conspecifics, the biology and psychology of our brains is statistically the same. We each have our quirks of genetics and development, but even those are statistically similar among people who share our quirks. Thus with just a few bits of data we can already record most of what makes you who you are.
I'm not confident in this part. Although a large percentage of human biology and psychology are identical, the devil is in the details. From a statistical perspective, aren't humans and chimps practical identical also? Percentage similarity of traits is probably the wrong metric, since small quantitative differences can have large qualitative impact.
Your idea of a generic human template, with various subtemplates for quirks, is also interesting, but still too low resolution.
Under what metric do we say that you and I have the "same" quirk, even if our phenotypes look superficially similar? How much data is discarded by the aggregation?
Even if we assume that the notion of a generic human template is meaningful, there are almost as many ways that people can deviate from the generic human template as there are people, and there would have to be that many quirky subtemplates. It's possible that we could compress human phenotypic deviation into a lower number of templates than the number of people, but I don't think we are anywhere near having a satisfactory way to do so. In the least, storing the deltas of human phenotypes might cut down on the data we all have in common.
The problem with lossy measures of phenotype such as memories and our current ability to measure quirky deviations from the average is that they discard too much information: the genotype, and other low-level aspects of phenotype.
Let's start with the genotype problem. In the future, we synthesize a human with the same phenotype as our crude records of you (your memories, and your quirks according to the best psychometric indexes currently available). We will call this phenotype set X. Yet since multiple genotype can converge towards the same phenotype (particularly for crude measures of phenotype), the individual created is not guaranteed to have the same genotype as you, and probably won't. Due to having a different genotype, this individual could end up having traits outside X that you didn't have. They will have the same set of phenotypic traits as you that were recorded, but they may lack phenotypic traits that weren't recorded (because you method of recording discarded the data), and they may have extra phenotypic traits that you didn't have because keeping those traits out wasn't in the spec.
Fundamentally, I think it's problematic to try to reverse-engineer a mind based on a record of its outputs. That seems like trying to reverse-engineer a computer and its operating system based on a recording of everything it displayed on its monitor while being used. Even if you know how computers and operating systems work in general, you will still be unable to eliminate lots of guesswork in choosing between multiple constructions that would lead to the same output.
If you know a system's construction, you may be able to predict its outputs, but knowing the outputs of a system doesn't necessarily allow you to work backwards to its construction, even when you know the principles by which systems of that type are constructed.
I think the best reconstruction you will get through this method won't be substantially more similar to you than taking your best friend (who has highly similar personality and interests) and implanting him with your memories. At best, you would get someone similar enough to be a relative of yours.
We really need to preserve your genotype; otherwise, future scientists could make an individual with all your memories and crudely measured personality traits, but a different personality and value system (in ways that weren't measured), who wakes up and wonders what they heck you were doing all your life. We would need a solution that has all the phenotypic traits recorded for you, with the constraint that the individual created has the same genotype as you.
Yet even such a solution would still be discarding too much information about biological traits that influence your phenotype yet are not recorded in your genetic code, such as your prenatal development. It's been shown that prenatal factors influence your brain structure, personality, and interests. So we need to record your prenatal environment to be able to create a meaningful facsimile of you. Otherwise, we could end constructing someone with the same genotype, memories, and psychometric measures as you, who nevertheless had a different brain structure; such a person would probably be less like you than a twin of yours who was implanted with your memories, because your twin shares a similar prenatal environment to yours, while your copy does not. A different brain structure would create a similar problem to having a different genotype: a different brain that has the same recorded phenotype as you will differ from you in unrecorded aspects of phenotype.
I would worry that a record of every single thought and behavior you have, both from yourself and observers, would still not be enough to reverse-engineer "you" in any meaningful way. Adding the constraint of matching your genotype would help, but we still don't have a way to compress biological factors other than genotype, such as prenatal development. We have no MP3 format for humans.
The best record of your prenatal development that we have is your body and brain structure, so these would have to be stored along with your memories. Preferably at a rather cold temperature.
At best, you would get someone similar enough to be a relative of yours.
Even if that's all the better we can do, that's much better than the nothing that will befall those who would otherwise have been totally lost because they didn't sign up for cryonics.
This is also a primary plot point of the Battlestar Galactica prequel Caprica. This also comes in Charles Stross's Accelerando up when some evil AIs decide to do this to most of humanity based on our historical records. The full text of the novel is at http://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando.html Search for "Frequently Asked Questions" to find the relevant section.
There is also another similarly interesting plot thread in the story which can be summed up by this excerpt:
The Church of Latter-Day Saints believes that you can't get into the Promised Land unless it's baptized you – but it can do so if it knows your name and parentage, even after you're dead. Its genealogical databases are among the most impressive artifacts of historical research ever prepared. And it likes to make converts.
The Franklin Collective believes that you can't get into the future unless it's digitized your neural state vector, or at least acquired as complete a snapshot of your sensory inputs and genome as current technology permits. You don't need to be alive for it to do this. Its society of mind is among the most impressive artifacts of computer science. And it likes to make converts.
I have no doubt that this sort of thing has been occasionally explored in fiction. That said, there's a big difference between considering an idea in fiction and considering acting on an idea in real life.
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
You have a link to an "article on constructive development", which you repeat no fewer than six times to encourage readers to go and read it.
However, the thing at the far end of the link is not an article on constructive development. It is an article about (1) two ways of responding to one's own misdeeds and (2) a notation for describing stages in the transition between two modes of thinking. (The notation is called "subject-object notation" but appears to have nothing specifically to do with the subject/object distinction. This doesn't seem to me like a good sign that the author is thinking clearly about things.)
There is a link from there to a summary of constructive-developmental theory by Peter Pruyn. It seems ... OK, I guess. I'm rather put off by the patronizing mealy-mouthedness with which the author disclaims the very idea that the later stages might be thought "better" -- in the same article in which he says that later stages indicate their capacity to cope with difficult situations, suggests that those at earlier stages are unfit for senior roles at work, calls the later stages "higher levels of consciousness", and of course classifies them as developmental stages which on its own pretty much gives the game away.
Still, congratulations on reaching level 4. (Though it seems to me there's something rather inappropriate about saying that.)
I agree that one of the problems with constructive development theory, as you seem to hint at, is that it sets off certain alarm bells in your mind because it matches the same patterns as things which we have now concluded to be incorrect or just instruments for abuse of power.
Explicit levels are a common tactic taken to try to give rationalizations of why this person or that person of higher status "deserves" that status against human egalitarian norms, so naturally any theory that includes something like them feels a bit icky, and nothing in the material presented here does much to clear that up.
I've also not read a good explanation of constructive development theory that would make sense to someone who doesn't understand the subject-objection notation (that is, I've seen nothing that does a great job of explaining subject-object notation to someone who doesn't immediately grasp the concept and then just needs some details filled in) or who hasn't started thinking at least some of the time at level 4.
However constructive development theory certainly hasn't been as vigorously researched and written about as many other topics in psychology, and lacking any strong disconfirming evidence I'm inclined to suspect we might find good evidence and helpful explanations if we spend some more time digging into it.
I'm woefully under-skilled for the task of both rigorous scientific studies and clear explanations that will satisfy a wide audience, so my main hope is that my insights spur on a few folks who are appropriately skilled to dig deeper.