Ignore this. After getting some much needed sleep I decided that the first half of this post, though necessary if I am to write the second half, isn't justified in being so complicated unless I actually use the added complications in solving the problems posed. First, I will use empiricism to see if this model is helpful in finding and verifying methods of attack on the problem of crafting a rationalist identity. If so, I will go back and try to simplify what I have here and make sure all parts are necessary to understand the solutions found.

---

 

Related to: Cached Selves, Cached Thoughts, A Rational Identity

Thesis: Harnessing the large pressures on thoughts and actions generated by innate drives for signaling, identity negotation, and identity formation by consciously aiming for instrumentally optimal attractors in identityspace should allow for structured self-improvement and a natural defense against goal distortion. First, though, we must see the true scope of the problem, and then find methods of attack.

Part One: Conceptualizing the Problem Domain

I: It Begins

Life is an odd, long journey. The difference between a young girl and the wizened professor she becomes is quite profound. One of those crazy things in life that seems so natural. The child surely has a very different set of aspirations than the professor. They look different. They have entirely different thoughts. They share the same genes, but the two are still way more different than the average pair of identical twins. The professor has very dull memories of the child, but that's a pretty weak link, and it only goes one way. They have different senses of self, different beliefs, different affiliations, different relationships, different goals. Different identities. Is the professor what the child would have wished to become, if the child had known more and thought faster, had become wiser and more enlightened by some magical process? It's hard to say; it's hard to say how well the child's theoretically reflectively endorsed goals were fulfilled, how many utilons were attained by that standard.

There is a long, long journey between the child and the professor, and the journey is frought with potential for both growth and goal distortion. Talk of genetic predispositions all you want, but Orwell wasn't that far off in claiming that man is infinitely malleable. That list of human universals you might have seen pertains to individuals much less than to cultures, and even less so to the individual splices of your average person's life. We may have a lot in common on average, but there's a heck of a lot of individual variance, and the memeplex of thoughtspace can do a lot of surprising things with that variance [homosexuality adaptation towards something else evo psych book]. Amplification, subjugation, deflection, attraction: mindspace is an interactive and powerful playing field, chaotic and multidimensional. If we are to think about it clearly, we should probably try to nail down some terminology.

 

II: Definitions: Mindspace, Attractors, Rationality

---

Mindspace is the configuration space of a mind. For humans at least, mindspace can be loosely broken down into thoughtspace and identityspace. Generally speaking, thoughtspace is ephemeral where identityspace is more constant, though changing. Attractors in the realm of thoughtspace tend to work on timescales like seconds or hours, where identityspace is what happens when patterns of thoughts, emotions, habits, et cetera persist for days or lifetimes. Humans are constantly moving through mindspace. Some are drifting lazily, some are propelled purposefully, some are tossed about chaotically. Ideally, we wish to move strongly and quickly towards the areas of mindspace that are most conducive to achieving our goals. We do so by thinking optimal thoughts and having optimal dispositions. Of course, it's never that easy. (Reference.)

---

Thoughtspace is a giant multidimensional web of knowledge and possible thoughts and insights. Most of it is handed to you in some form or another, from sensory experience or a book or a friend of a friend, probably mutated along the way, often for the worse. Thoughtspace is made up of thoughts, not dispositions or habits: those are in identityspace. But habits of thinking determine your movement through thoughtspace, and thoughts when thought habitually may quickly become part of your identity. The two spaces are connected at many joints, sometimes non-obviously.

Your web of beliefs makes up your starting point for exploration of thoughtspace. There are many destinations to aim for. Sometimes, we wish to gain new knowledge to update an old belief, or sometimes we spend time thinking to integrate new evidence. Sometimes we synthesize old concepts and come up with something new. Sometimes we formulate plans to reach goals provided by our identities. The part of thoughtspace in your grasp is your map; it reflecs reality, sorta, and allows you to navigate via your identity towards thoughts and beliefs that are likely to fulfill your goals, insofar as you know what your goals are.

---

Identityspace is a thousand-dimensional configuration space of thought patterns, emotion patterns, habits, dispositions, group affiliations, biases, goals, values, structured clusters of pre-conceived ideas, specific knowledge structures or cognitive representations of the self, mental frameworks centering on a specific theme that helps to organize social information, et cetera. The stability of identity varies from person to person. Most generally don't change all that much past a certain. Some, like those with bipolar disorder, cycle between two configurations in identity space. Some purposefully reinvent themselves every few months.

But at any rate, identity is obviously very malleable, even over short spans of time. Chaotic fluctuations are not uncommonly triggered by innocuous stimuli. Most often, though, significant shifts in identityspace follow noticeably large events. Long-held grudges are dropped, religions are broken with, et cetera. But more regularly, people slowly and suboptimally drift through identityspace, not taking care to carefully monitor their progress or their desires, and letting goal distortion creep in to the poorly kept garden of rationality.

Goals are a significant part of identityspace: for most people, they provide stable targets to work for, instead of just drifting about in the memetic sea. Thus, a great though incredibly difficult ability is being able to determinedly, methodically, swiftly, and carefully progress through identityspace towards strictly defined and significant goals. This article attempts to examine the problems and opportunities encountered while doing so.

---

Attractors are a class of thoughts, reactions, dispositions, emotions, social pressures, biases, and anything else that will push or pull or deflect or alter your grand voyage through mindspace, for better or worse. It might help to think in terms of 'positive' and 'negative' attractors, codified by their expected value in terms of successful navigation of mindspace. You can adjust for attractors, but it's difficult, and it takes a whole bunch of rationality before you start doing it right, instead of overcorrecting or trying to be too clever in using the gusts to your advantage.

Unpredictable storms of mind projection fallacy on the horizon will force you off course, and whirlpools of confirmation bias will try to suck you into the abyss. Sometimes people will try to mess around with your map just to be dicks, as if Poseidon and Zeus decided to get together to zap you with thunderbolts for fun and profit. I think I may have just confused the map with the territory. You see? It's difficult. (Inspired by these.)

---

Rationality is the method of navigation. It helps you figure out what the territory actually looks like, so you can plot a course of thoughts that leads roughly in the right direction. Navigating these insane monster-ridden hyperdimensional waters isn't exactly easy. I won't go into detail about rationality, as that is what the rest of Less Wrong is for.

---

 

III: The Evolution of Navigation

Rationality, though similar to past attempts at careful philosophy and social epistemology, is a new and different art form. Traditional rationality, the precursor to true rationality, was the product of the social process of science. Science cares not about what goes on inside your head when you navigate thoughtspace. It exists only to verify that what you find on your journey is of value in reality. Traditional rationality is what happens when you take care to hold your methodology for thinking to the same standard that would be demanded by peer review. The traditional rationalist dreams up a bunch of scientists sorta like himself, but much more critical, and puts his ideas under their imagined expected scrutiny. Sometimes the final judge is reality, but ideally you can eliminate most silly ideas before they get that far; it's best to not actually test your garage for invisible dragons.

The rationality that makes up Overcoming Bias and Less Wrong canon is what happens when the peer review council is instead made up of superintelligences that tolerate nothing less than exact precision. There is no such thing as an 'accepted' hypothesis: superintelligences don't speak binary. The number of 9's you trail your probability estimates with had better be carefully calibrated, or your Bayes score is going doooown. But such judges do not exist in reality, and so we have the Great Problem: we cannot easily verify rationality.

Bayesianism is the ideal of the perfected art, though it is incredibly difficult for humans to approximate, and the field, though growing, has few practitioners. Bayesian decision theory is the ideal method of achieving our goals, but we haven't fully worked it out yet, and humans are even further away from that ideal than just Bayesianism. It's hard to know what we're even striving for, or what methods could help us do so. We rationalists do not yet have as refined of tools for becoming maximally effective people as we do for finding truth. It is probable that the lore to help us do so is already hidden somewhere for us to find.

Buddhist meditation may be rationality's true spiritual predecessor. Both arts give the layman tools for navigating thoughtspace and, to a lesser extent, identityspace. Like meditation, it's hard to watch someone being rational. The vast majority of correct thinking is unremarkable stuff that goes on inside your head, and furthermore, again like meditation, most of it isn't verbal. It's intuitive instinctual reaction to patterns of thought that are either in harmony with the Bayes, or not. Distracting or biasing thoughts are pushed aside as the meditator or rationalist tries to see the world as it really is. Where meditation is concerned primarily with precise thought focused on the self, the rationality glorified by Less Wrong is precise thought focused on just about everything else. Perhaps this should change, and the two kin arts should become one, more precise and more effective.

I cannot speak with any authority on meditation -- I hope others will do so -- but it does not seem to me that rationality provides one with many tools for navigating identityspace. You are told things like, policy debates should not appear one-sided. Be wary when arguing your preferred position, for arguments are like soldiers, and you will feel uneasy about betraying your allies. The Greens and the Blues spilt blood over the most trivial and unimportant things; tribal affiliations revolving around sports matches within a single unified civilization. Not even Raiders fans are that crazy, generally. You are told the story of the cult of Ayn Rand, naively called 'the most unlikely cult in history.' You are told of the fearsomly powerful confirmation bias, positive bias, consistency effects, commitment effects, cached selves, priming, anchoring, and a legion of other ways for your identity to screw you over if you're not incredibly attentive. And these things you are told present a gestalt of bad habits of thought to avoid around this whole identity thing. But the tools are not precise, and they don't really allow you to do it right. Reversed stupidity continually fails to equal intelligence. Identity can be good. Perhaps we could harness the power of identity to correctly navigate identityspace, and thus thoughtspace?

 

IV: Common Attractors in Mindspace

There are way too many types of attractors, deflectors, warpers, and all sorts of weird creatures in mindspace to list exhaustively. Still, these are some of the ones that come most quickly to mind:

Group

• Social status (The mother of group attractors, for better or worse.)
• Political (Includes those who define themselves by their disinterest in politics.)[pretending to be wise]
• Religious (Atheism, though not a religion, sometimes gets treated like one by atheists.) [Note: this might be why agnostics are generally saner]
• Socioeconomic
• Clique (Colleges, golf clubs, dance clubs, and organizations.)
• Occupational
• Age
• Ethnic
• Philosophical (Including things like Bayesianism and rationality.)
• Sexual

Nongroup

• Emotional, instinctive (Fast perceptual judgment, often too fast to debias on the fly.)
• Priming and anchoring (Having specific thought patterns triggered by recent stimuli.)
• Cached selves (Being like one's past self due to consistency effects, etc.)
• Belief coherence (Having an idenity that avoids cognitive dissonance.)
• Heroic aspiring (Being like someone else, real, fictional, or imagined.)
• Negative heroic aspiring (Being unlike someone else.)
• Narcissistic (Having a sense of identity that is most self-glorifying.)

An exhaustive list of biases could also be made, but the above is a short attempt at listing the ones that trip people up most often, as well as the ones that people tend to use most effectively to motivate themselves to accomplish their nominal goals. The sword is double-eged, though the negative edge does tend to be significantly sharper. It is important to note that a sufficiently advanced rationalist could use any of the above to more effectively achieve their goals. Whether or not that is possible in practice is the main question of this post.

 

V: Rational Actors and Rationalist Attractors

From the perspective of an 'aspiring rationalist' (group identity), positive attractors in identityspace do not tend to be strongly group-based. The sorts of dispositions we would like to cultivate are not particularly tied up in any of the attractors listed in the 'group' section, and perhaps not really in the 'nongroup' section either. Dispositions like 'I stay focused and work carefully but efficiently on the task at hand' aren't really well-connected to the most common natural attractors found in identityspace: maybe if we were in House Hufferpuffer it'd be different, but as it is, there's only a vague connection with instrumental rationality, not nearly enough to draw on the power of signaling and consistency effects. Other dispositions, like 'I notice confusion and immediately seek to find the faults in my model', come more naturally to an 'aspiring rationalist', as such cognitive tricks are the focus of Less Wrong canon.

There are, of course, selection effects: Less Wrong is an attractor in mindspace that pulls strongest on those that already have similar dispositions and thoughts. Cascades are powerful. So it may not be surprising that Hufferpuffer dispositions do not come naturally to the aspiring rationalist.

The downsides of rationalist attractors are fewer than in other groups, but they do exist. The most common one I see is overconfidence in philosophical intuition. Being handed a giant tome of careful philosophy from (mostly) Eliezer and Robin, we then think that our additional philosophical views are similarly bulletproof. I had this problem bad during the first 6 months after reading Less Wrong; I didn't notice that in reaching the level of rationality I'd reached, I was only verifying the reasoning of others, not doing new and thorough analysis myself. This gave me an inflated confidence in the correctness of my intuition, even when it clashed with the intuition of my rationalist superiors. Having the identity of 'careful analytical thinker' can lead you to think you're being careful when you're not.

Also, typical groupthink. We're a pretty homogeneous bunch, and sometimes Less Wrong acts an echo chamber for not-obviously-bad but non-optimal beliefs, habits, and ideas. Even so, this part of the rationalist identity is countered by counter-cultishness, which itself is countered by awareness of cultish counter-cultishness and the related countersignaling. Oh yeah, that's another double-edged attractor in rationalist mindspace: we're very quick to go meta. Different people have different thoughts as to the overall usefulness of this disposition. I personally am rather pro-meta, whereas some of our focused instrumental rationalists think of der wille zur meta as rationalist flypaper.

Less Wrong, though, is an uncharacteristic island attractor of relative sanity amongst a constellation of crazy memes. Before we get too excited about our strengths, we should explore some ways that attractors can lead to our trajectory through mindspace going disastrously wrong.

 

VI: Social Psychology Meets the Mindspace

Social psychology is an interesting science. It is inconsistent in its accuracy, and the theories don't always carve reality at its joints. That said, there's a lot of interesting thought that's been put into it, and it is a real science. The field is very Hansonian; indeed, construal level theory belongs to this realm. By cherrypicking what seem to me to be the most interesting concepts, I think it may be possible to establish a framework for new ways to approach rationalist problems in real life by reasoning about social and psychological phenomena in terms of attractors and trajectories through mindspace.

---

Cognitive dissonance is

an uncomfortable feeling caused by holding conflicting ideas simultaneously. The theory of cognitive dissonance proposes that people have a motivational drive to reduce dissonance. They do this by changing their attitudes, beliefs, and actions. Dissonance is also reduced by justifying, blaming, and denying. It is one of the most influential and extensively studied theories in social psychology.

Experience can clash with expectations, as, for example, with buyer's remorse following the purchase of a new car. In a state

of dissonance, people may feel surprise, dread, guilt, anger, or embarrassment. People are biased to think of their choices as correct, despite any contrary evidence.

A powerful cause of dissonance is an idea in conflict with a fundamental element of the self-concept, such as "I am a good person" or "I made the right decision." The anxiety that comes with the possibility of having made a bad decision can lead to rationalization, the tendency to create additional reasons or justifications to support one's choices. A person who just spent too much money on a new car might decide that the new vehicle is much less likely to break down than his or her old car. This belief may or may not be true, but it would reduce dissonance and make the person feel better. Dissonance can also lead to confirmation bias, the denial of disconfirming evidence, and other ego defense mechanisms.

---

Self-verification is 

a social psychological theory that asserts people want to be known and understood by others according to their firmly held beliefs and feelings about themselves, that is self-views (including self-concepts and self-esteem). A competing theory to self-verification is self-enhancement or the drive for positive evaluations. 

Because chronic self-concepts and self-esteem play an important role in understanding the world, providing a sense of coherence, and guiding action, people become motivated to maintain them through self-verification. Such strivings provide stability to people’s lives, making their experiences more coherent, orderly, and comprehensible than they would be otherwise. Self-verification processes are also adaptive for groups, groups of diverse backgrounds and the larger society, in that they make people predictable to one another thus serve to facilitate social interaction. To this end, people engage in a variety of activities that are designed to obtain self-verifying information.

---

Construal level theory is the the study of near versus far modes of cognition. Robin Hanson's pet topic. It attempts to describe

the relation between psychological distance and how abstract an object is represented in someone's mind. The general idea is that the more distant an object is from the individual the more abstract it will be thought of, while the opposite relation between closeness and concreteness is true as well. In CLT psychological distance is defined on several dimensions - temporal, spatial, social and hypothetical distance being considered most important, though there is some debate among social psychologists about further dimensions like informational, experiential or affective distance.

Perhaps the most interesting implication of construal level theory is on behavior and signaling patterns: actions are near, goals are far. Thus we are susceptible to spending time thinking, talking, and planning in far mode about the people we will be and the goals we will attain, but when the opportunities actually show up, near mode pragmatism and hypocrisy are engaged, and rationalizations are suddenly very easy to come by. The effects are profound. Sitting at home I think it'd be ridiculous not to ask for that one girl's number. I'm a suave guy, I'm competent, and I'm not afraid of rejection. It's a really high expected value scenario. But when I see her on the street, it turns out my anticipations were wildly off.

Same with grand project ideas, like really jumpstarting a rationalist movement. That sounds great. I have a hundred ideas to try, and to email to other people to implement. But for some reason I never get around to doing them myself, even though I know the only thing stopping me is this weird sense of premeditated frustration at my own incompetence. It's hard to sabotage yourself more successfully than that.

One of the most amazing superpowers a rationalist could pick up is the ability to act in near mode to optimize for rational far mode preferences. A hack like that would lead to a rationalist win, hands down. More on this in Part Two.

---

Schemata theory should ring your pattern matching bell. I'll quote from the Wikipedia article:

A schema (pl. schemata), in psychology and cognitive science, describes any of several concepts including:

• An organized pattern of thought or behavior.
• A structured cluster of pre-conceived ideas.
• A mental structure that represents sex.
• A specific knowledge structure or cognitive representation of the self.
• A mental framework centering on a specific theme, that helps us to organize social information.
• Structures that organize our knowledge and assumptions about something and are used for interpreting and processing information.

A schema for oneself is called a "self schema". Schemata for other people are called "person schemata". Schemata for roles or occupations are called "role schemata", and schemata for events or situations are called "event schemata" (or scripts).

Schemata influence our attention, as we are more likely to notice things that fit into our schema. If something contradicts our schema, it may be encoded or interpreted as an exception or as unique. Thus, schemata are prone to distortion. They influence what we look for in a situation. They have a tendency to remain unchanged, even in the face of contradictory information. We are inclined to place people who do not fit our schema in a "special" or "different" category, rather than to consider the possibility that our schema may be faulty. As a result of schemata, we might act in such a way that actually causes our expectations to come true.

Or in our terminology, schemata are attractors in mindspace. I highly recommend reading the whole Wikipedia article about schemata theory for a more concise and organized version of this post. It's a gem.

---

 

VII: Introducing Goal Distortion

It is difficult to reason about what counts as goal distortion. Humans are usually considered pretty bad at knowing what sorts of things they want. There are many who lead happy lives as ascetics without thinking that it'd have been nice to give more than wisdom to the starving and poor they left along their paths. And there are many more who chase after money and status without really realizing why, nor do they become happier in doing so. It is important, then, to identify which goals we choose to define as being distorted, and what goal system changes count as distortion and not simply enlightened maturation. Thus we come to the concepts of reflective endorsement and extrapolated volition.

ga;bglamb;pl explanation explanation link to CEV bla bla bla

explain why goal distortion is a serious problem bla bla bla

explain mechanisms of goal distortion like commitment and consistency, avoiding cognitive dissonance, near/far, 

standard methods for dealing with goal distortion like not doing drugs bla bla bla

growth versus goal distortion

potential for rigidly controlled and consciously maintained identity to minimize goal distortion and maximize benefical change bla

 

VIII: Thoughts and Signals

You think what you signal, you signal what you think. Feedback processes like that don't often go supercritical, but as Eliezer points out many times in this subsequence, it is important to watch out for such cascades. That you end up thinking a lot about the things you wish to signal is one of those insights that Michael Vassar tosses around like it's no big deal, but if taken seriously, can be quite alarming and profound. I must confess, either I am significantly more aware of this fact than most, or, more likely, I have a rather strong form of this tendency. Being naturally somewhat narcissistic, I tend to have a bad habit of thinking in dialogue that is flattering to myself and my virtues. That is, flattering to the things I wish to signal. Like most people, I tend to want to signal good things, and thus by thinking about those good things I make them part of my identity and become more likely to actually do them, no?

Well, sometimes it works that way. But we like to signal things in far mode, and it's pretty easy to rationalize hypocrisy when near mode work comes up. Most of the time, thinking about the things I wish to signal is a form of wireheading. Having a dialogue playing in my head about how I'm such a careful rationalist is arguably more pleasant and undoubtedly a lot easier than actually searching for and thinking about my beliefs' or my epistemology's real weak points. And when I'm thinking these self-glorifying thoughts all the time, you can bet it comes out in my interactions with people.

You signal what you think; it's not easy to hide. And here enters yet another distortion to mislead you: the feeling and the belief of self-justification is a very strong attractor in mindspace, and like many such attractors, its danger is amplified by confirmation bias. Imagine that you signal your cherished disposition to a friend. Say, that you work hard on important problems. If your friend agrees and says so, you get a warm fuzzy feeling of self-verification (wiki link). The feedback loop gets more fuel. Which is great if you're able to use that fuel to actually do the important work you want to signal that you do, but not so much if the fuel is instead used to light distracting fires for your mind to worship all day instead. If your friend disagrees and calls you out on it, roughly the same things happen and the same rules apply. The responses are varied, but often people get offended, and dwell on that offense and how untrue it was, or dwell on ways to prove their attacker wrong with far mode thoughts of personal glory, or dwell on past examples of hard and diligent work done that prove their signals are credible. Such dwelling also provides fuel, though it's usually cruder, and even harder for the mind to use effectively.  

bleh

Your thoughts are bent by what you wish to signal. Choose your identity carefully.

 

IX: Signals and Identity

You signal what you wish to identify with. You identify with what you signal. As if one potentially catastrophic feedback cycle was all your brain would provide you with. Always remember, it's never too difficult to shoot your own foot off.

consistency effects literature summary bla bla bla

cached selves awesomeness summary bla cached selves is really awesome everyone should read it bla bla

near/far distinctions and effect on identity and signaling

 

X: Cascades: Thoughts, Signals, Identity

It is inevitable that cascades will cause gravitation towards suboptimal attractors in mindspace. The strength of the currents theoretically tells you how hard you must steer in the other direction to hold a true course, but realistically, humans just aren't that good at updating against known biases. You can see an iceberg and see its danger, but the whirlpools of confirmation bias have an annoying tendency to look like safe harbors, even when you're stuck in them going 'round and 'round... and 'round.

You think what you signal, you signal what you think, you signal your identity, you identify with your signals, you think about your identity, you identify with your thoughts. This... is scary. The mind is leaky, and these interactions are going on constantly. Priming, anchoring, commitment effects, consistency effects, and of course the dreaded confirmation bias and positive bias are all potential dangers. In one way, it is no wonder that people don't seem to change much. With such constant confirmation of identity, it's hard to see how one could change at all. But the untrained mind is chaotic and of potentially infinite malleability, and cascades can be very powerful. Drift happens, implicit navigation is undertaken. One twin joins a cult, the other joins Less Wrong, which is pretty much the same thing I guess but bear with me.

bla bla bla feedback loops recursion cascades bla egregious links to yudkowsky bla.

I will boast that I believe I have found a decent set of attractors in identityspace to aim for. Thus, though I spend a lot of time wireheading and not actually navigating towards my nominal ideal dispositions nor goals, I'm at least kinda aiming in what seems to be generally the right direction, as far as my limited rationality can see. I'm lucky in that regard.

But my map is not the best one to navigate by, and the vast majority of it is blank, including the most important parts; the parts where my goals and my ideal identity lie. I have but a vague sense of direction. Furthermore, nearly all of my map is the result of slovenly lines copied secondhand from the thirdhand notes of others, and a whole bunch of those scribbles seem to be legible only as "Here there be dragons." I can only imagine how much harder it'd be if I was less aware of the limitations of my map, or if I had not chosen a set of destinations to navigate towards, or if I'd accidentally got sucked into a whirlpool only to be eaten by Charbyddis. Or whatever the cognitive bias equivalent of Charbyddis is; probably faith.

 

XI: Harnessing the Winds of Change

Navigating identityspace is tricky business. Most people don't try to. You ask them what kind of person they wish to be or what goals they wish to accomplish, and they either admit to not really thinking about it or quickly query their far mode module for something that sounds really sweet and inspiring. Those who do try tend to do so implicitly, by either carefully monitoring themselves and the way they change, or carefully monitoring their goals and in what order they are achieved. Often these are the kind of people that purposefully go out on Friday night with the intent of coming back home with a story they can tell for years to come. It's difficult to track your life and your progress without having stories and milestones to go by. They do not regularly try to directly control their course through identityspace. It's not obvious how one would even attempt to do so. As aforementioned, the tools we rationalists do have are more naturally suited to navigating thoughtspace

Attractors pull. I've generally dealt with that fact in a negative light, because I tend to think mindspace has at least 7,497 dimensions, and the coordinates of the set of optimal thoughts and therefore optimal actions are in a tiny corner of that vast space. Your thoughts are being deflected and ricocheted and pulled and pushed by a swarm of memes and biases and cached selves and anchors and all sorts of things that we just can't keep track of, on timescales from seconds to decades. Some forces pulsate, others are erratic. You think you're sailing along fine when some stupid thing like the giant cheesecake fallacy blows you oh so slightly off course and causes your entire AI career to go along a totally hopeless trajectory without you're realizing it. Who wants their epic journey tripped up by something as stupidly named as the giant cheesecake fallacy? It'd be less pitiable to be eaten by Charbyddis, at least that's pretty epic.

Strong metarationality would have kept that from happening: metarationality keeps you from failing predictably in special domains. But strong metarationality can be aided. Hopefully, the things we happen to be aiming for in identityspace are stable attractors that don't randomly push you away or shift around. This is not always the case: some goals are ephemeral, some are cyclical. My friend wants a girlfriend one month out of every two. That sure strains the relationship after a month or three. With such a naturally chaotic mindspace, it's difficult to be sure that what you're aiming for is something that will be there when you get to where you thought it was. You want to be the cool girl at the party, thinking this is a terminal value, but then you succeed and become the cool girl at the party and it's just not all that fulfilling. It'd have been better to navigate towards a different destination.

Not that you can't set sail for multiple places at once: sometimes you just want to get to the New World, not a particular reef of the Bahamas. And sometimes you may wish to travel to two entirely different places. Do you contradict yourself? Very well, then you contradict yourself, you are large, you contain multitudes. But I think you'll find it difficult to have two very different destinations in identityspace to aim for all at once.

Alright then, enough, you've heard all the warnings, seen the scribbles that say 'non-negligible potential for dragons', and now want to do some positive thinking. How can we harness these variable winds of change on our journey through identityspace?

 

---

 

Part Two: Brainstorming Methods for Identity Optimization

[insert methods for carefully capitalizing on attractors, drawing on and fleshing out Cached Selves, among other things,bla.]