Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Further discussion of CFAR’s focus on AI safety, and the good things folks wanted from “cause neutrality”

36 AnnaSalamon 12 December 2016 07:39PM

Follow-up to:

In the days since we published our previous post, a number of people have come up to me and expressed concerns about our new mission.  Several of these had the form “I, too, think that AI safety is incredibly important — and that is why I think CFAR should remain cause-neutral, so it can bring in more varied participants who might be made wary by an explicit focus on AI.”

I would here like to reply to these people and others, and to clarify what is and isn’t entailed by our new focus on AI safety.

continue reading »

CFAR’s new focus, and AI Safety

30 AnnaSalamon 03 December 2016 06:09PM

A bit about our last few months:

  • We’ve been working on getting a simple clear mission and an organization that actually works.  We think of our goal as analogous to the transition that the old Singularity Institute underwent under Lukeprog (during which chaos was replaced by a simple, intelligible structure that made it easier to turn effort into forward motion).
  • As part of that, we’ll need to find a way to be intelligible.
  • This is the first of several blog posts aimed at causing our new form to be visible from outside.  (If you're in the Bay Area, you can also come meet us at tonight's open house.) (We'll be talking more about the causes of this mission-change; the extent to which it is in fact a change, etc. in an upcoming post.)

Here's a short explanation of our new mission:
  • We care a lot about AI Safety efforts in particular, and about otherwise increasing the odds that humanity reaches the stars.

  • Also, we[1] believe such efforts are bottlenecked more by our collective epistemology, than by the number of people who verbally endorse or act on "AI Safety", or any other "spreadable viewpointdisconnected from its derivation.

  • Our aim is therefore to find ways of improving both individual thinking skill, and the modes of thinking and social fabric that allow people to think together.  And to do this among the relatively small sets of people tackling existential risk. 

continue reading »

Double Crux — A Strategy for Resolving Disagreement

61 Duncan_Sabien 29 November 2016 09:23PM


Double crux is one of CFAR's newer concepts, and one that's forced a re-examination and refactoring of a lot of our curriculum (in the same way that the introduction of TAPs and Inner Simulator did previously).  It rapidly became a part of our organizational social fabric, and is one of our highest-EV threads for outreach and dissemination, so it's long overdue for a public, formal explanation.

Note that while the core concept is fairly settled, the execution remains somewhat in flux, with notable experimentation coming from Julia Galef, Kenzi Amodei, Andrew Critch, Eli Tyre, Anna Salamon, myself, and others.  Because of that, this post will be less of a cake and more of a folk recipe—this is long and meandering on purpose, because the priority is to transmit the generators of the thing over the thing itself.  Accordingly, if you think you see stuff that's wrong or missing, you're probably onto something, and we'd appreciate having them added here as commentary.

Casus belli

To a first approximation, a human can be thought of as a black box that takes in data from its environment, and outputs beliefs and behaviors (that black box isn't really "opaque" given that we do have access to a lot of what's going on inside of it, but our understanding of our own cognition seems uncontroversially incomplete).

When two humans disagree—when their black boxes output different answers, as below—there are often a handful of unproductive things that can occur.

The most obvious (and tiresome) is that they'll simply repeatedly bash those outputs together without making any progress (think most disagreements over sports or politics; the people above just shouting "triangle!" and "circle!" louder and louder).  On the second level, people can (and often do) take the difference in output as evidence that the other person's black box is broken (i.e. they're bad, dumb, crazy) or that the other person doesn't see the universe clearly (i.e. they're biased, oblivious, unobservant).  On the third level, people will often agree to disagree, a move which preserves the social fabric at the cost of truth-seeking and actual progress.

Double crux in the ideal solves all of these problems, and in practice even fumbling and inexpert steps toward that ideal seem to produce a lot of marginal value, both in increasing understanding and in decreasing conflict-due-to-disagreement.


This post will occasionally delineate two versions of double crux: a strong version, in which both parties have a shared understanding of double crux and have explicitly agreed to work within that framework, and a weak version, in which only one party has access to the concept, and is attempting to improve the conversational dynamic unilaterally.

In either case, the following things seem to be required:

  • Epistemic humility. The number one foundational backbone of rationality seems, to me, to be how readily one is able to think "It's possible that I might be the one who's wrong, here."  Viewed another way, this is the ability to take one's beliefs as object, rather than being subject to them and unable to set them aside (and then try on some other belief and productively imagine "what would the world be like if this were true, instead of that?").
  • Good faith. An assumption that people believe things for causal reasons; a recognition that having been exposed to the same set of stimuli would have caused one to hold approximately the same beliefs; a default stance of holding-with-skepticism what seems to be evidence that the other party is bad or wants the world to be bad (because as monkeys it's not hard for us to convince ourselves that we have such evidence when we really don't).1
  • Confidence in the existence of objective truth. I was tempted to call this "objectivity," "empiricism," or "the Mulder principle," but in the end none of those quite fit.  In essence: a conviction that for almost any well-defined question, there really truly is a clear-cut answer.  That answer may be impractically or even impossibly difficult to find, such that we can't actually go looking for it and have to fall back on heuristics (e.g. how many grasshoppers are alive on Earth at this exact moment, is the color orange superior to the color green, why isn't there an audio book of Fight Club narrated by Edward Norton), but it nevertheless exists.
  • Curiosity and/or a desire to uncover truth.  Originally, I had this listed as truth-seeking alone, but my colleagues pointed out that one can move in the right direction simply by being curious about the other person and the contents of their map, without focusing directly on the territory.

At CFAR workshops, we hit on the first and second through specific lectures, the third through osmosis, and the fourth through osmosis and a lot of relational dynamics work that gets people curious and comfortable with one another.  Other qualities (such as the ability to regulate and transcend one's emotions in the heat of the moment, or the ability to commit to a thought experiment and really wrestle with it) are also helpful, but not as critical as the above.  

How to play

Let's say you have a belief, which we can label A (for instance, "middle school students should wear uniforms"), and that you're in disagreement with someone who believes some form of ¬A.  Double cruxing with that person means that you're both in search of a second statement B, with the following properties:

  • You and your partner both disagree about B as well (you think B, your partner thinks ¬B).
  • The belief B is crucial for your belief in A; it is one of the cruxes of the argument.  If it turned out that B was not true, that would be sufficient to make you think A was false, too.
  • The belief ¬B is crucial for your partner's belief in ¬A, in a similar fashion.

In the example about school uniforms, B might be a statement like "uniforms help smooth out unhelpful class distinctions by making it harder for rich and poor students to judge one another through clothing," which your partner might sum up as "optimistic bullshit."  Ideally, B is a statement that is somewhat closer to reality than A—it's more concrete, grounded, well-defined, discoverable, etc.  It's less about principles and summed-up, induced conclusions, and more of a glimpse into the structure that led to those conclusions.

(It doesn't have to be concrete and discoverable, though—often after finding B it's productive to start over in search of a C, and then a D, and then an E, and so forth, until you end up with something you can research or run an experiment on).

At first glance, it might not be clear why simply finding B counts as victory—shouldn't you settle B, so that you can conclusively choose between A and ¬A?  But it's important to recognize that arriving at B means you've already dissolved a significant chunk of your disagreement, in that you and your partner now share a belief about the causal nature of the universe.

If B, then A.  Furthermore, if ¬B, then ¬A.  You've both agreed that the states of B are crucial for the states of A, and in this way your continuing "agreement to disagree" isn't just "well, you take your truth and I'll take mine," but rather "okay, well, let's see what the evidence shows."  Progress!  And (more importantly) collaboration!


This is where CFAR's versions of the double crux unit are currently weakest—there's some form of magic in the search for cruxes that we haven't quite locked down.  In general, the method is "search through your cruxes for ones that your partner is likely to disagree with, and then compare lists."  For some people and some topics, clearly identifying your own cruxes is easy; for others, it very quickly starts to feel like one's position is fundamental/objective/un-break-downable.


  • Increase noticing of subtle tastes, judgments, and "karma scores."  Often, people suppress a lot of their opinions and judgments due to social mores and so forth.  Generally loosening up one's inner censors can make it easier to notice why we think X, Y, or Z.
  • Look forward rather than backward.  In places where the question "why?" fails to produce meaningful answers, it's often more productive to try making predictions about the future.  For example, I might not know why I think school uniforms are a good idea, but if I turn on my narrative engine and start describing the better world I think will result, I can often sort of feel my way toward the underlying causal models.
  • Narrow the scope.  A specific test case of "Steve should've said hello to us when he got off the elevator yesterday" is easier to wrestle with than "Steve should be more sociable."  Similarly, it's often easier to answer questions like "How much of our next $10,000 should we spend on research, as opposed to advertising?" than to answer "Which is more important right now, research or advertising?"
  • Do "Focusing" and other resonance checks.  It's often useful to try on a perspective, hypothetically, and then pay attention to your intuition and bodily responses to refine your actual stance.  For instance: (wildly asserts) "I bet if everyone wore uniforms there would be a fifty percent reduction in bullying." (pauses, listens to inner doubts)  "Actually, scratch that—that doesn't seem true, now that I say it out loud, but there is something in the vein of reducing overt bullying, maybe?"
  • Seek cruxes independently before anchoring on your partner's thoughts.  This one is fairly straightforward.  It's also worth noting that if you're attempting to find disagreements in the first place (e.g. in order to practice double cruxing with friends) this is an excellent way to start—give everyone the same ten or fifteen open-ended questions, and have everyone write down their own answers based on their own thinking, crystallizing opinions before opening the discussion.

Overall, it helps to keep the ideal of a perfect double crux in the front of your mind, while holding the realities of your actual conversation somewhat separate.  We've found that, at any given moment, increasing the "double cruxiness" of a conversation tends to be useful, but worrying about how far from the ideal you are in absolute terms doesn't.  It's all about doing what's useful and productive in the moment, and that often means making sane compromises—if one of you has clear cruxes and the other is floundering, it's fine to focus on one side.  If neither of you can find a single crux, but instead each of you has something like eight co-cruxes of which any five are sufficient, just say so and then move forward in whatever way seems best.

(Variant: a "trio" double crux conversation in which, at any given moment, if you're the least-active participant, your job is to squint at your two partners and try to model what each of them is saying, and where/why/how they're talking past one another and failing to see each other's points.  Once you have a rough "translation" to offer, do so—at that point, you'll likely become more central to the conversation and someone else will rotate out into the squinter/translator role.)

Ultimately, each move should be in service of reversing the usual antagonistic, warlike, "win at all costs" dynamic of most disagreements.  Usually, we spend a significant chunk of our mental resources guessing at the shape of our opponent's belief structure, forming hypotheses about what things are crucial and lobbing arguments at them in the hopes of knocking the whole edifice over.  Meanwhile, we're incentivized to obfuscate our own belief structure, so that our opponent's attacks will be ineffective.

(This is also terrible because it means that we often fail to even find the crux of the argument, and waste time in the weeds.  If you've ever had the experience of awkwardly fidgeting while someone spends ten minutes assembling a conclusive proof of some tangential sub-point that never even had the potential of changing your mind, then you know the value of someone being willing to say "Nope, this isn't going to be relevant for me; try speaking to that instead.")

If we can move the debate to a place where, instead of fighting over the truth, we're collaborating on a search for understanding, then we can recoup a lot of wasted resources.  You have a tremendous comparative advantage at knowing the shape of your own belief structure—if we can switch to a mode where we're each looking inward and candidly sharing insights, we'll move forward much more efficiently than if we're each engaged in guesswork about the other person.  This requires that we want to know the actual truth (such that we're incentivized to seek out flaws and falsify wrong beliefs in ourselves just as much as in others) and that we feel emotionally and socially safe with our partner, but there's a doubly-causal dynamic where a tiny bit of double crux spirit up front can produce safety and truth-seeking, which allows for more double crux, which produces more safety and truth-seeking, etc.


First and foremost, it matters whether you're in the strong version of double crux (cooperative, consent-based) or the weak version (you, as an agent, trying to improve the conversational dynamic, possibly in the face of direct opposition).  In particular, if someone is currently riled up and conceives of you as rude/hostile/the enemy, then saying something like "I just think we'd make better progress if we talked about the underlying reasons for our beliefs" doesn't sound like a plea for cooperation—it sounds like a trap.

So, if you're in the weak version, the primary strategy is to embody the question "What do you see that I don't?"  In other words, approach from a place of explicit humility and good faith, drawing out their belief structure for its own sake, to see and appreciate it rather than to undermine or attack it.  In my experience, people can "smell it" if you're just playing at good faith to get them to expose themselves; if you're having trouble really getting into the spirit, I recommend meditating on times in your past when you were embarrassingly wrong, and how you felt prior to realizing it compared to after realizing it.

(If you're unable or unwilling to swallow your pride or set aside your sense of justice or fairness hard enough to really do this, that's actually fine; not every disagreement benefits from the double-crux-nature.  But if your actual goal is improving the conversational dynamic, then this is a cost you want to be prepared to pay—going the extra mile, because a) going what feels like an appropriate distance is more often an undershoot, and b) going an actually appropriate distance may not be enough to overturn their entrenched model in which you are The Enemy.  Patience- and sanity-inducing rituals recommended.)

As a further tip that's good for either version but particularly important for the weak one, model the behavior you'd like your partner to exhibit.  Expose your own belief structure, show how your own beliefs might be falsified, highlight points where you're uncertain and visibly integrate their perspective and information, etc.  In particular, if you don't want people running amok with wrong models of what's going on in your head, make sure you're not acting like you're the authority on what's going on in their head.

Speaking of non-sequiturs, beware of getting lost in the fog.  The very first step in double crux should always be to operationalize and clarify terms.  Try attaching numbers to things rather than using misinterpretable qualifiers; try to talk about what would be observable in the world rather than how things feel or what's good or bad.  In the school uniforms example, saying "uniforms make students feel better about themselves" is a start, but it's not enough, and going further into quantifiability (if you think you could actually get numbers someday) would be even better.  Often, disagreements will "dissolve" as soon as you remove ambiguity—this is success, not failure!

Finally, use paper and pencil, or whiteboards, or get people to treat specific predictions and conclusions as immutable objects (if you or they want to change or update the wording, that's encouraged, but make sure that at any given moment, you're working with a clear, unambiguous statement).  Part of the value of double crux is that it's the opposite of the weaselly, score-points, hide-in-ambiguity-and-look-clever dynamic of, say, a public political debate.  The goal is to have everyone understand, at all times and as much as possible, what the other person is actually trying to say—not to try to get a straw version of their argument to stick to them and make them look silly.  Recognize that you yourself may be tempted or incentivized to fall back to that familiar, fun dynamic, and take steps to keep yourself in "scout mindset" rather than "soldier mindset."


This is the double crux algorithm as it currently exists in our handbook.  It's not strictly connected to all of the discussion above; it was designed to be read in context with an hour-long lecture and several practice activities (so it has some holes and weirdnesses) and is presented here more for completeness and as food for thought than as an actual conclusion to the above.

1. Find a disagreement with another person

  • A case where you believe one thing and they believe the other

  • A case where you and the other person have different confidences (e.g. you think X is 60% likely to be true, and they think it’s 90%)

2. Operationalize the disagreement

  • Define terms to avoid getting lost in semantic confusions that miss the real point

  • Find specific test cases—instead of (e.g.) discussing whether you should be more outgoing, instead evaluate whether you should have said hello to Steve in the office yesterday morning

  • Wherever possible, try to think in terms of actions rather than beliefs—it’s easier to evaluate arguments like “we should do X before Y” than it is to converge on “X is better than Y.”

3. Seek double cruxes

  • Seek your own cruxes independently, and compare with those of the other person to find overlap

  • Seek cruxes collaboratively, by making claims (“I believe that X will happen because Y”) and focusing on falsifiability (“It would take A, B, or C to make me stop believing X”)

4. Resonate

  • Spend time “inhabiting” both sides of the double crux, to confirm that you’ve found the core of the disagreement (as opposed to something that will ultimately fail to produce an update)

  • Imagine the resolution as an if-then statement, and use your inner sim and other checks to see if there are any unspoken hesitations about the truth of that statement

5. Repeat!


We think double crux is super sweet.  To the extent that you see flaws in it, we want to find them and repair them, and we're currently betting that repairing and refining double crux is going to pay off better than try something totally different.  In particular, we believe that embracing the spirit of this mental move has huge potential for unlocking people's abilities to wrestle with all sorts of complex and heavy hard-to-parse topics (like existential risk, for instance), because it provides a format for holding a bunch of partly-wrong models at the same time while you distill the value out of each.

Comments appreciated; critiques highly appreciated; anecdotal data from experimental attempts to teach yourself double crux, or teach it to others, or use it on the down-low without telling other people what you're doing extremely appreciated.

 - Duncan Sabien

1One reason good faith is important is that even when people are "wrong," they are usually partially right—there are flecks of gold mixed in with their false belief that can be productively mined by an agent who's interested in getting the whole picture.  Normal disagreement-navigation methods have some tendency to throw out that gold, either by allowing everyone to protect their original belief set or by replacing everyone's view with whichever view is shown to be "best," thereby throwing out data, causing information cascades, disincentivizing "noticing your confusion," etc.

The central assumption is that the universe is like a large and complex maze that each of us can only see parts of.  To the extent that language and communication allow us to gather info about parts of the maze without having to investigate them ourselves, that's great.  But when we disagree on what to do because we each see a different slice of reality, it's nice to adopt methods that allow us to integrate and synthesize, rather than methods that force us to pick and pare down.  It's like the parable of the three blind men and the elephant—whenever possible, avoid generating a bottom-line conclusion until you've accounted for all of the available data.


The agent at the top mistakenly believes that the correct move is to head to the left, since that seems to be the most direct path toward the goal.  The agent on the right can see that this is a mistake, but it would never have been able to navigate to that particular node of the maze on its own. 

Several free CFAR summer programs on rationality and AI safety

18 AnnaSalamon 14 April 2016 02:35AM
CFAR will be running several free summer programs this summer which are currently taking applications.  Please apply if you’re interested, and forward the programs also to anyone else who may be a good fit!
continue reading »

Unofficial Canon on Applied Rationality

30 ScottL 15 February 2016 01:03PM

I have been thinking for a while that it would be useful if there was something similar to the Less Wrong Canon on Rationality for the CFAR material. Maybe, it could be called the 'CFAR Canon on Applied Rationality'. To start on this I have compiled a collection of descriptions for the CFAR techniques that I could find. I have separated the techniques into a few different sections. The sections and descriptions have mostly been written by me, with a lot of borrowing from other material, which means that they may not accurately reflect what CFAR actually teaches.

Please note that I have not attended any CFAR workshops, nor am I affiliated with CFAR in any way. My understanding of these techniques comes from CFAR videos, blogs and other websites which I have provided links to. If I have missed any important techniques or if my understanding of any of the techniques is incorrect or if you can provide links to the research that these techniques are based on, please let me know and I will update this post. 


Learning this material based solely on the descriptions written here may be unhelpful, arduous or even harmful. (See Duncan_Sabien's full comment for more information on this) It is because the material is very hard to learn correctly. Most of the techniques below involve in one way or another volitionally overriding your instinctual, intuitive or ingrained behaviours and thoughts. These are thoughts which not only often feel enticing and alluring, but that also often feel unmistakably right. If you are anything like me, then you should be very careful if you are trying to learn this material alone. For you will be prone to rationalization,  taking shortcuts and making mistakes.

My recommendations for trying to learn this material are:

  • learn it deeply and be sure to put what you have learnt into practice. It will often help if you take notes on what works for you and what doesn't. Also take note of the 'Mindsets and perspectives that help you in discovering potential situations that you could end up valuing' section as these are very important.
  • get the help of experts or other people who have already expended great amounts of effort in trying to implement this material like the people at cfar. This will save you a great amount of stress and effort as it will allow you to avoid a plethora of potential mistakes and inefficiencies. If you really want to learn this material, then you should deeply consider attending a CFAR workshop. 
  • get the help of or involve friends. As Duncan_Sabien has said:
    It is better on almost every axis with instructors, mentors, friends, companions—people to help you avoid the biggest pitfalls, help you understand the subtle points, tease apart the interesting implications, shore up your motivation, assist you in seeing your own mistakes and weaknesses. None of that is impossible on your own, but it's somewhere between one and two orders of magnitude more efficient and more efficacious with guidance".
  • be dubious of your mental models. Beware thoughts and ideas that feel unequivocally right especially if they are solely located internally rather than also being expressed or formulated externally. 
  • You might want to bookmark this page instead of reading it all at once as it is quite long.


continue reading »

Why CFAR? The view from 2015

46 PeteMichaud 23 December 2015 10:46PM

Follow-up to: 2013 and 2014.

In this post, we:

We are in the middle of our matching fundraiser; so if you’ve been considering donating to CFAR this year, now is an unusually good time.

continue reading »

Results of a One-Year Longitudinal Study of CFAR Alumni

33 Unnamed 12 December 2015 04:39AM

By Dan from CFAR


When someone comes to a CFAR workshop, and then goes back home, what is different for them one year later? What changes are there to their life, to how they think, to how they act?

CFAR would like to have an answer to this question (as would many other people). One method that we have been using to gather relevant data is a longitudinal study, comparing participants' survey responses from shortly before their workshop with their survey responses approximately one year later. This post summarizes what we have learned thus far, based on data from 135 people who attended workshops from February 2014 to April 2015 and completed both surveys.

The survey questions can be loosely categorized into four broad areas:

  1. Well-being: On the whole, is the participant's life going better than it was before the workshop?
  2. Personality: Have there been changes on personality dimensions which seem likely to be associated with increased rationality?
  3. Behaviors: Have there been increases in rationality-related skills, habits, or other behavioral tendencies?
  4. Productivity: Is the participant working more effectively at their job or other projects?

We chose to measure these four areas because they represent part of what CFAR hopes that its workshops accomplish, they are areas where many workshop participants would like to see changes, and they are relatively tractable to measure on a survey. There are other areas where CFAR would like to have an effect, including people's epistemics and their impact on the world, which were not a focus of this study.

We relied heavily on existing measures which have been validated and used by psychology researchers, especially in the areas of well-being and personality. These measures typically are not a perfect match for what we care about, but we expected them to be sufficiently correlated with what we care about for them to be worth using.

We found significant increases in variables in all 4 areas. A partial summary:

Well-being: increases in happiness and life satisfaction, especially in the work domain (but no significant change in life satisfaction in the social domain)

Personality: increases in general self-efficacy, emotional stability, conscientiousness, and extraversion (but no significant change in growth mindset or openness to experience)

Behaviors: increased rate of acquisition of useful techniques, emotions experienced as more helpful & less of a hindrance (but no significant change on measures of cognitive biases or useful conversations)

Productivity: increases in motivation while working and effective approaches to pursuing projects (but no significant change in income or number of hours worked)

The rest of this post is organized into three main sections. The first section describes our methodology in more detail, including the reasoning behind the longitudinal design and some information on the sample. The second section gives the results of the research, including the variables that showed an effect and the ones that did not; the results are summarized in a table at the end of that section. The third section discusses four major methodological concerns—the use of self-report measures (where respondents might just give the answer that sounds good), attrition (some people who took the pre-survey did not complete the post-survey), other sources of personal growth (people might have improved over time without attending the CFAR workshop), and regression to the mean (people may have changed after the workshop simply because they came to the workshop at an unusually high or low point)—and attempts to evaluate the extent to which these four issues may have influenced the results.

continue reading »

CFAR in 2014: Continuing to climb out of the startup pit, heading toward a full prototype

61 AnnaSalamon 26 December 2014 03:33PM

Summary:  We outline CFAR’s purpose, our history in 2014, and our plans heading into 2015.

One of the reasons we’re publishing this review now is that we’ve just launched our annual matching fundraiser, and want to provide the information our prospective donors need for deciding. This is the best time of year to decide to donate to CFAR. Donations up to $120k will be matched until January 31.[1] 

To briefly preview: For the first three years of our existence, CFAR mostly focused on getting going. We followed the standard recommendation to build a ‘minimum viable product’, the CFAR workshops, that could test our ideas and generate some revenue. Coming into 2013, we had a workshop that people liked (9.3 average rating on “Are you glad you came?”; a more recent random survey showed 9.6 average rating on the same question 6-24 months later), which helped keep the lights on and gave us articulate, skeptical, serious learners to iterate on. At the same time, the workshops are not everything we would want in a CFAR prototype; it feels like the current core workshop does not stress-test most of our hopes for what CFAR can eventually do. The premise of CFAR is that we should be able to apply the modern understanding of cognition to improve people’s ability to (1) figure out the truth (2) be strategically effective (3) do good in the world. We have dreams of scaling up some particular kinds of sanity.  Our next goal is to build the minimum strategic product that more directly justifies CFAR’s claim to be an effective altruist project.[2]

continue reading »

Practical Benefits of Rationality (LW Census Results)

16 Unnamed 31 January 2014 05:24PM

by Dan from CFAR


Abstract: Two measures of the practical benefits of rationality, one a self-report of the benefits of being part of the rationality community and the other a measure of how often a person adds useful techniques to their repertoire, were included on the 2013 Less Wrong survey. In-person involvement with LW/CFAR predicted both measures of benefits, with friendships with LWers and attending a CFAR workshop showing the strongest and most consistent effects. Online Less Wrong participation and background had weaker and less consistent effects. Growth mindset also independently predicted both measures of practical benefits, and on the measure of technique acquisition there was an interaction effect suggesting that in-person LW/CFAR involvement may be especially beneficial for people high in growth mindset. However, some caution is warranted in interpreting these correlational, self-report results.



Though I first found Less Wrong through my habit of reading interesting blogs, the main reason why I've gotten more and more involved in the rationality community is my suspicion that this rationality stuff might be pretty useful. Useful not only for thinking clearly about tricky intellectual topics, but also in ways that have more directly practical benefits.

CFAR obviously has similar interests, as it aims to create a community of people who are effective at acting in the world.

The 2013 LW census/survey provided an opportunity for us to probe how the rationality community is doing so far at finding these practical benefits, as it allowed us to survey a large cross section of the Less Wrong community. Unfortunately, there is not a standard, simple measure of practical benefits which we could just stick on the survey, and we were only able to use a correlational research design, but we sought to get some relevant information by coming up with two self-report questions to include on the survey.

One question was somewhat broader than the set of practical benefits that we were interested in and the other was somewhat narrower. First, there was a broad self-report question asking people how much they had benefited from being involved in the rationality community. Second, we asked people more narrowly how often they successfully added a useful technique or approach to their repertoire. We were primarily interested in seeing whether involvement in the LW community would predict practical benefits on these two measures, and (if so) which forms of involvement would have the strongest relationship to these benefits.

About 1400 people answered the relevant survey questions, including about 400 who have read the sequences, about 150 who regularly attend LW meetups, about 100 who have attended a full CFAR workshop, about 100 who interact with other LWers in person all the time, and about 50 who met a romantic partner through LW.  The survey also included a brief scale measuring growth mindset, and a question about age.

Some methodological notes: In the body of this post I’ve tried to put the results in a format that’s relatively straightforward to interpret. More technical details and additional analyses are included in footnotes, and I can add more details in the comments. Note that the study design is entirely correlational, and the questions are all self-report (unlike last year’s questions, which included tests of standard biases). This gives some reason for caution in interpreting the results, and I’ll note some places where that is especially relevant.


Background & Survey Design

The simple, obvious thing to do, in order to investigate how much people have benefited from their involvement in the rationality community, is to ask them that question. So we did: "How much have you benefited from your exposure to and participation in the rationality community (Less Wrong, CFAR, in-person contact with LW/HPMOR readers, etc.)?" There were 7 response options, which we can scale as -3 to +3, where +3 is “My life is MUCH BETTER than it would have been without exposure to the rationality community” (and -3 is “... MUCH WORSE…”).

This straightforward question has a couple of straightforward limitations. For one, we might expect people who are involved in almost any activity to say that they benefit from it; self-reported benefit does not necessarily indicate actual benefit. Second, it could include a broad range of benefits, some of which might not have much to do with the usefulness of rationality (such as meeting your current romantic partner at a Less Wrong meetup). So we also included a narrower question related to competence which is less susceptible to these issues.

A simple model of how people are able to become highly competent/productive/successful/impressive individuals is that they try lots of things and keep doing the ones that work. A person’s work habits, the questions they ask during conversations, the methods that they use to make certain kinds of decisions, and many other things can all be developed through a similar iterative process. Over time, someone who has a good process in place of trying things & sticking with the helpful ones will end up collecting a large set of habits/techniques/approaches/principles/etc. which work for them.

The second set of questions which we included on the survey were based on this process, with the aim of measuring about how often people add a new useful technique to their repertoire. There were 3 survey questions based on a streamlined version of this process: first you hear about many different techniques, then you try some fraction of the techniques that you hear about, and then some fraction of the techniques that you try end up working for you and sticking as part of your repertoire. We first asked “On average, about how often do you *read or hear about* another plausible-seeming technique or approach for being more rational / more productive / happier / having better social relationships / having more accurate beliefs / etc.?”, then “...how often do you *try out* another plausible-seeming technique..”, and finally “...how often do you find another technique or approach that *successfully helps you at*...”  This final question, about how frequently people acquire a new helpful technique, is our other main outcome measure of practical benefits.

In reality, people often generate their own ideas of techniques to try, and try many variations rather than just a single thing (e.g., many people end up with their own personalized version of the pomodoro technique). Focusing on the streamlined process of hear → try → acquire is a simplification which had two survey-specific benefits.  First, having the context of “hearing about a technique and then trying it” was intended to make it clearer what to count as “a technique,” which is important since the outcome measure is a count of the number of techniques acquired. Second, including the “hearing” and “trying” questions allows us to probe this process in a bit more detail by (for example) breaking down the number of new techniques that a person acquired into two components: the number of new techniques that they tried and the hit rate (techniques acquired divided by techniques tried).

One other predictor variable which we included on the survey was a 4-item measure of growth mindset, which was taken from Carol Dweck’s research (sample item: “No matter what kind of person you are, you can always change substantially”).[1] A fixed mindset involves thinking that personal characteristics are fixed and unchangeable - you either have them or you don’t - while a growth mindset involves thinking that personal characteristics can change as a person grows and develops. Dweck and her colleagues have found that growth mindset about a characteristic tends to be associated with more productive behaviors and more improvement over time. For example, children with a growth mindset about being good at thinking tend to seek out intellectual challenges which stretch their abilities, while children with a fixed mindset tend to avoid tasks that they might fail at and seek tasks which they know they can do well.

A blog based on the idea of becoming less wrong sounds like it would reflect growth mindset more than fixed mindset, and many aspects of the local idea cluster seem to match that. Ideas like: there are systematic methods that you can learn which will allow you to form more accurate models of the world. Complex skills can be broken down into simple trainable components. Don't get too attached to a particular image of who you are and what you stand for.  Mastering the right cognitive toolkit can make you more effective at accomplishing the things that you care about. Tsuyoku Naritai! In addition to these connections to LW thinking, growth mindset also seems like it could facilitate the process of becoming more successful by trying out various changes to the way that you do things and sticking with the ones that work. Thus, we wanted to investigate whether people who were more involved in the rationality community (in various ways) had more of a growth mindset, and whether people with more of a growth mindset reported more practical benefits.

The other main predictor variables were several different indicators of people's involvement in the rationality community:

LW background
A composite scale, which standardized and then averaged together four questions which all indicate a person’s amount of background with the lesswrong.com website (and which, as I found on previous years' surveys, all correlate with each other and show similar patterns of relationships with other variables). The four questions measured: having read the sequences (ranging from 1 “Never even knew they existed until this moment” to 7 “[Read] All or nearly all of the Sequences”), karma (log-transformed), LW Use (ranging from 1 “I lurk, but never registered an account” to 5 “I've posted in Main”), and length of time in the community (capped at 8 years).

Time per day on Less Wrong
“How long, in approximate number of minutes, do you spend on Less Wrong in the average day?” (log-transformed).

Meetup attendance
“Do you attend Less Wrong meetup?”
“Yes, regularly,” “Yes, once or a few times,” or “No” (categorical variable).

CFAR workshop attendance
Have you ever attended a CFAR workshop?
“Yes, I have been to a full (3+ day) workshop,” “I have been to at least one CFAR class, but not a full (3+ day) workshop,” or “No” (categorical variable).

LW friendships
“Is physical interaction with the Less Wrong community otherwise a part of your everyday life, for example you live with other Less Wrongers, or you are close friends and frequently go out with them?”
“Yes, all the time,” “Yes, sometimes,” or “No” (categorical variable).

LW romantic partner
Have you ever been in a romantic relationship with someone you met through the Less Wrong community?
“Yes,” “I didn't meet them through the community, but they're part of the community now,” or “No” (categorical variable).

I considered combining these four measures of in-person involvement with the LW community (LW meetups, CFAR workshops, LW friendships, and LW romantic partners) into a single scale of in-person LW involvement, but there ended up being a large enough sample size within these groups and strong enough effects for me to analyze them separately.

Respondents also reported their age (which was transformed by taking the square root).



I. Self-reported Benefit
"How much have you benefited from your exposure to and participation in the rationality community (Less Wrong, CFAR, in-person contact with LW/HPMOR readers, etc.)?"

The average response to this question was a 1.4 on a -3 to +3 scale (SD = 1.08), and 15% of people selected the scale maximum “My life is MUCH BETTER than it would have been without exposure to the rationality community.”

Which variables were associated with a larger self-reported benefit from the rationality community?

In short, all of them.

Each of the following variables was significantly related to this self-reported measure of benefit, and in a regression which controlled for the other variables all of them remained significant except for meetup attendance (which became p = 0.07).[2] For ease of interpretation, I have reported the percent of people in each of the following groups who selected the scale maximum.  I have sorted the variables in order of effect size, from largest to smallest, based on the results of the regression (see the footnote for more details).

Percent of people in each subgroup answering “My life is MUCH BETTER than it would have been without exposure to the rationality community”

61% LW romantic partner (n = 54)
44% attended a full CFAR workshop (n = 100)
19% age 25 or less (younger people reported more benefit) (n = 724)
50% LW friendships (n = 88)
28% above 3.0 on growth mindset scale (n = 277)
25% high LW background (n = 137)
35% regularly attend meetups (n = 156)
31% acquire a new technique every 3 weeks or more often (n = 213)
18% use LW for 30+ min per day (n = 218)
15% all respondents (n = 1451)

Three noteworthy results:

  • Each of the variables related to involvement in the rationality community was associated with reports of getting more benefit from the community.
  • The strongest effects came from people who were involved in fairly intensive, in-person activities: finding a romantic partner through LW, attending a full CFAR workshop, and being around other LWers in person all the time.
  • Three variables which were not directly related to community involvement – younger age, growth mindset, and acquiring new techniques – were all predictive of self-reported benefit from the rationality community.

One interpretation of these results is that getting involved in the rationality community causes people to acquire useful rationality skills which improve their lives, with larger effects for people who get involved in more depth through close relationships, shared housing, CFAR workshops, etc. However, as noted above, these effects could also be due to non-rationality-related benefits (e.g., finding friends or a romantic partner), a tendency to say nice things about activities & communities that you're a part of, or causal effects in the other direction (e.g., people who benefited the most from the Less Wrong website might be especially likely to attend a CFAR workshop or move into shared housing with other LWers).

It is worth noting that growth mindset and acquiring new techniques were both predictive of larger benefit from the rationality community even though neither variable is directly related to involvement in the community.  That makes these effects less open to some of the alternative explanations which could account for the community involvement effects and provides some validation of the self-report measure of benefits, although other causal paths are still a possibility (e.g., people who have changed more since they started reading LW may have come to have more of a growth mindset and also report more benefits).

II. Acquiring New Techniques

"On average, about how often do you find another technique or approach that successfully helps you at being more rational / more productive / happier / having better social relationships / having more accurate beliefs / etc.?"

The average response was a 2.23 (SD = 1.31) on a 1 to 8 scale where 2 is “About once every six months” and 3 is “About once every 2 months.”  This can be interpreted more intuitively as acquiring one new technique every 146 days (as a geometric mean).[3]

Which variables were associated with acquiring useful techniques more often?

Only some of them.

LW friendships and CFAR workshop attendance again had significant effects. The other two forms of in-person LW involvement, LW meetups and LW romantic partner, were also predictive of acquiring more techniques, but those effects did not remain significant in a regression controlling for the other variables. Time per day on Less Wrong had a weaker but reliable positive relationship with acquiring new techniques, while LW background had a significant relationship in the opposite direction: people with more LW background acquired fewer techniques. Younger age and growth mindset were again predictive of more benefit.

Based on the results of a regression, here is the number of days per new technique acquired (sorted by effect size, smaller numbers indicate faster technique acquisition).[4] In this list, both the number of days given and the order of the list reflect the results of the regression which controls statistically for the other predictor variables. (* = p < .05, ** = p < .01).

85 days: LW friendships *
87 days: Age (younger) **
95 days: Attended a full CFAR workshop **
114 days: LW romantic partner (p = .21)
118 days: Growth mindset **
174 days: LW background (negative effect) **
131 days: Time per day on Less Wrong  **
151 days: Regularly attend meetups (p = .63)
146 days: all respondents

The pattern that was apparent on the self-report measure of benefit from the rationality community – that in-person interactions were more predictive of benefits than online participation – was even stronger on this measure. Attending a CFAR workshop and LW friendships had the largest effects, and these effects seem to be cumulative. People who both attended a full CFAR workshop and interacted with LW friends “all the time” (n = 39) acquired a new technique every 45 days on average, while people who had no in-person interaction with LWers by any of the 4 variables (n = 824) acquired a new technique every 165 days.

Some of the alternative explanations for the effects on self-reported benefit seem less plausible here. For example, it seems less likely that people who have LW friendships would say that they try and acquire more new techniques out of a general tendency to say nice things about communities that you're a part of. Alternative causal paths are still a clear possibility, though. People who tend to try more things may be more likely to go to LW meetups, sign up for CFAR workshops, or move to a city where they can hang out in person with people from their favorite website.

III. The Process of Trying & Acquiring New Techniques

“On average, about how often do you *read or hear about* another plausible-seeming technique or approach for being more rational / more productive / happier / having better social relationships / having more accurate beliefs / etc.?”
“On average, about how often do you *try out* another plausible-seeming technique or approach for being more rational / more productive / happier / having better social relationships / having more accurate beliefs / etc.?”

On average, people heard about a new technique every 12 days and tried a new technique about every 55 days. That means that (at least according to the streamlined model: hear → try → acquire) people tried about 22% of the techniques that they heard about, and added about 36% of the techniques that they tried to their repertoire.[5]

Breaking down acquiring techniques into its two components, techniques tried and hit rate (techniques acquired divided by techniques tried) all of the effects discussed above involving acquiring techniques appear to be due to trying techniques, and not to the hit rate.  None of the variables discussed here were predictive of hit rate, and the variables that predicted acquiring techniques were similarly predictive of trying techniques (though in most cases the effect was slightly weaker).  In particular, trying techniques predicted self-reported benefit from the rationality community, and people with more LW background tried fewer techniques. People who both attended a full CFAR workshop and interacted with LW friends “all the time” (n = 39) tried a new technique every 13 days, while people who reported no in-person interaction with LWers (n = 849) tried a new technique every 65 days.

These data provide some evidence that, if CFAR workshops, LW friendships, growth mindset, and time on Less Wrong cause people to acquire more techniques, a substantial portion of the effect comes from getting people to try more things (and not just getting them to be more effective at trying the things that they already have been trying).

However, these data do not clearly pin down is different about people's process of trying things. One might expect that hit rate reflects how good a person is at choosing what to try and actually trying it (in a way that makes useful techniques likely to stick), so the lack of effect on hit rate indicates that the difference is just in trying more things. But if someone improved at the process of trying things, becoming more efficient at getting useful-for-them techniques to stick and setting aside the not-useful-for-them techniques, then that might show up primarily as an increase in number of techniques tried (as they cycle through the try things process more rapidly & more frequently). Or, a person who lowers their threshold for what techniques to try might start trying five times as many things and finding twice as many that work for them, which would show up as a drop in their hit rate (they'd also be adding useful techniques to their repertoire twice as fast).[6]

IV. Growth Mindset

Sample item: “You can do things differently, but the important parts of who you are can't really be changed” (reverse-scored).

Growth mindset – seeing important parts of yourself as malleable, and focusing on what you can do to improve – seems like it could be related to the process of benefiting from the rationality community in multiple ways.  Here are three:

  1. People with more of a growth mindset might tend try more things, acquire more useful rationality techniques, get more practical benefits out of the things they do.
  2. Being involved in the rationality community might cause people to shift towards a growth mindset from a fixed mindset.
  3. Relatively intensive involvement in the rationality community (such as living in a house with other LWers, or attending a CFAR workshop) might provide a bigger benefit to people with more of a growth mindset.

Item 1 is what we've been looking at in the analysis of acquiring new techniques and self-reported benefit, with growth mindset as one of the predictor variables.  The hypothesis is that people who score higher in growth mindset will report more benefit on those measures, and the data support that hypothesis (though these correlational results are also consistent with alternative causal hypotheses).

Item 2 identifies a hypothesis which treats growth mindset as an outcome variable instead of a predictor variable: do people who regularly attend LW meetups have more of a growth mindset? Or those who have more LW background, or who have attended a CFAR workshop, or who have LW friends, etc.? This hypothesis is relatively straightforward to examine with this data set, although the correlational design leaves it an open question whether involvement in the LW community led to a growth mindset or whether having a growth mindset led to people getting more involved in the LW community.

When looking at one variable at a time, each of the measures of in-person involvement in the LW community is significantly predictive of growth mindset. In order of effect size (given in Cohen's d, which counts standard deviations), growth mindset was predicted separately by LW romantic partner (d = 0.42), attending a CFAR workshop (d = 0.21), LW friendships (d = 0.20), and regularly attending meetups (d = 0.15). However, when controlling for the other predictor variables, only having a LW romantic partner remained statistically significant (d = 0.46, p = .03) and attending a CFAR workshop remained marginally significant (d = 0.18, p = .07); LW friendships and meetup attendance became nonsignificant (d < 0.10, p > 0.3).

LW background showed the opposite pattern: it was not related to growth mindset on its own (r = -0.04, p = .13), but it became a highly significant predictor of lower growth mindset when controlling for the other variables related to LW involvement (r = -0.11, p < .01). One plausible causal story that could explain this pattern of correlations is that people who are high in growth mindset who get involved in the website are more likely to also get involved in other in-person ways, while those lower in growth mindset are more likely to just stick with the website. This would lead to the negative relationship LW background and growth mindset when controlling for in-person LW involvement. According to this causal story, growth mindset is a cause of in-person LW involvement rather than a consequence.

Younger age was the strongest predictor of growth mindset, whether controlling for other variables (r = -0.15, p < .01) or not (r = -0.19, p < 0.01), and time per day on Less Wrong was not a significant predictor.

Item 3 from the list predicts an interaction effect between growth mindset and involvement in LW: the benefit of greater involvement in the LW community will be stronger among people high in growth mindset (or, equivalently, the benefit of growth mindset will be stronger among people who are more involved in the LW community). This hypothesis is particularly interesting because this interaction effect seems more plausible under the causal model where LW involvement and growth mindset both cause greater practical benefits than it does under the alternative causal theory that competence or a tendency to try things causes in-person LW involvement.

When predicting self-reported benefit from the rationality there was no sign of these interaction effects, whether looking at the predictor variables one at a time or including them all in a multiple regression. Growth mindset was an equally strong predictor of self-reported benefit for people who are closely involved in the LW community (by each of the various measures) and for people who are less closely involved in the LW community.

When predicting acquiring new techniques, these interaction effects were significant in several cases.[7] A growth mindset was associated more strongly with acquiring among techniques among people who regularly attend LW meetups (p = .003), people who are younger (p = .005), people who have attended a CFAR workshop (p = .04), and (with marginal statistical significance) among people with LW friendships (p = .06).  In a multiple regression that included each of these variables, none of these interaction effects was individually statistically significant except the age x growth mindset interaction (presumably because of the various forms of LW involvement were all associated with each other, making it difficult to tease apart their effects).[8]

These results are consistent with the model that the various forms of in-person involvement with the rationality community are especially helpful at producing practical benefits for people who are high in growth mindset.



With this correlational research design there is a limit to how well we can distinguish the hypothesis that LW involvement leads to benefits from other causal stories, but each of the three main variables that we examined were related to in-person LW involvement in ways that were consistent with this hypothesis.

People who have been involved with the in-person LW/CFAR community were especially likely to indicate that their life is better due to the LW community. They tended to report that they tried out and acquired new useful techniques more frequently, especially if they were also high in growth mindset. If spending time with LWers or attending a CFAR workshop leads people to try more rationality-related techniques, find more things that work well for them, and reap the benefits, then these are the results that we would expect to see.



[1] The 4 mindset questions on the survey were taken from Dweck's book Mindset (p. 13). These questions and others like them have been used to measure mindset in many published studies. Many of the questions that have been used focus more narrowly on mindset about intellectual ability, while these four questions deal more broadly with personal qualities.

[2] Unless otherwise noted, all reported effects are significant both in tests with only the single predictor variable and also in tests which controlled for the other predictor variables. A regression was run predicting benefit based on the LW involvement variables and age (growth mindset and acquiring new techniques were not controlled for, since they could be consequences of LW involvement which mediate the benefit). Though all three levels of the categorical variables were included in the regression, the effect size used to order the variables in the list was calculated as the standardized difference in least square means between the highest level of the group (e.g., regularly attend meetups) and the lowest level (e.g., never attend meetups), leaving out intermediate levels (e.g., occasionally attend meetups). To estimate the effect size of continuous variables, the correlation coefficient was translated into an equivalent standardized mean difference by the formula d = 2r/sqrt(1-r2).

[3] The 8 response options were coded as a 1-8 scale, which was used for all analyses. Each scale point indicates a 3-4x multiplier in how often a person acquires new techniques. This 8-point scale can be interpreted as a log scale for the variable "days per technique acquired" (they are associated approximately by the equation 7*3^(5-x)) so a mean on this scale is equivalent to the geometric mean of the number of days. For example, a 3.5 on the 8-point scale translates into 36 days, which is the geometric mean of 21 days (a 4 on the scale) and 63 days (approximately a 3 on the scale).

[4] For categorical variables, the number of days is based on the least squares mean for the highest level of the group (e.g., regularly attend meetups). For continuous variables, it is based on the regression equation predicting the values one standard deviation above the mean of the predictor variable.

[5] On the 8 point scale, “heard about” has mean = 4.48 (SD = 1.62) and “tried” has mean = 3.12 (SD = 1.56).  Rate of trying is simply “trying” minus “heard about,” mean = -1.37 (SD = 1.42), and hit rate had scale mean = -0.94 (SD = 0.84). These numbers can also be interpreted as being on a log base 3 scale, so -1 on the hit rate scale corresponds to an actual hit rate of 1/3 (1 technique acquired for every 3 techniques tried).

[6] Trying techniques can be further broken down into two components, hearing about techniques and percentage tried (techniques tried divided by techniques heard about). The data suggest that both are relevant, but they are harder to tease apart with the limited statistical power of this data set.

[7] When looking at a single categorical variable, I only looked at the highest level of the group and the lowest level, leaving out the intermediate level. For example, I tested whether growth mindset was more strongly related to acquiring techniques among people who regularly attend meetups than among people who never attend meetups (leaving out the group that occasionally attends meetups). In the regression including all predictor variables, I included the intermediate level groups (since otherwise it would have been necessary to exclude the data of anyone who was in an intermediate level group on any of the variables).

[8] When I combined the four variables related to in-person involvement into a single composite scale (scoring the highest level of involvement on each variable as a 2 and the lowest level as a 0), the interaction between growth mindset and this in-person involvement scale was statistically significant in a multiple regression predicting techniques acquired (p < .01).


71 AnnaSalamon 28 December 2013 11:25PM

Summary:  We outline the case for CFAR, including:

CFAR is in the middle of our annual matching fundraiser right now.  If you've been thinking of donating to CFAR, now is the best time to decide for probably at least half a year.  Donations up to $150,000 will be matched until January 31st; and Matt Wage, who is matching the last $50,000 of donations, has vowed not to donate unless matched.[1]

Our workshops are cash-flow positive, and subsidize our basic operations (you are not subsidizing workshop attendees).  But we can't yet run workshops often enough to fully cover our core operations.  We also need to do more formal experiments, and we want to create free and low-cost curriculum with far broader reach than the current workshops.  Donations are needed to keep the lights on at CFAR, fund free programs like the Summer Program on Applied Rationality and Cognition, and let us do new and interesting things in 2014 (see below, at length).[2]

continue reading »

View more: Next