paulfchristiano:
Desire for accurate beliefs. Many people don't feel that improving the quality of their beliefs is worthwhile. This is a combination of a failure to recognize exactly how wrong many of their beliefs are and a general perception that it doesn't matter anyway.
This is a tremendously important issue, to which I think the post doesn't give the necessary attention. (It's also generally neglected on LW whenever topics like these are discussed.)
Here on LW, there is often a tacit assumption that changing one's beliefs towards greater accuracy is always a good and rational thing to do. However, this neglects two crucial factors. First, re-evaluating and changing one's beliefs has a cost in time and effort, which must be offset by some benefits to make it rational (in the economic sense at least). Second, when it comes to beliefs whose relevance is primarily of signaling (rather than instrumental) character, one may well be better off having an incorrect belief with superior signaling properties.
Therefore, when people feel that improving the quality of their beliefs is not worthwhile, there is at least a possibility that they might be right about it, and it's completely unjustified to dismiss this attitude as false or misguided out of hand.
This is related to the perception that there is no free lunch: as humans we resist the suggestion that we can get anything for free, and acting intelligently seems to be considered too easy.
But this is a very accurate heuristic. It's basically a correct application of the weak efficient markets hypothesis. When it comes to beliefs that have no instrumental implications, it's relatively easy to learn about biases from sources such as LW sequences and then use this knowledge to find and correct a bunch of incorrect beliefs. However, when it comes to beliefs that are relevant for practical action, figuring out how to improve those is extremely difficult. There is no straightforward way at all to apply LW sequence-style general intellectual skills profitably.
Improving your beliefs may not be the best way to become successful, but my goal isn't increasing the number of successful people in the world. What I care about (tautologically) are my goals, and those are well served by convincing intelligent people who care about the world that having correct beliefs is essential to doing good. The efficient market hypothesis has nothing to say about this assertion. Beliefs that have "no instrumental implications" are important to understanding the long-term consequences of our actions. Of course now I am rely...
I believe that the number of smart rationalists has a profound influence on the quality of the future, so I would like to spread rationality at the maximum possible rate. In fact I believe this is the most important problem which I can work on right now. This post describes some of my beliefs about rationality outreach. The purpose of this post is threefold. First, by exposing my beliefs to criticism I hope to improve them. Second, by communicating my beliefs I hope to help others with similar goals. Third, by engaging in more concrete discussion of rationality outreach I hope to increase the probability that others will work seriously on the problem.
What Should be Taught?
"Spread rationality" is a rather vague and unhelpful goal. Here are three behaviors / habits of mind which I believe would make the world a much better place if they caught on among smart people.
1. Desire for accurate beliefs. Many people don't feel that improving the quality of their beliefs is worthwhile. This is a combination of a failure to recognize exactly how wrong many of their beliefs are and a general perception that it doesn't matter anyway. Society as a whole is assumed to be "basically right," even by people who verbally acknowledge rather serious failures of humanity's collective rationality. This is related to the perception that there is no free lunch: as humans we resist the suggestion that we can get anything for free, and acting intelligently seems to be considered too easy. The other side of this bias is a reluctance to believe that incorrect beliefs can really do any damage.
2. Mindful living. Many people direct incredible intelligence towards solving problems they encounter in their work, but fail to apply that intelligence to understanding the sources of their own beliefs/emotions, clarifying/refining their goals, or considering how to direct their resources to obtain those goals.
3. Initiative and confidence. It is very easy to believe that the world is a crazy place but that an individual has no power to change it. Regardless of how true that may be (my experience suggests not very), the world would probably be a better place if more really intelligent people internalized the belief that their actions change the world. Believing that your life is high stakes--that the potential upside is large--seems to be well correlated with doing great good.
Why Would People Listen?
Unfortunately, the majority of smart people don't believe that improving their own rationality is worthwhile (related to point 1 above). Someone who considers rationality a waste of time is unlikely to engage seriously with material presented in the context of rationality training, and they are likely to be driven off rapidly by perceived proselytizing. They are unlikely to take seriously a website dedicated to rationality, read a book dedicated to rationality, go to a talk about rationality, etc. Moreover, spreading rationality is not really about transferring knowledge. In order to change someone's behavior, they need a good reason to file advice about "rationality" into the compartment that actually affects behavior, and in general "because rationality is useful" does not constitute a sufficiently compelling reason (even if it is accepted as fact in the compartment for controlling beliefs).
Finding ways around this problem is one of the main difficulties of rationality outreach. I know one approach which seems to work, at least anecdotally and from personal experience: attach rationality as a rider on content intelligent people want to consume, and which requires or encourages engagement on its own terms. Some examples:
1. Educational materials. Though smart people don't generally want to improve their own rationality, they generally do want to learn. Interesting topics presented clearly for an intelligent reader seem to have a wide audience. Many subjects do not yet have particularly excellent expositions on the internet, and good teachers/mentors are always in short supply.
2. Competition. Interesting forms of competition exert a strong draw on many smart people. I believe that the world could support more formal competitions aimed at a broad range of ages.
3. Entertainment. The success of HP:MoR is some indication of the desire for intelligent science fiction / fantasy. In the case of video games there is an incredible shortage of high-quality games targetted at a really smart audience. Filling either of these vacuums well can draw a huge audience.
My belief is that intelligent rationalists can produce content significantly above common standards (from the perspective of an intelligent young person) in any of these forms.
How Is Rationality Attached?
Having an audience is not automatically useful. How do you convert people engaging with your content into people changing their behavior?
1. Choose effective content. Most critically: choose content which communicates the lessons you want to impart. Teach classes in a way that suggests that more is possible, encourages curiosity, conveys the usefulness of the material and its ability to anticipate and control real things, and gives a really smart student enough background and confidence to attack some important questions on their own. Run competitions and design games at which rationalists really do win, in which defeat really does suggest that more is possible, and in which your opponent's / other participant's innate virtuosity is not an excuse.
2. Force decompartamentalization. My experience is that once a source manages to change any aspect of your life, it becomes immensely more likely that it will change your life in other ways. Though this wades into dark arts territory, exploiting it seems to be important. If you introduce a student to a field they come to care about and do work in, the probability of affecting their behavior in other ways is increased enormously. More subtly, if you run a program or competition that requires any material investment (perhaps just going to a physical location) then its probability of affecting behavior may be significantly increased. If you design content that forces the audience to step back and re-evaluate anything at all about their life, then your chances of having a serious impact are seriously increased.
3. Foster the impression that individuals influence the world, not just the other way around. Talking about rationality explicitly in general is probably not going to get traction except in special cases. Trying to convince someone that their actions change the world is flattering enough that it might work if there is not enough resistance. If you really believe this, then the quality of your beliefs starts to matter. The possibility of real improvement becomes concrete, and understanding why improvement is hard becomes a priority. I think I might be being a little too optimistic here by believing that you can do any good in this way, but it deserves trying.
4. Channel the community. If given the opportunity, communitites often crystallize around engaging content. I have little expertise at influencing communities, but I can say empirically that the direction of community discussion and involvement can be significantly influenced by authority, either directly or indirectly.
Who is the Audience?
I believe that outreach is more likely to succeed and more important when the audience is intelligent. Until I have a better understanding of effective outreach, I intend to focus on an extremely small slice of the population at the upper end of general intelligence. For example, I am interested in outreach to high school students. I currently intend to target materials at what I believe to be the top ~10,000 students based on my experience. Details this precise are likely to change, but this at least gives a sense of what I am talking about.
How?
Supposing I decide that trying desperately to spread rationality is the best use of my time: how do I get there from here? What do I actually do? Of course a reasonable plan (in great generality) is to earn as much money as possible and give it to someone you trust to spend it as well as possible. The question is whether it is possible to do better. I believe that in some cases it is. Here are my thoughts on what we can do.
1. Invest free time. Cultivate hobbies and interests that allow you to spread rationality for fun, or when you need a break from work. Teaching classes or engaging in mentorship can help you understand how people learn and how they react to attempts at spreading rationality; acquire evidence and share it. Develop online resources that interest you--games, fiction, interactive tutorials, blogs--which might spark an interest in rationality or self-improvement. More importantly, try and gain an understanding of how smart but not necessarily rational people respond to efforts. If you can't find anything that interests you enough to do for fun, if you think the problem is really important you can view the project as work on the side.
2. Make money. People pay money to participate in academic programs, advertisers pay money for access to a well-defined audience, and people consistently pay money for valuable content. However you feel about this, I currently believe rationality outreach is only going to succeed on a massive scale when it is backed by a combination of passionate rationalists and a good enough business model to keep them alive and support rapid growth.
3. Engage in meta-outreach. I have started by describing my thoughts on how to engage in effective rationality outreach rather than why I believe rationality outreach is important, but I would have started the other way around if the atmosphere on LW were different. Regardless, you should think carefully about the importance of rationality outreach. If you believe that the problem is really urgent and have a cogent argument, you should make every effort to encourage other rationalists to engage with the problem. I also believe that thinking seriously and concretely about how to best achieve a precise goal is an excellent and necessary drive for improving our rationality, both individually and collectively.
What are My Plans?
I have an unusually good background in theoretical computer science and mathematics, know people who have a lot of access to high school students (and have good familiarity with a large clique of brilliant highschool students in the US), and am financially stable. I realize that not giving the money to any fixed charity is tantamount to believing that charity should be giving money to me. I am currently on track to become a research scientist, and it is going very well. I have not been thinking about rationality outreach for long, but the calculus has come out overwhelmingly in its favor and I am rapidly updating my beliefs. I am not yet willing to disrupt my current life plan in a way that would be difficult to reverse.
I believe the best first step for me is to start using my spare time (which I can make as large as 30 hours a week) to develop and evaluate online learning/testing resources. Unless I encounter an unexpected obstruction, I would then like to start recruiting other types of talent, testing more broadly, and looking for a business model which can effectively support continuing development along these lines. I may well fail to execute this plan, but it would be either because my beliefs changed or a better plan suggested itself (or, most likely, both).
Independently, I am interested in running an academic program for smart high school students. The existence of successful summer programs with a strong academic focus suggests that the market exists, and I believe that I could get enough support to run a program at least once (with future outings facilitated by success). I believe the main obstacles are the non-academic difficulties of running such a program and the recruitment of talented faculty, and the main benefits are probably an increased understanding of how to run a program and how to engage with smart high school students.